Competing methods of measuring traffic
online leave advertisers, investors, and even Net companies almost flying
blind
Life is good for 27-year-old Seth J.
Sternberg. A year ago, he dropped out of Stanford Business School to work
full-time on Meebo Inc., an easy-to-use service that has solved one of instant
messaging's nagging problems: the inability to communicate with people who use
an IM service other than yours. Today Meebo is going gangbusters. It has raised
$3.5 million from the Silicon Valley crème-de-la-crème, including Marc
Andreessen of Netscape fame and venture capital heavyweight Sequoia Capital.
More impressively, the service attracts almost a million people every day, who
swap more than 60 million messages.
There's just one hitch: Sternberg
and his co-founders have a hard time proving the site is as popular as they say
it is. Look up Meebo's Web traffic using the comScore Networks Inc. service, and
you'll find that a European competitor eBuddy.com is four times as big. Alexa, a
competing Web measurement service owned by Amazon.com Inc. (AMZN ), shows Meebo is bigger.
Which is true? Probably neither. Sternberg's best guess is that the two rivals
are about the same size. Yet even he doesn't know for sure.
The dirty
little secret of Silicon Valley is that no one knows exactly who is going where
on the Web. That flies in the face of the impression that online advertising is
the most dependably trackable ad medium of all time, a big reason spending on
Web ads is expected to grow 33% this year, to $16 billion. But confusion over
traffic measurement could cast a chill over the Web 2.0 craze. Valuations for
startups such as Facebook Inc. and YouTube Inc. appear to be doubling every few
months, but those numbers are based on traffic figures that could be
misleading.
From the start, measuring online traffic was a juggling act.
Rather than simply relying on a Web site's traffic reports, advertisers
traditionally compared that data with information from Nielsen//NetRatings Inc.
(NTRT ) and comScore,
independent services that recruit Web surfers to record their mouse clicks.
Those outfits argue that there are many reasons not to just count the clicks off
a Web site's server logs. For instance, comScore points out that servers would
count pop-up ads as a page view if the tracking service didn't filter them
out.
Independent traffic analysis becomes more important
as bigger chunks of advertising flow online and the threat of "click fraud," which
inflates ad bills, grows bigger. No wonder that a host of newer services, such as Alexa and
Hitwise, are highlighting the weaknesses of the older traffic-measuring
companies and are muscling onto the scene with alternatives. By providing some
free traffic data via their Web sites, these outfits make it easier for anyone
to publish an estimate. But they also have their own blind spots and are making
side-by-side comparisons vastly more confusing.
To see why there's an
opening for new ways of measuring traffic, look at what has happened to the old
standard for gauging online growth, the page view. As the Internet evolved
during the 1990s, advertisers came to rely on the number of pages a site served
up each month as their most reliable metric. With the rise of new programming
and distribution technologies, however, page views suddenly look less relevant.
For instance, the beauty of a site such as Meebo is that it is built with
software tools called Ajax, which speed up Web surfing. When you log onto Meebo,
instead of loading a new page for every mouse click, only the log-in section is
loaded. But no matter how long people stay on Meebo, they're technically viewing
only one page.
Meebo isn't selling advertising yet. When it does, the
company's executives will be challenged to persuade ad buyers to consider
measurements other than page views. In June, Sternberg invited comScore's
researchers to his Palo Alto (Calif.) office to look at his company's server
logs. "Here is our data, and here is your data. Something is wrong!" he told
them, to no avail. Sternberg and his co-founders have thrown up their hands and
have decided to publicly disclose all their internal numbers. One thing they're
counting on is that people will take into account the amount of time members
spend on the service.
Ajax isn't the only technology that's upending
traditional Web measurement. Real Simple Syndication, or RSS, lets people sign
up to have news articles, blog posts, or audio interviews from their favorite
sites sent directly to their computers. But since they aren't surfing around,
none of the sites gets credit for the page view.
Then there are widgets,
interactive icons that can be installed on a wide array of sites. Install the
Meebo Me widget on a MySpace page, and people can just click on that to IM you
when they visit the site. More than 40,000 Meebo Me widgets have been installed.
Because the widgets don't direct you back to Meebo's site, they don't show up in
the Meebo traffic figures.
For advertisers, the problem is that while any
one method of measurement may capture certain Web technologies or demographics,
it misses others. An alternative to counting page views, for instance,
is to measure reach, which is a calculation of each site's usage as a percentage
of all Net traffic. Alexa, which tracks Web surfers through a downloadable search
toolbar, combines that approach with counting page views. But Alexa is limited
because it compares reach only among the sites its users visit. Results also
appear to skew more toward sites favored by techies rather than the wider Web,
says Ed Sim, managing director of VC firm Dawntreader Ventures. Niall
O'Driscoll, Alexa's vice-president of engineering, says the company doesn't
believe that's the case.
Web outfits seem to agree that Alexa is flawed,
but they continue to rely on it because the data are so addictive. Since Alexa's
numbers are free and available online, they can easily be plugged into a
PowerPoint presentation or onto a blog, providing a quick-and-dirty way to get a
competitive snapshot. Blogs cite Alexa as gospel, and its graphs are part of
nearly every startup's pitch to investors. "It's a giant pain," says George
Zachary of Charles River Ventures. "If someone came up with something better,
I'd fund them."
LIMITED
PICTURE If Alexa favors early adopters and techies, the
opposite complaint is made about comScore and Nielsen//NetRatings. Some VCs,
startups, and established companies don't think these ratings methods adequately
capture the growth of new services. Yahoo! Inc. (YHOO ) saw this firsthand with
del.icio.us, a Web bookmarking site the portal bought last year. ComScore showed
declining traffic at del.icio.us over the summer, but Yahoo says it had at least
10% monthly growth.
The best hope for clarity is for Web ad giants and
small fry alike to work with the established measurement services to improve
their tracking ability. ComScore and Nielsen//NetRatings are beginning to
respond. ComScore, for instance, now reports the number of requested videos at
online video sites, rather than page views or unique visitors.
Some
observers expect that in time a variety of metrics, such as time spent online,
will be applied to different services until one measurement that combines a set
of factors can emerge. Until that happens, though, the Internet will have to
deal with the discrepancies. And Web metrics, like company valuations, will
remain a
crapshoot.
