The answer is audits

Online advertising is 12 years old and the industry still can’t get its act together around standardized metrics and traffic measurement. This morning’s New York Times reopens the perennial wound with a lead story in the business section that features the requisite hand wringing by online publishers over the gulf between their server logs and the traffic reported by the rating agencies like Comscore and Nielsen/ NetRatings.

Begin by looking at the role of the rating agencies and why they persist. Rating agencies use panel-based reporting to sample internet traffic and extrapolate gross traffic scores for significant media. Why they exist is a mystery to me, a vestige of broadcast media when television and radio was essentially measured through statistical sampling as there was no machine connection between the device and the broadcaster. Is it laziness on the part of the media planners? The buyers who evaluate the traffic and demographic profiles of sites before building plans for their clients? A normalized, convenient view across all sites in one convenient, but ultimately inaccurate package?

“Other big media companies — including Time Warner, The Financial Times and The New York Times — are equally frustrated that their counts of Web visitors keep coming in vastly higher than those of the tracking companies. There are many reasons for the differences (such as how people who use the Web at home and at the office are counted), but the upshot is the same: the growth of online advertising is being stunted, industry executives say, because nobody can get the basic visitor counts straight.

“You’re hearing measurement as one of the reasons that buyers are not moving even more money online,” said Wenda Harris Millard, president for media at Martha Stewart Living Omnimedia and, until June, chief sales officer at Yahoo. “It’s hugely frustrating. It’s one of the barriers preventing us from really moving forward.”

Rating agencies may exist to provide some degree of third party verification to an industry marked by different server architectures with different traffic logging and measurement procedures. While Microsoft IIS and Apache based servers may log activity differently, the real measure comes from the traffic analysis packages deployed by the site owner. Log analysis – the old model of the mid-90s, has been replaced by beacon-based systems such as Hitbox and Omniture – however they are accurate only as far as those beacons/cookies are accepted by privacy paranoid users.

Publishers have made a sport of discrediting the rating agencies. They don’t accurately count at-work panelists, they don’t extend to international audiences, etc. The entire concept of panels – users recruited to install monitors on their browsers which report activity back to the rating agency – is very flawed and should be stopped. Take the best known public example of browser-based panels, Alexis, and look back to the last presidential campaign when site managers gamed the system by having their supporters download the Alexa client to overweight the statistical samples. While ComScore and Nielsen can control and pledge some normalization in their panel composition, the simple fact is this:

There is no place for a sample when the internet is precisely measured down to every call to a server.

The issue is how to get to an apples-to-apples basis for self-reported web traffic. Here is where the Internet Advertising Bureau could make an impact beyond its current role of establish technical standards. As I’ve blogged over and over – the magazine industry, the sloppiest reported medium there is, has two independent systems for verifying circulation – the Audit Bureau of Circulation and the Business Publishers Association. Publishers routinely get their circ lists scoured and scrubbed and the result is a standard reporting format. Why can’t the IAB bring the same to the web? I know I am hopelessly naïve on this – apologies to Randall Rothenberg in advance – but over a decade and this business is still being hobbled by a lack of transparent accountability. I only can assume the IAB has tried to get audits in place, but something – publisher resistance, technical hurdles, lack of expertise … something is holding the obvious from occurring.

So how can the IAB finally get audits established? 1) force participation 2) involve the Web Analytics Association and the metrics vendors and define the metric/analytics tool makers’ reporting standards and methodology 3) vigorously discipline anyone who is caught inflating reports 4) get the media buyers aboard 5) establish an advertiser sub-group to push for those standards, advertisers who will vote with their feet away from any site that refuses to participate or who is caught inflating 6) persuade ComScore and Nielsen to drop their panel process and morph into the role of auditors and demographic profilers, working to initiate third-party audience surveys with the publishers.

Here is the IAB congratulating ComScore for cooperating with an audit into ComScore’s measurement process. I have a proposal – get them to drop the measurement process and initiate an audit of the publisher’s web logs.


Randall Rothenberg at the IAB posts a great take on the Times article on his “clog” (column and blog) “I, A Bee” He, and Derek Slater at CSO point out the obvious which I overlooked — that it’s one thing to count, it’s another to profile the audience. On that, no server log will give an indication. However, to further diss the research model, every publisher is out there laying claim to the largest audience of “left-handed Latvians between the age of 18 to 23” based on their own survey research. I guess a media planner just wants to turn to the database and seek the right composition and go from there…..