Online advertising is 12 years old and the industry still can’t get its act together around standardized metrics and traffic measurement. This morning’s New York Times reopens the perennial wound with a lead story in the business section that features the requisite hand wringing by online publishers over the gulf between their server logs and the traffic reported by the rating agencies like Comscore and Nielsen/ NetRatings.
Begin by looking at the role of the rating agencies and why they persist. Rating agencies use panel-based reporting to sample internet traffic and extrapolate gross traffic scores for significant media. Why they exist is a mystery to me, a vestige of broadcast media when television and radio was essentially measured through statistical sampling as there was no machine connection between the device and the broadcaster. Is it laziness on the part of the media planners? The buyers who evaluate the traffic and demographic profiles of sites before building plans for their clients? A normalized, convenient view across all sites in one convenient, but ultimately inaccurate package?
“Other big media companies — including Time Warner, The Financial Times and The New York Times — are equally frustrated that their counts of Web visitors keep coming in vastly higher than those of the tracking companies. There are many reasons for the differences (such as how people who use the Web at home and at the office are counted), but the upshot is the same: the growth of online advertising is being stunted, industry executives say, because nobody can get the basic visitor counts straight.
“You’re hearing measurement as one of the reasons that buyers are not moving even more money online,” said Wenda Harris Millard, president for media at Martha Stewart Living Omnimedia and, until June, chief sales officer at Yahoo. “It’s hugely frustrating. It’s one of the barriers preventing us from really moving forward.”
Rating agencies may exist to provide some degree of third party verification to an industry marked by different server architectures with different traffic logging and measurement procedures. While Microsoft IIS and Apache based servers may log activity differently, the real measure comes from the traffic analysis packages deployed by the site owner. Log analysis – the old model of the mid-90s, has been replaced by beacon-based systems such as Hitbox and Omniture – however they are accurate only as far as those beacons/cookies are accepted by privacy paranoid users.
Publishers have made a sport of discrediting the rating agencies. They don’t accurately count at-work panelists, they don’t extend to international audiences, etc. The entire concept of panels – users recruited to install monitors on their browsers which report activity back to the rating agency – is very flawed and should be stopped. Take the best known public example of browser-based panels, Alexis, and look back to the last presidential campaign when site managers gamed the system by having their supporters download the Alexa client to overweight the statistical samples. While ComScore and Nielsen can control and pledge some normalization in their panel composition, the simple fact is this:
There is no place for a sample when the internet is precisely measured down to every call to a server.
The issue is how to get to an apples-to-apples basis for self-reported web traffic. Here is where the Internet Advertising Bureau could make an impact beyond its current role of establish technical standards. As I’ve blogged over and over – the magazine industry, the sloppiest reported medium there is, has two independent systems for verifying circulation – the Audit Bureau of Circulation and the Business Publishers Association. Publishers routinely get their circ lists scoured and scrubbed and the result is a standard reporting format. Why can’t the IAB bring the same to the web? I know I am hopelessly naïve on this – apologies to Randall Rothenberg in advance – but over a decade and this business is still being hobbled by a lack of transparent accountability. I only can assume the IAB has tried to get audits in place, but something – publisher resistance, technical hurdles, lack of expertise … something is holding the obvious from occurring.
So how can the IAB finally get audits established? 1) force participation 2) involve the Web Analytics Association and the metrics vendors and define the metric/analytics tool makers’ reporting standards and methodology 3) vigorously discipline anyone who is caught inflating reports 4) get the media buyers aboard 5) establish an advertiser sub-group to push for those standards, advertisers who will vote with their feet away from any site that refuses to participate or who is caught inflating 6) persuade ComScore and Nielsen to drop their panel process and morph into the role of auditors and demographic profilers, working to initiate third-party audience surveys with the publishers.
Here is the IAB congratulating ComScore for cooperating with an audit into ComScore’s measurement process. I have a proposal – get them to drop the measurement process and initiate an audit of the publisher’s web logs.
Update:
Randall Rothenberg at the IAB posts a great take on the Times article on his “clog” (column and blog) “I, A Bee” He, and Derek Slater at CSO point out the obvious which I overlooked — that it’s one thing to count, it’s another to profile the audience. On that, no server log will give an indication. However, to further diss the research model, every publisher is out there laying claim to the largest audience of “left-handed Latvians between the age of 18 to 23” based on their own survey research. I guess a media planner just wants to turn to the database and seek the right composition and go from there…..
That would be a good start. It will answer the question “how many?” though not the question “who?” which is of greater interest to me. But you can only eat the elephant one bite at a time, I guess.
David, you already did what I was going to do — refer folks to my own “clog,” http://www.randallrothenberg.com, and to http://www.iab.net, for a history of the audience measurement controversy and what IAB is doing to resolve it. In response to your specific suggestions, let me elaborate.
1) force participation: A trade association really can’t force much of anything without risking a violation of antitrust regulations. We exist to encourage, and to network groups and companies together openly in ways that can coalesce solutions. This we’ve been doing, and with great success this year, particularly in encouraging the major audience measurement vendors to undergo auditing and accreditation by the Media Rating Council.
2) involve the Web Analytics Association and the metrics vendors and define the metric/analytics tool makers’ reporting standards and methodology: The IAB has been the leading developer of standards and guidelines for measurement and other advertising elements for much of its 10-year history. The metrics vendors are, in fact, very involved with us on that, serving, for example, on our Audience Measurement Committee. The W.A.A., by contrast, specializes in developing definitions, not in the collaborative development of guidelines, which are actually operating principles.
3) vigorously discipline anyone who is caught inflating reports: We could, but this has not been an issue that’s risen to a significant level. In fact, this is the first time in my year’s tenure it’s come up.
4) get the media buyers aboard: The American Association of Advertising Agencies was a major supporter of our drive for auditing this year, and attended our May “Summit Meeting” with the audience measurement companies.
5) establish an advertiser sub-group to push for those standards, advertisers who will vote with their feet away from any site that refuses to participate or who is caught inflating: The Association of National Advertisers was also a vocal supporter of IAB’s auditing drive, and attended our May Summit, too. That said, until now, interactive advertising has been a relatively small part of advertiser budgets, and marketers’ general response to the discrepancy problem has been “we want the media and the agencies to work it out.” Now that budgets are growing significantly larger, we’re seeing more advertiser interest in resolution.
6) persuade ComScore and Nielsen to drop their panel process and morph into the role of auditors and demographic profilers, working to initiate third-party audience surveys with the publishers: As you indicate in your end note, panels remain necessary for important parts of advertising analysis, planning and buying. As I note in my “clog,” our goal should not be to seek a single way of measuring, but multiple, audited data sources, used for varying purposes, that can be brought together to create a richer picture of the truth for marketers.
Keep writing the good write!
r2