Lots more noodling in Blogistan about the “lies, damn lies, and statistics” of emerging media. As I draft my first column for a major business magazine’s online version on this very topic, I am gathering string.
First from Steve Safran at Lost Remote, via Scoble, is this commentary sparked by the zeFrank/Rocketboom nerd fight.
Lost Remote TV Blog
“There’s simply no way for us to measure viewership of podcasts. But we keep reporting numbers from the networks, big sites and podcasters without questioning them (guilty as charged) and we need to step back for a moment and ask: “How do we figure out who is really watching or listening to our podcasts?” Then we have to admit “We don’t know.”
Let’s back up a second and understand why this stuff is even measured in the first place (above and beyond bragging rights along the lines of mine-is-bigger-than-yours, which I call the “Time Warner PathFinder Effect” back in the day when Time Warner’s execs boasted about getting “millions” of hits the way McDonald’s quotes the nebulous statistic of “Billions Served). Why do we care about accuracy in media measurement? Especially since the old media measured crap like “reach” and “audience” based on the numbers of cars projected to crawl past a billboard on Highway 101 during rushhour and the number of Nielsen households who pressed the right button at the right time during the Beverly Hillbillies? Or magazines that claimed precision on completely freaky statistics like “pass-along” (which would seem to count a moldy copy in a dentist’s office about a gazillion times) or “recall?”
The reason that numbers matter, aside from the tyrannical rise of the “measure to manage” actuarials in the CFO’s office who worship at the altar known as “ROI”, is that marketers are still buying at the head of the long tail –where things like “mass” and “reach” seem to matter.
Now we find ourselves in the wonderfully mechanical world of web logs, when every hit, download, and interaction is logged by our Apache servers, and suddenly the Web world has been held up as the most accurately measured media in history.
Hah!
I’ve gamed web logs. Everyone has. I can pull some pearl out of a web log and say, “Aha, Left-handed Latvians prefer my site on Sundays!” Now, as we exit the era of Page Views and enter the era of Engagement, things get even squishier and gamier. Downloads versus views? Good luck.
My confrere, Jim Hazen, asks the simple question today about third party verification.
“Should there be some sort of official, universally accepted standards that all companies adhere to in determining true traffic? Like SEC accounting rules for Web Metrics? How bought a federally mandated web metrics tag for all sites?! Maybe some crazy alogrithm that can be based on Google searches or something, since they essentially run the web now. Not sure what the answer would be, but it’ll probably end up as another highly questionable reach calculation like the ones they’ve used forever with tv and radio.”
Jim, welcome to the world of ComScore, and Nielsen, and that total farce, Alexa.
I’ve said it before and I’ll say it again, the burden of proof and measurement can’t be abdicated to a third party measurement system. The Internet Advertising Bureau has fallen down in not presenting a solid set of standards for metrics reporting. The equivalent of the BPA or the ABC hasn’t emerged for online, and in the end, it comes down to the buyer has to beware. The only statistic that matters for a person renting eyeballs is this: did it work for me? Did the traffic to my site, the click through on that search term, the download of that funny viral yield anything of value to me.
In other words, stop pointing a radar gun at the pitchers. Worry about whether the catcher is on the ball.