Former IDG colleague Matt McAllister tweeted a link to a wonderful post by Stijn Debrouwere, a Knight-Mozilla OpenNews fellow who is “loosely affiliated with the Guardian’s data science team in London.”
Debrouwere tackles the futility of newsroom analytics and measurement, something I lived at Forbes.com and IDG and then Lenovo when I ran web analytics, mainly Omniture (now Adobe) and Google Analytics. As Debrouwere sarcastically notes, putting dashboards on big flat panel screens and making them really big makes them really important. He compares media executives who cling to their dashboards to New Guinea primitives waving at the sky and waiting for the cargo to come to them the way it used to come during WW II when the US Army was fighting the war.
Some zingers from his post:
- “If you’re like most people, you don’t stray very far from the dashboard you get when you log in. You stare and squint and hope insight will magically manifest itself.”
- “There’s nothing like a dashboard full of data and graphs and trend lines to make us feel like grown ups. Like people who know what they’re doing. So even though we’re not getting any real use out of it, it’s addictive and we can’t stop doing it.”
- “There’s enough social media analytics tools to merit listicles that helpfully introduce you to the top 8.”
- “You’re supposed to put these dashboards up on a wall, on a huge plasma screen. Because of course numbers are twice as persuasive if you make them twice as big.”
- “Metrics are for doing, not staring.”
- “I honestly can’t recall the last time I’ve looked at our pageviews. I know it wouldn’t get me anywhere.”
http://stdout.be/2013/08/26/cargo-cult-analytics/
This Big Data thing has a lot of people confused, myself included. And for good reason. We think of this big database in the cloud, doing something so big and difficult that it requires lots and lots of processing power and a thing called “Hadoop,” watching individuals interacting like so many ants with companies and stuff in real-time like some scary NSA spooky datacenter in Utah listening to all our phone calls (but respectful of our privacy of course), figuring out patterns and trends and opportunities and MAGICAL INSIGHTS in REAL-TIME. So let’s get ourselves some of that there Big Data and save the company, be like Google, A/B test the shit out of stuff, and get rid of the Highest Paid Person’s opinion, blah, blah blah…..
First, a lot of people, including me, suck at math and statistics and so we overcompensate by regarding any numbers and the word “quantitative” as mystical. If it has a number attached to it (not an adjective) it must be important.
Second, the legends around Big Data and the magical insights they deliver to retailers are kind of cool to consider and become mythical. Target can tell when a woman is pregnant based on her shopping history. Wal-Mart figured out 80% of its store visitors turn right when they enter the store. The promise of finding one of those awesome insights is just too compelling to miss. The problem is staring at a dashboard doesn’t equate to discovering an insight. Hence a lot of us are like the Cargo Cultists.
Third, modern management is obsessed with measurement — former colleague Lew McCreary called it the Tyranny of Metrics — and the Pokemon Model of Got To Get ‘Em All applies to equating Big Data with Big Data Collection, which yields the ugly phenomenon coined by Google metrics guru Avinash Kaushik: “Data Puking.” The admonition that, “You can’t manage what you don’t measure,” has built a corporate culture more concerned with looking buttoned-up, on the ball, and obsessively accurate than being intuitive, empathetic and innovative.
I was the guy who built these dashboards, peered at them for magical insights, puked them at my bosses, and over time I started to get really cynical and put tired old quotes pissing on measurement into my PowerPoint presentations:
Einstein: “Not everything that can be counted counts, and not everything that counts can be counted.”
Warren Buffett:”They studied what was measurable, rather than what was meaningful” –
I know Debrouwere’s post appealed to me because he was specifically addressing metrics in the newsroom — a place I spent most of my career. But it also struck a current chord with me because of my work for clients, all of whom cite Big Data incessantly as a force for disruption and transformation, yet haven’t the faintest clue of how to harness it or whom the Oracle will be in their organization who will study the digital tea leaves and come up with the single “AHA!” that will make them Measurement Legends.
I’m pretty sure when you ran Forbes you were one of the only web sites in the way early 90’s to have a Math Masters candidate handling your web stats.
I remember Christine often telling your sales guys to fuck off because the numbers they wanted to use were total BS. Battle of the Danish math geek versus the sales weasels.
Then again, I also remember her at a well known bank doing risk management and being assigned to CDOs in 2003 and saying, “these things are total shit and make no sense and no one understands the models at all. I can barely make sense of them and I’m the only one who has a math degree.” And oh, yeah, she found a ton of errors too.
The good ole days.
I don’t think anyone had anything else back in 1994 but metrics. Problem is that 20 years later, too many companies haven’t moved past them. And the bigger they get, the more they seem to rely on them. The Microsoft forced ranking system or the Google internal systems tend to poison from within.
Then again, we started with the first top ten list — thou shall not… — and haven’t stopped since