Malsurement: Measuring The Wrong Thing

measurement-by-seamstressMusic industry pontificator Bob Lefsetz highlights how incredibly easy it is to be lured in to measuring the wrong things for the wrong reasons.

The quick summary:  Billboard, the company that lets you know Katy Perry is again at #1 in the charts, recently announced plans to include downloads and streams when calculating the Top 200 tracks.  Sounds great, right?

Here’s where we learn about measuring the wrong things for the wrong reasons.

Part of Billboard’s new calculation is an attempt to squeeze the new into the old: Ten downloads of a track equal one album purchase.  When was the last time you purchased an album, let alone a physical CD?

Lefsetz captures the essence of this mistake. In an online world, streaming equals listening equals the right thing to measure…

“The only thing that counts is listens. Sales are irrelevant. Especially of albums…”

“But the whole [music] industry is based on albums so they don’t want to throw out the baby with the bathwater…”

“Do you see Netflix telling us how many views equal one DVD? Come on.”

Yikes.  He’s right.  I work in mobile today, which suffers from malsurement (that’s bad measurement) everyday.  Almost every data point measured in mobile is measured because that’s the way it’s always been done on your desktop website.

Best example: click-through rates.  You see an ad, you click on it.  This breaks down so fast on mobile, it’s just outrageous. Somewhere between 30-50% of all clicks are accidental, because you’ve got fat fingers, or you’re two martinis deep while checking tomorrow’s forecast.

Easy fix. Have everyone in the entire global ecosystem of mobile adopt better metrics.  Dammit.  That ain’t gonna happen, at least this decade.  Billions of dollars, millions of ad campaigns, and thousands of sales pitches all hinge upon click-through rates.

My point, if I have one, is that marketers, product managers, sales people, and the big bosses at companies large and small all struggle to measure the right data for the right reasons.

Measuring the right thing for the right reasons sometimes isn’t as easy as it’s made out to be.

But you should still try.


Comments are closed.