On Metrics
Ah, metrics. Everyone loves a horse race. Of course, in a horse race it is not as if the horses are allowed to take a hyperspace jump to various points in the track outside of the assumed finite length of a race track. Enter the dilemma of any so-called “marketing intelligence” service selling you a stopwatch to determine who wins the race.
Wait? What? See the article linked below my own footnotes. This mini-ramble is about Hitwise making a graph or something showing something growing or not growing or winning or not winning. I’m not picking on the article or Hitwise per se but the general notion of tracking service utilization in the fast paced world of iterative development with anything but the very broadest and most fluffy of metrics.
Specifically, I question the ‘moving target’ element that Google+ and any such property tracked would exhibit. Consider that the way Hitwise tracks [1] today may not necessarily be the way tracking is done as properties evolve into new signatures, A/B testing, deep personalization, mobile or closed loop networks, leverage of partners or complementary partners, or shifts in the delivery infrastructure to favor SSL enabled experiences throughout [2].
How do such services really compare the signature of today to the signature of next year? How do such services account for the possibility of egregious errors or failure to adequately capture the intentionally or unintentionally obtuse / obfuscated traffic indicators?
Indeed, the quality of the time spent and the sharing gestures (text updates vs. photos vs. movies) may be harder and harder to glean as this data is considered part of the privacy contract and the salable asset of the property. Remember, you are the one feeding the machine as a user. As someone much more clever put it, “if the product is free that means you are what is for sale”.
That may mean third party services that seek to classify and quantify user patterns are forced to disclose methodology in greater detail (ultimately owning up to how little they do know), striking deals with destinations to truly map such trends with a firehouse of data, or they are simply relegated to selling more art than science.
Perhaps being the ‘outside view’ will come to be an even more literal condition.
[1] http://www.hitwise.com/us/about-us/how-hitwise-works
[2] relegating most characterization to blunted techniques i.e. packet and flow based accounting
The article in question: http://venturebeat.com/2011/07/22/google-plus-ranking/
Enjoyed this post?
Consider supporting my sponsor!
🔓 Unlock Your Best Self with Lida Coaching! 50% off regular coaching rates until July 31, 2026!
A quick message from Lida:
With 18 years building, marketing, and scaling products at startups, I work with founders and leaders stuck between where they are and where they need to be. I help you cut through the noise, make sharper decisions, and get your team executing with clarity instead of chaos.
Most founders mistake complexity for strategy. They keep adding when they should be subtracting. When I'm brought in, the team is usually doing a lot right, and the real issue is focus and direction. I diagnose what's actually going wrong across product, positioning, and execution, then reset direction and get everyone aligned around one clear path forward that will lead to growth.
A bit of what I've done: I was the first PM on a fintech product that hit 10,000+ users within months, built a national incubator that supported over $500M in startup funding, and led teams at 23 Design before it was acquired by frog (the design team behind Apple’s first products). I've spoken at Google, Microsoft, and LinkedIn, etc.
🔓 Unlock Your Best Self with Lida Coaching! 50% off regular coaching rates until July 31, 2026!
