⬅️ My First Year at VCE 🧭 Private Clouds Ahead ➡️
On Metrics
Ah, metrics. Everyone loves a horse race. Of course, in a horse race it is not as if the horses are allowed to take a hyperspace jump to various points in the track outside of the assumed finite length of a race track. Enter the dilemma of any so-called “marketing intelligence” service selling you a stopwatch to determine who wins the race.
Wait? What? See the article linked below my own footnotes. This mini-ramble is about Hitwise making a graph or something showing something growing or not growing or winning or not winning. I’m not picking on the article or Hitwise per se but the general notion of tracking service utilization in the fast paced world of iterative development with anything but the very broadest and most fluffy of metrics.
Specifically, I question the ‘moving target’ element that Google+ and any such property tracked would exhibit. Consider that the way Hitwise tracks [1] today may not necessarily be the way tracking is done as properties evolve into new signatures, A/B testing, deep personalization, mobile or closed loop networks, leverage of partners or complementary partners, or shifts in the delivery infrastructure to favor SSL enabled experiences throughout [2].
How do such services really compare the signature of today to the signature of next year? How do such services account for the possibility of egregious errors or failure to adequately capture the intentionally or unintentionally obtuse / obfuscated traffic indicators?
Indeed, the quality of the time spent and the sharing gestures (text updates vs. photos vs. movies) may be harder and harder to glean as this data is considered part of the privacy contract and the salable asset of the property. Remember, you are the one feeding the machine as a user. As someone much more clever put it, “if the product is free that means you are what is for sale”.
That may mean third party services that seek to classify and quantify user patterns are forced to disclose methodology in greater detail (ultimately owning up to how little they do know), striking deals with destinations to truly map such trends with a firehouse of data, or they are simply relegated to selling more art than science.
Perhaps being the ‘outside view’ will come to be an even more literal condition.
[1] http://www.hitwise.com/us/about-us/how-hitwise-works
[2] relegating most characterization to blunted techniques i.e. packet and flow based accounting
The article in question: http://venturebeat.com/2011/07/22/google-plus-ranking/
