The right metrics - Lessons from baseball and web analytics
Spring is finally here and with it comes baseball's opening day! I'll go ahead and admit it, I am a baseball junkie. I learned to read because of baseball as my parents taught me how to read boxscores in the paper so I could figure out how the Braves did the next morning. Its probably not a huge surprise I gravitated towards a field that deals with numbers based on my incessant memorization of baseball statistics. The interesting thing is that over time, I've come to realize that a lot of the stats that many held dear for so long were in reality meaningless.
Let me clarify by what I mean by the term 'meaningless'. In this case, what I am saying is some of the stats that have long been used to determine whether or not someone is a good hitter or not doesn't necessarily correlate towards runs and more importantly wins. We were programmed for so long to evaluate a hitter as being good if they batted over .300 for a season. The problem is that same hitter might never get a walk, hit only singles or hit .100 with runners in scoring position. As a result, that hitter generated less runs for the team and ultimately less wins because they were creating more outs and not getting into scoring position. Essentially, batting average wasn't the leading indicator for how good a hitter is even-though it was the gold standard since the game was invented.
Over the last 20 years or so, a menagerie of stats geeks (called sabermetrics) started to re-swizzle how baseball players were evaluated and as a result developed new and interesting stats that do a better job of correlating to success (such as wins). Starting with Bill James and later documented in Michael Lewis' "Moneyball", newer models were developed to figure out which hitters are actually producing as opposed to using old metrics like batting average and runs batted in. In the last decade, we've seen a dramatic shift in how general managers are making player personnel decisions based on newer metrics like OPS (On Base Percentage Plus Slugging) and WAR (Wins Over Replacement). Most of those teams now view their data and how they analyze that data as competitive advantages against the teams that still use their own gut instinct for evaluating talent. Its not uncommon now to find a ton of stats geeks in the front office making player personnel decisions. The quants have taken over.
So what does this have to do with web analytics? On the surface, not a ton. But if you allow me to stretch a bit there is a nice corollary between the business world, web analytics and baseball. In the business world, many executives and managers still make decisions solely on their experiences and intuition, much like scouts did for 100 years in baseball. Many think they know what will work and go with that belief. Unfortunately, they probably make some sub-optimal decisions based on their own inherent biases and ego. As we've recently seen with the baseball world, some adventurous companies started using their data to help them drive more informed business decisions. Companies like Southwest, P&G, Google and Amazon use data to drive everything and ultimately a competitive advantage. The quants took over there as well.
The other tangent is the aspect of using the right metrics to determine success. As I mentioned earlier, for over a hundred years baseball used batting average as one of the sole metrics to determine a good hitter. That logic was flawed and probably lead to some equally flawed decisions by management. It was just the way it was done for so long that no one questioned its logic. Happens all the time in all decision making, its the way we've always done it, despite it not actually being right.
Same thing occurs to some extent with web analytics. I've heard organizations say things like I want to grow page views, time on site, page views per visit and my personal nebulous favorite 'engagement'. The problem is none of those metrics actually mean anything without context. In my opinion, no company should set out to grow these kinds of metrics without understanding their correlation to success. The obvious exception is for media sites who mainly get paid for page views (which is probably a flawed model as well).
Example, say the goal of your website is to drive leads by visitors filling out a form for a whitepaper. That is how a 'win' is defined for your organization. If you've been able to correlate that visitors who view more pages end up filling out that form, then I'd say its a decent metric to use as a barometer. But using a metric of page views without any idea whether its good or bad is probably meaningless. Because the increase in page views on your site could solely be result of poor navigation or the insertion of new pages into the flow and not an indication that visitors find the content of interest.
An alternative suggestion is to first define your 'wins' such as sales, leads, downloads, etc and then figure out which activities on your site lead you to that outcome. You'll hear the term 'micro-conversion' mentioned in doing this. Micro-conversions are those other events on your site that lead up to a final desirable outcome/win. Find those events and use them as your guides for success. With baseball it was looking at how often hitters get on base as opposed to making outs. With your company it might be smaller activities like signing up for a newsletter, creating a shopping cart, looking at a particular piece of content, etc. The other benefit of doing this is it will help illuminate the reason your site exists in the first place. I always like to ask the difficult/simple question: what are we trying to do? Figuring that out will lead you on the path for the right metrics you'll need to tell you if you're going to win or not.
Comments
No comments yet. Be the first.
Leave a Comment