If I had more Time I would’ve Created a Simpler Measurement Model

Mark Twain is often quoted as stating something similar: “I would’ve written a shorter letter if I had more time.”

It ends up that Mark Twain wasn’t the one who actually started this theme.  The originator of this theme, at least according to Quote Investigator, was a mathematician and philosopher named Blaise Pascal. He wrote something similar in a 1657 collection of letters called Letters Provincialies.  It seems that Mr. Pascal was really quoting the first case of analysis paralysis, which, of course, one might expect from somebody that is both a mathematician and a philosopher.

I can only imagine what a tug of war it must’ve been going on in Mr. Pascal’s brain: “Does this KPI represent the intended outcome or is the universe conspiring against me to look bad?”

Specifically Mr. Pascal said: “I have made this longer than usual since I have not had time to make it shorter”.

So my conclusion is that as far back as 1657, we’ve been struggling to make our measurement models simpler.  Let’s look at some ways that we can reconsider our understanding of measurement and create some simpler models.

How to Find the Obvious KPIs in a Sea of Data

The world of digital measurement has amplified this challenge because there is so much we can measure.  The question becomes not all the things that we can measure, but rather what are the right things to measure.

Was that a monstrous statement of the obvious?  Is it simple common sense?  Yes, both statements are blatantly true, so why do we struggle to figure out simple measurement models?  I would submit to you it’s because the simple models are just that SIMPLE.  They are so agonizingly simple that we hear a voice in our head SCREAMING “How could it be that simple!”

Now let’s be clear, simple doesn’t mean “not sophisticated”, simple doesn’t mean “not valuable”.  Quite simply, simple means understandable to your audience.  There is an important point of clarity here:

Understandable to your audience.

I was recently discussing this topic with a V.P. of Digital Marketing at a large consumer products goods company.  We were discussing all of the cool digital metrics that could be leveraged to create engagement, conversion, time on site, etc…  We debated and debated in the end we used two simple metrics to represent all that digital activity:

  • Reach (i.e. impressions)
  • Engagement (i.e. activity)

Yep, that was it.  Those were the two KPIs that this sophisticated digital marketer was going to use to share with his audience!

School of Hard Knocks

I’ve learned this lesson myself, many times actually.  We have so much information available to us that we forget that sometimes, heck most times, are audience only understands the very simple measurement models.

About 4 years ago, I was working with a customer who had some pretty sophisticated marketing going on:

  • Website with nearly 1,000,000 visits per month.
  • Conversion goals of 8% - 10% per month.
  • Email campaigns being sent to 4,000,000 consumers per month.
  • Facebook campaigns attempting to attract more than 1,000,000 “Likes”.

Guess how many KPIs we used on the executive dashboard? Three.

Oh, I wanted to share a heck of a lot more than that but, in the end, after getting the snot beat out of me, we ended up using three.

The Birth of the Simple Measurement Model

Let me show you an example of how simple it really was.  First we identified the key objectives (I’ve changed the nomenclature to protect the confidentiality of our client):

 

Objective Measurement
Drive early acquisition into the program at the early lifecycles of our customer. Increase registered users to 45% of total contacts.Increase early lifecycle consumer registration from 55% to 58%.Increase Facebook fan base to 300,000.

We identified two other key objectives and identified the simple measurement for those key objectives.  Given the relative sophistication of this particular client, we took this a few steps further and defined the audience of these simple measurement models. Here is an example of the definition for the Executive Audience:

Audience Deliverable Notes
ExecutivesBrand Managers Executive Dashboard

  • This dashboard will display the KPI’s as mentioned on the previous slide that are aligned with the objectives.  It will include a trend of the KPI where appropriate.
  • We will clearly state the observations that are critical to understanding the performance of the various CRM programs.
    • #,### of consumers acquired
    • #,### of website visits
    • #,### of facebook fans
    • Etc…
Given the timing of delivering the performance metrics in 2011 these dashboards will primarily focus on observable facts.Not every observable fact will have a conclusion, recommendation or implication, as these do not always become clear on a monthly basis.As questions are asked and exploration is performed any insights and recommendations will be developed and presented in separate analyses and presentations.

We went even further and identified a definition of layers of analysis:

Measurement
Term
Definition Notes
Observation An observation is a statement of a fact. Observations occur on almost all metrics to call out specifics of what has happened.Observations are used to provide transparency and context into the metrics.
Insight An insight is an understanding of relationships that sheds light on or help solves a problem. Some but not all observations will create an insight.Insights are often formed as a question or a hypothesis.  The insight can then be explored through further analysis.
Implication An implication would alter the way that we are executing a specific process. Some, but not all, insights will create implications.Implications should be weighed against our business objectives and our ability to execute.

 

Now, I’m not here to debate you on if the above definitions are accurate.  You might say tomayto while I say tomahto.  The important thing is that we gained agreement on what these items meant to us.  Whether it’s an agency to customer relationship, or an internal analyst to stakeholder relationship, the important thing is to get alignment.

Another Way the Old Logician Said It Another Way

After my trip through the school of hard knocks, I found that others had already established (or reinvented) similar models.  It ends up that there is a principle called Occam’s razor that is related to a 14th century logician named William of Ockham who basically said:

“When you have two competing theories that make exactly the same predictions, the simpler one is the better."

I discovered Avinash Kaushik, who writes a blog post of the same name, Occam’s Razor, who had created an even simpler model and soon we were leveraging a variation of that model with our internal work and with our clients.  If you’ve studied any of Avinash’s posts you’ll know his basic model looks like this:

Global World Domination Inc.

From: http://www.kaushik.net/avinash/digital-marketing-and-measurement-model/

It’s About the Journey, NOT the Destination

I said something that I think is relevant in my previous comment.  We leveraged a “variation of that model”.  It’s important to remember that your model will be based on what’s best for YOU.  You’ll need to experiment, succeed and fail, to truly identify what works best for you which is why we came up with a simple acronym to describe our approach.

The CAMM Model

CAMM is a simple acronym for Clarity, Alignment, Measure, Modify

  • Clarity makes sure you are all on the same page about why and what you’re going to measure.
  • Alignment gets everybody to understand how you’re going to measure.
  • Measure is the actual gathering of data and information and the analysis that you perform.
  • Modify is the ability to modify your execution; to change your outcomes or modify your measurement model if something is not working.

The Old Timers Were Right

It would seem like our two characters from the 16th and 14th century, Mr. Pascal and Mr. Ockham, were on to something. Perhaps we should try to leverage their 500+ year old advice:

  • Know your audience
  • Keep it simple
  • Implement the CAMM model

See if using these principles in developing your measurement models gets you to a simpler place!

Categories: