Making marketing data understandable (and the curse of false data precision)

   

Making marketing data understandable (and the curse of false data precision)

30 Second read 
  • Marketing data is only there to provide insight. To gain insight, there must first be comprehension.Inappropriate detail is called false precision and hinders comprehension.  
  • Instead of presenting number information to two decimal places, present it to two significant figures. 
  • Often number information to only one significant figure is required for decision making. 
     
 
Full read (5 minutes) 

Imagine that are you are driving a car and you take your eyes off the road for a second to glance at the dashboard. The well designed dashboard below quickly gives four pieces of important information: 

  • I am driving anearly 140 km/h 
  • Engine speed  fine ✔️
  • Engine temperature  fine ✔️
  •  Fuel - fine ✔️

 

 

Now, the data could have been presented in more detail: 


So why
don’t car manufacturers display data like this on the driver’s dashboard?    

There are two reasons:  

  • Accuracy: A speedometer is not very accurate.  It is based on estimating the distance travelled in a second by the number of times the car wheels have rotated. This will vary depending on how worn down your tyres are and the pressure in the tyres. Indeed, the EU/UK law for speedometers is that they must never read less than the true speed nor show more than 110% of true speed plus10 km/h. So to be compliant with the law, the true speed of our car could be anywhere between 117 and 139 km/h!    
     
  • Understandability: When glancing at your car dashboard, you have only a short time to grasp the information. Give too much information and you will not be able to understand it. You might even mis-read the speed and the RPM figure is meaningless without a reference point (the ‘red line’).

 

Marketing data 

Hands up anyone who has sat through a quarterly report meeting and a marketing agency has presented multiple slides looking like this: 

Date 

Website visits 

Leads generated 

Conversion rate 

1/1/20 – 3/31/20 

2110 

233 

11.04% 

1/1/19 – 3/31/19 

1523 

112 

7.35% 

Change YoY 

38.54% 

108.04% 

50.16% 

Source: Google Analytics 

 

Whilst you are trying to work out the meaning of the 11 pieces of data in the chart, the presenter attempts to be helpful by explaining As you can see - the campaign led to a year-on-year increase in website visits of 38.54%, 108.04% more leads and the conversion rate improved by 50.16%.” 

The audience responds with a bewildered silence and the presenter is left disappointed that the excellent results are not receiving their due applause 

 

I blame Excel and Google Analytics 

As marketers we use tools such as Excel and Google Analytics daily.  Both tools display numbers to two decimal places as standard.  This is simply a convention, not a directive from clever data scientists at Microsoft and Google. 

Let’s take an example to explain the problem of always using two decimal places. If I were to visit a sawmill to purchase a piece of wood 240.00 centimetres in length, the sawmill should refuse on the basis that its saws cannot cut wood to a precision of 1/100th of a centimetre (1/10th of a millimetre!)After all it would be bizarre for me to request a piece that is 239.98 cm long which demands the same precision as 240.00 cm. However, the sawmill might reasonably agree to cut a piece to 240.0 cm as this requires a precision of 1 mm.   

All web analytics toolsGoogle Analytics included, are never 100% accurateFor example, it is often difficult to get data from Google Analytics and Google Ads to fully agree (because of the  different ways they collect data). So, to present data from Google Analytics to two decimal places is misleading due to ‘false precision’. 

A few years’ ago, witnessed a colleague reassure a client with the statement “Compared with last month, your website’s bounce rate has reduced by 0.32% which is positive news!”.  This statement fails on both the false precision and the understandability criteria (arguably on meaningfulness too)They might more correctly have said Compared with last month, there has been no significant change in the website bounce rate”. 

I enquired why my colleague had felt the statement was appropriate and she answered that the data had come directly from Google Analytics and to have altered this would have lacked rigor and integrity. Sadly, this showed a misunderstanding of data integrity and a mis-understanding of why data is being collected in the first place (which is to bring meaningful insight for informed decision making).  

 

Don’t think decimal places, think significant figures 

Rather than presenting data with a fixed number of decimal places, it is better to use an appropriate number of ‘significant figures’A significant figure is not related to the number of decimal places. The numbers 430, 43, 4.3, 0.43 and 0.043 all have two significant figures. The location of decimal points is not a factor.   

In the example of the car speedometer, two significant figures is appropriate.  If I am driving in a town then knowing that I am driving at 27 mph rather than 31 mph is important; if I am driving on a motorway, then knowing that I am driving at 68 mph, not 74 mph is important. If I am on a racetrack then knowing that I am driving at approximately 16mph might be useful (but probably it is not useful to know that I am driving at 163 mph which has three significant figures).   

To reduce the number of significant figures, we simply use ‘rounding’.  When we apply rounding, it is good practice to state this.   

 

Applying this to marketing data 

When comparing marketing data, using two significant figures is nearly always sufficient. 

Two significant figures could be: 7.8%, 12% and 46%. 

However, if we consider that the sole reason for collecting the data is to aid decision making, then it is worth remembering that in many cases a decision requires numbers to just a single significant figure 

Rounding the above figures to one significant figure gives us 8%, 10% and 50%. 

Consider whether you would make a different marketing decision if you knew that the number of website visitors had reduced by 10% v 12% or if a campaign had delivered 46% v 50% more leads.  

 

Helping an audience to understand the data 

This is how a marketing agency could present the same information about the campaign performance: 

Time period 

Website visits 

Leads generated 

Conversion rate 

Jan – Mar 2020 

2110 

233 

11% 

Jan – Mar 2019 

1523 

112 

7.4% 

Change 

39% 

108% 

50% 

Source: Google Analytics 

Note: for ease of comprehension, calculated figures have been rounded to 2 significant figures 

 

Note the spelling out of the time period being reported – dates written as numbers are difficult to understand (with the added confusion between US and UK date formats) 

Replicating the simplicity of the car dashboard, a well-designed graph adds further clarity.  

The presenter can now summarise“In the first three months of 2020, the campaign generated double the number of leads compared to the same period last year (233 v 112). This was because of an increase in the number of website visits and an improved website conversion rate.  

We might hope that the client audience will applaud the marketing agency and praise the Search Engine Marketing and CRO teams.   

Sources: 

https://www.wewantanycar.com/news/car-speedo-how-fast-are-you-going/ 

https://link.springer.com/article/10.1007/s11669-018-0662-z 

http://ccnmtl.columbia.edu/projects/mmt/frontiers/web/chapter_5/6665.html  



If you are interested in getting more value from your marketing data, please get in touch. We’d love to help!

Get in touch

   

Contact Us

Do you have a challenge for us to solve?

get in touch

We're here to help

During this period of uncertainty we are still operating at 100%. Our brilliant team are working remotely at the same fast pace and to the same high quality. Whether you need support or are an existing client, we're here for you.

Find Out More
X