Customer Loyalty Analysis: Tapping Into the Power of Cohorts

We all know when it’s done right, loyalty programs are great for retail businesses – and also for their customers. But a true online retail professional is never pleased just by the word “great”. She wants to see the numbers to know how “great” that actually is. And that’s what customer loyalty analysis and cohort analysis can help you with.

This leads to many questions: how should we measure our loyalty program? What metrics should we look at? What are the benchmarks? What numbers count as “good” or ”bad”?

Retention

Let’s start with the obvious. When you are running a loyalty program, you want your customers coming back and shopping more often. You want to increase your retention.

The math is simple: when you retain customers, you save money on marketing budget (and related costs). You can give back some of that money to customers in the form of discounts/gifts/coupons/etc., so they don’t go to your competitor. Retaining customers is a win-win… in theory.

In practice it’s a win-win only if the repeat purchase actually happens. I want to make this really clear: to evaluate a loyalty program, you should not focus on:

• how many returning visitors you have on your website
• how many people have opened your loyalty-program emails
• how many people have seen your loyalty ads
• etc…

Of course these numbers are important indicators of other decisions, i.e. did you choose the right email subject line or do you have the right messaging on the right pages.

However, if you want to decide whether your loyalty program is successful or not, you should look at the sales metrics:

• Revenue
• Profit
• Number of items sold
• Number of sales
• Number of repurchasing customers
• Etc…

In retail, retention means repeat purchases.

Choose the right metrics

Focusing on five (or more) different metrics is never good. Pick one or two instead! The best sales metrics always depend on the given business. However, quite often these numbers are highly correlated to one another, so choosing either number of sales or number of items sold will lead to the same conclusion. At this point it’s up to your professional judgement, but if you’d like some more help in the decision, I can recommend following the One Metric That Matters principle.

How not to measure your metrics

Once you know which metric to focus on, the next step is measurement. Measuring retention is always tricky, because of two things: organic churn and seasonality. Let me explain, in detail.

At first sight, how would you calculate your retention rate? (Remember! Retention in this case means repeat purchases.) Many companies – incorrectly – use this simple formula: (eg. for retention rate of March)

The problem is that the result of this equation will be affected quite a lot by the organic churn of your users. (Plus it incorrectly includes the new customers too.)

Let’s take a look at the following table:

In the first column you see the different months. The second column shows the number of new customers in the given month (in this simplified example, we’re using 100 for each month). The third column shows how many of those 100 customers came back and bought something in the following month. The fourth column shows the same thing 2 months later, and the fifth 3 months later.

Let’s crunch the numbers:

Number of Unique Customers in March: 100+80+64 = 244

Number of Unique Customers Ever Until End of March: 300

Retention % in March: ~81%

Using the same method for April:

Number of Unique Customers in April: 100+80+64+51 = 295

Number of Unique Customers Ever, Up Until April: 400

Retention % in April: ~74%

See? Same performance, different numbers.

If you follow this model, your retention rate will correlate to zero in the long term. Why? Simply because of organic churn. In 10 years, you’ll have a lot of passive customers. (It’s not a pleasant thing to hear, but it’s normal.) So, if you always count in this passive customer base when you calculate your retention rate, it will really distort your numbers.

So, please don’t do that!

Cohort analysis

In my opinion the best way to measure retention is to use cohorts. What does that mean? Let’s refer back to our table again:

A cohort is a set of customers that we can select clearly based the date and time of a certain interaction they’ve made.

In this table, the row corresponding to January shows the cohort of those people who made their first purchase in January. It’s really easy to see that the monthly retention of this group is ~80%.

The same goes for February, March and April. This is actually a much more informative overview of our results, than any aggregated view.

It has a lot of advantages over the previous (incorrect) calculation:

1. You can see your actual retention rate and compare the different months/cohorts!
Nothing distorts your numbers, as you don’t count in the passive customers. That’s a big achievement in itself. But with cohort analysis, you can finally exclude the seasonality effect, too.
In the demo cohort analysis above, it’s really easy to see that January, February and March performed the same. But if you typically push hard in December (as most online retailers do), there’s a high chance that the customers you’ll bring on board that month will return less than your average customers do.

And now you’ll be able to see that clearly!

2. It makes calculating the return on your investment really easy.

Having the exact retention rate for each cohort helps you to calculate your return on investment more accurately. Maybe the first purchases from a Facebook ad didn’t cover your expenses… but if you see repeat purchases in the following months, you might change how you measure the promotion’s success.

3. It helps you to evaluate your loyalty program.

If you release a loyalty program (especially if you are doing it with A/B testing), you’ll see exactly how certain cohorts reacted to your program. With that, you can tell:

• How much extra profit you made with the loyalty program
• If the loyalty program worked at all
• Which cohort reacted the best (In other words, which stage in the customer life-cycle needs the loyalty program most?)

Cohort analysis is great, we can agree on that!

Reading a table full of numbers is never as simple as reading a line chart – I admit that. But in analytics, oversimplification can translate into the loss of information.

So, don’t be afraid to go with cohort analysis, because it shows you more detailed and accurate information about your retention numbers than any other analytical method.

(Note: By the way, you can calculate the average 1st month / 2nd month / 3rd month / etc… retention across cohorts and put it on a line-chart at any time.)

What’s the right timeframe?

Monthly retention? Weekly retention? Daily retention?

Good question. Some retail businesses have pretty stable retention. Others don’t. Think about a coffee capsule store. If people run out of coffee capsules, they go and buy – by default – without any further promotion.

Now the question still remains: how often are they re-purchasing? Do they buy every day? Probably not… Do they buy every week? Maybe. Every month? More likely…

You don’t have to guess around it. The simplest thing is to look at your historical data and calculate the average retention window. This will require some statistical brain power – as these different timeframes could be very spread apart, but once you have a good average, you can easily set up your cohorts using that number. For example, if you see that your retention is daily, your cohorts should be broken-down on daily basis as well. If your customers are coming back only once every quarter, it’s okay to have cohorts for Q1, Q2, Q3 and Q4.

I wrote about this topic in more detail and from a more analytical perspective on my blog in this article: Measuring Retention Using Cohort Analysis.

How can I do that?

If you want to use cohort analysis, you can do it by:

• using different 3rd-party tools (eg. Mixpanel, Kissmetrics, Google Analytics, etc.)
• pulling your raw data into an Excel or Google Spreadsheet
• using different visualization softwares like Tableau
• or by setting up a script to do the calculation on your data server automatically (if you’re interested, here’s an article of mine about SQL for Data analysis)

Conclusion: your business, your decisions

Here are three takeaways from this article:

1. Measure retention. And when you‘re doing so, focus on your sales numbers.
2. Use cohort analysis as an analytical method.
3. Always use common sense! Don’t follow market benchmarks, but look at your own numbers and think!

If you’re measuring correctly, you’ll be much more knowledgeable about your loyalty program – and you can measure and optimize it far more easily. You can also use the data to improve customer satisfaction surveys.

Interested in real results? Then download the case study about one of our clients, LUISAVIAROMA, in which we detail how the fashion retailer increased their revenue by €16m within two years.