How personalization and A/B testing can work together to transform your business

Personalization is one of the hottest topics in ecommerce right now, as it can help boost customer engagement, improve conversion rates and grow your revenue. While the temptation might be to plow all of your resources into this latest digital trend, it’s vital to remember the other cogs in your well-oiled marketing machine too.

Personalization is not a broad-brush solution that can be applied to every part of your online store; it still needs to be done right. And it needs to work in tandem with your other techniques and optimization strategies to ensure you’re getting the most value.

Conversion rate optimization (CRO) itself is still seen as a relatively new discipline in the world of digital marketing. But while A/B testing is CRO’s best-known activity, it is at risk of being left behind by those who have seen the potential that personalization offers.

After all, why would we need to split test design solutions on the masses, when we can just personalize the experience for each user? That’s the thought process that is driving some marketers to abandon split testing altogether, expecting personalization to provide all their optimization wins.

Independently, split testing and personalization are both powerful tools that can boost the performance of your website. But there’s no reason why they should be mutually exclusive. When used together, they can provide an even more potent combination.


Moving beyond split testing

Split testing targets the average user. It finds optimal points where more people are likely to buy. But this is often based on treating users as homogenous conversion rates. It is the middle ground but doesn’t necessarily improve the experience for everybody.

That’s just how averages work; you might have a block of users who hate your ‘optimal’ design and will never convert. Some might ‘love it’ and convert like a hot knife through butter, with the results averaging out somewhere in the middle to give you an ‘optimal’ conversion rate.

Bear in mind that most sites have conversion rates below 5%, so there is plenty of room for improvement. That’s more than 95% of visitors leaving your site without buying anything or giving you their details.

Even if you’ve aced your split testing program and have raised conversion rates to record levels, personalization can now help push you to another level. When optimizing based on averages, you can never achieve optimal performance – only by catering for the unique behaviors of different users will you be able to push the boundaries further. This is what personalization helps you to do.


Evolving split tests into personalized content

Let’s look at a very simplified example for how your optimization can evolve with both split testing and personalization working together. Say you run an ecommerce store selling sweaters.  You have a hero image on your homepage showcasing your most popular red sweaters, but decide you want to test out the impact of showing different colors. So you run a split test where a third of your users see the red sweaters, a third see green, and a third see blue.

Conversion rates soar for the users who see the blue version. Green sweaters don’t do too bad either. Red are the worst performing. In traditional A/B testing, these results might lead you to display the winning blue sweater image to all users, as this has the best chance of leading people to a conversion.

Let’s say 60% of your users preferred blue sweaters, 35% green, and just 5% red. So even though you’ve optimized for the majority, there’s still 40% of visitors that are not instantly attracted by your hero image and in danger of bouncing straight off your site.

What if you could identify which color each user preferred and present a hero image relevant to their tastes? That is, in essence, what personalization allows you to do. The latest personalization platforms use machine learning and data from various interactions and touchpoints to intelligently identify what is most likely to appeal to each user, ensuring the content on your site is individual and tailored just for them.

But your A/B test has still served a purpose here. It has demonstrated the importance of that hero image on the path to conversion, while using personalization to make that content specific to each visitor has helped ramp the persuasion up to another level. Testing can provide proof of concept and help direct your optimization strategy, before you decide where to focus time and resources on delivering personalized experiences.


Personalization should be tested too


It is important to keep the principles of split testing front of mind when attempting personalization. Test different things and use the data to make informed decisions about what is the most effective course of action. A simple example could be testing a personalized version of a page against a ‘non-personalized’ page. Or perhaps you want to test different levels of personalization, for example, individual versions of a page compared to a cohort-based design, whereby different types of content are presented to different buckets of users, such as new versus returning visitors.

There may be situations where intelligent, one-to-one personalization doesn’t work for you, or you might want to adjust the way you use it. For example, say you use intelligent product recommendations to always lead each customer to exactly what they want to buy. You might find you make more sales, but that the value of those sales is lower than it was before. Perhaps having such a personalized user experience means you are missing out on the chance to promote those high-value luxury products that people rarely come looking for, but are the most profitable ones to sell.

In a situation like this, you might want to think about how split testing different cross-sell or upsell opportunities could help you to still promote these items, without negatively impacting the customer experience that personalization can help to provide.


Ensuring a positive ROI

Cost is a factor when deciding on a strategy. Personalization on a simple level can be very affordable to implement. For example, many email platforms have subject line personalization baked in as a standard feature, while cohort-based personalization is readily accessible for low-traffic sites. But costs inevitably rise as the sophistication levels, and traffic levels grow – so you at least want to be sure that any gains you make through personalizing the content on your site more than cover the costs of the software or platform you are using.

When measuring the impact of any optimization strategy, remember to look beyond just the simple ‘conversion rate’ metrics. Quality of leads is often more important than the number of leads. Fewer orders can actually be worth more to your business if those orders are of a higher value. Ultimately you need to be looking towards higher level metrics such as revenue and profit margins, so you can be sure that whatever personalization platform or techniques you use are working for you and your business.


Don’t upset the data

What puts some people off from using both split testing and personalization is that it often seems safer to assume that testing and personalization can’t work together, because they will crossover too much and lead to skewed data. But ’test bleed’ is a risk in any traditional A/B testing program. This is where tests on different sections of the site that crossover in their user journeys can ‘pollute’ each other’s results. For example, say you’re running two A/B tests on your site one a new ‘buy now’ button on your product page, and a new layout on the checkout page, it can be difficult to tell which change affected the conversion rate.

This is why you need to be disciplined in how you set up your tests and in how you analyze them. Avoiding crossover altogether is one way to ensure purer test results, but there are other ways to ensure you still get reliable results. For example, making tests mutually exclusive, or collecting big enough sample sizes to allow the results to proportionally even out in statistical terms.

The point is that if you are aware of the dangers to your data and apply the same disciplined approach to your A/B tests and personalization, and use the same considerations when analyzing your results, then there’s no reason why the two can’t play nice together.


Optimization is never finished

If you’re used to running A/B tests, you’ll know that a winning test should not be the end of your efforts. Next you’ll want to think about how to apply those learnings to other parts of your site, or how you could test different variations for different segments of users.

The same applies to personalization. Don’t just turn it on and think all your work is done. It is not a set-it-and-forget-it strategy. Like many elements of web design, you can make improvements through testing.

It is important to remember that personalization is often an evolutionary process. The more a user interacts with a site, the more the personalization engine can learn about their shopping habits. That means it might take a few clicks or swipes to hone in on what they might want to buy or learn about their tastes.

Therefore, optimizing that very first landing page is vital, as you need them to stick around so they can taste the secret sauce. Unless your bounce rate is 0%, then there’s still optimization work to be done. Split testing those crucial first touchpoints can be a vital way to increase the number of people funneled into your personalization-powered buyer journey.

Personalization is no magic wand. It is a powerful tool, but is it just one element in your toolbox. And it still needs to be used properly to be most effective. Laser-targeted product recommendations are no good if your checkout page is leaky, or if your images are not persuasive, or if there are too many other distractions on the page for the personalized products to have an impact. These are parts of the journey that can be improved with split testing, allowing the personalized elements to have an even bigger impact.


Each technique has a time and a place

Personalization is more affordable than ever, thanks to recent advancements in the artificial intelligence that powers it. But it is still a significant investment for many retailers. That’s why it’s important to prioritize how and where you use it. And, like more traditional forms of website optimisation, you still need to do your research and have solid hypotheses behind your design decisions. This is where split testing can help you, to test, form and confirm your hypotheses, and help define your personalization strategy. The two techniques should complement each other.

You can’t personalize everything. Plus, even if you could, then the resources required would be significant. Split testing can confirm your ideas about which elements are most important to your users. This can in turn help create a target list of elements to try and personalize for maximum impact.

There are various elements and steps in the user journey that can dictate whether or not a person buys. Personalization might offer a bespoke service, but you need all your ducks in a row for it to work. Split testing can help you get there, ensuring you can use the persuasive power of personalization to its full potential.