Every email marketer wants higher open and click-through rates for their campaigns, since these result in more leads and a higher conversion rate. The best way to improve your email marketing campaigns is to run A/B tests on them.
The concept behind A/B testing is simple. A/B testing email marketing involves sending one version of your campaign to a subset of your mailing list, and another version to a different subset. This will allow you to compare the two versions and see which one generates better results.
Just like any kind of experiment, you can make A/B testing as general or as specific as you wish. For example, you can test different subject lines and see which ones have higher open rates. You could also test two or more radically different templates to check which one has more click-throughs. And so on.
Why Email Marketers A/B Test Their Content
While industry observers have been predicting the demise of email marketing for a few years, its status as the king of digital marketing is still undisputed. McKinsey and Company reported that email marketing is still around 40 times more effective than social media marketing. And despite Facebook and Instagram investing heavily in their ecommerce platforms, customers say they can make a purchase through email 3 times faster.
We can attribute email’s effectiveness as a marketing medium to its self-selective nature. When someone signs up for your email newsletter, we can infer that they are interested in your brand. Email marketing’s return on investment (ROI) is unmatched:
Source: Smart Insights
The cost of doing A/B testing on your email marketing campaigns also tends to be lower than the cost of running similar tests for landing pages or social media pages.
Performing A/B testing on your email marketing content allows you to get the most out of your campaign. It will show you what works best and allow you to avoid the things that just waste your subscribers’ time. A well-optimized email marketing campaign lets your brand stand out in your subscribers’ inboxes and gets them to click that all-important “Buy Now” button.
Types Of A/B Testing In Email Marketing
While there are multiple elements you could test in an email marketing campaign, most A/B tests look at one of three main variables:
Subject Line
This type of test compares the performance of two emails that are identical except for the subject line. There are numerous things you can do with your subject lines, including applying personalization, using different focus topics, testing different kinds of promotions, and even adding emojis.
Sender Name
Most people won’t open an email from a sender they don’t know. Therefore, make sure to include the sender’s name in email marketing.
There are several ways to A/B test your sender’s name. You can change the sender name and keep the email address the same, change the email address and keep the name the same, or change both the sender name and the email address. A good email marketing software will let you toggle between sender names easily.
For example, you might find that your customers are more inclined to open an email from “Stacy from Brand X Marketing” than one from “Brand X Marketing Promotions”. This is because the former sounds more personal.
Email Content
This type of A/B testing for email marketing encompasses a wide array of elements, including the template, header images, text content, and the call to action (CTA) button. You could also do A/B testing on two or more completely different email templates to see which one results in more click-throughs and conversions.
Here’s an example we tried: we tested two completely different email templates. The first used a customized cartoon character as a header image, and the other used a stock photo. The text reflected the style of the header image – the cartoon character corresponded with a more conversational tone, while the other email was written more formally.
The emails with the cartoon character header image got more click-throughs, although we can’t be sure whether the image or the more conversational tone was the reason. We’d need to run the second batch of tests to be sure.
Remember: you don’t have to either test everything all at once or test just one thing at a time. The beauty of A/B testing email marketing campaigns is that you can experiment with different things until you come up with a combination that works. However, before you can find something that works for you, you need to define your criteria for success.
Pro tip: when you write new email content, use one of our favorite Grammarly alternatives to check your copy. Poorly written copy littered with grammatical mistakes can impact your results and lower audience confidence in your brand.
Defining Criteria For Success
Even the most advanced A/B testing procedures are ineffective if the criteria for success are not well-defined. To define your criteria, you will need a good amount of historical data to serve as your baseline. If you’ve been using the same email template for a few months or the same seasonal campaign for the past few years, you can use the data from these campaigns as your starting point.
Source: SEObility
However, if you don’t have any previous email marketing campaigns, I suggest aiming for the following numbers as a starting point:
- An email open rate of around 15-25%
- A CTR of 2.5%
- A click-to-open rate of 20-30%
The numbers above are for email marketing campaigns in general, but different industries have different performance benchmarks. For example, the average CTR for hobby websites is 5.01%, while the average for restaurants is 1.34% as of August 2020. Study your industry to establish what are reasonable benchmarks.
You can also look at how well your emails perform in different email clients, such as Gmail, Outlook, or the iPhone email app. These three email platforms alone account for more than two-thirds of all email users worldwide, so you must optimize your templates to ensure your emails render correctly in each client.
A/B Testing In Email Marketing: The Process
To people who are not familiar with A/B testing, it might seem like guesswork combined with trial and error. However, A/B testing an email marketing campaign follows a process that can be done repeatedly. In that way, it is very similar to the scientific method of doing experiments. A/B testing might not win you a Nobel Prize, but it sure will help you create more effective campaigns!
Here’s the process I suggest you follow as you set out to A/B test your email marketing campaigns:
Choose One Variable At A Time
There are many things you could A/B test in your email campaigns, from the subject line to the copy or even the images. But to really assess the effectiveness of each version, you will need to take one variable at a time, change it, and see how that change affects the overall success of your campaign.
You might be surprised to discover that a simple change, such as replacing the words in your CTA button or including a banner image, could result in huge improvements to your results. When you break down your campaign into chunks that you can easily change and monitor, you will be able to identify what works and use this information to improve future campaigns.
Identify Your Success Criteria
You are probably doing A/B testing to improve the overall performance of your email marketing. However, the most effective A/B tests focus on just one metric. For marketing campaigns, this metric will be either the open rate or the CTR. While an improvement in the open rate might result in an improvement in the CTR, you should focus on one or the other.
Choose the metric you’d like to track, make an educated prediction about the impact of your proposed change, then run your test. You’ll find that focusing on one key metric makes it easier to achieve your goal and determine the things that improve performance.
Define Your “A” And “B” Templates
Now that you have a variable and a target, you can come up with your “A” and “B” templates. The “A” template is usually the one that you’re currently using, while you use the “B” template to implement your suggested changes.
Think of it as a science experiment. You need a control sample – one that is already existing and won’t be changed during the experiment – and a “challenger”, which aims to improve on the performance of the control. We normally use the control sample as a base for the challenger, changing elements one at a time until we get the encouraging results we want.
For example, let’s imagine you want to do A/B testing on the email sender you’re using to send out a series of sales emails. You might use the current “Brand X Notifications <notifications@brandx.com>” as your control, then use the new email address “The Best Picks from Brand X <notifications@brandx.com>” as your challenger.
If you see that this sender name change doesn’t make much of an impact on your open rate, you might try including a human name, such as “Brandi from Brand X”. Your previous “B” template could become your “A” template, and so on.
Split Your Sample Groups Randomly
To get the most reliable results, you need to use a sample audience that is split into two random and equal-sized groups. First, define the size of your sample audience.
When you do A/B testing on email campaigns, you can use a relatively smaller sample group. 1,000 recipients are usually more than sufficient, and fewer are sometimes fine. You don’t need to test your campaigns on every single person in your mailing list to get a statistically significant result that you can use to choose the winning version.
You’ll also need to determine the statistical significance that will indicate a winner. We normally require a confidence level of 95%. Think of it as a bit like placing a bet. If you’re 70% or even 80% sure that a result is significant, there is still a 20% to 30% chance that you’ll choose the wrong version to run.
However, for major redesigns, like completely changing copy or replacing your usual structure with a graphics-heavy template, you can get away with a lower confidence level. You only need to be extremely stringent when you’re testing small and very specific elements, such as the color or wording of a CTA button.
Use An A/B Testing Tool
Before A/B testing tools were created, it was a big task to create A and B templates, put them up online, and track the results. Now, A/B testing tools exist to make your life easier.
If you use a professional email marketing software, you will also have access to features that are useful for A/B testing email marketing campaigns. HubSpot Enterprise, for example, allows you to set the size of your sample group with a slider, determine the metrics to track, and set the duration of your test at the click of a button.
Use Both Variations At The Same Time
Timing plays an important role in determining campaign results, especially in a world where most people check their email every day. If you run Version A today and Version B next month, you cannot be certain whether your audience’s reaction is determined solely by the change in your campaign. Timing and external factors beyond your control may also play a role.
The only exception to this rule is if you’re actually A/B testing the days and times you send out a promotional email (trust us, this can have a major impact on your open and click-through rates!). There are no hard and fast rules as to the optimal day and time to send emails. It will largely depend upon your audience’s working hours and your industry.
As you do your A/B testing, you must fight the temptation to look at variables different from the metrics you decided to track. If you decide to focus on open rates, choose the version that has a higher open rate. You can then use it to do further A/B testing for click-through rates.
How A/B Tests Can Increase Engagement And Boost Profits
Email marketing is still one of the most cost-effective marketing methods available. Its ROI and engagement rates are unmatched. For that reason alone, your business cannot afford to get email marketing wrong.
Therefore, you should take the time to improve on your email marketing campaigns by running A/B tests on various aspects of your emails. A/B tests can increase your CTR by around 2.5%. In real terms, if you have 100,000 people on your mailing list, that means 2,500 more site visits that could generate revenue for you. Who would say no to that?
Lessons Learned
A/B testing email marketing is a bit like the digital marketing equivalent of a science lab. To get truly credible results, you need to be clear about your objectives, test one variable at a time, use equally sized and randomly selected sample groups, and use the right tools.
A/B testing email marketing campaigns will also keep you on your toes for a long time, as each new successful test becomes your new baseline. It should be an ongoing process of continual testing and improvement. As your audience grows and evolves, your email campaigns should also change with them. Doing effective A/B testing will allow you to build campaigns that are responsive to the needs of your market.
Here’s to more click-throughs and conversions than ever!
Bio
Owen Baker is a content marketer for Voila Norbert, an online email verification tool. He has spent most of the last decade working online for a range of marketing companies. When he’s not busy writing, you can find him in the kitchen mastering new dishes.
