Creating a split test of your email campaign

If you wish to test the effectiveness of an element of your campaign, you can to run a split test. This allows you to test an element of your email content by creating several different versions of your email.

When you split test a campaign, the campaign is sent to a small proportion of your email list; the results of the split test then determines which version of your email is sent to the rest of the email list. 

It is important to monitor the results of your split test to identify the most effective variation of your campaign. You can use performance data to refine and improve your campaigns for the future.

How to create a split test

When you create a new email campaign, on the first page you see - the 'Campaign Detail' page, you can choose to create a 'Standard campaign' or an 'A/B test campaign'.

You can then choose the type of test you want to conduct:

  • Subject line: Effective when you need to improve the open rate of an email campaign. You may wish to change one word in the subject line or test an entirely different approach.
  • Email content: Test the content in your email body. For this split test, you can change any element in the email. You may wish to change the copy, an image, the layout, dynamic content, personalisation features, or the CTA.


Testing the subject line

When you select the subject line split test, you will be taken to 'Subject' stage of the campaign build. 

Enter the first version of your subject line. Then, to add another version, click the green 'Add' button. You can add up to six variations of your campaign to each split test. 

Testing the email content

When you select the email content split test, when you're in the 'Design' stage of the campaign build, create the first version of your design.

You can then click on the green 'Add' button. This will create a duplicate of the email. You can change any element of the email that you want to test.

To determine which element of your email campaign impacts performance, it is important to test only one element of the campaign at once. You can add up to six variations of your campaign to each split test.

Setting up the split test

Once the campaign and split tests have been created, when you’re scheduling a campaign at stage 4, you can select an initial percentage of your email contact list to send the split test to. 

A split test can be sent to up to 60% of the list you are sending the email to. The percentage of the list that you choose determines how large your sample is. Whereas a large percentage of the list allows you to collect a more representative sample of your recipients, you also run the risk of sending a sub-standard campaign to more contacts. However, be careful to ensure that you select a large enough sample to gain meaningful split test results. We recommend your initial batch to be at least 30%.

At this point, you are also able to choose which measure, opens or clicks, you define as success. For a split test concerning the subject line we suggest opens as the appropriate measure, whereas for email content split tests, clicks is more appropriate.

Schedule how long you wish to conduct the split test for by selecting a time period up to 24 hours. We recommend a minimum of 2 hours for testing to ensure the email is delivered and engaged with by your initial audience. After this time, the split test will end and the best performing campaign will be sent to the remainder of the full email list.

A/B test campaign reports

After your split test is finished, you can view the campaign reports to see how the split test has performed, including which variation won, to introduce further learnings into future campaigns. 

For example, you may find that using personalisation such as the first name inside your subject line gathers a better open rate – introduce this into all of your campaigns to help boost engagement.

To view the reports for your campaign, navigate to 'Reports' > 'Campaign Summary', click on the name of the relevant campaign, and click on the 'A/B Split Summary' tab.

We use cookies to personalise content and ads, to provide social media features and to analyse our traffic. We also share information about your use of our site with our social media, advertising and analytics partners. Cookie Policy