Everything You Need To Know About A/B Email Testing

Email marketing is not a hit or miss strategy.

Businesses constantly test their findings in order to increase email open rates and online revenue. One of the most effective and oldest testing methods is called A/B testing.

This technique highlights the effectiveness of a campaign in real-time. In a nutshell, customers are shown different samples of an email or landing page. The results are then compiled and the top performing variables are used for the final sample.

Below elaborates more on this testing method and how you can apply it to your email marketing campaign.

How Do I Create A Control Email?

Your control email is the original or basis for comparison. In A/B testing, you first need to create your control email (the original, or version A), and then create your test email (version B) by choosing the one factor from the control that you are testing for.

When choosing a group to send to, you’ll see a lot of suggestions in using 10% of your list. Unfortunately, this doesn’t take into account your list size. In order to get statistically reliable feedback, it is recommended to use at least 1,000 subscribers for each message. This should give you significant results from which you can confidently draw conclusions. Once you’ve reviewed the data, send the winning version to the remaining contacts in your list.

There are numerous factors you can test. The following are the most common variables that businesses change during testing:

  • Call to action icons, wording, size, color and placement
  • Headline or product description
  • Form length and types of fields
  • Layout and style of website (for landing pages)
  • Product pricing and promotional offers
  • Images on landing and product pages
  • Amount of text on the page (short vs. long)

How Long Should I Run Each Test?

Unless you are testing to learn the best time of day, or day of the week to send your emails, schedule all of your campaigns to go out at the same time and on the same day. If you don’t, this may be another factor that affects your results. It is best practice to let a test run for a full 24 hours, if possible. About 80% of all engagement from an emailing will occur in the first 24 hours, unless you are sending out emails on a Friday or Saturday.

Other Tips For Accurate Results

When doing A/B testing, never ever wait to test the variation until after you’ve tested the control. If you test one version one week and the second the next, you’re doing it wrong. It’s possible that version B was actually worse but you just happened to have better sales while testing it. Always split traffic between two versions.

Next, don’t conclude too early. There is a concept called “statistical confidence” that determines whether your test results are significant. Lastly, don’t surprise regular visitors. If you are testing a core part of your website, include only new visitors in the test. You want to avoid shocking regular visitors, especially because the variations may not ultimately be implemented.

Alternatively, you should show repeat visitors the same variations. Your tool should have a mechanism for remembering which variation a visitor has seen. This prevents blunders, such as showing a user a different price or a different promotional offer.

To conclude, A/B testing is a reliable method that should be used for enhancing one’s email campaign. Instead of guessing what works, you can now rely on hard data and prove the effectiveness of your marketing strategy.


Sourced Article: