14 Best Practices For Effective A/B Testing

When you want to know if things are working the way you want them to, the only way to do it is by collecting data. However, you want the right data, collected under rules that you define. Define your rules using some common sense, get your A/B testing in process and sit back and watch.

14 Best Practices For Effective A/B Testing

1. Define Your Test Hypothesis

Start your test with a hypothesis – a short statement that defines what you’re trying to prove or refute. Keep this hypothesis in front of you while you test, so you don’t forget it. Your hypothesis should include the variables you’re testing and the success metric you want to achieve.

2. Determine The Success Metrics

Determine the success metrics that you’re looking to achieve, and align your test variables to your success metric. For example, you’re aiming for lower shopping cart abandonment, and are testing the steps in the shopping cart process. In this case you’re aiming for the perfect number of steps for maximum successful purchases. Include that in your hypothesis – the lower the steps, the greater the purchase completion.

3. Ensure Sufficient Data Volume

Make sure you have enough volume in terms of test groups and results to make the data statistically significant. There’s no point in measuring clicks on an email if you’re sending it out to 10 people. If you’re looking at a 10% success rate, you want to send out the emails to a 1000 people and look for 100 clicks.

4. Use The Right Audience

While testing, take the right audience into account. The test traffic sources and audiences you use must be a close match to the real target audience. If possible, get a select group of your actual audience to participate in your major tests.

5. Test Only On New Visitors

If you’re testing an important part of your site, make sure only new visitors see the variations. Don’t surprise your regular visitors with changes they may not be comfortable with. You may or may not implement those changes, and you don’t want to shock regular visitors into quitting.

6. Start Simple

A/B testing requires some skill, never mind how experienced you are. So begin your test with something simple, such as adding Trust symbols to your home page and watching for increased conversions. If you find that adding them increases your conversions, add them to other pages and watch what happens.

7. Make Minor Changes

Know that even minor changes, such as changing the color of your headline can impact your results positively or negatively. You don’t have to make major changes in order to obtain results. Keeping that in mind, make planned, simple tweaks and check for improvements.

8. Test All Variables One By One

You don’t have to limit yourself to one variable on a page. For example, if you’re testing just the background color or font size on a page, you won’t get the results you want. Consider all the variables that can impact your conversions, such as language, bullet point, images, videos, headlines and so on.

9. Keep Testing

Don’t be satisfied with the primary test results. Change other variables and test again. Make more changes and keep testing. There’s plenty of room for optimization on any website. Test results don’t remain static; keep testing features to increase leads and conversion rates.

10. Change One Variable At A Time

A/B testing is all about testing a single variable while keeping all else constant. If you happen to change more than one variable at one time, you won’t know to which variable to attribute your success. This will only cause confusion, so test one variable at a time.

11. Don’t Test Everything

You don’t have to test everything, even though they can be tested. Use your common sense, existing data and best practices to evaluate what’s already working and what needs testing. Focus on those variables that you’re sure will give you greater and not marginal results.

12. Document Your Test Results

If you forget to document your tests and results, you will not be able to build upon your past lessons. Nor will you be able to repeat successful tests and educate your team. Start with a hypothesis; name each test and document the processes, variables, tools and results accurately.

13. Do Simultaneous Tests

Test both the control and the variation simultaneously. Don’t test the control first, wait for a few days and then test the variation. Sales are impacted owing to various reasons – holidays, festival times and discounts. So it’s quite possible that your variation was worse but the sales timing was better. You’ll never know unless you test both control and variable simultaneously.

14. Don’t Go By Instinct Alone

Your variations may sometimes seem stark to you, such as a bright red CTA button on a pastel-shaded site. Don’t shy away from the color just because you feel it stands out too much. Allow the test results to let you know, instead of going by your personal code of aesthetics.

Leave a Reply

Your email address will not be published. Required fields are marked *