The Absolute Beginner’s Guide to A/B Testing

by RubenCorbo on July 4, 2013

A/B Testing

In this day and age, uncertainty often is the main element that drives businesses to adopt a series of operational measures as diverse as product differentiation, benchmarking, risk management and competitive intelligence tracking.

The point is to reduce—if not eliminate—potential losses that could arise if you make the wrong decisions.

This is where A/B testing comes in, helping you evaluate different scenarios before making a final decision. If performed properly, A/B testing can help enhance your business’ reputation, attract customers, generate a long-term positive return on investment, and polish your corporate reputation.

First—The Basics
A/B testing is performed exactly as it sounds: evaluating whether Option A is better or worse than Option B. You choose what Option A and Option B are. In other words, you have complete control over the variables, and the same is true for the time period during which you want to run your experiments.

Variables can be anything of operational interest to your business, be it email marketing, website design, seasonal promotion or discount deal packaging. The idea, though, is to pick one variable and stick to it, adopting the same analytical approach for the time period as well.

For example, you can decide to gauge the public-relations effectiveness of an email marketing campaign. Create a newsletter, which will become the control version, and send it to, say, the 500 email addresses in your database.

One week afterwards, do the same with a second newsletter, Option B, and then compare the response rates for both newsletters.

Second—How Do You Perform A/B Testing?
A/B testing initially came from the world of statistics, but you don’t have to be an Einstein or an arithmetic whiz to perform the testing. The important thing is consistency, consistency, consistency.

Whatever variable you decide to assess, make sure it is in sync with your operational goals and that it fits nicely in your analytical approach.

Then, choose the attributes you want to test on the variable. List them, either by operational or chronological order, and set a time frame for testing—which could be a specific time, say, Monday at 2:30 a.m., or during a specific period, such as Monday and Tuesday.

Finally, choose the “test singleton,” which is the variable you want to change when evaluating both options.

You’re thinking this is a bit convoluted, and I agree with you. So, let’s break it down further.

Let’s go back to our newsletter example.

- Variables: Layout, font size, font color, content, product discount of 5%
- Test singleton: Newsletter layout
- Option A: formal, professionally looking layout
- Option B: casual, excitement-prone, “show-biz”-like layout
- Test period: Option A: Friday at midnight; Option B: one week after the first test, at the same time

After running both tests, you may find that Option A had a response rate of 25%, whereas Option B had 18%. Consequently, you can infer that Option A was the most effective newsletter, and it is the one you should adopt in your broader marketing campaign.

Third—What Are the Pros and Cons of A/B Testing?
The positives of A/B testing include convenience, ease of implementation and affordability—no need to possess an advanced degree in Stochastic Theory to perform the testing, and no need to hire someone with that academic pedigree, either.

The main negative is that you can only test one variable at a time in order to get an effective technique.

So it can be costly if you have, say, 10 variables and need to have a full picture of those variables before making a final decision.

One way to mitigate this negative is to use online tools to speed up the process and take the guesswork out of all testing procedures.

The Bottom Line
A/B testing is straightforward and can be performed manually, but don’t hesitate to use online tools if needed. All you need is a list of variables as well as a specific time period and a test singleton to modify in Option A and Option B.

Image Credit: We Liqid

78152dd34f56caeff0c313c74f2b9957delicious

Article by Ruben Corbo

Ruben Corbo is a freelance writer that writes about technology, gaming, music, and online marketing where you can read more about A/B Testing. Ruben is also an avid gamer and music composer for short films and other visual arts. You can read more about this particular subject at Maxymiser as well as traffic to sales converting topics.

Ruben has written 1 awesome articles for this blog.

If you like This post, you can follow WebTrafficROI on Twitter.

Subscribe to WebTrafficROI feed via RSS or EMAIL to receive instant updates.

MaxBlogPress Ninja Affiliate

WebTrafficROI runs on the Thesis Theme for WordPress

Thesis WordPress theme

Thesis is the search engine optimized WordPress theme of choice for serious online publishers. If you’re a blogger like me who does not understand any code or technical elements, Thesis will give you a ton of functionality without having to alter any code. For the advanced, Thesis has incredible customization possibilities via Thesis hooks.


With so many design options, you can use the template over and over and never have it look like the same site. The theme is robust and very flexible, the Thesis Theme helps WebTrafficROI to run far more efficiently than ever before.

{ 4 comments }

Kevin Chapman July 5, 2013 at 6:05 am

Great post, I’ve been looking in to split testing quite a bit recently so this has really helped!

Thanks.

Alan Smith July 6, 2013 at 5:59 am

Don’t limit yourself to doing only one A/B tests. Do many tests for better results. You can also use some tools to perform A/B testing such as Google Website optimizer, Unbounce, Visual Website Optimizer, etc.

Rew July 7, 2013 at 9:49 am

A/B testing is so important these days. Especially for someone like me who has so little to work with. I split-test literally everything I do. From text ads on up to follow up emails. Every part of your marketing can and should be split tested. And I urge everyone to adopt this phrase into their lives. “The important thing is consistency, consistency, consistency.” I preach that like its going out of style.

Space Goat July 11, 2013 at 12:51 am

Really informative post, i was looking for a beginner to start with AB testing and hence i got complete basics of it.

thanks for sharing.

Comments on this entry are closed.

Previous post:

Next post:

Webtrafficroi RSS Feed