Stealing Google’s Fire: Powerful A/B Testing for the Solopreneur

Much like the titan Prometheus stole fire from the Greek gods to bring civilization to humanity...

...Thrive Themes just stole figurative fire from those YUGE online companies that dominate the internet.

And just like mythological Prometheus, we want to share that competitive fire with you, the solo-preneur.

Monsters like Google and Amazon not only have massive marketshare and capital advantages, but they leverage expensive A/B testing technologies – run by the world's nerdiest split testing geeks – to squeeze every last drop of conversion juice from their webpages.

Well, that powerful A/B testing advantage is now at your fingertips!

Simple, fast and scary affordable landing page, sales page and lead generation page A/B testing can be harnessed on your own WordPress website through our new Thrive Optimize plugin.

Now you can steal this A/B testing fire and wield it like a conversion optimization flame thrower...as long as you implement the following 10 A/B Testing Principles to compound your gains over time, just like the current titans of the internet do!

1. State a clear hypothesis for every A/B test:

If you don’t have a clear idea of what you’re testing, how will you know if an A/B test succeeds or fails?

To avoid aimlessly running A/B tests you can’t learn anything from, always write down the hypothesis you want to test first. Then design your A/B test around trying to answer that clearly stated hypothesis.

To randomly spitball A/B tests is unhelpful and will cause you much frustration, missed learnings and slow growth.

2. Never A/B test more than one variable at a time:

The number of elements you will A/B test simultaneously...is one.

Let your clearly stated hypothesis guide what single variable is best to test and hopefully answer that hypothesis.

Design your testing variation in a way that only significantly alters a single variable from your control.

For example, if you decide to test a 2 column vs. 3 column pricing table, don't also test different value propositions between the two variations at the same time. If there's a significant change in conversions, how will you know what element was actually responsible for the difference?

Testing a single variable will help you avoid attributing causation to the wrong elements and prevent harmful decisions in the future based on incorrect interpretations.

3. Always test two conspicuously different versions of your single variable:

Do your conversion rates really depend on what color your call-to-action buttons are?

No...likely not. And even if button color does matter, the impact is minimal.

More importantly, button color tests won’t teach you anything about your customers.

Accept that most of your A/B testing results will cause negligible changes and offer inconclusive results. But if you ever want to score the occasional big win, you must test conspicuously different variations of your chosen test variable.

Use this Paperclip vs. Bowling Ball analogy to help yourself design A/B tests that attempt to clearly answer your hypothesis like in the following examples:

• Does a 2 column or 3 column pricing table convert better?
• Does an extremely long sales page convert better than a short sales page?
• Does [Value Proposition A] or [Value Proposition B] convert better?

After you state a clear hypothesis and decide on a single variable to test, make sure your A/B test variations are conspicuously different so it's usually clear what's driving the final results.

When you A/B test landing pages, make the single variable you're testing conspicuously different to help clearly understand if it has a strong effect on your results. Using the above Thrive Architect templates as an A/B test example, your hypothesis could be: "Changing the length of my sales page from a short video sales page to a long copy, video sales page will increase my conversions due to the high complexity of my product (i.e. customers require more information before purchasing). If the longer sales page does measurably better than the short sales page, it's a strong indicator that the hypothesis is valid. Notice that the video, overall design and headlines between the two variations are identical.

4. Never stop A/B tests before achieving statistical significance:

Do not allow yourself to stop any A/B test before enough time has passed or sufficient conversions have been collected to pick statistically significant winners.

The phenomenon of random chance must be paid its proper respect by never ending A/B tests too early and selecting what appears to be the initial winner.

It’s common that the leading variation at the start of a test is not the winning variation over time.

Notice how the green "No Opt-out" test initially outperforms the yellow and blue tests. The underperforming blue "No thanks" test was ended on August 2nd as it hit statistical significance of being a clear loser. However, the yellow "Wasting time" tests eventually outperformed the green "No Opt-in" test by a small conversion rate margin. Small result variations like this are much more common than the big wins showcased in your typical case study post.

You must let the random testing noise seen at the start of most tests die down before selecting your winner. Otherwise you risk selecting losing variations that will compound your conversion losses over time.

In A/B testing tools like Thrive Optimize, Thrive Leads and Thrive Headline Optimizer, we recommend that you let your A/B tests run for 2 weeks collecting a minimum of 100 conversions – not impressions – before selecting a winner.

To be confident that your testing results are real and not random chance, your “Chance to beat original” should be greater than 95% (your variation wins) or less than 5% (your control wins) – after the minimum time and conversion thresholds have been met.

Screenshot of an a/b testing result showing the "chance to beat original" at 94.59% and a conversion rate improvement of +26.44% after 186 conversions. This a/b testing result meets our recommended minimum statistical significance of 95% after at least 100 conversions.

Of course, if you want to remove your human emotions from the process altogether, all Thrive A/B testing plugins have automatic winner settings. you can use to set-and-forget your A/B tests without ever having to check-in on them!

5. Always be A/B testing:

Your A/B tests don’t need to be super complicated.

If you use every A/B test as an opportunity to learn something about your customers, the value you gain from increased conversions and engagement will compound over time.

To do this, commit to running A/B tests every time you publish a new page on your website. You don't have to spend more than a few extra minutes to accomplish this each time.

With Thrive Optimize, you just clone your control variation in 1-click, quickly modify your variation page and then start running your A/B test!

Just make sure to ask yourself:

6. Avoid perfection and run “good enough” A/B tests instead:

Don’t ever let yourself waste an A/B testing opportunity because you don’t have the PERFECT testing idea.

There is no perfect A/B test.

The best you can do is test ideas that are “good-enough” to learn something meaningful about your customers.

Take what you currently know about your audience, use that knowledge to develop a useful hypothesis, create an A/B test to answer that hypothesis, and start collecting data.

For better or for worse, take what you learn and let it guide your future A/B testing ideas.

The compounding benefit of this rapid feedback loop will lead you towards the promised land of increased sales and online business growth!

7. Don't be disappointed by inconclusive test results:

You must accept the fact that many of your A/B tests will be inconclusive.

This does not mean that such A/B tests are a waste of time. Indeed, they will provide you with many learnings you should appreciate!

Inconclusive A/B tests simply mean that your hypotheses didn’t really matter and should be de-prioritized in favor of more promising experiments.

These inconclusive learnings can be quite beneficial as they will provide you with new insights and shift your thinking towards more valuable ideas.

Celebrate and learn from your inconclusive tests just like you would from a big conversion optimization win.

8. Don't freak out about missed conversions during your A/B tests:

Your human emotions must be set aside while running A/B tests.

Although it’s easy to let your emotions run wild while monitoring low performing variations, you must not indulge your impulse to end tests too soon.

The emotional roller coaster of extrapolating how many leads or sales you’re missing while letting "losing" variations continue will be hard. However, you must reach that recommended benchmark of 95% statistical significance before naming any winners or losers.

An A/B testing example showing the control losing to the variation initially only to become the statistically significant winner over time. Don't underestimate the phenomenon of random chance affecting the initial stages of your test. Wait at least 2 weeks with at least 100 conversions before selecting a winner.

The foolproof solution to sidestep irrational emotions during A/B testing is to use the set-it-and-forget-it automatic winner settings Thrive Optimize, Thrive Leads and Thrive Headline Optimizer provide.

9. Make sure your conversion funnel experience is congruent:

Realize that the results of your A/B tests are not strictly dependent on your copy and page layouts alone.

The congruence from your Facebook or Google ads to your webpages plays an important role in your overall conversion rates.

Thus, you must always take your traffic sources into account when launching any A/B test.

For example, do not run a Facebook ad A/B test while also running a landing page A/B test.

You can listen to Shane and our in-house Facebook ad expert Dave discuss this congruence issue on the ActiveGrowth podcast here.

If your customer’s funnel experience blends seamlessly, your conversion rates will benefit. But if any part of your ad to website handoff clashes, expect conversion rates to suffer.

10. Don't blindly follow the A/B testing claims of your marketing gurus:

Have you ever been excited by case study headlines like:

• How A Single A/B Test Increased Conversions by 336%
• How AMD Used A/B Testing to Achieve 3600% Increase in Social Sharing
• How Server Density A/B Tested Pricing Plans and Increased Revenue by 114%

Do you just blindly follow A/B testing recommendations based on such claims?

Or do you use case studies to help generate your own A/B testing ideas that fit within the context of your own online business?

Remember that there's a publication bias in case studies. They only share impressive results because those are the headlines that get clicks. The more typical single digit testing gains or inconsequential results never make it to press.

The reality is that small wins can compound into significant conversion rate gains over time, but only if you make A/B testing a habit every time you publish.

Do not underestimate the benefit of small, but frequent A/B testing wins compounded over time. The other benefit to frequent A/B testing? You're more likely to land a big conversion optimization win at some point. If you don't test, don't expect anything to change...

If you learn to appreciate your more frequent, but small A/B testing gains, expect compound conversions and business growth as a result!

Now It's Your Turn

If you're in need of an A/B testing tool that is both affordable and super simple to use, Thrive Optimize is the fire from the gods you've been waiting for.

Go forth and learn those valuable lessons about your customers you can use to compound your sales – and profit – over time!

And if you want some help getting started creating a useful A/B test that follows these 10 A/B testing principles, post your questions and testing ideas in the comment section below so the Thrive Themes Team can help you with some rapid feedback!