We Spent $5,311.48 A/B Testing 4 Landing Pages—Here’s What We Learned
...and the revolutionary-business-changing-secret-sauce-landing-page-variation was...
Wait, wait, wait, did you just read we spent $5,311.48 for “Inconclusive”?!
Yes, but it’s not surprising and was actually a good thing because of the experiment’s primary goal.
I’ll explain the primary goal of the experiment at the end, and you’ll understand why.
However, first I’m going to give you:
- The 2 mistakes we made so you don’t waste your money making them yourself.
- All the landing pages we tested.
- The best Facebook ads we used to drive traffic to the pages.
You can use it all for your own business so you don’t throw your money away, because even though we didn’t find a higher converting landing page doesn’t mean we didn’t learn anything...
- Test Duration: 86 days
- Landing Page Variations: 4
- Total Conversions: 704 Leads
- Total Spent = $5,311.48
- Total Impressions: 4,787
Google Analytics Test Results
Lesson 1: Test BIG Differences & a SMALL Number of Versions
Here at Thrive Themes we focus on data driven results. It’s why A/B testing is not only a cornerstone of Thrive Themes’ company strategy, but also a cornerstone of most of the plugins we release for our customers.
However, A/B testing can get tricky when it comes to understanding how much data you actually need, and how to interpret that data to make a statistically significant decision.
The 4 Landing Page Variations Tested:
While they all look different—different colors, different wording, different layout—there’s nothing MASSIVELY different about them.
They are all above-the-fold landing pages, they all hit similar pain-points and they all showcase similar benefits.
But most importantly, they are all promoting the SAME opt-in offer and they all do a GOOD ENOUGH job at it by taking advantage of good landing page practices.
Because of how many variations we were testing, the fact they were all for the same opt-in offer and because of where all the traffic for this test came from, it’s not surprising the test was inconclusive.
This brings us to our next lesson...
Lesson 2: Know How Your Visitors Arrive
For this experiment, all of the visitors to these pages came from paid Facebook traffic.
More specifically, they arrived because they clicked on a Facebook ad only shown to a specifically targeted Facebook audience.
This means: only highly targeted, qualified leads saw these landing pages.
In order to see these pages you had to meet the following criteria:
- Be part of an audience similar to people who have already bought Thrive Leads (the opt-in offer of this experiment shows you how to build better opt-in forms). This audience was automatically created by Facebook and is called a “lookalike audience”.
- “Liked” specific Facebook pages indicating interest in building better opt-in forms. These included (but aren’t limited to): Digital Marketer, Jon Loomer Digital, Ryan Deiss, Infusionsoft, Instapage or Unbounce.
- Found the Facebook ad interesting enough to actually click it.
So by the time someone saw any of these pages, there’s already a good chance they would be interested in signing up.
So what did the Facebook ads actually look like?
Well, we were also running a test on the Facebook ads at the same time as this landing page experiment. Here are the top three performing ads by the highest click through rate (CTR):
As you can see, the copy is somewhat similar to the landing pages, but the images we use in the ads are nowhere to be seen on any of the landing pages.
This is because of the nature of the experiment. We wanted to find the best performing landing page, independent of the Facebook ads, i.e. we tried to minimize how big of an impact the ads had on the landing page test results.
To do this, the ads and landing pages were built independently of each other.
This isn’t the best practice. Generally speaking, you want your landing page to resemble the look and feel of the ad sending them there.
You want them to be congruent with each other because this is a quick indication for the visitor they are in the right place.
Why We Didn't Care
As you’ve been reading this post you might have said to yourself: “for a company focused on conversion optimization and A/B testing, it seems like they made some pretty fundamental errors with this experiment…”
Yes, that’s right.
Here are at least three ways we could have designed this experiment to be more successful from the beginning if our primary goal was to increase conversions for our opt-in offer.
- Only testing two landing page variations instead of four.
- Creating more massively different variations since we were already sending very qualified traffic to the pages.
- Making the landing pages more congruent to the Facebook ads.
However, our primary goal for creating four different landing pages wasn’t necessarily to find a higher converting landing page for our opt-in offer (although that would have been a nice bonus).
Our primary goal was to give our four Thrive Themes apprentices a chance to learn, by doing.
Building a landing page for one of our opt-in offers gave them an opportunity to:
- Get hands on experience using Thrive Landing Pages and Thrive Content Builder—two of the plugins we use the most.
- Practice building an effective landing page that arouses curiosity, communicates the USP and focuses on benefits over features.
And because this wasn’t just a theoretical exercise, we were able to:
- Use the data to see if anyone was particularly good (or particularly bad) at building landing pages and give appropriate feedback.
- Generate 704 qualified leads we can potentially turn into customers.
- Use the experiment as a case study to provide you, our audience, with something valuable.
Rapidly building effective landing pages is a fantastic skill to have, and it’s why we wanted our apprentices to learn it. So even though our test came back with inconclusive results, it was far from a failure.
If you’d like to learn this skill yourself, you should sign up for Thrive University and take our RAPID landing page course...it’s free!
And if you want us to spend more money on tests so you don’t have to...leave us your ideas in the comments below!