A while ago, our CEO was interviewed by the wildly successful podcaster John Lee Dumas for his show Entrepreneur on Fire. During the interview, we agreed to help John with his lead generation efforts.
Since then we’ve been working on incrementally improving the opt-in conversions on eofire.com, using the Thrive Leads plugin.
In this case study you’ll discover the step by step process we followed, which led to a 268% improvement in opt-in conversion rates.
You’ll see what worked (and what didn’t) and our take on why we got these results.
More...
First Step: Get The Control In Place
To make sure our test led to actual improvements, we first needed to get a control in place. There were already opt-in forms showing on the Eofire site, so when we switched to Thrive Leads, we created a clone of the opt-in form we were going to test.
This control we created looked exactly like the previous form, showed on the same pages and had the same trigger. Specifically, it was triggered on exit intent. This means that it never showed for mobile users because the trigger is based the movement of the mouse pointer.
The conversion rate on this lightbox was 2.09%, which became our “score to beat” for this test.
Control: Exit Intent Triggered Lightbox: 2.09% Conversion Rate
Don’t Exclude Mobile Users
The very first test we ran was to use the exact same design, but with the SmartExit feature in Thrive Leads activated. This will trigger the lightbox on exit intent for desktop users but will trigger after a certain time for mobile users.
Instead of ignoring the mobile visitors, we made sure they also saw the opt-in offer.
Smart Exit Triggered Lightbox: 3.28% Conversion Rate
By simply showing the opt-in offer to mobile users, we increased conversions by 58%!
Get a Micro Commitment First
Without changing anything else about the trigger or on the animation, we decided to add a multiple choice opt-in form.
Smart Exit Intent Triggered Multi Step Lightbox: 3.79% Conversion Rate
When somebody wants to leave the site on desktop (or after a certain time on mobile), they are presented with a yes/no choice in the lightbox. If they click on “yes”, the opt-in form is shown.
This allows the visitors to first answer a question (something the brain loves to do) before being asked to enter their information. When you click on the button, you confirm that you do indeed want the thing that was advertised. And that click is a small commitment.
Having made that micro commitment, you are then more likely to fill out an opt-in form when presented with one.
Adding this extra step increased conversion again and now this opt-in form is already converting 81.34% better than the control.
From this first test we learned that it’s a big mistake to ignore mobile visitors and to respect the power of micro commitments.
Here’s the full test data:
From 2.09% Conversion Rate to 3.79% Conversion Rate, a 81.34% Improvement
The Right Timing
Up until now, we only tested (smart) exit intent triggers. It’s time to test if this is indeed the most efficient way to convert visitors into subscribers.
We decided to test the same two-step lightbox with different triggers:
- The control = SmartExit
- Showing the lightbox immediately on page load
- Showing the lightbox after 10 seconds
Here’s the raw test data:
To be clear: the exact same opt-in form is being shown in each instance. It’s only the timing/trigger that we’re testing here.
Here’s an overview of how different triggers affected the conversion rate:
The test shows that both showing the lightbox after a delay or showing it instantly on page load are significantly more effective at converting visitors into subscribers than the exit intent trigger is. In this case, showing the lightbox immediately on page load led to the highest conversion rate.
Having a lightbox shown immediately on page load might be annoying and you might choose not to use this on your website (it’s totally up to you), but this test shows that in terms of conversion rates, it can make a positive difference.
Changing the trigger from exit intent to immediately on page load got us to a 4.79% conversion rate
Have Some Fun With Animations
Now that we’ve seen a big improvement in conversion rates, it’s time to tweak the opt-in form a little bit. We wanted to see if the entry animation of the opt-in form would make a difference.
We tested:
- Zoom in (the control)
- Zoom out
- Rotational
And we saw some interesting results indeed.
One word of caution about this test. This typically is the kind of test we do not want you to start out with. Often this kind of tweaking does not give any significant results, that’s why we advise to test big changes first.
On the EOFire website (due to the high amounts of traffic) we knew that even if the results were inconclusive we wouldn’t lose a lot of time waiting for results.
The rotational animation got a 4.99% conversion rate and beat the zoom in and zoom out animations.
After running this 3 tests we went from a 2.09% conversion rate to a 4.99% conversion. That’s a whopping 238% improvement.
| “WOW, a 238% increase in conversions when I switched to Thrive Leads. This is the most powerful list building plugin I’ve used and the team behind it is ON FIRE! Thrive Leads absolutely gets a recommendation from me.” | ||
 | John Lee Dumas , Entrepreneur on Fire | |
But why stop there, right? After doing a special opt-in campaign for John’s book The Freedom Journal we continued the tests and got even more amazing results.
What Opt-in Offer Is Most Appealing?
The previous tests were all run with the same opt-in offer, the 12 resources for entrepreneurs. But John has more than one opt-in offer on his website.
Because testing big things gives the best chances of improvement, we decided to start a new test and see which one of the 5 opt-in offers would get the highest opt-in rates.
- Top 12 Resources PDF (control)
- $500 and 7 days ebook
- 11 habits of successful entrepreneur PDF
- Podcast Course
- Webinar Course
To make sure that the design didn’t influence on the conversion rate, we kept all opt-in forms the same and only changed the copy.
From this test, we discovered that the most appealing opt-in offer for the EOFire audience was the 500$ and 7 days ebook.
This offer converted at 5.66%
But not only is it interesting to see that this opt-in offer is outperforming the original one, we can also learn a lot from the underperforming opt-in offers.
One might have thought the “11 habits of successful entrepreneurs” would have been a compelling opt-in offer to the EOFire audience but it is clearly a lot less effective at converting visitors into subscribers that the "12 resources" or the "$500 ebook".
The podcast mini course and the webinar mini course also converted less, which is not as surprising as those offers have more of a niche appeal.
Think about this: John already had a more effective opt-in offer than the one he had been advertising for a long time. This just shows how important testing is.
Switching this made conversion go from 4.99% to 5.6% which might seem like a small improvement but for a website like EOFire this is literally hundreds of more subscribers per month.
Incremental Improvements
The current state of testing on the EOFire website allowed us to help John go from a 2.09% conversion rate to a 5.6% conversion rate.
This is a 268% improvement.To make that more practical, on a site that gets 1,000 visitors a day, this improvements means going from 8,000 leads generated in one year to over 20,000 leads per year!
But this did not happen in just a single test - and not every variation we tested outperformed the control. Each test helped us to improve a little bit and to discover what works and what doesn’t.
The Power of Relentless Testing and Incremental Improvements
As much of a cliché as it is, Rome really wasn’t built in a day and I hope that from this case study, you can see how continual, incremental improvements really pay off over time.
As always with case studies, if you simply copy the exact same things that worked in this example, you may or may not get good results. The “trick” to getting excellent conversion improvements like the ones presented here doesn’t lie in a specific tactic or that perfect opt-in form template. The trick is to turn A/B testing into a habit.
Now it’s Your Turn
Unlock the bonus PDF: 10 Things You Should A/B Test To Get Higher Conversion Rates On Your Opt-in Forms.
You're just a click away from accessing the bonus PDF. Click the download button below to instantly get the bonus:
And start testing.
What was your takeaway from this case study? Let me know in the comments below!
Excellent Post! Thanks 😀
Thanks Diogo
Awesome post, thank you!
I never heard of eofire.com but it looks like they have some interesting podcasts. Going to go listen now!
Hi Eric, a good podcast for sure! You’ll hear inspiring stories of lots of entrepreneurs.
Wow it is great. My conversion rate is 1.39%.
Time to start testing Binit. Hope this article give you some ideas about where to start!
Hi Hanne,
This is very interesting and useful, I’m just tweaking my Blog so this post will help me a lot.
The share to unlock box you used is part of an old Shane product (Sharely) is isn’t ?
Hi Claudemir,
Happy to hear this will help you improve your blog. We use Social Locker for now but that will probably change pretty soon.
Love what you guys keep delivering with all the Thrive products.
I have to admit that I have been very slack in improving my opt-ins. I know I need to test more and try all the features of Thrive Leads. It’s such a no-brainer to get more and more of your site visitors on to your list. I don’t really know why I haven’t been testing!
Anyway, this has kicked me into action.
Thank you!
Hi Colin,
Glad to hear this article kicked you into action to start testing. You’ll see it’s really only a few extra minutes of work but can change your results significantly.
And it is actually a lot of fun to do (although that might just be me getting all geeky about this stuff 😉 )
Excellent case study! Thank you for the insight 🙂
Linden
Thank you Linden
Excellent case study!
Thanks Arel
Great post Banners. BTW how do you get the “share to unlock PDF” widget in your post?
Hi Ron, thanks, we use Social Locker for the moment
Hanne,
Thanks for sharing this. I’m so glad A/B split testing has been baked into the Thrive products. It makes a huge difference in the success of a website.
One question: how do you determine statistical significance for a test?
Cheers!
Mike
Portland, OR
Hi Mike,
You can use online tools to calculate statistical significance. This is a pretty complicated one or this is a more simple one.
In general though you should set up a test and forget about it.
I know this is hard 🙂 I really have to force myself to do this, but the beginning of the test often gives different results than the final outcome! Constantly looking at a test will make you panic and stop tests before getting real results.
When you set up test and then “forget about it”, you can use the automatic winners function of Thrive Leads to make sure you’re always showing the highest converting opt-in form.
Thanks Hanne for such a finr case study. It will help beginners like me to give a proper direction to conversation tactics.
Hi Ashu, thanks. Hope this article shows that all levels can use A/B testing (even beginners!)
I found this Post pleasant because written as a story, and represent also for me a very good training.
Thanks Michel 🙂
Once again, members who are already PAYING for thrive themes should not be asked to tweet and share stuff to get access to lead magnets. Please start providing these resources in the Members section.
Hi Pulkit,
This blog is not only read by customers so you’ll keep seeing share to unlock lead magnets but we are working towards a resource area. We want to do things right so it will not be implemented tomorrow but it’s coming.
Posts like this truly let me know that I made the right decision to go with Thrive Themes. I’m definitely going to use this info to start tweaking and experimenting with the opt in forms I currently use on my site.
Hi Fred, stoked to hear that!
It’s a nice post, but lets point out the obvious – the final opt-in rate is just above 5%, which is a known standard among online marketers with targeted traffic.
“Known standard”? Where did you get that from?
Perhaps more importantly, this is not targeted traffic. This is all kinds of traffic coming to all kinds of blog posts. Of course, conversion rates on a landing page for targeted traffic could be much higher.
Thanks for opening my eyes to the possibilities here!
Our pleasure, John
muy bueno!!!!
Thanks Alan
Yes l love Case Studies too – provides highly actionable steps & also reminds users of the many functions a aisle within the Thrive tool box. Cheers
That’s exactly right, Alison 🙂
Great post! It really helped me get clear on a viable progression for A/B testing.
Thanks Tomas
Real world case studies life this have a lot to offer. Thanks for sharing.
Bayo
Happy you liked it Bayo
Great post Hanne. I like the step-by-step approach taken. One thing on animations. On testing this on a number of sites, we found that animations locked up on the viewing computer. That is, the screen just halted. We removed the animations and the forms showed up again as they should. For the small increment in optins I don’t think the animations are worth it. Maybe it’s the CDN, maybe it’s the site structure, but I’m not taking the risk. I found overall they they affected site speed quite badly. Speaking of site speed, I also notice that there is a lot of render blocking and a heavy load on browser caching when running pagespeed tests through google console. How can we fix this and is it something that Thrive should look at on it’s themes? It seems to affect all of them (well, the one’s I’ve tried anyway). Keep up the good work. Thrive is definitely the best product on the market bar none.
Great information and very useful. I’d like to know how you did the math for 373% improvement though… I get 168% ((5.6 – 2.09)/2.09)=1.68 or 168%. A 100% improvement over 2.09% would get you to 4.18% and a 200% improvement would get you to 6.27% so 168% sounds correct.
Hi Charles, you’re totally right and I messed up the math on this one! Corrected now 🙂
I LOVE this and I’m super motivated to continue on with A/B testing in Thrive Leads (I’ve only started some time ago), thanks a lot Hanne and TT team, you’re awesome!
There’s something that bothers me a bit tho. AS a user, if I’m reading an article and I get the optin form pop up on top of it, I *won’t* subscribe (even if I want to get the optin bribe) because I don’t want to miss the article I’m reading. Normally after opting in I’ll be taken to the optin thank you page, and then I’ll have missed what I was reading (which was the actual reason I was there in the first place).
Is there a way around this?
Hi Maricarmen,
I see what you mean and I think here you have to ask yourself as the website owner do you prefer people who opt-in or people who read the article?
For those people, who act like you (and will not opt-in not to loose track of the article) I suggest adding an opt-in form on the bottom of the article that repeats the offer.
But you’ll probably notice that the lightbox will get more leads 🙂 (But you should test for sure!)
Another thing to monitor closely is conversion by traffic source.
We have a squeeze page that is converting at 31.22% for Adwords traffic. That’s pretty good, but a lot of that is the focused nature of the keywords we’re targeting.
You’re 100% right Simon, the traffic source does make a big difference! Here we are talking about all traffic to the website (so not to a specific landing page). But the difference in conversion rate between cold traffic (people who never heard about you before and come through eg ads) and warm traffic (an offer to people who already know, like and trust you) is huge.
31,22% on adwords traffic is really good though!
Very helpful – thank you for the excellent break down of the process and the results tracking.
Thank you Rob
Anne,
s’il vous plait, une chose que je ne comprend pas,
– On demande l’adresse email pour envoyer le rapport, mais en fait la Lightbox renvoie seulement sur une page de téléchargement, n’est-ce pas ?
– Aurait-il été possible de faire une Double Opt qui déclencherait l’envoi du mail par la Lightbox, qui donne alors le lien de téléchargement ?
Bonjour Michel,
Oui, le light box n’envoie pas de mail, Thrive Leads n’envoie pas de mails (c’est le service d’emailing, comme Active Campaign par exemple qui le fait).
Vous devez paramétrer votre service de mailing pour que le mail de bienvenue envoie le lien ou pour qu’une automation démarre à l’inscription de ce formulaire.
Le premier mail dans l’automation sera un mail avec le lien de téléchargement.
What is the Program that You show the testdata with ?
Hey Jan, we use Thrive Leads (and the built in A/B testing fonction).
Another outstanding post, Hanne – thanks very much. We’re looking at this right now, so once again, you’re psychic. 🙂
Hihi Thanks Matthew, I have a crystal ball to help me 😉
Excellent Case Study. I love Thrive Leads & also A/B Testing features it provides. In your opinion from your past tests which of Thrive Leads Opt-In Form converts best?
Hi Mohit,
Sorry no real answer there. In our tests interruptive opt-in forms (such as a light box, screenfiller or a scroll mat) perform better than less interruptive opt-in forms. But for the design you’ll have to test yourself 🙂
Another incredible blog post. This is exactly what I needed to know today. EOF is a great example that we can all learn from.
Would love to have a demo of the rotational thingie and the exit intent. Or, put that pointer to other places on your site as an end note.
Hi Mark,
You can see the rotational effect here . The Exit intent is simply when people go with their mouse to the close or back button.