AWeber User Gets 1000% Increase in Opt-Ins Using Popover Form

Over at MarketingSherpa there’s a great case study of how AWeber user Leo Notenboom of increased his opt-in conversions by more than 1000%. The part that might surprise a lot of our readers (but not us)? He did it using a popover form.

Over at MarketingSherpa there’s a great case study of how AWeber user Leo Notenboom of increased his opt-in conversions by more than 1000%.

The part that might surprise a lot of our readers (but not us)? He did it using a popover form. As in, a simulated popup form.

Popover forms? But won’t they annoy visitors?

I can’t count the number of times I’ve heard someone say something to the effect of, “I just won’t use a popup. I don’t like them, and neither will my visitors.”

This makes me sad, because it shows an unwillingness to split test, which as I discussed recently is key to running effective email marketing campaigns.

This ‘annoyance factor’ is something Leo addresses in the case study — he found no negative effects of running the popover forms.

How’d he do it?

While I don’t want to hijack MarketingSherpa’s article, I think we need to highlight one key element of his success:

Leo rigorously tested the time delay he used for his form — it didn’t display immediately when visitors landed on his site. He set the delay based on the average amount of time visitors spent on his site.

For more details on the study, go read the writeup on MarketingSherpa. It’s only available to the public until July 9th 2008 (after that it’s members-only), so act fast!

To see Leo’s popover form in action, swing by and hang out a bit. 😉



  1. Robin Heppell

    7/3/2008 11:38 am


    Thanks for the insight and proof that they do work. My clients say the same thing. Now your article can be part of my arsenal.

    Funny, when I went over to MarketingSherpa to read the article, I was greeted with a popover!

  2. Roger Haeske

    7/3/2008 8:55 pm

    I read the article at Marketing Sherpa and there was something mentioned with the form. I’m wondering if this can be done via Aweber.

    I’ll post it below.


    -> Step #3. Set restriction

    They set a restriction on the hover box. Regular readers who didn

  3. Leo Notenboom

    7/3/2008 10:08 pm

    Hi Justin,

    Wanted to mention that the only complaint I actually DO get is when people have cookies disabled. I have the popover set to only display once every 6 months, which of course requires that the visitor be cookied. So far all complaints (all 2 or 3 of them, out of hundreds of thousands of visitors) have been related to people getting repeated popovers because something was interfering with cookies. Since I’m a Q&A site I posted an article on it:

    Quite acceptable. I’m very pleased.

  4. Jim

    7/4/2008 8:41 am

    Great topic and insight.
    I have been playing with Hover pop-ups and see they work around blocker settings.. I have tried as much as 200 seconds on my newsletter, for example.. but 200 seconds doesn’t seem to work. Respecting the client, I believe delayed pop-ups are subtle.

    I read the Sherpa article and now understand how the hover pop-up works within the html.. Rats though.. I didn’t get the Sherpa pop-up..
    Imagine wanting a pop-up and not getting one!

  5. Haywood

    7/5/2008 5:45 pm

    I am going to apply this to my site right now!
    Very informative.

  6. Tarik

    7/7/2008 7:46 am

    Thanks for the headsup. I used a pop box that activates when the user leaves the page months ago, and it produced a very high opt-rate.

    However, it was so abtrusive that I deleted it from my pages. The Hover Box is a nice alternative that produces similar effects.

    I’m so glad you published this case study. I really need to kick my e-mail marketing into gear!

  7. Tim McHyde

    7/7/2008 8:39 am

    Well I did not quite get 1000%, but I’m very happy with around 150% improvement I got out of this simple tweak.

    I used the default 5 second popup delay. For my site I tried 5, 30, 60, 40 and 45. 45 so far is the winner.

    I never thought to experiment, never knew we had split tests. Now I’m hooked. Can’t wait to see what happens when Isplit test different signup teaser texts. Since after the delay tweak I’m running at only 6% signup rate, I figure I had lots of room to improve my text next.

    Thanks for this very helpful tip.

  8. Justin Premick

    7/7/2008 10:43 am

    Thanks for sharing your experiences everyone!


    Same thing happened to me – guess they apply lessons from their own case studies 🙂


    There’s not a feature of the Web Form tool at this point, and according to Leo’s comment above it’s not something he did (looks like Sherpa misspoke).

    However, it is possible to do that. You’d have to cookie visitors when they land on your thank-you page. Then, check for that cookie when someone visits your site and if it’s present, exclude the form JavaScript from your page.


    150% is nothing to sneeze at… congratulations!

    Split testing is certainly addictive…

  9. Tim McHyde

    7/8/2008 12:47 pm

    Update, my numbers were wrong. My historic signup rate with my 5 second delay popup is 1.9% (yes, I know, abysmal). Now I’m seeing 6.9% using 40 seconds (it started beating 45 seconds consistently when I let it go a few days longer) so I figure that’s a 363% increase. I encourage everyone to use popups and try this tip if you have not already.

  10. Tim McHyde

    7/8/2008 1:16 pm

    Adding to my last comment, I just realized that the article’s 1000% increase claim is based on going from no popups to using popups with the optimal delay setting. So I really feel great getting 1/3 of that increase given that it is based on optimizing popups I already had.

  11. Jennifer Hofmann

    7/8/2008 3:35 pm

    Popups do get a bad rap. Just reading the title of this post, I had to cringe at the idea of inflicting popups on my cherished readers.

    But the delayed popup has a certain appeal – it seems more nuanced than the glaring, blinking, in-your-face blitz that most of us are used to.

    In other words, all you’re really doing using a popover is changing the screen view so that the reader can make a choice – and then go on reading.

    Whether you use a pop-up or popover, I still believe that the website’s copy – honest, compelling copy – is the way to grab readers interests and get sign-ups that will stick around.

    I’d be interested to know from that 1000% increase, what percentage of opt-ins unsubscribe within a given time frame. We all know that it’s the ones that stick around who equal both income and referrals.

  12. Lori Titus

    7/8/2008 7:24 pm

    I am one of those people who hate popups so much, I either close them automatically or leave the website in disgust. However, after reading numerous articles about how effective they are, I had to try on my own site.

    I started using a hover form solely on my final check-out page (ie, after a customer has placed an order, the hover appears on the thank-you page). I get about 200 visitors a day, and prior to this was getting maybe 1 new email list client a day. Now I am getting 2-4 on a regular basis. Maybe not a huge increase, but certainly significant, given my traffic.

    I did have one issue with Leo’s article. I have been too lazy to crunch the numbers, but is .13% (60 seconds) vs .11% (75 seconds) really a significant increase in a signup rate, given the traffic he receives? A "small but detectable difference" is different from a significant difference. Still, given other data (like clients spending an average of 60 seconds on the site), the 60 second delay is as good as any.

  13. Alex

    7/16/2008 8:36 am

    It looks like this format is definitely worth some testing. I was one of those who adheres to the point that any sort of pop-up annoys users…

  14. Rick Leland

    7/21/2008 3:43 pm

    I installed a pop up like Leo’s. My sign up rate went off the charts.

  15. Oaks

    7/28/2008 5:12 pm

    I stopped using pop-ups years ago because Google Adwords wouldn’t allow you to put them on a landing page. Is it still a no no with Adwords?

  16. Martin

    7/29/2008 4:03 am

    Hi all…

    You know I have just (in the last 3 days) put the lightbox web form on one of my sites and noticed a distinct increase in sign ups. Previously it was about 1-2 a day and now it’s like 5+

    I know these are small number but you’ve got to start somewhere. 🙂

    So I recommend you do at least try the lightbox form or different ways to approach sign ups, especially once you have a flow of traffic…

    You really need to try stuff out – that’s really the only way you can find out if it’s going to work or not…

    Great stuff Aweber…

  17. Get More Subscribers With These 3 Popup Form Split Tests - Inbox Ideas: Email Marketing Tips by AWeber

    10/9/2008 3:51 pm

    […] they’re not always the right solution for all sites, some businesses have found they can increase opt-in rates significantly by adding popup […]

  18. Steve

    10/9/2008 11:20 pm

    is there a way to customize the web form in the popups beyond just adding fields? Since we are forced to use the javascript it doesn’t seem like we can add anything to the popup beyond what we get with the Aweber web form creator

  19. Justin Premick

    10/22/2008 10:49 am

    Just came across another example of popovers yielding some great results:

    Check it out!

  20. aWeber Popups Increase Opt-Ins | Get A New Browser

    12/29/2008 6:18 pm

    […] verify the effectiveness of popover requests… And guess what – it works as advertised. aWeber featured a post stating a 1000% increase in subscriptions using this method. ProBlogger wrote a post regarding the […]

  21. Popup, popunder, hoverover, or inline

    9/3/2009 10:15 am

    […] month. then try a hover fly in ad. Guess which one gives you the most sign ups? Yep. Here are two examples of […]

  22. AWeber User Chris Garrett on Autoresponders, Incentives and Testing

    5/13/2010 8:47 am

    […] he doesn’t use popover forms as a list building tool, even though other users have had success with […]

  23. Neil Hocking

    8/21/2011 7:13 pm

    I know this is an old thread but I wondered if this was still relevant.

    Since February, the IAB have delisted popovers and popunders. Is that a sign that they don’t think people like them anymore?

    What do you think?

  24. Amanda Gagnon

    8/22/2011 9:16 am

    Neil ~ Popovers can certainly be abused in ways people don’t like (several in quick succession or filled with irrelevant offers), so they do have a bad reputation with some people. That said, if yours is a single, relevant form with a helpful offer, you’ll be combating that reputation.

    And of course, it always helps to have an inline form available as well.

  25. Pauline

    7/7/2013 6:03 pm

    This is an older post about pop-ups and pop overs, but it’s definitely got good information – thanks much! I’ve been toying with using a third party widget for this versus my existing AWeber account – guess I’ll play around with both and see!