Why Should You Split Test Email Subject Lines?

Did you know split testing is actually part of everyday life?

It can happen when you try on different outfits. Or if you try a different ingredient in your recipe. Or maybe you’re comparing ways to tell people you want to take one of those vacations to the moon so you know how you should break it to your spouse. Split testing is all about testing different options to find what works the best.

Just like you don’t want to wear the suit that everyone thinks is ugly, you don’t want to send an email newsletter with a subject that doesn’t sound appealing.

Why You Should Split Test Subject Lines

“Split testing allows you to find what will work best” is a rather broad statement, so let’s take a closer look at the benefits:

Increase conversions
When you have a subject line that performs well, more subscribers will open that message. More subscribers opening means more subscribers reading. And more subscribers reading means more subscribers are likely to respond to the call for action.

Learn about your audience
You want to get to know your subscribers so you can send information they’ll love. There are a number of ways to get to know your subscribers, and looking at the subject lines they like is just one of them. It can teach you if they like your sale emails, or if they open your newsletters more.

Adjust to people changing
Just because something has worked before, doesn’t mean it will always continue to work. As your subscribers’ needs and wants change, you’ll want to continue testing to make sure you’re aware of what they want.

What You Can Split Test

There are a number of things you can split test in your subject line:

Length
Short subject line vs. long subject line? In general, you want to use the least number of characters possible while still getting your point across. Sometimes you may find that it’s more important to get your point across, and that requires a longer subject line.

Personalization
Should you use subscribers’ names or other personal information in the subject? There are mixed reviews on personalization. It’s up to you to find out how your subscribers feel about it.

Company name
Should you include your company name in the subject? We’ve seen results on how this might be a good move. In most cases, the more your subscribers see your brand, the better. You should still test to see how subject line branding performs for your list.

Wording
How you say what you need to say is important. For example, do subscribers like “% off” or “$ off?” Do they respond to psychological trigger words like “secrets?” You can test these in your subject line to find what wording you should be using, and it can even help you with writing the rest of your messages.

Appearance
Your capitalization and punctuation can also influence subscribers. You don’t want your subject line to look like spam, but at the same time you want it to pop. What you think looks good and what your subscribers think looks good can be different, so you need to test!

Split Tests by Tumbleweed Houses

The Tumbleweed Tiny House Company used the AWeber split test feature regularly. Check out some of the split tests they’ve done on the subject line:

Winning subject: “It’s FREE. All the tiny houses on our site and more”
(26% more opens)
Losing subject: “It features all the houses on our website plus more…”
Our take: People love free things, so using the word “free” right up front may have been key here. Other than that, the subjects are very similar. Just one word can cause a huge difference.
Winning subject: “SOLAR LIVING GOES TINY”
(18% more opens)
Losing subject: “Solar Living Goes Tiny”
Our take: Some people might think using all-caps looks like spam, but sometimes it grabs attention. If you’re in your subscribers’ address book and consistently sending quality content, then using this tactic could be beneficial.
Winning subject: “Not so nuts after all”
(17% more opens)
Losing subject: “Can Evan and Gabby stay sane in 117 square feet?”
Our take: The winning subject doesn’t give a lot of information; it’s more of a teaser. It’s a good strategy, because it can make the subscriber curious. Who’s nuts? Why did people think they were nuts?

Have some of Tumbleweed’s results surprised you? That’s why it’s important to split test! You’ll never know what will work best for your campaign until you set up some split tests.

We Want To Hear About Your Split Tests!

So what do you plan on split testing? Feel free to discuss it!

And if you’ve run some tests before, we want to know results you’ve seen!

By:
Education Marketing Associate (Crystal Gouldey Moore) on Google

Become a Better Email Marketer

Subscribe to This Blog by Email
Why Subscribe?

15 Comments

  1. We hear all the time that the word “free” is one of the biggest spam ID triggers, guaranteed to get your email chucked in the Junk folder, yet you recommend it as a winning word for the Subject line.

    How do you reconcile this contradiction?

    8/24/2011 2:05 pm
  2. Roberta,

    “free” in the subject line being a large spam flag is urban myth these days.

    It may have been true 4-6 years ago when spam filtering was far less sophisticated, but there are no major ISPs today that are going to junk folder your email for simply putting “free” in the subject line. They have far more
    accurate reputation measures to quantify what is and is not spam these days.

    8/24/2011 3:04 pm | Follow me on Twitter
  3. Is there a way to split test in Follow-up Messages? i know I can split test for Broadcast messages. I am offering a 28-Day free eCourse on website building. Even though it is free, I have trouble getting people open their emails. I want to test different subject lines.

    Thanks!

    8/28/2011 8:04 am
  4. Paul –
    We don’t currently offer a feature for split testing follow up messages. Thanks for the suggestion though. We’ll keep it in mind!

    8/29/2011 8:22 am
  5. An

    I would love to be able to split test the follow up messages, too!

    12/16/2011 10:31 am
  6. Dan

    One more vote for follow up message split testing! I would think that everyone would want this. Who’s with me?

    12/22/2011 10:01 pm
  7. Is there a way to Split test follow up messages?

    5/30/2012 8:01 am
  8. Mimi

    Sorry Tom but that’s some bad information you are pushing there (but I know with the best intentions). I own an ESP. The way SPAM filtering works today is on a scoring system. ALL the major ISPs, Email Hosts & ESPs basically use Spam Assassin as the backbone of their customized filtering. Emails are scored from 1 to 5, 5 being the worst in this case. You want your email to get the lowest possible spam probability score.

    Where FREE comes in is that it is INDEED on a list of trigger words that are often associated with spam (along with “Viagra”, “Winner”, etc.). It is not that a particular ISP or email host spam filter seeks out these words and routes the email to the SPAM filter. That is not how it works.

    The filters compile a score in Nanoseconds comprised of numerous factors that could suggest that the email is SPAM. If the score is too high, meaning too many factors, the email is indeed routed to the spam filter instead of inbox. The word “FREE” is heavily weighted as a spam indicator. You are right that this alone likely will not cause the email to be routed to SPAM, but . . . couple that with other indicators such as use of punctuation, special characters and uppercasing in the subject line or more than a very small ratio of underscoring, boldfacing and colored type and your email is likely a goner. The worst of all is an email with a high ratio of images to text. That’s almost always going to reach a 5.

    The site escapes me at the moment as it’s been a long time since I’ve had need to reference it, but there is a Open Source site that provides the long up-to-date list of spam-trigger words. When a new Internet scam goes around (like the Old Nigerian “Dear One, I need help claiming my zillion dollar inheritance”), in Wiki style, new spam trigger words are added to the list. Check out the open source spam assassin site too.

    There are also sites where you can get your subject line and email “spam scored” in advance, so you’ll know what if anything should be changed before launching.

    hope I’ve helped! :-)

    7/30/2012 12:36 pm
  9. Mimi -

    Thanks for your comment. We’ve actually looked into the issue of using trigger words like “Free” in subject lines and published a post about what we found, which you can read here: http://www.aweber.com/blog/email-marketing/yes-you-can-put-free-in-your-subject-line.htm

    It’s still important to be aware of content filtering, and hopefully the insights in your comment will be helpful to other readers as well.

    7/30/2012 2:36 pm
  10. Mimi

    Hi Rebekah,

    Unfortunately the answer in that thread is not quite right and could easily mislead the email campaigns of some to their doom.

    The research is sound but the conclusions aren’t. This is just one piece of it.

    I don’t want to bog down this thread with a copy of what I posted there so just look here for my explanation.

    http://www.aweber.com/blog/email-marketing/yes-you-can-put-free-in-your-subject-line.htm

    I truly hope it helps and i commend AWeber and you for calling attention to this important area!

    7/30/2012 3:02 pm
  11. Thanks for your comments, Mimi. It’s a good discussion to have, especially as spam filtering technology changes and improves.

    7/31/2012 7:50 am
  12. The information I posted yesterday to the other link did not post properly. I have reposted and i think it should b235452e very helpful to getting your emails past the SPAM filters. here is the link to the other AWEBER post:

    http://www.aweber.com/blog/email-marketing/yes-you-can-put-free-in-your-subject-line.htm/comment-page-1#comment-

    7/31/2012 8:31 pm
  13. Hi There,

    Another vote here for getting a “testing function” for “follow-up messages”. I keep building follow-up messages, I hardly ever broadcast.

    But here is a way to test different follow-up messages the “manual way”…

    If you are using paid advertising – as I do, it would be quite simple.
    Duplicate an ad (have them show equally) and duplicate the landing page, but link your opt-in box to a different list (two opt in boxes linking to different follow-up sequences and lists).

    If you have website/blog just rotate different list/boxes over a period of time if you can’t do it automatically.

    The fast way to duplicate your follow messages is to copy all the messages over to a new list (you can do this automatically in aweber). Then your done.

    Cheers

    5/8/2013 10:15 pm
  14. When will come the feature to split-test follow-up titles? It looks like a main feature to me.

    7/11/2013 8:12 am