Advice Articles
Top Articles
- Successful Affiliate: Becky Turner
- "Show us something better and we'll give you $50,000"
- Affiliate Marketing Success Story
- How to make $1000 a month online from scratch
- Free Affiliate Masters Course
- Best Affiliate Programs
- SBI Value Exchange - free reciprocal links
- Affiliate Program Tutorial
- How to boost your AdSense revenue
- Web affiliate programs experiment 2
Stay In Touch
Split testing newsletter subject lines
Let's look at some interesting results together.
I'm going to show you a few stats that have just come in from a series of our recent email blasts.
To get the show on the road allow me to point out the criteria under which each of these split tests were conducted.
- Each email blast was sent to a list with 5000 or more subscribers. This means that the results are fairly stable and reliable.
- The 2 competing newsletter subject lines were split between the same list. It other words, if one of our lists has 20,000 subscribers, 10,000 on that list received exactly the same email with newsletter subject line a, and the other 10,000 received exactly the same newsletter, except these guys got newsletter subject line b. This means that split tests were conducted on an equal and fair playing field.
- Test 1 focuses on opens (the number of times that the newsletter was opened) while tests 2 – 5 reveal the number of times that links within the newsletter were actually clicked.
With the criteria for the tests on the table, let's get down to business.
1. Here are our first two candidates.
a) Spying on your competitors properly
b) Top 5 Advantages of Spy Tools
In this test we're analyzing opens. Our split testing revealed that the winner is...
a) Spying on your competitors properly
It was actually opened 12.5% more than b.
This is a little bit interesting to me because "Top 5" is usually a strong performer. I can only assume that people out there want to spy on their competitors. This must be why "Top 5" went down in this bout. Perhaps the title caps on b) are too official?
Any ideas? What do you think caused this outcome?
Moving right along...
2. The 2 contenders are going up against each other this time based on clicks (the number of times that links within the newsletter were actually clicked).
Here they are:
a) You snooze, you lose... :)
b) 4 critical factors for online profits
OK, I've got my money on one here, how about you?
Snare drum please. The winner by TKO...
a) You snooze, you lose... :)
And by a landslide too. Up by 22.3%
Now this one here I believe taps into the human condition. Snooze and lose with a smiley face versus critical factors.
My verdict is that "critical factors" is a bit too heavy and gloomy. The world's got enough problems and troubles as it is. Feels like that the light hearted approach won hands down here.
There may be a lesson to be learned when it comes to heralding good news? I don't know. Maybe my diagnosis is up the creek? What do you think?
3. Here they come. Two newsletter subject lines going head to head.
a) Sorry for the late notice
b) Late notice I know - sorry!
But hang on, they're the same aren't they? Well, the question is how do they perform? We want to make improvements in our email marketing. We're deeply committed to "good, better, best, never let them rest!"
They're off and racing...
And the winner is.... Oh no, an upset. a) Sorry for the late notice has won by a nose.
We'll actually, links on the exact same email but with newsletter subject line a) Sorry for the late notice were clicked on 7.8% more than the same newsletter with b) as the subject.
How can it be? We'll here's my take on it. Sorry with an exclamation mark is too in your face. Someone's preaching at me. They're trying to demand my attention so I retaliate with a no click. I vote with my click finger. Is it a possibility?
4. What's the difference with this pair? Well, one's been personalized with the subscribers name and the other hasn't. So in this test b) would have read Bill's exclusive …. or Mary's exclusive …. and so on.
a) Your exclusive keyword research tool discount coupon – private
b) {!firstname_fix}'s exclusive keyword research tool discount coupon – private
I'll be keen to see the outcomes here because I've heard a lot about the power of personalization so here we go.
The outcome? a) Your exclusive keyword research tool discount coupon – private up by 3%
Now this is a strange one. I have no idea why this occurred. The personalized newsletter subject line ran last when it should have hosed it in. Why?
I'll have to take a stab in the dark and go with the only idea I can come up with. Perhaps the personalization approach has grown old and people know all about the concept of mail merging from a database through a token. Perhaps they subconsciously say to themselves, “Here we go, another machine trying to be a human... I'll teach it a lesson.” Could it be? What do you think?
And finally, the last of the newsletter subject wars.
a) Keyword Magic Pro Available Now
b) I can save you days of keyword research time
This will be interesting. Without further ado.
Well, this was neck and neck. b) I can save you days of keyword research time came home by just on 2%.
It's really not a big difference at all.
I would say that pushing the brand would have pros and cons depending on the reputation of the product out there. Meanwhile, a) focuses on the benefits – time saving which seems to have the edge on this occasion.
This concludes the analysis of our most recent newsletter subject split tests.
Again, we would love to hear your feedback on any of these and your own experiences.
For those who want to run a bit of testing in this regard we use AWeber which will also allow you to roll out split test broadcasts with ease.
It's a wrap and we'll keep you in the loop regarding other Associate Programs findings soon. Cya!
Newsletter Sign-up
Your Host

Learn from an affiliate veteran.
Your host, Allan Gardyne, has been earning a good living from affiliate programs since 1998.






Really great post on split testing subject lines. I don't think people split test them enough in my opinion. Also, I feel that a lot of the lists I'm on the marketers (mostly outside IM) are missing the whole idea of subject lines. Your one goal is to get them to open that email.
I definitely am going to be testing the "you snooze, you lose...:)" link for an email Im sending out this weekend that is time sensitive. I will report back on my results with click thrus.
Completely agree with you on the personalization approach. I have honestly found that it is not that effective anymore and I have received similar results as you with opens when using personalization. Personalization within the email copy has actually decreased conversions too. That might be another test you want to look at.
Another thing I would suggest people test is adding (open asap) or (time sensitive) to appropriate subject lines I have had those give me increases in open rates of between about 5%-12% or so.
Thanks again,
Ryan
Yer, the personalization approach was a bit of a wake up call for me. I think a lot in IM are stuck in 'that's the way we've always done it.'
We'll certainly give your two suggestions (open asap) or (time sensitive) a go to.
Thanks again. Mike
Great article! So many marketers simply don't test. I do have one question, how statistically significant are the results? Did you do any further analysis (using standard deviations etc) to the results. Given some were at the 2 and 3 percent range it would be interesting to know if these results are significant or require further testing. Again, thanks for a great article!
best, Jack
Thanks for the boost.
I believe that these results are fairly accurate given that they were split evenly. Same list, same day etc.
2 - 3% can equate to 1000's so it's worth measuring :)
I reckon that they're pretty solid but I may run the same tests again soon to see if the results are consistent. Like you say there could be other factors that influenced them.
Thanks again, Mike