Surprising Email Test Results: Text vs HTML Survey Versions

"Everyone talks in general about text vs HTML, but hardly anyone has specific test data they can share on it," Joanne Blatte, the editor of our upcoming Email Metrics Guide 2nd Edition, complained to me last week.

I was about to email about 50,000 of our readers a survey to gather data for our media kit, so I figured, why not toss a test cell into it? Heck, why not toss in two?

Here's how the test worked and what I learned:

We randomly generated three cells (I prefer this to slicing the list into parts so there's less chance of skewing results.)

Group A were sent a text-only note with a link to the survey form online.

Group B were sent an identical note with link, except a graphic of our logo was at the very top which meant the message was HTML.

Group C were sent a virtually identical note with our logo at the top, but they also received the actual survey form in the body of their email. If the form worked in their email system (Lotus Notesy in the UK). I also expected the HTML "lite" to win overall for no good reason beyond gut.

Actual results were:

Group A text-only:
Opens - can't tell with text
% click on link - 8.5%
% of sent completed survey - 7.5%

Group B HTML lite:
Opens - 35%
% click on link - 8%
% of sent completed survey - 7%

Group C HTML form:
Opens - 35.4%
% clicked link to use form online - 1%
% of sent completed survey - 7.2%

Color me completely stunned. Never in a million years did I dream the tests would be so similar.

If the list was not regular readers, I suspect there would
have been more profound differences. Perhaps if people have a strong enough relationship with your brand, your
email format doesn't affect response as much as it would a
newbie?

Anyway, if you've ever conducted a text vs HTML test,
please do let me know what you learned at
anneh@marketingsherpa.com. Thanks