April 02, 2003
Article

Surprising Results from HTML Newsletter Tests: Now it's Safe to Use Cascading Style Sheets

SUMMARY: After months of testing, this Monday Rich Tatum at ChristianityToday.com took the first of the site's many email newsletters from text-only to HTML.

In this exclusive interview he shares his test results (including useful metrics) and insights. Also includes screenshots of all 15 different tests he conducted to make sure the HTML would turn out ok.
The folks at ChristianityToday.com who publish a variety of email newsletters to their 600,000 subscribers, have been yearning to switch from text-only to HTML for years now.

They held off until this Monday morning.

"In previous years we didn't feel we could send HTML and reliably expect them to appear correctly," explains Web Site Manager Rich Tatum.

Rich had split off the subscriber database into two files: One file were the AOL users and the other were everyone else.

He sent AOL addresses an HTML-version that was basic enough for the older AOL versions to read. There were no graphics, but he could at least put in bolds, italics and some color. Then every few months, he would check AOL's reported data and his own site visitor logs to see if a critical mass of AOL users had upgraded to a newer version that would allow full HTML display.

Last year, that number hit critical mass. Rich began to test full HTML on both AOL and a dozen other email clients including:

AOL 7
AOL 8
Eudora 5.2
Juno (web-based)
Netscape Standard
Netscape Composer 7.02
Outlook 2000 version 9
Outlook Express 6.0
Outlook Express 6.0 -- via Hotmail
Outlook Express 6.0 -- forwarded from Hotmail
Web-based: AOL
Web-based: Hotmail
Web-based: Hotmail printer friendly
Web-based: Yahoo.com
Explorer 6 - Standard

The results in many systems were terrible. The problem was that Tatum wanted to use cascading style sheets instead of "old style HTML."

Why? "Cascading style sheets reduce the overall file size so the message is smaller, and there's a lot more predictability about what something's going to look like." Your messages bounce less, may cost less to send (if you are paying for bandwidth), and often end up looking nicer.

Frustrated with results, Tatum held off on switching to HTML until just this Monday when a new series of tests proved that most email clients had changed and would now support the style sheets. "I was pleasantly surprised," he says.

The only problems were in Hotmail, Juno, and AOL's Web-based mail. These were ripping out the style sheet tags and displaying a really ugly email instead. (In addition, Tatum suspects Lotus Notes, older Eudora versions, and Pegasus mail will not work with style sheets, and we bet he is correct.)

Tatum decided not to worry about Juno and AOL Web-based users because these were "a miniscule portion" of his list. However, Hotmail users are 20% of his list, so he posted queries to MarketingSherpa's EmailTech discussion group (info below) to see if anyone had an answer.

InfoWorld columnist Brian Livingston stepped forward to share his secret solution. Tatum is sworn not to reveal exactly what it is, but says you can have your techies figure it out by subscribing to one of his site's Books & Culture newsletter and viewing the code.

Finally after test after test (link below to actual screenshots from tests), Tatum launched his first full-HTML newsletter this Monday. At the same time, he also launched a subscription management form that people could use to switch from HTML to text if they wanted to, or switch email addresses.

Here are some quick results metrics from Monday's mailing:

7,122 sent
7,036 "successfully" delivered (Note: AOL does not reveal bounces so you think you got 100% delivered even though that is impossible.)
2,399 "opened" (Note: Outlook preview function skews open rates upwards. It shows some messages were opened even though the recipient did not actually click to open them.)
257 recipients visited ChristianityToday.com after "opening" the email. This is based on clickstream cookies and not just newsletter clicks, so even if you went to the site on your own later in the day, it counted. 4 people used the account management function to switch back to text-only email

Tatum notes that the clickstream analysis is especially helpful when you have a large Hotmail list, because Hotmail's annoying clickthrough header and its habit of killing all links after a message has been open for 4:59 minutes has trained recipients not to click on links in Hotmail. They just use their browser instead.

We will check back in with Tatum in a few months when he has had time to switch the rest of his lists to the new format and track more results.

In the meantime, here is a link to samples of his HTML tests we posted for you. Be sure to look at the Juno one, ewww! http://www.marketingsherpa.com/cht/ad.html

Also, if you are an email techie in charge of sending out a large amount of broadcast email and you would like to join our (very) small discussion group for techies, email AHolland@MarketingSherpa.com. There is no cost to join, but you must be a full-time email tech professional working with large broadcasts.

Improve Your Marketing

Join our thousands of weekly case study readers.

Enter your email below to receive MarketingSherpa news, updates, and promotions:

Note: Already a subscriber? Want to add a subscription?
Click Here to Manage Subscriptions