Email Metrics for Marketers 101

Everyone says they love email because it is so "measurable."

The only problem is, although email marketing is rife with
numbers, metrics, and data, much of it is confusing or even
misleading.

Many marketers put undue emphasis on numbers that do not matter
(such as unsubscribe rates), others use common terminology with
very different meanings. Also, metrics reporting systems are not
standardized, so there is confusion there too.

To start the new year off right, I have created a handy 101
guide to email metrics basics for you:

1. Open Rate: Never a Certainty
2. Click Through Rate (CTR): Beware of Confusing Math
3. Conversions: Are You Counting Everything?
4. Unsubscribes/Opt-Outs: Unimportant!
5. Delivery Metrics: Tech Hell

-> 1. Open Rate: Never a Certainty

This is the percent of people you sent an email to whose inboxes
opened your message. Only emails containing some HTML can report
open rates. No one can tell you what the open rate of a text-
only message is.

You need to be aware of two big distinctions. First of all, it
does not mean the percent of people who received your email, just
the percent of emails that were sent. You can never know exactly
how many people actually got your email (see Delivery Metrics
below).

Secondly (and more importantly), an open rate *does not* mean
these people actually personally opened your message because they
wanted to read it.

That is because email programs often automatically open email for
people. Think about how many times a piece of unwanted mail has
popped open automatically in your Outlook preview box, or on your
Hotmail screen as you went through your inbox. Yes, all those
count as opens. Even though you did not click on the subject line
yourself to open the message.

No, no program today can tell the difference between an
Outlook preview box "open" and a full-screen open.

Nobody knows exactly what percent of opens are from email
boxes doing it automatically, and what percent from a human being
saying "Oh I want to open this."

This means "opens" are a useless number to look at except in a
broad-brush fashion. (i.e. an "open" rate of 90% is certainly
better than one of 10%.)

You should, however, always look at your open report to see if
it is freakishly low compared to your normal range. A big
variance can indicate that your message did not go out properly or
it was filtered at an unusually high rate before it got to
recipients.


-> 2. Click Through Rate (CTR): Beware of Confusing Math

Many times I have had this conversation with a marketer I am
interviewing:

The marketer, "Our click rate is xx%."

Me, "That's great. What's that a percent of? Total sent vs.
opens? Unique clicks vs. non-uniques?"

The marketer, "Oh, I'll have to check on that."

Just like statistics, click through rates are all about math. There are plenty of misleading click rate metrics out there in
the press, vendor data, and other reports about metrics. Here
are the five critical questions to ask about click metrics:

Question #1. What is the click a percent of?

Is it a percent of all messages sent, or it is a percent of just
the messages sent that were "opened." If the number seems
abnormally high, suspect the latter.

Question #2. Is the click unique?

If one recipient of your email message clicks on more than one
link, or they click on the same link repeatedly, is it counted as
multiple clicks even though they are one person? If the number
seems abnormally high, suspect the latter.

Question #3. Which link did they click on?

If there were multiple links in the message (and there usually
are), what percent of total emails sent got clicks on each of the
different links. Watch out on this number, people can confuse
you by talking about what percent of total clicks each link got,
which would be a completely different (vastly higher) number.

Question #4. Was it a house list or a rented list?

Click percentages from house lists (names that are owned by the
marketer sending to them) are generally more accurate than
percentages from rental lists, because you know exactly how many
names you are starting with a house list and you often have a
general idea of how much filtering to expect.

Rental files are completely different because usually the company
renting you the names will mail a few more than they tell you
they "officially" did, just to make up for any undeliverables,
bounces and potential filtered names. This is a perfectly
acceptable practice in the rental industry when done within
reason (yes, there are some scoundrels out there that do it to
excess when you're testing a list).

Question #5. Which broadcast firm sent the broadcast?

Different email broadcast firms have widely different delivery
rates, which in turn will affect your click rate as a percent of
messages sent because more or fewer messages actually got
through.

For example, some firms are "whitelisted" with AOL and other
major email systems so their mail gets through with less
filtering. Others are well-known in the ISP world as being
companies that allow opt-out marketers to use their systems to
send broadcasts, in that case they may be filtered heavily.

If you are judging the results of a campaign sent using
multiple broadcast firms (such as when you rent a variety of
lists), make sure you base your click math on percent of opens
instead of percent of total sent, because you just do not know
exactly how the various broadcast firms affected your results.


-> 3. Conversions: Are You Counting Everything?

Conversions is the number showing how many people actually ended
up taking the action you wanted them to take (perhaps buying
something from you, registering, or signing up for a newsletter).

Unfortunately most marketers do not have the tech back-end they
need to really count conversions properly. It can be difficult
and expensive, especially if you are dealing with legacy systems
and multiple customer/prospect databases.

However, your conversion stats are by far the most important
numbers you can gather. More important than clicks, opens,
deliveries, anything!

At the very least, you should be tracking the percent of click
throughs, by list and by creative, who took any of the actions
available on the landing page (especially abandoning it!).

If possible, you should also track the number of people who took
action via a different channel, perhaps calling your phone
number.

Remember: If anyone ever tries to wow you with click rates,
your reply should be, "Oh yeah? What about conversions?"


-> 4. Unsubscribes/Opt-Outs: Unimportant!

This is a huge pet peeve of mine. So many marketers brag, "Oh
the campaign was great, people loved it, our opt-out rate was
tiny!"

Guess what? Your opt-out rate is practically meaningless.

Think about it. When you get email you do not want, do you take
the time to opt-out of that list, or do you simply delete it?

People have been trained to delete, to filter, to complain to
their IT manager or email system, to switch mail boxes, to do
about everything but opt-out. Sometimes it is because they have
tried to opt-out in the past and it "didn't work," or they heard
the urban legend about opting-out letting spammers know you are a
live account.

Yes, watch your opt-out rates if you have a regularly sent
mailing to a house list. If there is an unusual blip, you need to
know about it.

Also, be sure to check with your email vendor to find out what
their opt-out report really indicates. (For example, the
SparkLIST unsubscribe report bundles requested opt-outs together
with auto-unsubs for accounts that have been undeliverable for
too many sends in a row.)


-> 5. Bounces/Undeliverables: Tech Hell

The terms "bounce" and "undeliverable" are used pretty much
interchangeably by email senders.

Most email systems will give you a delivery report when you
broadcast a message. That report is guaranteed to be inexact,
so do not trust the numbers as gospel. They are rough guidelines
rather than definite deliveries.

a. Bounces:

Lynda Partner, CEO GotMarketing (an email broadcast firm), told
us a couple of weeks ago she had identified 32 different kinds of
bounces. Can you say tech hell?

A bounce could be from a bad address, a full mailbox, a downed
computer system or ISP, or it could be a formal bounce by an ISP
that has decided not to accept any email from your IP address (the
place your email comes from).

Although your techies may, marketers do not need to worry
enormously about the distinctions between the various types of
bounces, aside from checking your overall rate occasionally.

For house lists mailed twice a month or more, the rate should not
be higher than 3%, and generally runs lower. If it is in the
double digits, the problem is probably with tech and not your
list. Call your broadcast firm immediately!

b. Transient problems:

Often included as "bounces," these are emails that could not get
through for a variety of reasons but your system will keep trying
for a few more sends. Should not be higher 3% for a regularly
mailed house list.

c. Filtered messages:

You *can not* get a report that tells you how many of your messages
have been filtered out or redirected into "junk" folders by all
ISPs and corporate IT departments before reaching the recipient's
mailbox.

You also *can not* get a report of how many messages are filtered
by the recipients themselves who may be using programs to keep
junk out of their email box.

This means anyone who promises you they know their delivery rate,
or that they know their messages are not filtered is not telling
you the truth. It is technically impossible right now to know how
much is being filtered. Period.

Related Sherpa links:

Executive Summary: Email Marketing Metrics Survey Results - 1,711
Marketers Reveal Data (No-cost article)
http://www.marketingsherpa.com/sample.cfm?contentID=2137

How to Track Open Rates of Text Messages With a Teeny-Tiny Bit of
HTML
http://www.consumermarketingbiz.com/barrier.cfm?CID=2127

How PC Mall Increases e-Store Profits with Database Marketing
http://www.consumermarketingbiz.com/barrier.cfm?currentID=2210

The Three Major Problems with Online Marketing Metrics (And What
You Can Do About Them)
http://www.consumermarketingbiz.com/barrier.cfm?CID=2004

Key ROI Metrics B-to-B Marketers Should Use for Online Campaigns
http://www.consumermarketingbiz.com/barrier.cfm?CID=1589