How to Conduct an Online Survey, Part II The Art of Question Writing

****
Note: See Part I of this series, How to Set Goals:
http://www.contentbiz.com/sample.cfm?contentID=1980

See Part III: Testimonials at:
http://www.contentbiz.com/sample.cfm?contentID=1995
****

Last week, we discussed how to set a focused goal for your
survey. You already know the LAST thing you want to do is to
call everybody (editorial, ad sales, marketing) together to
brainstorm up a bunch of questions.

Instead, you are going to put together a survey for just one goal,
and one goal only. Here are some pointers on writing it:

-> Do you really, really, really need that question?

Even when focusing on a single survey objective, most publishers
still write way too many questions. This hurts you in two ways:

1. The more questions, the less likely you are to get answers.
People who click to your survey will leave in an instant if it
looks like "too much work." If you want a good response
rate, ask as few questions as possible.

2. Remember your back-end. It is so fun to whip up a bunch of
questions, but so much work to analyze answers afterwards.
(Especially open-ended questions where folks can write any answer
they would like … it can easily take you hours to organize their
answers into a remotely useful format.)

Think of survey writing as though you are an avid gardener
going to a garden center in the springtime. There is so much
temptation to buy all those plants! You must exert an iron will,
and only allow yourself to get items that you already dug the
hole for in the yard back home.

What is the perfect number of questions? Generally you can get
your goal accomplished with three to five. Sometimes it can take
as many as seven. If you go over 10, go back and try again.

No, it does not help to split the questions onto multiple pages so
folks only see a few and then have to click on a "continue" or
"next" button. You will just annoy everyone with a slow Internet
connection, and/or people who like to see how long a survey is
before they decide to take it.

No, it does not help not to number questions. Numbering questions
is a common courtesy to avoid at your peril.


-> How to get participants to answer easy questions

An easy question is one that requires no or little thought.
"What's your job function?" and "What's your age group?" are both
easy questions.

The problem is keeping them easy. Most survey writers yield to
the temptation to add lots of options (such as 27 different job
functions participants must pick from), or to ask for information
that might be easy, but feels too personal (such as "what's your
age?" vs. "Age group?")

Keep easy questions as easy as possible to get maximum response.
In particular, do not ask for contact information such as fax
number or street address unless you really, really need it.
Even then, tell them why you need it. ("What's your address so
we can send your free thank you gift to you?")

Also, consider end-results when you are making your list of
options people can choose from. Surveys only yield useful
information when you get enough responses to each option within
questions. If you have asked participants to pick one of 12
different items to describe themselves, what is the likelihood
you will end up with a big-enough pool of responses in each item to
make that slice worthwhile studying on the back-end??

For example, if you only get 3 answers out of 800 that say they
are "VP Trucking Platinum Pistons Operations," how much value do
you get from studying the rest of their answers? Well, not much
because 3 is not a viable pool to draw any conclusions from.


-> How to get participants to answer hard questions

Hard questions are questions with requiring multiple decisions
("Tell us how you'd rank these 10 items") and open-ended
questions ("What challenges do you see lying ahead for the
trucking industry?")

To get those hard questions answered, first organize your
questions in order of easiest-to-hardest.

Just because someone has entered your online survey does not mean
they are going to take the whole thing. With each question, you
have to sell them on the idea of continuing; "hey this isn't so
hard, hey this is kinda fun"

You do that by using an old sales trick of asking an easy no-
brainer question first -- such as what is your title? Or a quick
yes/no question. Once they have answered that first easy question,
they are emotionally involved and more likely to go on and answer
the harder questions.

If you have several hard questions in a row, it is best to scatter
a super-easy question between them. Just as your respondent
is feeling like "this is nasty hard work," you sock them with a
"what the heck, this one's easy" question.

You can also get higher response for hard questions by
constructing them in an easier-to-answer fashion without reducing
their value to you. For example:

* Rather than asking participants to rank items on a 1-10 scale
(or worse yet, to assign a list of items numbers in order of
importance -- yuck!), ask participants how important items are in
words. For example:

a. Hard work way -
Please give these items a ranking in order of
importance from 1-10 (10 being most important)
Trucking regulations 1,2,3,4,5,6,7,8,9,10

b. Easier way -
How important are these items to your job? Please pick
one choice:
Trucking regulations (not very) (somewhat) (critical)

* Rather than using a big open box for each open-ended question
(which can be intimidating for folks who do not like writing) use
a smaller, one-line box they can type a simple phrase or word
into. For example:

a. Hard work way
What in your opinion are the two biggest challenges facing
the trucking industry today?
(big open box to type in)

b. Easier way
Briefly in your opinion, what are the top two challenges
facing the trucking industry today?
Challenge #1 (little one-line white box)
Challenge #2 (little one-line white box)

* Break up long lists into two or more questions. Any list with
more than 5-7 items is too long. It looks dense to the eye. For
example:

a. Hard work way
How important is news on these items to you?
Routes
Marketing
Pistons
Regulations
Engines
Gears
Pricing
GPS
Wheels

b. Easier way
How important is news on these business topics to you?
Marketing
Regulations
Pricing
Routes

How important are technical and operations topics to you?
Pistons
Wheels
Gears
Engines
GPS


-> Wording your questions to get better answers

Publishers often make one of two mistakes when writing survey
questions -- unclear wording, and non-compelling wording.

Obviously your answers will be suspect if participants did not
understand what your question really meant. Often survey writers
sacrifice clarity to the alter of "keep it concise." Actually,
as long as you do not have too many questions, you can get wordier
in the questions themselves than you might think.

Be sure to define acronyms and any terms that your editor
might use a lot, but are not necessarily commonly used by readers
on a daily basis. For example, "Do you approve of Act 27B?"
should be replaced by, "Do you approve of Act 27B, the Texas law
that truckers must take a break every 3 hours?"

Then, show your survey draft to several people about the office.
Do not just ask them if the survey is easy to understand, ask them
to define in their own words what each item listed means. Almost
always you will be surprised to learn something you think is
crystal clear, really is not.

Because surveys are very important, so many publishers fall into
the trap of writing them with equally important language.
Their survey then sounds like their regular editorial voice put on a
three-piece, navy blue, business suit and is presenting at a
Board meeting. Unnaturally stiff, formal, and almost pompous.

Remember, your readers find your site or newsletter compelling
because of its voice … make sure your survey matches that voice.

Plus, just as with good marketing copy, surveys get more
responses when you include the word "You". For example, "Please
rank the following business issues in order of importance…" is
less likely to be answered than "How important are these business
issues to you?"


-> A few powerful open-ended questions

Here are some question ideas that have been proven to work in the
past. In each case depending on your audience you may want to
add a phrase like "Remember, this survey is completely
anonymous." Also, do not load up your survey with a whole bunch
of these. Using more than one or two will decrease response
dramatically:

"If the world's greatest trucking business expert was on the
phone right now with you, what would you ask him or her?"

"If you had the services of a trucking research team at your
office for a week, what would you ask them to research for you?"

"Come on, rip us apart! Trucking News can only improve if you
share your honest criticism. What can we do better?"

"What trucking issue is your biggest business problem right now?"

"What's your biggest fear about the next six months?"

"If you were the new publisher of Trucking News, what would you
immediately change?"

"Trucking News' editors want to do an in-depth special report for
you in May. What topic should it be on?"


-> Next week, How to ask for testimonials

Using a survey to get good testimonials is harder than you think.
But there are a few tricks that make all the difference. We will
share them with you in next week's ContentBiz.