Third Year Results from Hallmark's Online Market Research Experiment

Two years ago, we contacted Tom Brailsford, Manager of Advancing Capabilities at Hallmark Cards, on how his group was developing Internet-based research tactics for Hallmark Cards. In How Hallmark Cards Conducts Online Market Research: Successful Tests and Biggest Mistake (see link below) we learned that online communities presented a valuable, if yet unproven, tool for the Business to Consumer marketplace.

We wanted to find out if this practical research, The Hallmark Idea Exchange, has been successful as a private members-only community for certain consumers, as well as a research lab for Hallmark. We quizzed Brailsford on whether the online communities work for research purposes, what Hallmark does with the resulting information, and the key lessons learned since the Idea Exchange began. Here is what has been learned in the last two years:

-> How online communities work for research

Although online communities are different than panels, still, "You can use them successfully for research," Brailsford insists. When they began the effort, its capacity for research was virtually unknown.

During the last two years, Brailsford's team has done extensive parallel testing on the Idea Exchange versus traditional market research. Due to its considerable success, it's now an accepted research tool within the company.

"We're sort of an internal research supplier. People come to us and we do the research for them, quickly and less expensively than through other means," he says.

Research from the online communities is useful in a number of areas:

1. Faster, better, cheaper

Most importantly this is a tool that allows you to have it all!

Brailsford explains, "You can do three focus groups and hear from about 24 consumers and it'll cost you two or three weeks and $10,000. We can hear regularly from 150 consumers in 36 hours on a particular issue. We have evidence from linguistic analyses that suggests that the content we get from the communities is far richer than what comes from a focus group. We're trying to get faster, better, and cheaper."

2. Probing for understanding

It's nearly impossible to probe for understanding of respondents on a panel.

But with the communities, when an individual member has a response about something, you can probe for their reasons. "You can send something back to a threaded discussion on a bulletin board and ask, 'Why do you feel that? What makes you say that?'" This is where the inherent value of the system becomes apparent.

3. Bring consumers to life

The power of the communities to describe consumers and their lifestyles can't be underestimated.

"They post pictures of their homes, kitchens, cars. They can talk to us about the things on their mantelpiece," says Brailsford. "They give us an unprecedented view into their lives."

4. Learn consumer language

"Another thing that's really powerful about the communities is they let consumers describe issues in their own terms," Brailsford explains. "Lots of times companies use jargon internally."

At Hallmark they used to talk internally about "channels" of distribution. But consumers talk about stores, not channels. It is much clearer to ask consumers about the stores they shop in than what channels they shop.

For example, Brailsford clarifies, "We say we want to nurture, inspire, and lift one's spirits. We use those terms, and the communities have defined those terms for us. So we have learned how those things play out in their lives. It gives us a much richer vocabulary to talk about these things."

-> When *not* to use the communities for research

Because the members do not make up a representative sample, the Idea Exchange doesn't work for all projects.

If you have 10 designs and you want to know the top two, "We can do that on the Idea Exchange," says Brailsford.

"But if you want to rank them in order from top to bottom, and the middle ranks are really important, then traditional techniques work best," Brailsford reveals.

Brand awareness or brand loyalty research is not adequately measured here since members of the communities become more favorably disposed to the brand over time.

-> Examples of how Hallmark uses the Idea Exchange

Brailsford sends a quarterly newsletter, which goes to about 700 employees, that shares key learnings from the communities. The research library also sends out a weekly email bulletin to about 200 employees, which gathers data from ongoing research projects, many of which come from the Idea Exchange.

Here are 3 examples of how the Idea Exchange has been used:

o Example #1. Brailsford's team has conducted product tests in which they took pictures of a photo album display, and then digitally superimposed new artwork onto the albums. Members saw the new designs in the display and then took a survey telling which design they liked and why.

Testing products without actually having to create prototypes "saves huge," he promises.

o Example #2. "One of the first things we did was to ask people to take pictures of their Christmas trees and tell us a story about it," he says. "We learned a lot about Christmas ornaments and decorating from those photos," maintains Brailsford.

o Example #3. The Specialty Retail Group asked Idea Exchange team to help bring the card shop consumer to life. "We had them take us on a video tour of their home, and provide us visual representations of who they are," he says. "We put them together as collages in an effort to better understand our consumers. The presentations of these visual images have been very impactful. The visual images have also been useful to people in the Gift business to look at these collages and get a different look at what consumers considered memorable versus what we considered memorable," according to Brailsford.

-> Lessons learned

o Lesson #1. Most Importantly: Online communities work

"When we began, we didn’t know if people would actively participate," Brailsford says. "We regularly have 50% participation and sometimes as high as 75-80%."

It has been interesting to his team to watch how the communities have created their own social network. "It's a kind of growing phenomenon within our society. When I left high school and went to college you lost track of all your friends, but kids today keep those connections with email and instant messages and cell phones. The whole fabric of social networking is changing and the communities have watched that happen."

o Lesson #2. It was hard to get men to participate

Because women make 75% of purchases in Hallmark stores, Brailsford's team initially tried to involve 75% women and 25% men. "But we could only get three men to participate and they eventually dropped out. Women do such a better job of staying in touch, sending cards, communicating. So all members are women now, the facilitators are women, the one-and-a-half people who participate [in-house] are women. I'm just a visitor."

However, Brailsford says, this may only be true of the Hallmark brand and products.

o Lesson #3. Write your code of conduct carefully

Brailsford's team had to rewrite their code of conduct based on some "unpleasant experiences."

"On the discussion boards participants had the opportunity to post comments anonymously, and that caused us problems once, so we had to rewrite the code and be more specific about what could be posted anonymously," he acknowledges. "We've had to un-invite people for unsportsmanlike conduct." Uninviting people can be a problem, because by its nature, the communities get people intimately involved with the brand. If you offend them, they're likely to spread negativity as vigorously as they would positive things. You have to be careful, he warns. "It's a relationship. You need to treat it carefully, as with any relationship." Still, it is very important that the members feel like the community is a safe place.

Useful links related to story:

Hallmark Cards http://www.hallmarkcards.com

MarketingSherpa article: Hallmark Cards Conducts Online Market Research: Successful Tests and Biggest Mistake http://www.marketingsherpa.com/sample.cfm?contentID=2113 (small fee for access)