Marketing Research Chart: How frequently do marketers run A/B tests?

by Daniel Burstein, Director of Editorial Content

A/B testing can produce powerful discoveries about customers, like this example from a marketer who completed the Website Optimization Benchmark Report survey:

"Determined that a carousel with three images in rotation on the homepage was more effective at driving overall conversion than one with five images in rotation."

But you can't rest on your laurels with testing. What worked well a year ago may not produce results today due to an ever-shifting landscape of the economy, your competition, technology and myriad other factors.

In the Website Optimization Benchmark Report survey, we asked marketers:

Q. How frequently does your organization complete an optimization test?

Click here to see a printable version of this chart

One-third of marketers are not testing frequently

As you can see, almost one-third of marketers test quarterly at best — with 17% testing quarterly and 14% testing annually.

Only 8% of marketers are testing weekly. Also, 45% of marketers are testing at an "other," perhaps random, frequency.

Testing frequently helps you keep up with shifts in the marketplace that may change customer response since you ran your last test. For example:

• Have competitors radically changed the price of products? How does this affect your price and/or value combination?


• Has a competitor's (or supplier's) technological breakthrough changed the marketplace?


• Has a shift in the fundamentals of the economy changed what customers are looking for?


• Has a major news item had an impact on how customers perceive your product?


• Are customers researching and shopping for products in different ways because of new technology?


• As technology evolves, how do customers act differently on different devices?

Numbers can be revealing, but they can also lead us astray with false assurance. Keep in mind, just because you ran a test and achieved a result doesn't mean that what you learned about your headlines or button copy, for example, will always hold true. Revisit these discoveries from time to time.

As one Website Optimization Benchmark Report survey respondent said, "Fundamentals never change, but the technologies and usages do frequently."

Related Resources

Web Optimization Summit 2014 — New York City, May 21-23

Marketing Research Chart: Does A/B testing deliver a positive ROI?

Marketing Research Chart: Measuring website optimization ROI