Website Optimization: Simple A/B test adds 'thousands and thousands of dollars' to Cars.com’s bottom line

SUMMARY: Some marketing campaigns are complicated, while others are relatively simple and produce almost immediate results. As an example of the latter, see how an e-commerce company added an online security badge and ran a one-off A/B split test.

The company changed one simple element on an e-commerce homepage that resulted in a boost in conversion, and translated into thousands of dollars in monthly revenue.

by David Kirkpatrick, Reporter

CHALLENGE

A large proportion of MarketingSherpa case studies involve many steps, and cover a wide range of marketing activities. And for good reason, as most marketing changes that make an impact are very complex.

However, sometimes an effort comes along that was relatively simple, but ended up making a significant difference for the highlighted marketer.

Today’s case study is one of those efforts that looks simple, and in hindsight maybe even obvious, but resulted in significantly improving the online conversion of one business.

Cars.com has three websites -- CarPriceSecrets.com, WhyPaySticker.com and NewCars.com -- that are all lead generation websites with the goal of creating consumer leads for car dealers.

The business model is not complicated. The company drives high amounts of traffic to the site, so any boost in conversion has a direct benefit to the bottom line.

CAMPAIGN

This marketing effort involved an A/B test with a security badge on CarPriceSecrets.com designed to reduce visitor anxiety and improve conversion.

Step #1. Drive traffic to the website

Vince King, Director of Consumer Publishing, Cars.com, said the company does a lot of search engine marketing (SEM) to drive traffic to the websites.

A major portion of Cars.com’s marketing budget is spent on SEM, and any increase in conversion is significant because the three consumer websites receive around 1,000,000 unique visitors combined each month.

King said, "We’re always looking to figure out how we’re going to increase conversions. We define conversions as somebody who comes to our website, goes to the form page, chooses some dealers and hits ‘submit.’"

Cars.com’s search engine marketing is based around paid ads tied to automobile makes and models for all three consumer-facing websites.

"If you were to search for prices for a new Ford Mustang, you would probably see an ad for one of our three websites on Google and Bing," explained King.

Step #2. Test the website for security issues

Before Cars.com was able to place the security badge on its CarPriceSecrets.com website for testing, the vendor supplying the badge tested the site for security vulnerabilities. This test scanned from the network perimeter, to the Web application layer.

The scan also looked at potential problems, including:

Unintentional links to malicious websites

Browser exploit code

Accidental sharing of email data with spammers

Hacker testing

SQL injection

King said an additional benefit for Cars.com was that after its websites passed the security test, the vendor had a browser plug-in that placed an additional security icon next to search engine ads for anyone who had installed the plug-in.

Step #3. Run the A/B test

After CarPriceSecrets.com passed the website security test, the security badge was placed on the website for the A/B split test.

For 35 days, website traffic was randomly driven on a 50/50 basis to a homepage with the badge, and to a homepage without the badge.

The test reached statistical confidence after one month, but ran for an additional five days.

King said the vendor provided a number of different badges to choose from and made it very easy to set up the testing protocol.

RESULTS

The homepage treatment with the security badge had a 2.66% increase in online lead conversions.

The badge was then added to all three consumer websites, and given the high volume of website traffic, that level of lift in conversion represents "thousands and thousands of dollars" in monthly revenue, according to King.

"I heard that security, or trust, labels were important to help conversion on e-commerce and lead generation sites, so logos are important to consumers to give them the confidence in moving forward with their personal information," said King.

If you found this case study useful, you can sign up for our free bi-weekly Consumer Marketing newsletter.

Useful links related to this article

CREATIVE SAMPLE: Security badge on CarPriceSecrets.com

Cars.com

CarPriceSecrets.com

WhyPaySticker.com

NewCars.com

McAfee -- Cars.com’s security badge vendor

Optimization 101: How to get real results from A/B testing

Quick Lift Ideas: 8 test ideas to help you increase conversion across your site

Website Redesign: Customer surveys and testing help increase conversion by 21.6%

Anxiety: Use privacy as a competitive advantage

7 signs that you’re overvaluing search engine optimization

Marketing 101: What is conversion?