Marketing Testing: How marketers tested an email form, product bundle, and advertising channels to increase marketing performance

This article was published in the MarketingSherpa email newsletter.

“Test. Let the customer tell you the best way to communicate your message,” Flint McGlaughlin teaches in Sales Funnel Design: How to combine an “old school” journal, a web page builder, and DALL·E.

To give you ideas for your own marketing tests, today we bring you case studies with examples of three kinds of testing:

• Information Collection – The world’s largest animal health company tested where to ask for email information in its lead generation funnel.
• Product Presentation – A commercial gym equipment manufacturer tested product bundles based on customer service inquiries.
• Channel Selection – A SaaS company tested which channels are most effective for short-form video advertisements.

Quick Case Study #1: Global animal health company increases quiz completion rate from 80% to 92% with transparency

Is it better to collect an email address at the beginning or end of an information-collecting quiz? Let’s take a look at an A/B test that sought to answer that research question.

Zoetis is the largest animal health company in the world. Its Building Healthy Herds website asked ranchers to share information about their herd so Zoetis could create a custom plan specific to their needs as well as send timely reminders and tips via email later.

Creative Sample #1: Website for global animal health company aimed at ranchers

The completion rate for the plan builder had been 80% for the previous eight months. The team decided to conduct an A/B test on the plan builder quiz to explore if increasing transparency might improve their digital lead-gen initiatives.

CONTROL – Begin quiz by asking for name

They sent 50% of Building Healthy Herds traffic to a control page with the original structure, where email information was the final request.

Creative Sample #2: Control – first page of quiz for global animal health company

TREATMENT – Begin quiz by asking for name and email

The other 50% was sent to a page that requested name and email information first.

Creative Sample #3: Treatment – first page of quiz for global animal health company

The team used Google Optimize to split the landing traffic on the survey start page between the control and treatment groups. The A/B test ran for 10 weeks (70 days).

RESULTS

After 10 weeks and 338 collected sessions, the treatment group emerged as the clear winner.

Moving the email component to earlier in the form improved bounce rates from 23.4% with the control group to just 8.5% with the treatment group. The treatment group also navigated to more pages per session and stayed on the Building Healthy Herds website longer. Best of all, the treatment group achieved a form completion rate of 92% — significantly outperforming averages from the previous eight months.

“We are focused on digital marketing excellence, and optimizing our digital content is paramount in that journey,” says Clint Mefford, Head of U.S. Cattle Communications and Operations, Zoetis. “Analyzing how content performs and implementing A/B testing on our websites, in our email marketing and in our digital and social ad placements helps us ensure we’re putting the right effort in the right places. But, more importantly, it also helps us tailor content to effectively engage our audiences online.”

Paid social content directing users to Building Healthy Herds ran concurrently with the test. New users during this timeframe increased by 200% over the previous period. The fact that conversion rates didn’t flatten on account of the traffic increase generated by paid social further indicates the success of the A/B testing.

“The beauty of A/B testing is that it is so simple,” said Ned Brown, Chief Creative Officer, Bader Rutter (Zoetis’ advertising agency). “With enough responses, you get direct proof that something works better. It’s not hard to decide when you have concrete results.”

Quick Case Study #2: Commercial gym equipment manufacturer tests product bundles, increases quote inquires 26%

Yanre Fitness manufactures fitness training equipment based on its own research and development. Its ideal customer is a commercial fitness gym.

BEFORE – Customers have too many options to choose from

The manufacturer was getting a decent volume of visitors to its gym equipment product page. Although there were a lot of options for visitors to choose from, the team was getting quote requests for only a small portion compared to the available options on the website.

Creative Sample #4: Gym equipment product page, before (231 results)

There was a lot of back-and-forth communication with potential customers before making a sale. So, the team took all the communication data from the past 30 days, analyzed it, and discovered a pattern. The sales team was getting a lot of queries related to available space in the customers’ gyms. Potential customers did not know what exactly they needed or if the equipment would be suitable for their available space.

AFTER – Product bundles based on available gym space eliminate guesswork for customers

So the team decided to add a page that showed visitors the most suitable pieces of equipment based on the available space in their gym. This eliminated the customer having to guess what they should order. If the team received any queries for recommendations, they referred customers to this page.

Creative Sample #5: Gym equipment product page, after (4 featured choices, plus more options)

The team was able to make bulk sales to customers without putting extra effort into upselling additional equipment. The company received 26% more pricing/quote inquiries. Plus, they received significantly fewer inquiries like: ‘Will ABC product be suitable for XYZ m² space?’

“Previously customers got a bit confused and couldn't figure out what they could fit in their available space. Now customers had a baseline product recommendation for their specific needs. Plus, if they wanted to make modifications based on their liking, they could easily find a product model and replace one of the recommendations to get a specific quote,” said Cindy Liu, Sales Manager, Yanre Fitness.

Quick Case Study #3: SaaS company experiments with best short-form video advertising channels, gets 366 visitors from YouTube Shorts Ad (versus 165 for TikTok, 44 for Instagram, and 5 for Pinterest)

Creatopy is on several social media platforms. “We have a presence on TikTok, Instagram, YouTube, and Pinterest, as they are oriented towards visual content (videos and photos). In addition to being some of the most popular social media platforms out there, they can also be a source of inspiration for our target audience,” said Laura Trif, Digital Media Manager, Creatopy.

On the organic side, they noticed that each platform excels in different areas. “YouTube works great for informative webinars and podcast snippets. TikTok is great for replicating trends and showing the product's capabilities in a more playful way. We noticed that vlog-style videos work very well on Reels, while on Pinterest we usually post Idea Pins in the form of multiple slides,” said Răzvan Hercuț, Digital Media Assistant, Creatopy.

The team at the SaaS company decided to experiment with paid advertising in addition to organic efforts – specifically to test the performance of short-form video ads on these social media platforms.

Following TikTok’s success with vertical, short-form videos, the team at the ad design automation platform noticed a big push towards this type of content. Instagram launched Reels, Pinterest launched Idea Pins, and YouTube launched Shorts to compete with TikTok. So, the team wanted to know which of these social media platforms would be able to bring more traffic to its website through a paid advertising campaign.

Campaigns setup

The team invested $3,000 in the experiment, equally split between the platforms. The team set up four similar campaigns on TikTok, Instagram Reels, YouTube Shorts and Pinterest Idea Pins, all having traffic as a goal.

“While we initially considered comparing the campaigns based on a brand awareness objective, not all four platforms had this as an option. Instead, we went with traffic as an objective so the comparison could be as fair as possible,” said Diana-Alina Aldea, PPC Team Leader, Creatopy.

These campaigns used the same parameters, including the same ad (a 20-second, 1080x1920-px video), same locations (US and Canada), same audience (graphic designers), and same landing page.

Creative

The video used in the campaign aimed to show that designing ads in the ad design platform is fast, even faster than making a coffee. It used a split-screen technique to present both actions side-by-side, with the designer completing their banner before the espresso was done.

The social media team’s goal was to create a video that felt native to these platforms, which are often used for entertainment.

Creative Sample #6: TikTok Ad

Creative Sample #7: Instagram Reels Ad

Creative Sample #8: YouTube Shorts Ad

Creative Sample #9: Pinterest Ideas Pins Ad

Results

“As traffic was the goal for all four campaigns, we didn’t optimize for conversions (we also didn’t have any from these campaigns). On the other hand, we did look at metrics such as engagement rate and video completion rate, as we wanted to look beyond just quantity and get a better understanding of the quality of the traffic these platforms can bring you,” Aldea said.

The team found that each platform led in a specific area:

• The YouTube Shorts Ad brought by far the most users to the landing page – 366 (versus 165 for TikTok, 44 for Instagram, and 5 for Pinterest)
• The Pinterest Idea Pins Ad had the highest video completion rate – 20.16% of people who viewed that ad watched 100% of the ad (versus 7.42% for YouTube, 1.97% for Instagram, and 0.82% for TikTok)
• The Instagram Reels Ad had the highest engagement rate – 41.17% of people who engaged with the ad engaged for at least 10 seconds (versus 22.39% for YouTube, 20% for Pinterest, and 15.66% for TikTok)
• The TikTok Ad had the lowest cost for impressions – a CPM, cost-per-thousand impressions, of $4.74 (versus $7.15 for Pinterest, $13.50 for YouTube, and $16.67 for Instagram)

“We continue running ads on Instagram, as the platform brings us quality traffic. We also advertise on YouTube Shorts due to the higher volume of traffic it brings us. This traffic can later be retargeted with the aim to convert,” Aldea said.

Related Resources

Offer Sequencing Infographic – Get this free thought tool to help you decide how to sequence your offer

Marketing Research Chart: Clear communications and unconscious customer bias

Customer Theory: How to leverage empathy in your marketing (with free tool)