Marketing Efficiency and Cost Savings: A low-cost testing method, streamlined asset management, and AI-driven personalization

Quick Case Study #1: How real estate agent tool made data-driven decisions on a small (even teensy) budget, found a headline with 85% higher CTR

How do small businesses get some data to understand if they’re on the right track? This can even be a challenge for medium-sized businesses, or smaller departments and lines of business in giant corporations.

We’ve written extensively on overcoming validity threats when A/B testing. And that is the gold standard. But it takes a healthy budget.

So when I came across a pretty clever (though imperfect) method for gathering some data to help with business and marketing decision-making, I thought it might be helpful to you.

The vast majority of case studies we bring you on MarketingSherpa have nothing to do with us (like the second and third case study in this article). We simply scour the globe for the most helpful and detailed stories for marketers and entrepreneurs.

This first one is a little different, since I was personally involved. I had a front-row seat on this one and can provide more details and context than on most case studies – where I ask a ton of followup questions to get all the gooey details for you, but only know what the sources tell me or what I can personally find online.

Finding the right messaging to serve the ideal customer

I met Ken Ducey in the AI Guild, and he is an active participant in a live training/group coaching track I lead called MEC300: Develop Your Creative.

He co-founded HamletHub and built an engaged audience seeking hyperlocal information in a cost-efficient way.

“However, the revenue side of the equation continued to be a challenge. I painfully admit that I did not respect the importance of marketing. An engaged audience alone does not a media company make!”, said Ken Ducey, co-founder, HamletHub.

The team realized they needed to focus on a specific ideal customer and chose real estate agents. A few months later they developed a product to help agents easily post branded local content. However, when they started to market, they had several strategic questions about what value would most resonate with the ideal customer:

• A time-saving service?
• A service so people can think of the real estate agent as a pillar of their local community?
• A service that helps them keep in touch with their customers, who typically only use them once every five years?

Getting the answer to this questions is essential, but, it is also costly and time-consuming.

He told me, “A/B tests are great but require a large sample size to be statistically significant, and you can only run one test at a time. Other methods such as questionnaires and focus groups are expensive, and if you don’t run them correctly you can come up with bad data, which will cost even more money as you execute your plan”

Ducey met Mike King of in the AI Guild, who taught him the following low-cost testing method.

“I needed something that could get me some initial signals as to what my prospects might respond to. I might want to learn what kind of language or what they are motivated by – hope, fear, greed, pride, etc. If you start with your ICP (Initial Customer Profile) the way Meclabs teaches you should have a pretty good idea of all of this and it will make it easier for you to determine a starting point,” said Mike King, Owner, King Family Consulting.

He continued, “It's why I'm able to go from zero to spending six figures a day on campaigns in little time at all for clients like Grant Cardone or how I've been able to cut a client’s lead cost from $41 to $13 while spending $1.1 million promoting an event with a four-week timeline. I don't say this to impress anyone but to impress upon you the importance of what having a solid testing methodology can do for you.”

Step #1: Write headlines with the help of artificial intelligence, and subjectively rank them

The HamletHub team created 50 headlines with subheadlines using three different AIs. They put each headline in a row on a spreadsheet and ranked them. The ranking was subjective, based on the headlines they thought were the clearest. They slightly modified each headline as well. The second headline in the table below was the most modified, since it came from group coaching we conducted in the AI Guild.

“For what it’s worth, MeclabsAI was the clear winner of the three we used, with six of the eight headlines we chose for testing. Marketing Professor was the best of the different MeclabsAI experts,” Ducey said (MeclabsAI is the parent organization of MarketingSherpa and runs the AI Guild).

The below table shows the headlines Ducey’s team selected for testing.

Revised Headline	Original AI-generated Headline	Source
10 minutes a week to hyperlocal fame: The real estate game-changer	"10 Minutes a Week to Local Fame: The Real Estate Game-Changer"	MeclabsAI
Exclusive area opportunity: Transform your social influence into being seen as the neighborhood specialist	Exclusive Opportunity: Transform Your Social Influence into Local Leadership	MeclabsAI (and AI Guild group coaching)
Introducing next-level real estate marketing: Hyperlocal custom content with a singular click	"Next-Level Real Estate Marketing: Custom Content, Singular Click"	MeclabsAI
From content chaos to community leader in 10 minutes a week	"From Content Chaos to Community Leader in 10 Minutes a Week"	MeclabsAI
Gain unprecedented hyperlocal influence in your area without the time sink	"Gain Unprecedented Local Influence Without the Time Sink"	MeclabsAI
Stand out from competitors with 145 local sources: One click to becoming a local expert	"Get Ahead with 145 Local Sources: Your Blueprint to Becoming a Local Expert"	MeclabsAI
Stay top of mind in your community:hyperlocal content for real estate agents	Stay Top of Mind: Consistent, Quality Content for Real Estate Agents	ChatGPT
The ultimate tool for busy realtors: hyperlocal, exclusive content to increase your online presence	The Ultimate Tool for Busy Realtors: Enhance Your Online Presence Quickly	ChatGPT

 

“I'd say I have a good understanding of prompting so I've built GPT agents to help me with writing copy and headlines, but I actually prefer MeclabsAI because it's built off of Meclabs proven methodology and 10,000+ tests – so it's basically like having Flint [McGlaughlin] as your CMO,” King said.

Step #2 Create ads, avoiding extraneous variables

The team sought to run a single variable test where no other variable in the creative of the ad (like an image) impacted the results.

“We used Canva to create eight generic ads. Mike King has a rule to use no more than three lines for each message. We wanted to avoid any variables (such as colors) that may distort the results – so Arial font, black letters, white background,” Ducey said.

Creative Sample #1: Ad example

And, while Ducey focused on headline testing, this methodology isn’t only for headlines.

“Personally when it comes to testing I like to focus on the biggest needle movers first. For Facebook ads that's the creative, then the primary text, then the headline, then opening hook, then the CTA. On YouTube, it's the hook, the body, then the call to action. When I'm starting a campaign on Facebook for a client I'll typically have two testing campaigns. One for the creative and one for the copy (primary text),” King said.

King is trying to discover what the ideal customer responds best to. For example, color versus black-and-white images. “Black-and-white images can work better because they stand out in the Facebook feed when someone is scrolling,” he advised.

Step #3: Target ads in Facebook

“We used Facebook ad manager to make sure we were targeting the generic ads to our specific audience,” Ducey said. Here are the settings the team used:

• Campaign Objective: Conversions
• Conversion Location: Website
• Bidding Strategy: Maximum number of clicks
• Dynamic Creative: Off
• Ad Format: Single image or video
• Optimization for Ad Delivery: Link Clicks
• Primary Text: [blank]
• Description: [blank]
• Call to Action (Button): No Button
• Tracking: [blank]

“These are automatically checked and give them the power of changing your ads (adding text overlays for example) if they think it will deliver you a better result,” King said. “I like to remove these traffic traps and then focus on testing only one ad element at a time, so it ensures as clean of a testing environment as possible.”

Step #4: Create custom rule in Facebook

“Facebook will automatically optimize delivery for your different ads, which does not work for this test since you would like each ad to be shown an equal amount,” Ducey said.

To prevent this, he set up a custom rule that will stop showing each ad after a certain number of impressions.

“On Facebook specifically (but true for most networks) if you create three different ads and you let the 'machine' run wild, it will latch on to one ad and spend the majority of your budget on that because it thinks that's the best ad to get you the outcome you indicated you wanted,” King said. “I should mention that you can't use rules like this on other platforms like YouTube for example.”

Creative Sample #2: Settings for ‘impression shutoff’ custom rule

The number of impressions you need to get a statistically significant answer will vary based on the difference in performance. You can see an example when we share the results below.

The test cost less than $175 and took about a day and a half to get to the maximum number of impressions (2,600). “Facebook went way over on some titles, I don’t know why,” Ducey said.

BEST: Headlines with the highest clickthrough rates

The best-performing headlines were:

• Gain unprecedented hyperlocal influence in your area without the time sink
• 10 minutes a week to hyperlocal fame: The real estate game-changer            
• Introducing next-level real estate marketing: Hyperlocal custom content with a singular click
• Exclusive area opportunity: Transform your social influence into being seen as the neighborhood specialist

WORST: Headlines with the lowest clickthrough rates

The worst-performing headlines were:

• From content chaos to community leader in 10 minutes a week
• The ultimate tool for busy realtors: hyperlocal, exclusive content to increase your online presence
• Stand out from competitors with 145 local sources: One click to becoming a local expert
• Stay top of mind in your community, hyperlocal consistent content for real estate agents

RESULTS: Highest performer had nearly double the CTR of the lowest performer

“To set expectations correctly, you will likely get more accurate results with the traditional methods. Some of the clicks may be bots, but I am assuming those clicks are distributed equally among the different ads and therefore will not corrupt the results. However, this is a great way to learn about your customer in a few days for under $175,” Ducey said.

 

Title for Test		Impressions	Clicks	CTR
Gain unprecedented hyperlocal influence in your area without the time sink		2925	35	1.20%
10 minutes a week to hyperlocal fame: The real estate game-changer		2790	28	1.00%
Introducing next-level real estate marketing: Hyperlocal custom content with a singular click		6150	54	0.88%
Exclusive area opportunity: Transform your social influence into being seen as the neighborhood specialist		4706	38	0.81%
From content chaos to community leader in 10 minutes a week		2888	22	0.76%
The ultimate tool for busy realtors: hyperlocal, exclusive content to increase your online presence		3453	26	0.75%
Stand out from competitors with 145 local sources: One click to becoming a local expert		2859	21	0.73%
Stay top of mind in your community, hyperlocal consistent content for real estate agents		4161	27	0.65%

 

 “You can then put the results into MeclabsAI and ask it if the results are statistically significant,” Ducey said.

So I did just that, and asked the Data Science expert in MeclabsAI, which said the difference between the best and worst performers was statistically significant, but the different between the top two performers was not (at a 95% level of confidence). I created a shared chat in MeclabsAI and you can read it for yourself here.

Analysis

“In some cases, there is a very clear winner, which is not the case with the above because the percentages are close together. However, you can certainly get some ideas for further testing. I also put the results in the MeclabsAI Marketing Professor expert,” Ducey said.

Here’s the analysis from MeclabsAI:

• Strong value proposition and clear benefit regarding time efficiency and local influence are more compelling to viewers.
• Messaging centered around ease of use, local expertise, and exclusive opportunities resonates well with the target audience.
• The ones with clear, specific benefits (‘Gain unprecedented hyperlocal influence’) tend to outperform more generic or broad appeals (‘Stay top of mind in your community, hyperlocal consistent content for real estate agents’).

And here’s an analysis from Ducey’s perspective:

• Time saving certainly seems to be the biggest benefit, what his customers are buying.
• The top two headlines had two things in common – the words ‘Gain...’ and ‘10 minutes to...’ were action oriented. They had the action combined with the main benefit – time. The bottom two were also action oriented, but did not have the main benefit of saving time
• He was surprised that ‘exclusive’ did not test higher, and that ‘top of mind’ was so low. His conversations with agents would have put those two on top.

Going forward, Ducey would like to test headlines with more of a stark contrast in their specificity and quantification. “For example, it is hard from my results to tell if the exact numbers (quantification) in some of the headlines made a difference,” he said.

Limitations

We thoroughly debated the pros and cons of this testing methodology in a session of MEC300: Develop Your Creative in the AI Guild. I took that hour-long transcript, fed it into the Marketing Professor expert in MeclabsAI, and asked it to explain the pros and cons. It did a very good job. Click this link to see the shared chat.

Moving forward

I’m sure you wouldn’t be surprised to hear that King’s methodology isn’t set in stone – he’s still, well, testing it.

“From a technical standpoint I'm testing different bidding strategies inside the networks. For example, on Facebook you can do spend-based bidding (most common), goal-based bidding and manual bidding. I've been experimenting with manual bidding as a means to test more efficiently Without getting too far in the weeds, a manual bid means I can say, I only want leads for $20. Facebook will then only spend my budget if it thinks it can deliver this result,” he said.

King continues, “With this bidding strategy in theory I can add 10 different ads to a campaign and Facebook will only deliver impressions to the ads it believes will give me the result I want at the cost I want. In this scenario if one ad gets the majority of the spend, then in theory that's the best ad. I've only been testing this for about a month now. The results look promising, but I still need to prove it out more as it doesn't follow every guideline I normally like to adhere to.”

Marketing is really hard...

I’ll give Ducey the final words…

“The more I learn about marketing, the more respect I have for good marketers. Any technology is still a long way away. Yes Dan [Daniel Burstein, that’s me], Flint [McGlaughlin], and the Meclabs team seem smarter to me every day!

I don’t like things that are an art form. To me that means you don’t necessarily get better by just working harder at it. Marketing is chasing that elusive Holy Grail, and you do not have unlimited time and strength to find it, and you never know if your efforts are taking you in the right direction.

Testing is the answer – it is your compass. But that makes it sound easy, and testing is not easy. It is crazy hard to get data that is not contaminated in some way. You can misinterpret the results, or mistakenly correlate the wrong data for a specific result. You need a lot of data to get accurate results for each specific item you want to test. And anything other than the right result will screw up your compass and get you spending a lot of valuable time and energy traveling away from your target.

This is a great hack to help you with testing. Thank you, Mike King! Also, I am extremely appreciative of Dan, Meclabs and the MarketingSherpa team for helping me understand headlines.”

Quick Case Study #2: How streamlining digital asset management and product information saves a CPG brand $390,000 a year

Kinder’s Premium Sauces and Seasonings sells a wide selection of seasoning blends, rubs, sauces, and marinades through national retailers (like Costco and Target) and directly to consumers. Founded in 1946, it is the third-largest seasoning brand in the United States.

BEFORE: 1 hour each day just searching for the right marketing assets

With a recent expansion of its product lines, the food brand was ramping its marketing activities to scale up national distribution. However, the creative and marketing teams didn’t have the processes and infrastructure to support the company’s growth.

Kinder’s team spent an hour daily searching for assets. Their manual processes – printouts, spreadsheets, binders – hindered productivity, slowed communication, and risked errors.

The team envisioned replacing those manual processes with a strategy that combined digital asset management (DAM) and product information management (PIM) workflows. “We believed that getting this right would markedly improve our efficiency, make our agency partner and customer communication and asset distribution far more seamless and accurate, and free up our employees for more growth-oriented (and less frustrating!) tasks,” said Caleb Brown, the Creative Director at Kinder’s Premium Sauces and Seasonings.

AFTER: Thousands of current marketing assets, ready to go

The seasoning company now utilizes a combined DAM+PIM ecosystem. It stores 13,000+ digital marketing and branding assets, and 8,000+ product support files representing 200+ product SKUs.

Creative Sample #3: Habanero Pineapple Rub Seasoning product details in new DAM+PIM platform

“Having a single source of truth for brand assets can get complex at scale,” said James Fox, Senior Librarian, Image Relay (Kinder’s digital assets and product info vendor). “You have brand videos, product shots, continually-updated product information, etc – all those files not only need to be immediately accessible, but marketers need to ensure outdated assets aren’t floating around.”

From the brand’s decades of historic images to its latest (and quickly growing) collections of new brand content, all assets are now immediately available on-demand to whoever needs them.

Creative Sample #4: Habanero Pineapple Rub Seasoning PNG image in new DAM+PIM platform

Here are some key lessons the team shared from their implementation:

Prototype to ease change anxiety

Change isn’t easy. The status quo is tenacious. Any change in any organization will cause anxiety even when the current state has significant pain points.

While Brown’s colleagues agreed that Kinder’s existing system had major issues, they were nevertheless uncertain about investing in the new strategy. To overcome that hesitation, Brown worked with a digital librarian at the vendor and they built a sample Kinder’s Digital Asset Library. The intuitive interface and clear opportunity to eliminate wasted time, errors, and frustration quickly convinced the team.

Prioritize user adoption training

While the intuitive nature of a DAM is core to the technology’s purpose, user adoption training remains an essential – if too often overlooked – component of successful DAM onboarding. Getting the team comfortable using a new system can be intimidating, and early comfort is key to a smooth transition.

Kinder’s was mindful of this in its own onboarding process, with employees attending live webinar trainings, and getting access to step-by-step instructional videos and documents stored in the DAM library itself.

Offer diverse search methods

DAM user testing has demonstrated that any business’ users will be almost equally divided in preferring to locate assets in one of three ways:

• using keyword search only
• keyword search + refining tools
• search via a folder structure.

Kinder’s Digital Asset Library included all three methods, allowing all users – internal and external – to leverage their preferred method.

Empower leaders, not committees

The food manufacturing brand placed a designated leader in a clear-cut role, responsible for the DAM, PIM, and all product photography. Empowering a handful of highly invested leaders – rather than a nebulous committee – enables brands to get results faster, and with far less friction.

In one such example, Kinder’s bottle labels previously all included the same QR code, which pointed to the brand’s website. However, the Kinder’s leader working on the DAM+PIM implementation (the company’s Creative Image Specialist) used the latitude he was given to invent a unique use case.

Kinder’s bottles now have different QR codes for each flavor, with plans in the works to link customers to product pages for each SKU, powered by the DAM in a public-view mode. Those pages will include specific recipes as well as information on allergens, certifications, and other content that enhances the customer experience.

Manage and maintain your DAM system

DAMs cannot be thought of as set-it-and-forget-it tools. DAMs require careful management and regular maintenance in order to remain organized and keep asset availability optimized.

Share access with customer-facing employees

Support teams can be saddled with expired data if marketing or product teams don’t think to inform them of updates. Instant access to a single source of truth ensures always-accurate marketing assets and product information.

For example, Kinder’s customer support team leverages the combined DAM+PIM solution to make sure they’re (literally) on the same page and looking at the same product as their customers.

RESULTS: $390,000 in annual cost savings

“Considering where we were, the level of organization we have now is absolutely transformative,” Brown said.

By increasing productivity and eliminating the hour a day of tedious work that employees previously spent simply locating materials, Kinder’s estimates that its transformation represents around $390,000 in annual cost savings (calculating 50 employees at an average wage of $30/hour).

Quick Case Study #3: How food delivery company used AI to improve recommendations 10%

As Brazil’s largest food delivery service, iFood manages 80 million orders monthly. Beyond just delivering food, they offer groceries and payment solutions. But with growth came a challenge: how to personalize millions of orders daily.

BEFORE: Data overload, increased operational costs, and missed personalization opportunities

The marketing team at iFood faced an uphill battle: delivering personalized recommendations based on time of day, user preferences, and location – all while managing an overwhelming amount of data.

The team created an online matrix-based model to manage its recommendation systems. However, as the online food ordering and delivery platform grew, this online-only approach did not scale or perform as expected.

This model required significant computational resources, which increased operational costs and complexity. Additionally, the system couldn't fully capture the intricate patterns and relationships in the data required for highly personalized recommendations.

"Managing the vast amount of data and providing real-time, personalized recommendations was a significant challenge,” noted Luiz Mendes, Head of Data Science, iFood.

AFTER: AI-driven personalization with manageable computational costs

The team transformed its recommendation system with artificial intelligence (Kumo AI), leveraging advanced collaborative filtering techniques. This approach allowed them to predict what users would crave based on the behavior of others with similar tastes.

Creative Sample #5: Desktop restaurant recommendations on food delivery platform

Creative Sample #6: Mobile restaurant recommendations on food delivery platform

Here are key steps the team took to implement the AI-driven recommendations:

Integration

The team integrated AI into their existing infrastructure, utilizing Amazon S3 for central data storage, various data processing pipelines, and multiple machine learning platforms.

Data selection and configuration

They carefully chose datasets with rich user interaction and contextual data, ensuring all tables were interconnected for comprehensive analysis.

Rapid experimentation

Data models were tested to quickly evaluate their effectiveness, focusing on convergence and comparative analysis with previous experiments.

Qualitative analysis

The team examined specific examples to verify the accuracy and reliability of the model outputs, ensuring the setup captures all necessary data nuances.

Advanced tuning

An extensive AutoML hyperparameter search allows for experimenting with different model configurations, optimizing based on performance metrics from initial tests.

Model evaluation

Newly trained models were tested against a champion model on a shared holdout dataset to measure performance improvements and readiness for deployment.

Controlled deployment

Promising models underwent live A/B testing with a subset of users, closely monitoring the impact on user experience to ensure enhancements align with customer needs.

RESULTS: More usage of recommendations by customers

Users began relying more on recommendations instead of the search and last-orders components.

They were more likely to find relevant food and restaurant suggestions, leading to increased order volumes and higher conversion rates. The recommendations experienced increases in conversion rate ranging from 1% (for already high-performing components) to 10% (for less popular components).

The team also used this AI methodology for sponsored recommendations. These ads (recommended restaurants) saw an increase in performance with click-through rates (CTR) improving by up to 2% and conversions rising by approximately 3%.

“The improvements in recommendation accuracy, user engagement, and conversion rates collectively contributed to an increase in the company’s revenue,” Mendes said. “By providing more relevant and timely recommendations, we were able to drive higher sales and improve customer satisfaction. The enhanced recommendations not only boosted order volumes but also encouraged users to explore and purchase more frequently.”

Related Resources

How Can You Quickly Get a Higher Conversion Rate on Your Website

Conversion Optimization Strategies: Ways to get started overcoming 6 CRO challenges

Online Marketing Tests: How do you know you’re really learning anything?

Straightline Testing Method (in this video, King shares an example that cut lead cost 38%)

Top HARO (Help A Reporter Out) Alternatives: How to find primary sources for business journalism and content marketing

Mastering the Art of the Business Pivot: Real-life examples and success stories about an AI chatbot, ChatGPT prompts, and a business model shift

MEC300: Develop Your Creative (Link will only work for AI Guild members. Click here to get a free, three-month scholarship to the AI Guild)