|
SUMMARY:
You don’t need to have all the answers. You just need to know how to learn the answers…from your customers. One way to do that is by testing. To spur your next great test idea, in this article we show you how a classifieds website enhanced user engagement and how a lending platform optimized email campaigns. |
Action Box
Looking for test ideas? Try this prompt using the Conversion Pro expert assistant in MeclabsAI. It's totally FREE to use (for now):
"Generate A/B testing ideas for [specify the webpage, feature, or product] focusing on [specify the element, such as headlines, call to action, visuals, user flow, etc.]."
And get more test ideas from these case studies…
999.md is a classifieds website with, on average, 300,000 people visiting every day to buy or sell products and services.
As a two-sided marketplace, there are both sellers and buyers. Sellers list their items and goods, while buyers search for and buy them.
The site’s revenue is derived from promotional services used by sellers, like the Autorepublisher service. It re-lists advertisements at specified times and days. This service effectively boosts listings to the top during predefined times, contributing significantly to the platform’s overall revenue when compared to other services it offers.
“It is crucial to continually test and seek improvements for this service as it has great potential,” said Aksinia Chumachenko, Product Analysts Team Lead, Simpals (999.md’s holding company).
Let’s take a look at one of the team’s tests to give you ideas for your own product testing.
The team could only adjust a few parameters for the Autorepublisher, such as pricing and frequency.
Initially, the Autorepublisher was set to activate three times per day. Users had the option to adjust these hours by either reducing or adding more at their discretion.
Creative Sample #1: Control Autorepublisher settings with 3 daily options
The team analyzed customer usage and noticed that some users were adding extra hours themselves. So in the treatment variant, the team altered the number of hour options per day, adding two additional slots – from three launches a day, they moved to five.
Creative Sample #2: Treatment Autorepublisher settings with 5 daily options
The test ran for two weeks.
With the addition of two extra options per day, many users decided not to remove these new slots. This adjustment led to a rise in weekly republications. Because of this modified engagement, revenue also increased since payment is collected for every republication.
Weekly republications went from 12.6% for the control to 16.2% for the treatment – an increase of 22%. Revenue from this service also increased, by 13%.
The support team regularly receives user feedback, and they haven’t seen any negative reactions to the change.
I like this test because it is a great example of a simple change based on user activity that produced a significant lift. I also like it because the team actually tested it and didn’t add more product options arbitrarily. After all, a company could easily be tempted to let the additions get out of hand. If five increases revenue, wouldn’t six be even better? Or for that matter, 24?
When I asked Chumachenko about this subtle threat of overdoing it, she told me, “Indeed, adding too many hours could eventually stop revenue growth or even decrease it if users were to resist the change and not engage with the service at all. That’s why we closely watched the conversion rate. We don’t have a precise hour count where this effect might occur, but it’s about balancing conversions and the resulting revenue.”
“Before conducting an A/B test, it is essential to agree on clear objectives for how and to what extent you want to influence metrics, calculate unit economics, and anticipate potential trade-offs,” she advised.
There are two types of MarketingSherpa case studies. There are the ones that you probably assumed would work and aren’t shocked about the outcome. We share those with you to trigger a similar idea in your own head and explore how the team actually executed it. Quick Case Study #1 in this article is a great example.
Then there are the ones that are counterintuitive. If you see the before and after, you would pick the wrong one as the winner. We publish these to challenge your assumptions.
And this next case study is a great example. At least it challenged my assumptions, because I would think the educational-style email would outperform the flyer style. But I asked some pretty in-depth and direct questions, and the story vetted out (frankly, for each case study we actually do publish, there were many other stories that didn’t hold up to scrutiny).
Even better, this story goes a level deeper. When I saw this team had sent 460,100 emails over the past 15 days, I surmised they’ve probably built a pretty sizable opt-in list. No easy task. So I also asked some pretty detailed questions, and they were gracious enough to provide a behind-the-scenes look.
Without further ado, let’s dive in.
Niche Capital describes itself as a ‘consumer-first lending platform’ and promises ‘Personalized Business Capital without the Spam.’
The team was sending a newsletter-style email with the intent of educating their opt-in list on lending products.
Creative Sample #3: Newsletter-style email
But the team wasn’t happy with the open or clickthrough rates, or the delivery rate (below 50%).
They changed their email format to more of a flyer style and stopped sending the educational-style emails. “We found our opt-in list was much more receptive to this,” said Dayten Rynsburger, Chief Revenue Officer, Niche Capital CO.
Creative Sample #4: Flyer-style email
“Everything is so expensive and with inflation through the roof, market pressure, etc., we found business owners have a lot less time for educational content and really see value in ‘to the point’ style of email. This will not always be the case but in today’s market the simple ‘do you need money?’ is really out-performing the ‘let me educate you,’” he said.
The team realized the following results:
The team also made deliverability changes. "If no one sees the email, no one can click the email," he said.
They noticed a 40% increase in delivery rate when they added rDNS records and did not bulk send on just one IP. “We rotated already warm IPs and changed to a different MTA called Power MTA [from] SparkPost now MessageBird [which has been rebranded to just Bird],” Rynsburger said. With PMTA the team set ISP rules to ensure they were not over-sending to just one ESP.
“As far as DMARC, SPF, and DKIM, we set all that up with our domains and subdomains via Cloudflare and in our PMTA server,” Rynsburger said.
Good stuff. But I was still itching to go deeper and wanted to know how they built that opt-in list to begin with. Let’s go back to the beginning of this funnel and see how they built it.
The team creates their own content. “We put a lot of time into SEO and posting articles frequently to ensure we are number one on Google when people go looking for us,” Rynsburger said. And there are a few opt-in funnels on the company’s website, like on the main blog page. “However, the majority of our email subscribers are acquired using lead magnets and Facebook ads,” Rynsburger said.
The funnel is simple – from ad to landing page to lead magnet deliverable. Core to its success is tapping into a real customer need and delivering value. “Our strategy is very simple: give away something valuable in exchange for an email address,” he said. “It is important the deliverable is good. A bad deliverable is worse than no deliverable.”
Here’s is an example of one of the lead magnet funnels they used to acquire B2B consultants and service professionals.
First, they identified messaging that would resonate with business consultants – effectively using LinkedIn by improving your profile and sharing helpful content on it. “As we all know, Linkedin is a goldmine to find working business professionals and decision makers, however most people us it incorrectly. So, we leveraged this knowledge to add to our list using a Linkedin-centered ad.”
Before we look at the ads, let’s talk about the channel. I asked him why he used Facebook ads for a LinkedIn-focused lead magnet.
“We advertise everything on Facebook. It’s what we know and much more cost-efficient and tried-and-true than LinkedIn. In addition, most people, including business owners, use Facebook or Instagram. People scroll these platforms aimlessly every day, so we come to them where they hang out. On LinkedIn, people log on, do or find what they need, and then log off,” Rynsburger explained. “For the most part, Meta ads are what we know, and if it ain't broke, don't fix it!”
They test all of their ads, one variable at a time. Here’s an example split test where the only variable that changed was the headline.
In this test, “GRAB THIS FREE LINKEDIN MASTERCLASS” was the top-performing headline.
Creative Sample #5: Best-performing Facebook ad used to grow opt-in email list
“YOUR FREE GUIDE TO LINKEDIN MASTERY” was in second place, not too far off the winner.
“MONETIZING LINKEDIN THE RIGHT WAY” was the worst, with cost-per-click roughly 4x more than the top performer.
Creative Sample #6: Lowest-performing Facebook ad used to grow opt-in email list
“If I had to guess it is because it doesn’t have the words ‘Free’ or ‘Your’ which are two things people love: free and themselves. The other ads had much smaller variance but ultimately ‘Grab’ was the winner,” he said.
They aim for $0.50-$1.00 CPC max before worrying about the op-tin rate, then shoot for 50-70% opt-in rate once clicks are cost efficient. The team keeps targeting broad and lets the algorithm find their audience.
“Once a user clicks on your ad, it is crucial that your opt-in page relays the same message your Meta Ad does,” Rynsburger said.
The landing pages page they used are quite simple. Below is the entire landing page. This landing page (again, the winner after testing) achieved an opt-in rate of 65% to 70%.
Creative Sample #7: Email opt-in landing page for previously mentioned ad campaign
The team only asks for the email address. In their experience, every additional question can lead to a 50% increase in cost per lead.
As mentioned above, they optimize the landing page to get a 50-70% opt-in rate once clicks are cost efficient. They’ve found that using generic email newsletter pages or minimally relevant copy significantly lowers opt-in rates, potentially below 20% or even 10%, as it fails to maintain user interest from the ad.
And since they are not just using a single generic landing page, they track which campaign the new subscriber came in on to help with email segmentation down the road. “Use a platform like ConvertKit to capture and tag your subscribers based on which ad they were acquired from,” Rynsburger advised.
Upon entering their email address on the landing page, the lead receives an email with the lead magnet.
Creative Sample #8: Triggered email for subscribers
You’ll notice they encourage subscribers to engage in this very first triggered email – “Please reply YES to let me know you got it!”
Their email list’s unsubscribe rate ranges between 1% and 2%. “Back in the day we cared much less about who was signing up and how. It was more about getting the numbers up,” he said. “Now we are a lot more targeted in our approach. Running up quick numbers always ends up hurting you in the end, and that is true for most things – from social media to email lists. Having the wrong audience subscribing also hurts your email list and deliverability in the long run so we put much more effort into getting the right people to the lists.”
Join our thousands of weekly case study readers.
Enter your email below to receive MarketingSherpa news, updates, and promotions:
Note: Already a subscriber? Want to add a subscription?
Click Here to Manage Subscriptions
Get Better Business Results With a Skillfully Applied Customer-first Marketing Strategy
The customer-first approach of MarketingSherpa’s agency services can help you build the most effective strategy to serve customers and improve results, and then implement it across every customer touchpoint.
Get More Info >MECLABS AI
Get headlines, value prop, competitive analysis, and more.
Use the AI for FREE (for now) >Marketer Vs Machine
Marketer Vs Machine: We need to train the marketer to train the machine.
Watch Now >Free Marketing Course
Become a Marketer-Philosopher: Create and optimize high-converting webpages (with this free online marketing course)
See Course >Project and Ideas Pitch Template
A free template to help you win approval for your proposed projects and campaigns
Get the Template >Six Quick CTA checklists
These CTA checklists are specifically designed for your team — something practical to hold up against your CTAs to help the time-pressed marketer quickly consider the customer psychology of your “asks” and how you can improve them.
Get the Checklists >Infographic: How to Create a Model of Your Customer’s Mind
You need a repeatable methodology focused on building your organization’s customer wisdom throughout your campaigns and websites. This infographic can get you started.
Get the Infographic >Infographic: 21 Psychological Elements that Power Effective Web Design
To build an effective page from scratch, you need to begin with the psychology of your customer. This infographic can get you started.
Get the Infographic >Receive the latest case studies and data on email, lead gen, and social media along with MarketingSherpa updates and promotions.
