by Adam T. Sutton, Senior Reporter
Webpage testing and optimization is powered by technology. For most marketers, creating this technology in-house is not an option. You will have to assess your needs, research vendors, and select a tool.
Do not take this decision lightly. Selecting the wrong tool can limit the impact of your tests or waste some of your investment. Even free tools can bring surprises.
"Despite the tool being free, it does come with a cost," says Bill Bruno, President, Stratigent, an analytics consultancy. "That cost is the resources required to get it up and running."
To help you choose the best testing technology for your company, we spoke with Bruno and two other experts:
o Eric Peterson, Senior Partner, Web Analytics Demystified, an analytics consultancy
o Boris Grinkot, Associate Director of Product Development, MECLABS, the parent company of MarketingSherpa. Grinkot also wrote the MarketingSherpa
2011 Landing Page Optimization Benchmark Report From these experts, we distilled five questions to ask yourself before choosing or upgrading a testing and optimization solution:
Question #1. Do I really need a tool?Bruno and Peterson emphasized that choosing software should be one of the easiest steps to building an effective strategy. The larger question, they say, is whether you can develop the processes, culture and resources necessary to achieve ongoing success.
"It's not just an investment in technology," Bruno says. "It is an investment in changing the culture, becoming more data-driven, and it requires technical resources to implement these tools and business resources to analyze the results and grow from them."
- Gather IT resources
About 73% of marketers who test and optimize webpages identify competition for IT resources as an at least "somewhat significant" challenge, according to the MarketingSherpa
2011 Landing Page Optimization Benchmark Report. About 38% say the challenge is "very significant."
Before choosing a tool, ask yourself if your team has the will, resources and time necessary to effectively use that tool to consistently optimize pages.
Question #2. Do I need a paid tool?For smaller companies dipping their toes into optimization, the answer to this question is a flat "no." All three experts emphasized a free solution is enough to get you started with basic A/B split landing page tests.
Starting with a free tool avoids having to justify an upfront financial investment internally. You will also get a sense of the skills and resources you'll need to consistently run, manage and interpret tests before you invest in a tool.
While MarketingSherpa is vendor-neutral, each of our experts noted Google's Website Optimizer as the best free testing software available. Even when compared to paid solutions, Google's product is the most popular with 61% of marketers who test counting on it as their sole tool, according to our benchmark report.
- Upgrade when necessary
A free tool will get you started and help improve your pages through split tests. Once you feel limited by the tool's personalization, reporting or other features, it's time to consider investing in a paid tool, Bruno says.
"Then you have matured to the point where you are ready to take full advantage of a paid solution and ultimately get the most ROI for the money you are about to invest."
- Larger companies should invest
Enterprise-level companies are better off investing in a paid solution from the outset, Peterson says. Marketers at these larger companies will want contractual agreements, account managers and experts they can count on.
"If something goes wrong, you want to have a phone number to call and a throat to choke, and Google doesn't give you that," Peterson says.
- How much is this worth?
Executives are typically more willing to invest in technology that they expect will bring a high rate of return. They are interested in the bottom line, not clickthrough and bounce rates.
If you've been using a free tool for a while:
o Look at the results you've achieved
o Estimate how broader capability would help improve results further
o Estimate how that improvement would translate into more revenue
Marketers in the online retail space tend to have larger potential gains in immediate revenue from improved site optimization, Peterson says.
“In retail, the question is 'if that testing practice can increase your conversion rate on your website by one-tenth of a basis point, what is that worth to you?'" Peterson says.
"Lead-gen marketers can make a similar calculation, but only if they understand what each new lead is worth based on some data they are collecting," Grinkot says.
Question #3. What separates the platform vendors?A tool’s price can range.. Pricing is typically set by the number of tests you run and the volume of site traffic you test, Bruno says, which often translates into page-view-based pricing. Total costs can range from thousands to hundreds-of-thousands of dollars annually, Bruno says.
"This is not something that you can pay $19.99 for and actually earn long-term results," Peterson says.
So when you're evaluating vendors, be sure to ask about their prices and estimate the costs you'll incur. Here are two other areas in which testing vendors often differentiate themselves:
- Personalization
Some solutions are better able to integrate outside data sets and use them to improve personalization during a test. Such systems can use a visitor’s demographic, geographic, behavioral and other information to determine the content displayed on the page.
Say, for example, a city was in the middle of a two-week stretch of rain and a marketer wanted to test an image of umbrellas only for visitors from that region. Some vendors would be better-suited to accomplish this task than others, Peterson says.
- Architecture
Another way testing solutions differ is in how they deliver the content to the audience. Some systems use a tagging platform, where changes must be made to a webpage's code to signal which elements will be switched during a test. Other systems require only a single line of text to be added to a page, and the elements are selected in the software's dashboard.
Question #4. Is this vendor right for me?Determine your exact needs before shopping for a vendor. If you've done your homework and have been using a free tool for months, then you'll have a much easier time identifying the right paid solution for your team.
Pull out your list of requirements and available resources, and match them against vendors' claims. Make sure you ask about the availability of their account managers and experts if your team needs initial or ongoing help.
- Get a trial
You would never buy a car without test driving it, and some of these solutions cost much more than a new car. Make sure you ask for a proof-of-concept trial, perhaps for 30 days.
Set up the software, run a few tests, get the results, and make sure the software is meeting each of your requirements. Also, call your account rep a few times during this period to test their interest in your challenges and response times.
Question #5. Why would I build a tool in-house?Some companies are fond of building technology in-house, but each expert we spoke with warned against building a testing solution from the ground-up. Effective split and multivariate testing is just too complicated, and the benefits of having your own solution can greatly outweigh the costs.
"It would be crazy to start building your town tools when Google has free ones. And if you want to invest money, there are plenty of commercial tools," Grinkot says.
Bruno referred to building the technology internally as "reinventing the wheel."
"It is such a sensitive area because of the statistical background required to make the right decisions … There are a whole lot of variables you have to keep in mind if you are going to try to build your own, and I would say probably nine times out of ten, you forget one of those variables and are now compromising the data and the decisions you're making off of it."
Crazy or not, some companies have traveled down this road only to make a U-turn.
"They say 'oh yeah, we can do this.' And I say 'Okay, call me when it doesn't work.' Inevitably, they call me six months later and say 'yeah, it didn't.' There is a lot to this [technology]," Peterson says. "In almost all cases, you don't want to start there."
Useful links related to this article:MarketingSherpa 2011 Landing Page Optimization Benchmark ReportLanding Page Optimization: 36 articles and resources to help you complete your next LPO project Website Optimization: Testing program leads to 638% increase in new accounts Optimization and A/B Testing: Why words matter (for more than just SEO)Test Plan: Build better marketing tests with the Metrics PyramidOnline Testing and Optimization Solutions: Quick guide to LiveBallStratigentWeb Analytics Demystified
