Interview with Google Website Optimizer's Tom Leung
Google's Website Optimizer, currently in beta, is a free site content testing tool offered inside of Google AdWords. Designed to improve conversion rates on paid ads and thus keep advertisers happy (and, not coincidentally, keep them spending money), Website Optimizer allows advertisers to test out various combinations of original and varied headlines, images, etc. in order to determine which combination has the highest conversion rate.
Tom Leung, Business Product Manager of Google Website Optimizer, was kind enough to answer my questions about Website Optimizer. As someone who’s been doing PPC for a few years, I was actually quite excited to speak to him and learn more about a product that even a picky client who swears by numbers would like. (I’ll let you know how it works out once I convince my picky client to use it.)
What’s so fascinating about this product is that it has the potential to provide clients with NUMBERS to back up what the best chance for the highest conversion rate is. As you’ll see below, the product has the potential to increase conversions by 30%, which is quite significant. If you’ve ever dealt with a client who didn’t trust your recommendations because you didn’t really have actual numbers to back it up, Website Optimizer could seriously make your day.
And, just for all the mathletes out there, check out their section on Fractional versus Full Factorial Analysis for a jolly good read.
Julie Joyce: “What are the basic principles behind Google Website Optimizer?”
Tom Leung: "It's a tool that lets you test different combinations of content and web designs. You tell Website Optimizer, for a given page, what things you're curious about testing to increase its effectiveness: this headline could be better, one of these three images should work, this button should be located here ... Website Optimizer lets you test all of those ideas by distributing the traffic that hits that page, showing each visitor one of the various permutations, and providing reports that help you make more informed decisions and maximize conversion rates and visitor satisfaction."
Julie Joyce: “Where did the idea for this product come from?”
Tom Leung: "It came from the overall goal of what we do with Google AdWords and Google Analytics: increasing transparency and marketing effectiveness. With AdWords and Analytics, we've done a good job of helping website owners get targeted traffic to their sites. Website Optimizer closes that loop; it's the third leg of the stool. Website Optimizer will show what you can do to make pages perform better. The only way to figure out the ideal page is to try a bunch of combinations simultaneously and see, with great precision, what caused conversions."
Julie Joyce: “What does Google stand to gain from the improvement of PPC conversion rates?”
Tom Leung: "Google benefits from doing big things: making a big impact on the Web and the world, helping people design better pages so that users have a better experience. Websites can do a better job of converting and getting people more engaged and involved. It's also very scientific. Website owners will get a higher ROI, which can help people decide if they want to invest more with AdWords. Everyone wins."
Julie Joyce: “What feedback have you received so far on the tool? Has the initial response been what you anticipated?”
Tom Leung: "Great feedback. As a result of doing testing, a lot of users tell us their companies will not make any permanent changes to their site unless they've done at least one test. People aren't just designing pages by guessing and gut feel. A 20% or 30% increase in conversions is not uncommon! We encourage people to test all the time; it's a free tool. And marketers feel like they've gotten some power back. They used to be beholden to the designers and the IT team. Everybody feels like it's not about opinion and politics anymore, which makes it great buy-in across the company."
Julie Joyce: “What is the maximum number of combinations that can run for one page?”
Tom Leung: "From an engineering point of view, we limit the number to 10,000 for a single experiment, but it depends on your traffic. The rule of thumb is that for every 100 conversions, you can test 1 alternate version of your page. If you have 300 conversions per week, test 2 or 3. That's suggested for statistical relevance. We also recommend that people start small."
Julie Joyce: “Currently the tool doesn't allow testing of database-driven content. Are there plans for doing so in the future?”
Tom Leung: "Website Optimizer will work fine for a dynamic page, as long as the tags get inserted on the page by your CMS or webserver. We hold webinars about once a month where we explain how it works. As your site gets more sophisticated, the tool is still flexible enough but it requires a bit more investment. We also have Website Optimizer Authorized Consultants who offer additional support and marketing strategy."
Julie Joyce: “How long is a typical experimentation period?”
Tom Leung: "Even if you have a ton of traffic and you're testing 2 versions, let it run at least a week, but generally 2 weeks because of seasonal effects that need to be normalized."
Julie Joyce: “Any potential abuse that could happen from the tool? Can black hats use it for anything nefarious?” [Editor’s note: I was hoping to get some black hat tips but Tom was too clever…]
Tom Leung: "Testing is good if you're improving user experience, but if you're using any testing tool to deceive search engines or users, that is subject to the same penalties. Using Website Optimizer does not buy you any protection for wrongdoing, but if people are worried about being mistaken for cloaking, they can read on our Help Center about how testing for the good of users and making pages more effective are good things."
So there you have it...Google's Website Optimizer is quite a fascinating tool so if you're doing PPC, give it a go.
Comments