Not all A/B tests are about turning buttons red or green. We use statistical prowess and a team full of sharp engineers to think outside the button when it comes to website optimization. Here’s a rundown on one of the more interesting experiments we’ve run lately.
Problem: Long application form = low conversions on mobile = everyone feels bad.
Mobile is great for many things. Filling out forms isn’t one of them. Depending on the nature of your site, there may be no escaping this. Even if you’re already following best practices by paring down your forms to the bare minimum, mobile visitors may still abandon your site without completing their applications.
Solution: Modified mobile home page to capture emails for remarketing.
We hypothesized that for mobile visitors, a home page that captures email addresses would increase overall conversions because it offers the option of sending reminder emails asking these visitors to complete the funnel when at a desktop.
We see over and over that people browse on mobile and convert on desktops. This test was designed to see if we could combine them into one powerful conversion tool.
We took a clean, simple design for a mobile hero section with an email input and implemented it as an Optimizely experiment. That makes it sound so simple, though. Maybe more detail is in order.
First, we set up a mobile-only audience. When the experiment activates, the existing mobile home page hero is hidden and the new design is shown in its place. A custom event allows us to track how many people start typing into the email input, giving us a more detailed picture of visitor behavior.
Upon submission, we validate the email address, and if it passes muster we submit it to a custom Google Sheets API wrapper — we only allow for rows to be added, so even the most nefarious hacker can’t get to those email addresses. But your marketing department can!
Out of courtesy to visitors, we store the email address on their device and pre-populate the email field when it appears later in the funnel — even if they leave the site for a few hours of intense Facebooking. No sense inconveniencing those die-hard applicants who are not intimidated by long-form mobile text entry.
We put the test through a brutal round of QA. (Who tests the tests? We test the tests.) Then we launched it, sat back and watched the leads pour in.
The data gathered from an experiment like this leads to an interesting business decision — if collecting emails on the home page results in a lower conversion rate for the next step, you’ll have to balance this loss with the benefit of the additional leads you’re capturing. Which means you’ll have to quantify the value of those leads. Get ready to math!
Suppose you capture 350 emails a day, but notice that mobile conversions from the home page to the next funnel step are down by 2%. You’ll need to quantify the value of those email leads by tracking their conversion rate after remarketing, and balance this against the hit you’re taking on conversions. If all that sounds a little dizzying, give us a call. We can implement the test, analyze the results and put them into a dazzling slide deck. All you have to do is show up at your next board meeting with a mic to drop.