Some of the simplest website experiments (e.g., changing how a few form fields look) only take ~3 hours to launch from concept to execution. But how much lift could a test like this possibly generate? The answer might surprise you.
Watch this episode of The Cro Show, a game show for conversion rate optimization and marketing experimentation fans, and see if you can guess which test variation performed better during a recent Cro Metrics client experiment.
Subscribe to the Cro Metrics newsletter to get future episodes sent to your inbox.
Full Transcript:
Jennifer Lindsay: Hey, everyone, thanks for coming to watch this very interesting experiment talk.
So something that you need to keep in mind, of course, is it doesn’t matter how complicated an experiment is, you can see the biggest swings from the smallest changes.
So got a really interesting experiment here for a it’s a senior facility search. So if you have a parent that you need to find a home for them to go and visit, this is going to be the service for you.
And one of the main things we wanted to experiment on was the lead form here in the right rail. You’ll see that there is a label here for full name and then some placeholder copy that is “first and last, please.”.
And in our experiment, what we changed it to was first and last name, and then we just provided an example.
That’s all we did.
Y’all, this experiment took three hours total to write a spec build and QA, like, unbelievable.
It took very little to actually execute on this.
And the inspiration – I’ll go ahead and link a Medium article here – it’s all about text fields, form designs, UI, all that kind of stuff.
Really great read for folks that are looking for some interesting but more simple tests to run.
But yeah, do folks have any questions about either variant?
Again, all that changed was the label and then the placeholder copy.
Desktop and mobile.
Cristi Alvarez: Desktop and mobile. That was going to be my question.
Jennifer Lindsay: Yeah.
Cristi Alvarez: Wow, I’m excited to see the results for this one since it is such a small change. But I think… I think V1 is going to win. Or is the winner because it’s detailing out first and last and getting more like direct action. I could be wrong because it seems like I’ve been wrong a lot today.
Christine Nguyen: Do you happen to have any data or or does the client have any data on, like, the names that are actually submitted and if they’ve seen, you know, only partial names have been submitted there?
Jennifer Lindsay: No, no. They just, yeah, they’ll just grab the name and basically submit it. That’s it, yeah.
We also have to be fair – that’s – I’m glad that you brought this up – a big piece of what we actually look at to make sure if a test is a win or not is what we call a referred lead, and that actually occurs off the site.
So not only am I looking for people filling out this form, we’re also looking for how they behave on a call in a call center.
So do they actually reach out and talk to a human? Because that’s when they’re the most apt to actually convert and become a paying customer.
So I’m taking into consideration both. I’m glad that you you brought that up.
Cristi Alvarez: Alrighty… let’s go ahead and open it up for voting. So, again, thumbs down If you think hypothe…*sigh* I keep saying that if you think the control won, and thumbs up if you think variation won, and then in between, if it’s inconclusive.
So go ahead and cast your votes. Alright.
Let’s go ahead and see the results.
Jennifer Lindsay: We saw a 7.3% increase in that really down funnel metric!
This is probably one of the biggest wins that we’ve had for three a three hour test, you guys.
We got seven hundred… Unbelievable. Absolutely unbelievable.
We also saw a really good increase in actual, like, the lead form submits as well.
But we – this was the first time that we saw something hit stat sig this quickly on that offline or down funnel metric.
What a phenomenal win. Couldn’t get over what a simple test this was and what an impact we had.
Again, I’ll go ahead and link this article for y’all.
Really worth surfacing with clients. Great one.