Tom Sharkey: All right.
So the test I am going to walk us through is for a telehealth client, they specialize in helping patients treat migraines – So if there are any migraine sufferers among us, ping me, I can get you some great treatment from one of our clients.
And what we’ve tested here is based on some of the learnings that we were gathering as we were onboarding with them, they’ve got quite complex user journeys on their website.
The main entry point that they were using to get folks ultimately to purchase a consultation with one of their migraine doctors was a quiz.
And so what we’ve tested has been incorporating more CTAs in the homepage hero that more directly kind of navigate to the different user journeys on the site.
So you can see here from a design perspective, not too much change between the control and the variant.
The difference being we added two additional CTAs in the variant, and we did this both on desktop and mobile.
So take the quiz, that was the CTA in the control experience, and in the variant, we’ve added two more.
One of them is consult with a doctor.
This led to essentially a checkout page.
The thought here was there are certain folks who land on the website that are really high intent.
They know no matter what, they want to buy a consultation with a doctor and just kind of get their solution right away.
The second CTA we added was get a prescription.
And this directed to a build your own treatment plan page where they could browse the different medications that the doctors prescribe.
And we tested having all three of these CTAs in the homepage hero, so the metrics that we tracked overall, looking at just engagement with CTAs from the homepage.
So for comparing the two, we looked at cumulative clicks for the control.
Just all clicks of either the CTA or the link below it.
And then in the variant, we looked at all clicks of any of the CTAs that were in the hero to get a sense of engagement.
And then we also, of course, tracked conversions of purchases of the consultation
Any questions?
Dave Albert: And are you considering a win ultimately conversions or more top of the funnel, like getting people into the funnel?
Tom Sharkey: That’s a great question.
They were definitely more concerned about just conversions ultimately and I think the rationale behind that was their conscious as well.
It’s a pretty complex user journey.
And you know, the client and I think we agree a big thing as we’re starting to test for them which is figuring out if what we do here ends up resulting in more purchases of consultations.
It is a success and it’s something we can learn from and build on.
So definitely conversions was the primary metric of interest.
Dave Albert: Cool.
Tom Sharkey: What do people think happened? Any guesses?
Katie Green: I feel like from experience of running tests like this similar myself, where you’re trying to create more pathways to conversion.
When you add more options for clicking, I think you increased engagement overall, but you start to lose the you know…I’ve seen throughout multiple clients that I’ve tested this on that you lose that conversion impact, that even if the conversion is just flat, you’re still able to increase engagement just because there’s more buttons.
But…sorry, did you mention if this is mostly mobile versus desktop? Can’t remember.
Tom Sharkey: Almost almost 90% of their traffic is mobile.
Katie Green: Mobile, okay.
Dave Albert: I think, Katie, you’re you’re probably right.
Like for most instances, the one thing that leads me to believe that maybe the variant won here is that take the quiz is somewhat ambiguous.
And so it’s like what is the real value proposition of that CTA?
So the fact that I have more clear value propositions than the other two CTAs leads me to believe that maybe there was a lift in conversion.
Katie Green: And the control was take the quiz right?
Tom Sharkey: Yup.
Katie Green: Yeah, Dave, I totally see that.
James Buo: In the control, are there those same options, the prescription and the doctor that we just can’t see on the screen, are they only in the variant?
Tom Sharkey: That’s such a great question.
Evidenced by this tiny link with white text does say browser medications, which does lead to this get a prescription page and in the control there wasn’t as clear of a way to get straight to check out in the way that this consult with the doctor CTA drove folks there.
Should we take a vote?
Katie Green: I think we’re ready.
Thumbs up for V1, thumbs down for control.
Tom Sharkey: All right.
So the folks who said it was inconclusive, you’re wrong, unfortunately.
We actually saw pretty big hits to both engagement and conversion in this test, which was surprising for me.
You know, Katie, you made a point about when you’ve test this with other clients, seeing like conversions really take a hit just because maybe there’s too many options there.
So I think I’m less surprised by the negative impact on purchases and more surprised by the negative impact on homepage CTA clicks and engagement.
The one metric that was higher was visits to the checkout page.
And that also makes sense because our variant had a CTA that drove folks directly to that page and, you know, to James’ question the control did not.
So we did see a lift in checkouts, but that didn’t translate lower in the funnel, unfortunately.
Katie Green: Thank you so much for watching.
Be sure to subscribe to have more test ideas sent directly to your inbox.