Do you think you could figure out which design variation performed the best when it came to subscription referrals for a B to C to B company? The answer may surprise you.
Watch this episode of The Cro Show, a game show for conversion rate optimization and marketing experimentation fans, and see if you can guess which test variation performed better during a recent Cro Metrics client experimentation.
Full Transcript:
Hannah Brown: So this is a B to C to B company, so we are a platform for teachers to upload resources and then sell them on a marketplace.
And the subset of that – that we’re trying to that we’re trying to like advertise for is the subscription service that they provide.
So a school can sign up for a subscription and then all of the resources that a teacher normally would pay for it would be covered with this subscription.
So we’re doing some call outs on the home page.
This is as user logs in.
Then this pop up comes up, it calls out the personalized school name as well as the number of resources that are utilized that all of the teachers in the school have utilized this academic year.
So some cool bits of personalization, the CTA here “share the details”.
So we did a couple variations of this one.
They’re testing into a bit of a rebrand, so we updated just the kind of formatting and colors.
So this is our V0, we have V1 which is a periwinkle color but keeps similar aspects.
And then we have V2, which is a periwinkle color and a more of a purple, I guess with the logo kind of stripped of all of its branding.
CTA – primary CTA here would be like a referral form submit.
So users who click the CTA and then actually fill out the form that refers their principal to the subscription service.
Any questions?
Zeinoun Kawwass: Can you show the V0 and the variations again, please?
This is the V1?
Hannah Brown: This is V0.
The one with the blue.
Zeinoun Kawwass: OK.
Hannah Brown: V1 is purple and V2 is purple with no logo.
Zeinoun Kawass: And then we also changed like the 814 color from green to black.
Hannah Brown: Yeah.
Cara Binsfield: I think the variations are not going to perform as well because having that number pop out is like social proof in a way.
And I feel like by in spite of the branding, like that, one small change would have actually had a significant negative impact because people wouldn’t feel as much solidarity in like clicking on the CTA.
Also, blue beats out lavender.
You know, side note.
Michael Leydet: Is there a reason you went with lavender?
Is that like some other part of the branding or…
Hannah Brown: That, it was a like client requested color.
I think they’re moving towards more lavender in general.
It was less like dark blue for a while that we are using, but yeah.
Michael Leydet: That the blue just feels familiar to me.
Hannah Brown: Yeah.
Michael Leydet: When I see it like it’s the same blue that’s in when I type a comment in the chat here, like it’s the same blue.
So I…
Cara Binsfield: To be…
Michael Leydet: Yeah, I don’t know.
Cara Binsfield: To be fair, Mike, I was being glib when I said blue is better than purple, but there actually is a whole psychology –
Michael Leydet: Oh yeah.
Cara Binsfield: – of color and blue has like a much different emotional trigger than something like a lavender.
And so in terms of like what resonates with users and puts them at peace, blue is actually like typically your best color from a psychological perspective.
Drew Seman: That made me think of.
Cara Binsfield: Turning off my branding side.
Hannah Brown: OK, let’s go ahead and vote.
Katie Green: This one because there’s multiple variations let’s do numbers.
Let’s do it, Hannah.
Hannah Brown: So this was a win actually, and V2 was the winner. so not as much stat sig here, but, but we saw an almost 36% increase in referral form submit.
So that means that users clicked the CTA on V2 with – this is the no branding – and then actually filled out the form as well as overall site wide increases.
So not just like form submits are coming from this modal.
We have a couple other modals around the site that could have driven people there.
We also saw drop in clicks to close the modal and an increase in sharing that those details.
The thought that we kind of have around this is that maybe it’s different enough from the branding of the marketplace that teachers are unfamiliar with this new product that we’re trying to show them.
So by taking out this this branding, it more aligns with their current brand standards on the marketplace.
So it doesn’t feel as different of a product so that people are more interested.
So, we’re starting to test that throughout the other site, throughout the site.
And we saw similar results on a PLP page recently.
So I also thought that V0 was going to win for the same reasons that Cara said, so this was a surprise to me.
*mind blown*
Katie Green: Thanks for watching and be sure to subscribe to have more test ideas sent directly to your inbox.