Steve Meyer: All right.
So this client is – has a popular SAAS scheduling tool.
For this hypothesis, very simple test.
Basically, they have two websites, a marketing website and then a product website.
Um, the the thought is that existing customers that are already logged in when they hit the marketing site should just immediately be put into product.
Since from the data, we can tell at least 30% of these customers log in immediately.
So we thought, hey, why make people have to go through another step and click a button to get into the product?
So we went ahead and created this redirect test.
The goal is to increase subscriptions.
Um, but there, you know, we were checking also like product usage and then also some frustration metrics too.
Any questions?
*questions begin*
Drew Seman: Oh, whoa.
So many questions. Too many questions.
Katie Green: Too many questions.
Somebody else.
Jamie Brown: Device split. Go.
Steve Meyer: Device split.
It’s like mostly desktop.
Cool.
Jamie Brown: OK, so my question was um, what percentage of logged in users already have subscriptions or like, are there upsells that you call subscriptions?
Steve Meyer: So, yeah, they could upsell, that could also be considered a subscription in this test that they go to a different plan level.
But overall, I think it’s roughly I don’t know, I think it’s like 70, 80% of visitors don’t have a paid account.
Jamie Brown: Oh, wow.
Steve Meyer: Yeah, it’s a lot. Yeah.
The reason I asked that is because if you were like, oh, 95% of logged in users already have a subscription, I’d be like, why are you running this test?
But very different story.
So thanks.
Matt Vincent: Sorry.
Can you explain just one more time?
So after a user logs in, where exactly do they go?
Steve Meyer: Not when they – not when they log in, when they actually hit the marketing site.
So obviously people can be cookied and stuff.
So they are already technically authenticated when they hit the marketing site and then we immediately ship them over to the product.
So they don’t even see the marketing site, really it’s a redirect test.
So they type in *honk*.com and their in product.
Josh Braaten: Is it possible for people to get back to the marketing site if that’s where they want to go?
Steve Meyer: The only way they can get back is if they log out.
Jamie Brown: Is that made clear in the test?
Seems like that could rage click people to death.
Steve Meyer: I mean, we we added some frustration metrics that check how that might be going.
Katie Green: I guess it’s about that time, huh?
Anybody want to explain why they’re going thumbs up or thumbs down?
Jamie Brown: I went thumbs up because a lot of people that are a logged in user may not actually have a subscription.
So I – if I was a lot of them user and I wanted to actually get in the product and use it, I would think I would be more prone to pay if you don’t make me do silly stuff like click a button when you already know who I am.
Make it easy.
Drew Seman: Yeah.
It also just felt to me like probably your offer could be slightly more relevant to the customer and where they are on their price stuff, to Jamie’s point.
Matt Vincent: So I was actually I put a thumbs up, but now I’m thinking maybe comes down.
Drew Seman: Did Jamie convince you that that Jamie was Jamie that not persuasive or was I not that persuasive?
Matt Vincent: Well, so and maybe I’m maybe a misunderstanding the flow, but I’m actually wondering if there are more value props on the marketing page than like if you’re going straight into the product.
And I don’t know if there are a lot of opportunities to maybe sell if you don’t already have the subscription.
So maybe seeing some of the value props on the marketing page before you get into the the product would actually get you more excited to subscribe.
So I’m thumbs down.
Steve Meyer: All right. Big reveal.
We have a loss.
So what we saw is an increase in frustration and also a downturn in subscriptions.
So 4% decline in subscriptions at 60% significance.
Um, we saw that log outs were up 8% with full significance.
And then also clicks on the logo in products were up 5.6% as well.
We also – that’s totals too.
So it was like people were rage clicking on the logo to get back to the marketing site.
So definitely people really do want to go back to the marketing marketing site.
Maybe like, you know, you thought Matt to look more deeper into the product and kind of understand all the different features available because there really isn’t a great robust content set inside the product for that type of thing.
But I thought those are really interesting results.
Drew Seman: Yeah. Good job, Matt.
Steve Meyer: Yeah.
And the current – the CEO is currently wanting to actually implement this anyways, and it’s kind of an internal battle.
So, fun times.
Katie Green: And that’s why we test.
Thanks for watching and be sure to subscribe to have more test ideas sent directly to your inbox.