Tinder got famous by letting people swipe their way to new relationships. Could a streaming subscription service use a similar interface to “swipe left” on customer churn? The answer might surprise you.
Watch this episode of The Cro Show, a game show for conversion rate optimization and marketing experimentation fans, and see if you can guess which test variation performed better during a recent Cro Metrics client experiment.
Subscribe to the Cro Metrics newsletter to get future episodes of The Cro Show sent to your inbox.
Full Transcript:
Jennifer Lindsay: Hey everyone. This experiment is the long awaited we like to call it the Tinder test, and you’ll see why in just a minute here. So I have got a client who is a streaming service provider, they’re top 10 streaming content provider, they have a ton of really creative ideas, some of them are great, some of them not, but of course, who am I to judge? We wanna test everything.
So, the hypothesis of the test, we actually did find some data saying that if people add more content after they first sign up for the service, if they add more content onto their playlist that it’ll actually increase LTV. Or it’ll reduce the likeliness of them canceling after a free trial.
It’s probably been a while since everybody has signed up for a streaming service, I feel like everybody has their number one or two, but what their onboarding flow currently looks like is you sign up, and then you land on this page. And this page let’s you pick devices, so this would be your V0.
We created another variant that actually introduced genres, so you’re thinking oh, okay, I’ll pick some genres, like this is gonna add some information to the algorithm. And then after the genre step, there’s actually a playlist section, so it actually have titles of content. And then you could add that to your playlist. So that was the first variant.
And then we had a second variant that was the swiper here, so you enter the flow, and then you get this neat little thing where you can flip through movies, and you can add them or not, and you can kind of go through them, and we saw a lot of engagement on this one as well.
And then lastly, we had no onboarding. You know what that means? You sign up, you pay, bam you go right back to the homepage but everything’s unlocked and you can just navigate it as you want.
So, curious what you all think would have been a winner. Again, our control, just the flat page that tells you the devices, and then a genre step and a playlist step, and then we have got our swiper, which would be a V2, and then the fourth variant which is no onboarding whatsoever. And just, again, to be clear about the primary metric, we’re looking at a decrease in cancellations over time.
Bonnie Buchanan: So, if the user was served V2 or V3, then after that they would go to their homepage which is then curated, right?
Jennifer Lindsay: No, it does not inform an algorithm.
Bonnie Buchanan: Oh, okay.
Jennifer Lindsay: You’re literally just adding stuff to the page.
Drew Seman: It does add to your queue.
Jennifer Lindsay: Yes.
Ally Kuhn:- The algorithm would add a whole other exponential level of complexity.
Jennifer Lindsay: Right.
Grant Tilus: I’m choosing V0 because I think there’s additional buy-in if you’re getting people to download the actual app itself onto your device.
Dave Albert: I’m choosing V2 because married people typically don’t use Tinder, or at least we hope they don’t, and so this is a way for them to live that experience.
Ryan Lucht: I’m going with V3 ’cause I don’t think users wanna jump through all the silly hoops that we construct for them.
Jennifer Lindsay: Did you get insider info, Ryan?
Ryan Lucht: No, I’ve been, I’ve had this feeling for a long time.
Emily Schrieber: I’m also voting for the Tinder variation since that test has predated my employment at Cro Metrics and it’s been around for a while.
Jennifer Lindsay: Okay, are we ready? So, funny enough, the no onboarding, Ryan, kudos to you, Kudos to Katie, that actually did the best out of all of them. With a close second being the Tinder swipe, which is kind of odd, but if you think about the level of effort it would take to build that experience, it just, it’s not worth it.
Actually removing this device step, and just plopping people right back into the flow as soon as they sign up, that’s what they preferred, and it was very interesting too ’cause we were also thinking well, if folks are adding content, so in that V1, V2, you’re able to add more content, more stuff to your playlist, we thought that would also increase LTV, and it didn’t. It didn’t at all. And again I think that there’s something to be said around adding those items would inform an algorithm, it didn’t, it just populated a playlist, and I think that’s a really interesting call out.