This e-commerce fashion brand wanted to increase mobile newsletter signups by giving users multiple opportunities to sign up for the newsletter. Could adding a simple Remind Me Later button on a modal pop up fix this? The answer may surprise you.
Watch this episode of The Cro Show, a game show for conversion rate optimization and marketing experimentation fans, and learn from the Cro Metrics team on how to bring experiment-led growth to your marketing efforts.
Full Transcript:
Elaine Koh: Alright.
So on the theme of mobile, this is also a mobile only test.
And the goal of this test is increasing mobile newsletter subscriptions.
So the client here is a fashion e-commerce brand, and the goal was to increase mobile newsletter signups by giving users multiple opportunities to sign up for the newsletter.
So we have found that there had been a lot of desktop newsletter signups, but that was lacking in mobile.
Ultimately, this company also wanted to increase revenue, so the hope was that the email signups would later translate into down funnel metrics.
So this is the control.
On the left hand side is the homepage, when you first go to the site on a mobile device.
And then if you scroll down to the bottom of the page, that’s where the email signup comes in.
So then in the variation, this is what it would look like.
So when you first log in on the left hand side, there would be a little modal that pops up at the bottom.
And they would – there is a little Remind Me Later under the email submission bar.
So if users didn’t want to put in their email just yet and wanted to peruse the site first they would press Remind Me Later and the newsletter sign up would appear again after the first ad to bag.
And then what users would see after they pressed the Remind Me Later button.
Cool.
So any questions so far.
Zeinoun Kawwass: Are we looking also to see how this impacts sign ups, but also how it might impact checkouts as well?
Elaine Koh: Yes.
So this was tracking newsletter sign up.
So unique email submissions as well as add ad bags.
So when users – so also tracking down funnel.
So add to bag order completion pages and revenue billion.
Alex Gray: Just curious, where did the idea come from for this
Remind Me Later, sort of option?
Elaine Koh: Yeah.
So I think the idea was that if users weren’t ready to submit an email yet, they wanted to look through the page but there weren’t necessarily committed to wanting a 20% off.
So the remind me later was an option for them to add their email later and not feel pressured to do it right when they log into the home page.
So we give them time to look through the website to make the decision of whether they wanted the 20% off for their purchase intent once they add to bag.
Shanteé Hope: Elaine, is this something that we tested across multiple client’s sites?
Elaine Koh: That is a great question.
I’m actually not sure off the top of my head.
Shanteé Hope: I’m just asking because it’s a great feature.
I wish I saw it more.
Definitely.
Zeinoun Kawwass: Um, Elaine, do you know, if this was done for users who are not logged in versus locked in or does anybody get this experience when they land on the page?
Elaine Koh: Anybody gets the experience when they land on the page.
Zeinoun Kawwass: Okay, that’s interesting.
Elaine Koh: And then it would have – if somebody had already signed up when they type in their email into the email bar, they would get a different pop up at the bottom.
So instead of saying you’ll receive a reminder, it’ll say that they’re already signed up and would encourage them to log in.
Alex Gray: Elaine, just curious, does this client often do these sort of promotions like is this an evergreen feature for this client or was this a more timely testing need?
Elaine Koh: The 20% off for email is an evergreen feature.
Josh Braaten: What happens if you click the X?
Does that make the pop up go away completely or is that have the same functionality as the Remind Me Later?
Elaine Koh: It does make the model go away completely.
Zeinoun Kawwass: Okay.
And I know that like the sign up, really, somebody just is creating an account, right?
They’re not just signing up for…
Elaine Koh: So for this they are just signing up for that.
Zeinoun Kawwass: There’s just signing up for the newsletter.
Okay.
And that is the primary metric the others were just looking at to see if it impacts down funnel.
Elaine Koh: Mm.
Zeinoun Kawwass: Correct. Okay.
Elaine Koh: Yes.
Zeinoun Kawwass: Huh.
Interesting. I don’t know what to think.
I think that this might be a little disruptive.
I think we might be seeing signups, but it might be like negatively impacting checkouts.
So I’m conflicted on this one.
Elaine Koh: So for this one, we can do sign ups.
So I have both of them in terms of the results.
Zeinoun Kawwass: Okay.
Elaine Koh: But you can vote on whether you think this helped with sign ups or not.
Zeinoun Kawwass: Okay.
Elaine Koh: Now we can vote again for order completions.
Oh, just to throw…
Shake things up!
Since you brought it up.
Zeinoun Kawwass: All right. Are we ready to vote?
Should we do the signups first?
Okay, let’s do it.
Oh, James thinks it went down.
All right.
Do we want to do checkouts next?
Lots of mixed reviews.
Alright.
Elaine Koh: So, yes, Z was absolutely right.
There’s a huge lift in email submissions.
But in terms of order completions and also that’s the same percentage drop with add to bag.
So what we found with the Remind Me Later feature is that it seemed to remove urgency and demotivate purchase intent during the current session.
So one thing that we noticed is that the Remind Me Later feature didn’t come with copy to say when they would be reminded later.
So we think that users thought that the remind me later would be in a later session or potentially later during their visit in the site and not necessarily during the first add to bag moment.
So we found that this did distract users from having purchase intent.
Katie Green: Thanks for watching and be sure to subscribe to have more test ideas sent directly to your inbox.