What kind of messaging should nonprofits use in lightbox pop-ups to increase donations? We recently tested that and the answer might surprise you.
Watch this episode of The Cro Show, a game show for conversion rate optimization and marketing experimentation fans, and see if you can guess which test variation performed better during a recent Cro Metrics client experiment.
Subscribe to the Cro Metrics newsletter to get future episodes of The Cro Show sent to your inbox.
Full Transcript:
MJ Castells: All right, so this is a test we ran for a non-profit client that we have, they’re focused on environmental justice and climate change action.
So, one of the teams that we work with is their magazine site which is essentially just an online version of their print and magazine that all members get. This magazine site gets a lot of traffic particularly coming from emails, their newsletter that they are constantly sending folks to read the articles here or ads. So they get a lot of traffic but we haven’t really tested a lot into a donation ask specifically. So there’s this kind of like untapped potential here. So this particular test that we decided to run is basically a donation ask on the magazine site on their light box.
When we did this test, we decided, okay what else can we do here to kind of collect donations? So, our hypothesis was essentially if we change the light box from the newsletter sign up to a donation sign up, we’re gonna see an increase in the national donations coming in from users coming from the magazine site.
So this is what the V0 looks like. So, the user comes in they fill out their email they click the sign up and they’re signed up to the newsletter.
The V1 which is the donation ask removes that image and has this kind of like specific ask about championing the environment and helping to fund a crucial environmental journalism.
And then the CTA here that says join today that takes the user to one of the donation forms.
So basically that was really the test itself. It kind of went against the newsletter to see if we saw an increase or decrease in newsletter signups as well. But ultimately the goal was to increase donations but yeah that was the test. Does anybody have any specific questions or any kind of like thoughts on who won here?
Paul Swinand: So you’re measuring sign ups for the magazine and also donations in the first one but only donations in second?
MJ Castells: Right, yeah. In the first one there is no in the visa there’s no direct way to get to the donation forms. ‘Cause it’s not in the light box but if you’re in the treatment and you have to make a donation through a different they have donation sign ups throughout their site. So you can make a donation but it’s not a direct ask like that.
Paul Swinand: Do they track if you signed up like they would know if you signed up for the newsletter and then they’re tracked down?
MJ Castells: No, not if you signed up directly for the newsletter and then also made a donation. We were tracking clicks to this CTA here which is the sign up for newsletter. And then separately tracking clicks to this chunk database.
Grant Tilus: Is this a landing page experience like upon land or is this exit intent?
MJ Castells: This is upon land and it ran on their entire site wide. So, whichever if you came in from like an article, this particular layout the background here’s the homepage of the magazine. But if you came from an article you’d also see it.
Grant Tilus: Thumbs down if you think control one, thumbs up if you think the variant one.
MJ Castells: Okay, so this one actually saw a pretty nice lift in the V1 for donations made. So we have this metric here called national donations which basically tracks all donations across the site which is slightly different than this other metric that we have here monthly donations and one time donations. But all these donation metrics have a little bit of a lift here with some stats sig in the primary metric which was really exciting because sometimes it’s not that easy to get that stat sig on specifically on donations.
So overall, I mean this test really performed really great. And we saw that users are coming in here, making donations on that V1, clicking through about 0.86% on that CTA on the V1, which compared to the V0 CTA which was the newsletter sign up which we had 1.31 and a conversion rate here. So it wasn’t a super big drop in clicks into that. But the clicks that were being made were converting at a higher rate in terms of donations.
This is pretty nice win here because now we’re kind of opened up this new swim lane to be able to test donations that otherwise we weren’t really collecting or testing into.