Philippa Boyes: We’re going to talk about email.
So this is for one of our nonprofit clients in the education space that focuses on recruiting and supporting teachers.
So during the school year, they run a biweekly email newsletter for teachers that focuses on tools, tips, resources and opportunities for both teachers and school administrators, and we were noticing that the overall click rate for this newsletter was decreasing over time and we wanted to improve it since this is a newsletter that could genuinely provide support and guidance for folks.
So this is the control, the original newsletter.
We noticed that the majority of clicks were happening in this little section here called ‘Resources and Opportunities’…and so we wanted to test and see whether if we restructured the template of the newsletter, we put all of the content in the resources and the opportunity section, whether that would improve the click through rate as the key metric.
You’ll see the one on the left is the control, which has some of the resources kind of up front with a picture and a larger CTA button and then the less important resources in this quadrant box over here and it ends with the with the interactive form and in the variation, it’s much shorter and cleaner…It is just all of the content listed under the resources and the opportunities…Because of some coding issues, we weren’t able to have the interactive form in the variation.
So that’s very interesting, too.
And this was the test.
Any questions?
Shina Hall: Well, I do have a question.
So I’m noticing that each of the boxes within the resources and opportunities has a CTA in it, and just wondering if you measured the clicks to those CTAs separately and kind of determine which ones should be in which places or if that affected your overall click through rate for it, V0, then also V1.
Philippa Boyes: So we did look at the click, through rates for each specific CTA, more so for just learning purposes.
That did not factor into what we call the winner of the test…um in terms of the order in which they appear.
That was a decision by the client in terms of what they thought was the most important content.
Cara Binsfield: Is the variation email like markedly shorter?
It looks like it would be, but I want to understand the scrolling.
Philippa Boyes: Yeah, it is definitely shorter.
So. Okay.
One scroll, two scrolls.
So the variation is done in two scrolls, the control is done in like two and a half scrolls.
Yeah, and obviously you would multiply that for mobile.
Any other questions?
Are we ready to go to a vote?
Okay.
So thumbs down if you think that the control won.
Thumbs up if you think the variation won.
Katie Green: Ok, Phil, show us the results. Yeah.
Philippa Boyes: The results were that the variation won.
This was a pretty big win.
We saw a 30% increase in the click through rate by putting all content through the resources and the opportunity section.
We also saw a 43% increase in the click to open ratio, which signified to us that more people were actually reading through the entire email.
So to your point, Cara, the fact that it was shorter we think could be incredibly helpful going forward and we’re doing more testing in regards to the length of emails and just visually signifying what things are.
The other thing that I was really excited about, the winner for this test is one of the resources that this newsletter offered was a free year of mental health through a corporate partnership, and so the fact that we got many more teachers and school administrators to be able to click for their free year was just a really heartwarming win as well.
Unknown: Thank you so much for watching.
Be sure to subscribe to have more test ideas and directly to your inbox.