Cara Binsfield: I will be talking to you today about a test we ran for a food delivery service.
So the goal of this was to understand how to promote self-serve products.
And so we’re working with a food delivery service, as I mentioned, and we work with a lot of different business units with this food delivery provider.
But the business unit we were working with this time around was focused on the the restaurant side of the business so those merchants or vendors that then provide the food that’s like hosted on the food delivery site.
And our goal was to work with them to understand how could they get more merchants or restaurants to purchase and engage with their self-serve products.
So they have products like an online ordering platform that you can put on your website so you can send orders to your existing customers using their delivery drivers, or a product like an ad and promotions similar to how you can bid on Google ads for your own ads.
In this food delivery service as a restaurant, you can bid to have your listing posted higher.
Those kinds of things, or you can offer discounts to customers to get more purchases.
So the goal was to understand where they should be promoting those products within the merchant portal and also to understand like, did merchants know what products were available to them and where on the site would indicate a readiness to take action and engage with those products.
So in the control experience, one of the places within the merchant portal that they can go is a page that displays all of the orders they’ve ever posted through – or all of the orders they’ve ever had placed with them through this online delivery site.
And we thought, OK, when you look at this orders page, there’s really high traffic volume.
A lot of restaurants want to know how many orders have I gotten, when were they placed, et cetera.
And we thought, if you’re in the mind frame of looking at your orders, that’s probably a pretty prime spot to tell people that they can get more orders by using some of these self-serve products.
Specifically, the option to use ads and promotions.
So we tested putting a call out box at the top, sort of just below the had header of this page, letting them know want to increase orders, get noticed by new customers with ads and keep them coming back with promotions.
And then we also let them know about this promotion that the food delivery service had to give them $100 in free credits towards their first ads and promotions.
So I will stop there and say what questions do you all have on this test or thoughts?
Emily Martin: How would they know about it other than the banner?
Cara Binsfield: A couple of ways, so this tab here marketing.
They can click on that and it hosts all of them and sometimes they will have like a little call out highlighting it.
If people have never done one, so the tab will have like some sort of notice on it.
There is also a call out on the dashboard, which is typically one of the first pages that merchants go to for new merchants letting them know about this credit and about ads and promos.
So there are a couple of different ways, but they wanted to understand what we’re testing into really is, which products to surface and where the surface in the portal, because right now they only get about 5% of total merchants visiting the marketing tab to look at ads and promos
Katie Green: And Cara, if they exit out, it does not come back ever?
Or does it come back after a certain amount of time?
Curious about those returned visitors.
Cara Binsfield: Yeah. So initially we were going to have it be once per session, but the client requested once every seven days.
So we can do that functionality differently.
But in this test, we chose to do every seven days.
James Buo: And, Cara, is a success metric whether or not they actually created that campaign or just visited that marketing tab?
Cara Binsfield: So we looked at both and the, like goal for statistical significance was page visits to that marketing tab because that 5% is only visits to the marketing tab, not even campaign creations.
But we wanted to track campaign creations to understand if that awareness would directly translate to revenue for the company or if it would only like drive more visits without actually being in the right state of mind to convert.
All right, everybody, if you could vote if you think the test variation won with a thumbs up and if you think that test variation using the control won with a thumbs down.
It looks like a lot of people voted for the test to be the winner.
So this test was a very big success.
We – just to give reference like I said, we’re testing a couple different products and a lot of different places.
And so sometimes we’re actually not seeing even a lift in visits to the self-serve product page with some of these placements.
It’s very clear that contextually it’s not the right spot.
Sometimes we’re seeing a lot of friction with like three times as many users clicking to exit the banner as clicking on the CTA.
And so in the case of this test, it was really positive.
It had a 28% lift in page views to the marketing page, and then a 26% lift and campaign success.
When we looked at the total volume, basically about half of the users that we were driving to the page ended up converting and creating incremental revenue for the company.
So it was a really big win and they felt really excited about understanding that this was a really good placement in terms of contextual desire for users to lift their orders.
Katie Green: Thanks for watching and be sure to subscribe to have more test ideas sent directly to your inbox.