There’s nothing more humbling when you run an A/B test, supremely confident you know the outcome, only to be shocked when it loses.
It’s why we say, “It’s why you test.”
I’ve often said if we could predict winners, we could print money. Imagine having a crystal ball—“If I make this change, my conversion rate goes up, my Google Ads just got more effective, and I’ve 5x’d my sales.”
Given that we’ve run thousands of optimization experiments across a wide variety of verticals—including subscription services, e-commerce, social good, fintech, and hospitality, driving over $1B in incremental growth and invaluable customer insights for our clients, we asked ourselves, what if we could use that data, and leverage AI and machine learning to give us that crystal ball?
The Challenge: Prioritizing Experiments for Maximum Impact
Website experimentation (A/B, multi-variate, and personalization testing) is an art as much as a science. While ideation and strategy are critical, one of the most challenging aspects of optimization is determining which ideas are worth pursuing. With finite resources, choosing the right experiments is crucial to maximizing velocity, impact, and win rates.
Prioritization frameworks rely on subjective input or historical performance data to estimate potential value. While helpful, these approaches often fall short in predicting the likelihood of success, especially for entirely new ideas, often leaving optimization teams with decision paralysis and risking opportunity cost.
The Solution: ImpactLens
Recognizing this gap, we set out to transform how we approach experiment prioritization. By augmenting our data with AI and training predictive models on our rich dataset of past experiments, we can now predict outcomes approximately 70% of the time. While not (yet) a crystal ball, it is certainly better than most industry win rates in the 20 – 40% range.
This means we can leverage this tech to assign win probability scores and project weighted impact estimates for experiment ideas, allowing teams to:
- Elevate Win Rates: Identify experiments with the highest likelihood of success. While win rate is ultimately a vanity metric, elevating it can have an outsized impact when paired with a weighted impact calculation.
- Increase Velocity: Confidently prioritize experiments that deliver faster, meaningful results and deliver customer learnings and preferences more quickly.
- Maximize Impact: Focus on ideas that drive measurable business outcomes.
ImpactLens goes beyond traditional analytics, offering prescriptive recommendations that empower teams to make guided optimization decisions.
Why We Built ImpactLens
The idea for ImpactLens was born from our commitment to continuously improve the impact of testing for our clients. As we launched thousands of experiments across industries, we amassed a treasure trove of data, but we knew the potential value of that data extended far beyond retrospective analysis.
We envisioned a solution that could predict outcomes and prescribe the most effective course of action for future experiments. We turned this vision into reality by leveraging AI and machine learning, enabling data-driven decisions with unprecedented accuracy and confidence.
How It Works
ImpactLens analyzes historical experiment data, identifies patterns, and learns from past successes and failures. When introducing a new experiment idea, the system evaluates it against these patterns, assigning a probability-to-win score and estimating the potential business impact weighted against that probability.
This technology helps clients streamline their testing pipelines, eliminate guesswork, and focus on initiatives that align with their goals—improving conversion rates, boosting retention, or enhancing customer experience.
The Future of Experimentation
ImpactLens represents a significant step forward in the evolution of experimentation. By combining data, machine learning, and predictive modeling, we’re unlocking new opportunities for businesses to innovate faster, smarter, and more confidently.
We’re thrilled to announce that ImpactLens is now in beta. Early adopters are already seeing promising results, and we’re excited to refine the technology further based on their feedback.
Join the Beta Program or Waitlist
ImpactLens is just one part of Iris, our exclusive growth marketing platform—a powerful suite of tools designed specifically for our clients to maximize experimentation success, optimize their growth marketing programs, and drive measurable business impact.
Currently, Iris is available only to Cro Metrics clients, ensuring that the brands we partner with gain a strategic advantage through faster, data-informed decision-making, predictive insights, and continuous optimization.
If you’re interested in learning more about how Iris and ImpactLens can help you prioritize high-impact optimization and accelerate your organization’s growth, let’s talk. We’d love to understand your business needs and explore how we can work together to unlock smarter, faster, and more effective digital growth strategies.