Experimentation

You can't A/B test your way out of a confusing experience.

Testing optimises what exists. When the experience is unclear, fix the structure first.

DS

Dayo Samuel · Founder · 6 min read

You can't test your way out.

What a test can and can't do

An A/B test compares two versions of a decision that already makes sense. It will tell you which headline, which price frame, which button wins. It cannot turn a page nobody understands into one they do — you're just measuring degrees of confusion.

You can't A/B test your way out of a confusing experience.

Fix structure, then optimise

When research shows the experience is fundamentally unclear, the move is a structural rebuild, validated qualitatively, before the testing programme starts. Test the details once the shape is right. Test the shape and you'll iterate forever at 2% lifts.

Key takeaways

  • Tests refine decisions that already make sense.
  • Structure comes before optimisation.
  • Validate the shape qualitatively first.
DS

Dayo Samuel

Founder of Schemerr. 400+ experiments shipped across eCommerce, SaaS and fintech.

Your next move starts here

Have an opportunity worth exploring?

Tell us what you're trying to change, build or understand. We'll come back with an honest first view of where the opportunity is — and how we'd find out.

Let's talk