experimentation
Articles
Experiments at Airbnb
Airbnb uses controlled experiments to learn and make decisions at every step of product development, from design to algorithms. They are equally important in shaping the user experience.
How to Build a Strong A/B Testing Plan That Gets Results
A strong A/B testing plan will allow you to increase your revenue. You will also learn valuable insights about your customers, because you will know their preferences instead of guessing what your customer may want.
Iterative A/B Testing – A Must If You Lack a Crystal Ball
You have a hypothesis, and run a test. Result – no difference (or even drop in results). What should you do now? Test a different hypothesis?
How Booking.com A/B Tests like Nobody’s Business
A/B testing is meant to help create the best version of a product that’s specifically tailored towards the consumers’ needs and wants. What appeals to your end consumer, and how exactly do you optimize your app for the largest group of people?
A/B Testing - You’re doing it wrong - How to effectively A/B test in a way that drives long-lasting results
So, your company wants to increase revenue and adoption by making some marketing site tweaks. They want more conversions, more clicks, more shares, and more users. What do they tell you to do first? Well, A/B test! Compare two versions of a page, define a key goal (ex. clicks), and see if you get more clicks. But, does this actually work? Is it really the approach you should take? Let’s look at the data.
4 Principles for Making Experimentation Count
Practicing these principles will not only save your org a ton of engineering time and money, but will allow incredible insight into your users and product.
A/B tests that do more than validate
A/B tests provide more than statistical validation of one execution over another. They can and should impact how your team prioritizes projects.
The golden rule of A/B testing: look beyond validation
A/B tests provide more than statistical validation of one execution over another. They can and should impact how your team prioritizes projects.
Videos
Podcast
Research-Driven A/B Testing with Nick Disabato
When does A/B testing have a positive impact on your business? How do you make intelligent, informed decisions? How do you prevent your ego and past experience from interfering? Nick Disabato, a world-class expert in A/B testing, talks about his research-driven work process and his humble philosophy.
Research as the Key to Conversion Rate Optimization with Momoko Price
You’ll learn why research is key to any CRO activity, why elimination testing is awesome, what tools you can use to split-test your pages, and how to plan the entire testing process.
Experimentation is Product Management
Tristan Kromer shares the differences between qualitative and quantitative user feedback, how to effectively formulate hypotheses, and the role of statistical significance in interpreting results.
High-Impact Experimentation is Product Management
Holly Hester-Reilly, Product Consultant and former Product Owner at Shutterstock, shares how she runs experiments to test critical assumptions and make better decisions throughout the product lifecycle.