experimentation

Articles

Experiments at Airbnb

Experiments at Airbnb

Airbnb uses controlled experiments to learn and make decisions at every step of product development, from design to algorithms. They are equally important in shaping the user experience.

How to Build a Strong A/B Testing Plan That Gets Results

How to Build a Strong A/B Testing Plan That Gets Results

A strong A/B testing plan will allow you to increase your revenue. You will also learn valuable insights about your customers, because you will know their preferences instead of guessing what your customer may want.

Iterative A/B Testing – A Must If You Lack a Crystal Ball

Iterative A/B Testing – A Must If You Lack a Crystal Ball

You have a hypothesis, and run a test. Result – no difference (or even drop in results). What should you do now? Test a different hypothesis?

How Booking.com A/B Tests like Nobody’s Business

How Booking.com A/B Tests like Nobody’s Business

A/B testing is meant to help create the best version of a product that’s specifically tailored towards the consumers’ needs and wants. What appeals to your end consumer, and how exactly do you optimize your app for the largest group of people?

A/B Testing - You’re doing it wrong - How to effectively A/B test in a way that drives long-lasting results

A/B Testing - You’re doing it wrong - How to effectively A/B test in a way that drives long-lasting results

So, your company wants to increase revenue and adoption by making some marketing site tweaks. They want more conversions, more clicks, more shares, and more users. What do they tell you to do first? Well, A/B test! Compare two versions of a page, define a key goal (ex. clicks), and see if you get more clicks. But, does this actually work? Is it really the approach you should take? Let’s look at the data.

4 Principles for Making Experimentation Count

4 Principles for Making Experimentation Count

Practicing these principles will not only save your org a ton of engineering time and money, but will allow incredible insight into your users and product.

A/B tests that do more than validate

A/B tests that do more than validate

A/B tests provide more than statistical validation of one execution over another. They can and should impact how your team prioritizes projects.

The golden rule of A/B testing: look beyond validation

The golden rule of A/B testing: look beyond validation

A/B tests provide more than statistical validation of one execution over another. They can and should impact how your team prioritizes projects.

Videos

Podcast

Research-Driven A/B Testing with Nick Disabato

Research-Driven A/B Testing with Nick Disabato

When does A/B testing have a positive impact on your business? How do you make intelligent, informed decisions? How do you prevent your ego and past experience from interfering? Nick Disabato, a world-class expert in A/B testing, talks about his research-driven work process and his humble philosophy.

Research as the Key to Conversion Rate Optimization with Momoko Price

Research as the Key to Conversion Rate Optimization with Momoko Price

You’ll learn why research is key to any CRO activity, why elimination testing is awesome, what tools you can use to split-test your pages, and how to plan the entire testing process.

Experimentation is Product Management

Experimentation is Product Management

Tristan Kromer shares the differences between qualitative and quantitative user feedback, how to effectively formulate hypotheses, and the role of statistical significance in interpreting results.

High-Impact Experimentation is Product Management

High-Impact Experimentation is Product Management

Holly Hester-Reilly, Product Consultant and former Product Owner at Shutterstock, shares how she runs experiments to test critical assumptions and make better decisions throughout the product lifecycle.

Tools

Optimizely

Optimizely

Optimizely is the world's leading experimentation platform, enabling experimentation and personalization across websites, mobile apps and connected devices.

Books

Designing with Data: Improving the User Experience with A/B Testing

Designing with Data: Improving the User Experience with A/B Testing

This practical guide shows you how to conduct data-driven A/B testing for making design decisions on everything from small tweaks to large-scale UX concepts. Complete with real-world examples, this book shows you how to make data-driven design part of your product design workflow.