Video: Experimentation Driven Development

Hi, I'm Graham, co-founder of GrowthBook, and in this lightning talk, I'll cover experimentation-driven development. We'll explore how we build products today, the issue of knowing if our shipped products actually work, and the solution: experimentation-driven development. A-B testing is a controlled way of measuring the impact of changes on real users. It involves starting with a hypothesis, assigning users to different groups, exposing them to different variants, and tracking their behavior. Examples from Airbnb and Netflix show the varying success rates of A-B tests. On average, only one-third of tests are successful in moving the desired metrics. Without testing, you're just guessing. Common objections include relying on before and after data without controlled experiments. A-B testing helps control for variants and noise in data, allowing you to determine causation. User testing with small sample sizes may not provide accurate insights. Integrating A-B testing into the development process helps define hypotheses, track metrics, and iterate quickly. Use feature flags to easily test and roll out changes. Feature flags add safety by separating code deployment from feature release. A-B testing allows conditional feature release and provides statistical results. A-B testing replaces differences of opinion and celebrates learning from failures. Hypothesis testing is crucial for determining the success of a project. Experimentation driven development is easy and should be done on every project.