As developers, we constantly ship new features to production, but how do we know their impact? In this talk, we’ll discuss why it’s important to adopt experimentation-driven development, how to get leadership buy-in, and ways this can go wrong.
This talk has been presented at JSNation US 2024, check out the latest edition of this JavaScript Conference.
Experimentation-driven development is a process that focuses on shipping features that positively impact users and metrics by integrating A-B testing to validate product changes.
Graham McNicol is the co-founder of GrowthBook and a speaker on experimentation-driven development.
GrowthBook is the most popular open source platform for feature flagging and A-B testing.
A-B testing is a controlled way of measuring the impact of changes on real users by comparing different variants and tracking user behavior.
A-B testing is important because it provides a causal way to determine the impact of product changes, helping to avoid guesses and assumptions.
Challenges include a low success rate of about one-third for tests in moving intended metrics and the difficulty of predicting user preferences accurately.
Feature flagging allows conditional releasing of features, enabling safer deployments by separating code deployment from feature release and facilitating A-B testing.
The key takeaway is that 'done' should not mean shipped; instead, every project should involve hypothesis testing with A-B testing to ensure success.
Common objections include relying on before-and-after data analysis and small-sample user testing, both of which lack the control and scale of A-B testing.
Companies can integrate experimentation-driven development by defining success criteria, hypothesizing outcomes, and using feature flags and A-B testing in their product development process.
