Summary. For years, online experimentation has fueled the innovations of leading tech companies, enabling them to rapidly test and refine new ideas, optimize product features, personalize user experiences, and maintain a competitive edge. The widespread availability and lower cost of experimentation tools today mean that most organizations -- even outside the technology sector -- conduct tests. After initial adoption, however, many of them restrict experimentation to just a handful of carefully selected projects. That's because their data scientists are the only ones who can design, run, and analyze tests. Vastly increasing the capacity to conduct online experiments is becoming more critical as the expanding capabilities and applications of artificial intelligence -- particularly generative AI -- reshape innovation. Scaling up experimentation entails moving away from a data-scientist-centric approach to one that empowers everyone else on the product, marketing, and sales teams to run experiments. The authors suggest how to do that.
For years online experimentation has fueled the innovations of leading tech companies such as Amazon, Alphabet, Meta, Microsoft, and Netflix, enabling them to rapidly test and refine new ideas, optimize product features, personalize user experiences, and maintain a competitive edge. Owing to the widespread availability and lower cost of experimentation tools today, most organizations -- even those outside the technology sector -- conduct online experiments.