← Research / Experiment Design

Controlled experiments on the web: survey and practical guide

Read full paper →
AuthorsRon Kohavi, Roger Longbotham, Dan Sommerfield, Randal M. Henne
JournalData Mining and Knowledge Discovery
Year2009
DOI10.1007/s10618-008-0114-1

What Problem It Solves

Provides a canonical overview or reference point for the relevant DoOperator research area.

What problem it solves

Provides a canonical overview or reference point for the relevant DoOperator research area.

How it works

A practitioner-oriented survey of online controlled experiments, web experimentation systems, metrics, pitfalls, and deployment issues.

When to use it

Use when orienting a new paper, blog post, benchmark, or research plan in this area.

Limitations and failure modes

Do not cite the overview as evidence that a specific method works in a specific deployment setting without checking the underlying primary paper.

Read full paper →More Experiment Design

Related papers

Paper

The Design of Experiments

Ronald A. Fisher · 1935

Paper

Statistical Challenges in Online Controlled Experiments: A Review of A/B Testing Methodology

Nicholas Larsen, Alex Deng, Jiheng Zhang +2 more · 2024

Paper

On Causal Inference in the Presence of Interference

Eric J. Tchetgen Tchetgen, Tyler J. VanderWeele · 2012

Paper

Always Valid Inference: Bringing Sequential Analysis to A/B Testing

Ramesh Johari, Leo Pekelis, David J. Walsh · 2015