The Lady Tasting Tea Revisited: Insights on SUTVA Violations from a Canonical Example

Music City Center 

Fisher's Lady Tasting Tea is often one of the first examples of an experiments considered when teaching a design-based causal inference course. However, we show that Lady Tasting Tea violates the stable unit treatment value assumption (SUTVA), a foundational assumption made by many commonly used causal inference models, and whose violations are typically associated with studies containing units that ``interact'' with each other (for example, those on social networks). We suppose that the Lady has knowledge that half of all cups receive tea first and the other half receive milk first, and that this assignment is completely randomized. For each cup, the Lady obtains a ``likelihood score'' of that cup receiving milk first that may vary depending on whether that cup is actually given milk first. After tasting tea from all cups, the Lady guesses that the cups with the largest likelihood scores are given milk first. We show that SUTVA may be violated under this model, even when the Lady's milk-first likelihood is always higher when that cup actually receives milk first. We then discuss how this may impact best practices in teaching an introductory causal inference class.

Lady Tasting Tea

Stable Unit Treatment Value Assumption

Causal Inference

Experimental Interference 

Section on Statistics and Data Science Education