Sensitivity Analysis for Observational Studies of I by J Contingency Tables

Music City Center 

Contingency tables are a popular approach to study associations between categorical variables in an observational study. Unfortunately, most association tests for contingency tables assume no unmeasured confounding, an unrealistic assumption, and sensitivity analysis, a popular procedure to assess the bias from unmeasured confounders, often assumes a binary categorical variable. This paper proposes a sensitivity analysis for contingency tables. Specifically, we extend Rosenbaum's sensitivity model to non-binary categorical variables and present an algorithm to compute the exact, worst-case null distribution for a family of association tests. Results accelerating the computation in a sensitivity analysis when the test statistic is (a) permutation-invariant (b) a sum score test with non-binary outcome (c) a sum score test with binary outcome are also presented. We also complement the exact approach with a derivation of the asymptotic null distribution.

We also compare the power to detect an ordinal treatment effect between our approach and two simpler alternatives: (a) a sensitivity analysis that binarizes a multi-level outcome (i.e., binarizing) and (b) a sensitivity analysis using Fisher's exact test, which reduces an I by J contingency table to a 2 by 2 table by combining categories (i.e., collapsing). Our results demonstrate that retaining the original I by J table yields greater statistical power than the two simpler alternatives.

Finally, we extend our findings to stratified I by J tables (i.e., I by I by K tables) and illustrate our approach through a re-analysis of the impact of pre-kindergarten care on math achievement using data from the Early Childhood Longitudinal Study, Kindergarten Class of 1998-1999.

exact test

causal inference

sensitivity analysis

contingency table

power analysis

observational studies