Scholars increasingly prioritize credible causal designs by testing their assumptions through balance and placebo tests. However, traditional approaches often misapply statistical methods with a null hypothesis of no difference, potentially equating insignificant findings to acceptable homogeneity.
Instead, this paper proposes researchers should adopt an initial hypothesis that data may contradict valid research design, requiring strong evidence for its existence. By shifting the null hypothesis to represent difference and testing for equivalence instead, these 'equivalence tests' offer a more accurate assessment of causal identification assumptions.
This approach better incorporates substantive considerations about what constitutes good balance on covariates and placebo outcomes compared to conventional methods. The paper demonstrates these advantages through applications focused on natural experiments in political science contexts like US congressional elections (using data from the ANES).
Emphasizing robust methodology over simplistic statistical significance, this work advances best practices for evaluating research designs.
