For decades, political science has debated null hypothesis significance testing (NHST). Critics argue it focuses too much on statistical outcomes while neglecting practical relevance. This article introduces a new approach that balances both concerns by incorporating subject-area expertise.
Context-Driven Framework: The method moves beyond standard NHST to consider what parameter values matter in political contexts, based on existing theory and research knowledge.
Instead of automatically rejecting the null hypothesis at p < .05 or accepting it otherwise, this framework asks: Given sampling error, is a statistically significant finding practically meaningful for understanding politics?
Key Findings: By integrating substantive expertise with statistical analysis, researchers can better assess real-world political significance while still accounting for uncertainty.
The approach provides guidance on determining what parameter values should be considered 'interesting' or consequential in political science research.
Why It Matters: This method helps bridge the gap between statistical results and their practical implications. By grounding testing in context, it encourages more nuanced interpretations of findings that align with political theory and real-world relevance.
| Testing What Matters (If You Must Test at All): A Context-Driven Approach to Substantive and Statistical Significance was authored by Justin Gross. It was published by Wiley in AJPS in 2015. |
