Small Studies, Big Effects: Publication Bias in Voter Mobilization Experiments

If journals favor statistically significant results, the published literature can suffer from publication bias.

🔎 How the test works

Statistical significance depends on sample size: smaller samples must show larger observed effects to reach significance. That implies a clear empirical test—if publications are biased against statistically insignificant findings, average reported effect sizes should shrink as sample sizes grow.

📚 What evidence was examined

• Published experimental studies on voter mobilization

📈 Key findings

• Reported effect sizes decline as sample sizes increase.
• This pattern is consistent with publication bias against statistically insignificant results.
• The proposition that significance-driven selection produces larger effects in small samples was tested and confirmed in the experimental voter mobilization literature.

⚖️ Why it matters

The observed relationship between sample size and reported effect size indicates that the published literature on voter mobilization may overstate true effects when significance-driven selection operates. The described test offers a practical way to detect such bias in empirical literatures.