Weather Instruments Under Threat: Widespread Exclusion-Restriction Violations Found

🔎 What Was Reviewed

Instrumental variable (IV) analysis relies on the exclusion restriction: the instrument should affect the outcome only through the independent variable of interest. Any other causal pathways are exclusion-restriction violations and invalidate the instrument. Weather has been widely used as an IV across social science to predict many different independent variables. The use of weather to instrument different independent variables therefore represents strong prima facie evidence of exclusion-restriction concerns for studies that rely on weather IVs.

📚 A Survey of Studies Using Weather as an Instrument

• A review covered 288 empirical studies that used weather as an instrumental variable.
• The review identified 192 distinct variables previously linked to weather; each of these represents a potential exclusion-restriction violation.

🧪 How Potential Violations Were Evaluated

• Sensitivity analysis was used to assess the plausible size of direct weather→outcome pathways that are not mediated by the targeted independent variable.
• The analysis shows that the magnitude of many of these plausible violations is sufficient to overturn numerous published IV results.

⚠️ Key Takeaways

• Weather’s widespread use as an instrument creates substantial, systematic concerns about the exclusion restriction across many studies.
• The large set of previously documented links between weather and a wide range of variables means many weather IV studies face credible threats to validity.
• Sensitivity results indicate that several published causal claims based on weather IVs are potentially unreliable.

🛠️ Practical Guidance for Researchers

Concludes with practical steps to systematically review existing literature and identify possible exclusion-restriction violations when using IV designs that rely on weather instruments.