Identify Direct Effects in Survey Experiments Without Risky Assumptions

🔍 Research Problem

Many studies of causal mechanisms in survey experiments estimate indirect effects by conditioning on nonrandomized variables. That approach relies on a “selection-on-observables” assumption, which undermines the central advantage of random assignment and raises concerns about the validity of mediation claims.

🧪 How the Design Reveals Mechanisms

A factorial experimental design is proposed that randomly provides or withholds information about potential mediators. Key features:

• Randomly manipulate both the primary treatment and whether subjects receive information about a putative mediator.
• Identify the overall average treatment effect (ATE) through the randomized treatment.
• Identify the controlled direct effect of the treatment when fixing the mediator via the information-manipulation arm.
• Do not identify indirect effects on their own without additional assumptions.

📊 Key Advantages and Limits

• Avoids the selection-on-observables assumption required by standard mediation analyses that condition on nonrandomized mediators.
• Preserves the inferential benefits of randomization while furnishing evidence about how treatment operates.
• Does not by itself recover indirect effects, but enables a broader investigation of mechanisms that captures both indirect pathways and treatment–mediator interactions.

📚 Illustrative Applications

Examples demonstrate the approach in two substantive domains:

• Evaluations of U.S. Supreme Court nominees
• Perceptions of the democratic peace

⚖️ Why It Matters

This design offers researchers a principled way to learn about causal mechanisms using experimental design rather than relying on untestable selection-on-observables assumptions. It informs when and how survey experiments can credibly speak to direct effects and to more complex mechanism questions that involve interactions and indirect pathways.