When List Experiments Mislead: Fixing Measurement Error in Sensitive Surveys

🔍 Problem Identified:

Measurement error undermines survey validity, particularly for sensitive questions. List experiments reduce deliberate misreporting but remain vulnerable to nonstrategic measurement error from poor implementation and respondent inattention. Such error violates the assumptions of the standard maximum likelihood regression (MLreg) estimator and can produce misleading inferences—especially when the sensitive trait is rare.

📊 How the paper evaluates robustness and detection:

• Shows that the nonlinear least squares regression (NLSreg) estimator introduced in Imai (2011) is robust to nonstrategic measurement error.
• Proposes a general model misspecification test that flags divergence between MLreg and NLSreg estimates as a diagnostic for measurement error.

🛠️ How measurement error is addressed directly:

• Introduces new estimators that explicitly model nonstrategic measurement error.
• These estimators retain the statistical efficiency advantages of MLreg while improving robustness to the kinds of implementation and attention failures that bias standard estimates.

📁 Empirical checks and practical tools:

• Reanalyzes published list-experiment studies previously found to exhibit nonstrategic measurement error and demonstrates that the proposed diagnostics readily detect bias and that the new estimators mitigate it.
• Offers a set of practical recommendations for applied researchers on diagnosing and correcting measurement error in list experiments.

💻 Implementation:

• All proposed methods are implemented in an open-source software package to facilitate adoption and replication.

Why it matters: The suite of diagnostic tests and estimation tools allows researchers to detect when list experiments are compromised by inattentive or flawed implementation, to choose estimators that remain valid in those settings, and to recover more reliable estimates—critical when the target trait is rare and the risk of bias is greatest.