This note examines whether low-quality online survey panels can produce misleading inferences about acquiescence bias in measures of conspiratorial beliefs and political misinformation. The authors reanalyze and replicate an earlier survey experiment, identify a coding error that led to the surprising result that respondents with postgraduate education appeared most prone to acquiescence, and then run two preregistered replications to probe that pattern.
Data and methods
The paper reanalyzes an original survey experiment and then conducts two preregistered replication studies. One replication used the same low-quality survey platform as the original study and produced nearly identical results; the second used a larger, higher-quality survey panel and did not replicate the apparent effect. The abstract reports the discovery and correction of a minor data-coding error that motivated the replications.
Main findings
The authors report that the initially observed relationship between postgraduate education and higher acquiescence was an artifact of inattentive and fraudulent responses in the original panel. They find that attention checks alone did not fully resolve the problem and that using a higher-quality panel eliminated the puzzling effect.
Key elements: reanalysis of a survey experiment, two preregistered replications, artifact from inattentive/fraudulent responses
- The same low-quality platform reproduced the original puzzling result.
- A larger, higher-quality panel did not show the effect, suggesting it was spurious.
