NOW AVAILABLE: Essentials of Poltical Analysis 7e | R Companion 4e | SPSS 7e | Stata 5e | Excel 1e 🎵
NOW AVAILABLE: Essentials 7e | R Companion 4e | SPSS 7e | Stata 5e | Excel 1e 🎵
NOW AVAILABLE: Essentials 7e | More Textbooks 🎵
If this link is broken, please
You can also
(will be reviewed).

Amazon Mechanical Turk's Data Quality Crisis: New Insights into Detection

New research reveals a growing problem of fraud on Amazon's Mechanical Turk platform. Using VPNs, respondents are gaining unauthorized access to studies and submitting low-quality data.

🔍 What We Did

We analyzed 38 existing MTurk studies to understand the scope of this issue.

📊 Our Findings

  • Data quality is declining due to fraudulent access via VPNs, a problem that has worsened recently.
  • Fraudulent respondents provide significantly lower-quality data than legitimate users.
  • This low-quality input can weaken treatment effects in experiments.

🛠️ How We Help

Researchers now have two tools: an application to identify fraud in existing MTurk datasets and a method to block fraudulent access during Qualtrics surveys.

Political Science Research & Methods
The Shape of and Solutions to the MTurk Quality Crisis was authored by Ryan Kennedy, Scott Clifford, Tyler Burleigh, Philip Waggoner, Ryan Jewell and Nicholas Winter. It was published in PSR&M in 2020. Find on Google Scholar10.1017/psrm.2020.6
Methodology subfield banner