Article summary
Internet-based surveys have expanded public-opinion data collection but have weakened researchers’ ability to monitor respondent attentiveness, creating a growing need for ex-post evaluation of data quality.

Internet-based surveys have expanded public-opinion data collection but have weakened researchers’ ability to monitor respondent attentiveness, creating a growing need for ex-post evaluation of data quality.
New research provides guidance on effectively using screener questions to identify inattentive respondents during self-administered surveys—a common problem that introduces noise into datasets.
Surveys require attentive respondents. The article introduces "Screeners" as a tool to identify lack of attention but argues they have limitations.
Manipulation checks are often overlooked in political science experiments despite their value. This article introduces factual manipulation checks (FMCs) as an effective method to assess individual-level attentiveness.
This article compares online recruitment via Facebook, Mechanical Turk (MTurk), and Qualtrics panels across two countries: India and the United States.
Does social network data gathered online accurately reflect real-world offline relationships?
Data come from a recent online survey that used instructed-response items (so-called trap questions) to identify inattentive respondents. These items flag participants who fail to follow simple instructions embedded in the instrument.
Local governments dominate U.S. public administration: of the country’s 90,106 governments, 99.9% are local. These jurisdictions vary widely in institutional features, descriptive representation, and policy-making power, but political scientists have been slow to exploit that variation because comprehensive local data are often hard to obtain—unavailable or costly, difficult to replicate, and rarely updated.
New evidence challenges old concerns about survey experiments conducted on Mechanical Turk. Results obtained through these online platforms appear to generalize well across diverse populations.
New research reveals a growing problem of fraud on Amazon's Mechanical Turk platform. Using VPNs, respondents are gaining unauthorized access to studies and submitting low-quality data.
Online surveys increasingly shape political discourse. But do sponsors influence how people answer? This article examines sponsorship effects in survey responses.
Online crowdsourcing platforms like Witmart have become common tools for subject recruitment. This study examines whether samples drawn from these sites are representative of the general population or just internet users. By benchmarking Witmart data against national surveys, we find online respondents differ significantly from those in traditional polls but align closely with internet users specifically. Importantly, when weighted appropriately using post-stratification techniques, our findings show that crowdsourced samples can produce statistically similar coefficients to those of standard internet-based studies. This suggests these platforms may effectively represent Chinese netizens for researchers focused on digital populations. The study also investigates potential political and social biases in online responses, revealing notable desirability effects among subjects recruited through Witmart-like mechanisms.
Participants in online surveys and experiments are often inattentive, which can undermine substantive and causal inferences. Many practitioners therefore include multiple factual or instructional closed-ended manipulation checks to identify low-attention respondents. Closed-ended checks are binary (correct or incorrect), which makes them easier to guess and limits variation in measured attention.
Dropping subjects after random assignment because they fail a manipulation check is common across the social sciences, usually intended to restrict estimates to those thought to understand the prompt. This practice can, however, introduce serious bias and obscures what the experiment actually reveals unless handled differently.