New evidence challenges old concerns about survey experiments conducted on Mechanical Turk. Results obtained through these online platforms appear to generalize well across diverse populations.
Replication studies conducted over the last fifteen years demonstrate that findings from convenience samples are often similar to those derived from national surveys. This suggests either treatments elicit consistent responses regardless of subject type or that convenience and representative samples share common characteristics relevant to treatment effects.
Despite differences in background demographics among participants, treatment effect heterogeneity remains relatively limited within these experiments. These findings offer reassurance about the validity of results obtained through MTurk studies.
