This article evaluates Google Consumer Surveys (GCS) as a low-cost option for rigorous social science. The platform’s relative strengths and weaknesses make it most useful for researchers who identify causality through randomization to treatment groups rather than through selection on observables. The real cost advantage of GCS over alternatives is largely confined to short surveys with only a few questions.
🧪 How the Platform Was Tested
- Replicated four canonical social science experiments and one study focused on treatment heterogeneity using GCS.
- Assessed balance across randomized treatment groups, the feasibility of manipulation checks, and the quality of provided inferred demographics for weighting and heterogeneity analyses.
- Tracked whether each replication reproduced the usual directional finding reported in the original studies.
🔍 Key Findings
- Randomization on GCS achieved adequate balance across treatment groups for experimental inference.
- Treatment heterogeneity can be explored on the platform, and manipulation checks are feasible within short instruments.
- Inferred demographic variables provided by GCS appear sufficiently reliable for weighting and exploratory heterogeneity tests in many cases.
- The platform’s cost advantage is real but limited: savings accrue mainly for brief surveys with few questions.
- Given these trade-offs, GCS performs best for research that relies on randomized assignment to identify causal effects rather than for research that depends on selection on observables.
- Crucially, the usual directional finding from each replicated experiment was reproduced using GCS.
⚖️ Why It Matters
- GCS is a practical and affordable platform for survey experimentalists seeking randomized causal inference, especially for quick, short-question studies. Researchers should, however, be cautious about using GCS for longer instruments or for studies that require strong observational covariate adjustment.
