Many political scientists measure dispositions—like populism, conspiracism, racism, and sexism—using agree/disagree statements, but that format can produce acquiescence bias that inflates endorsement rates. This article asks how acquiescence affects criterion validity and evaluates common remedies. Drawing on 160 tests and an internal meta-analysis, the authors examine how bias changes estimated relationships between constructs and assess balanced scales and measurement models as potential solutions. They end with design advice and recommendations for redesigning several popular measures of political and psychological dispositions.
Bolded scanables: agree/disagree format, 160 tests, internal meta-analysis, balanced scales and measurement models
Data and methods
The authors compile evidence from 160 tests and conduct an internal meta-analysis to study the impact of acquiescence bias on measurement. The abstract identifies the study corpus (160 tests) and reports an internal meta-analytic approach; it also evaluates alternative measurement strategies such as balanced scales and explicit measurement models.
Main findings
Across the 160 tests the authors report that acquiescence bias can severely distort the magnitude of relationships between constructs and, in some cases, produce sign errors. Their meta-analysis indicates that the size of acquiescence effects varies across samples and constructs. When they evaluate balanced scales and measurement models, they find that estimated relationships between constructs are sensitive to these measurement choices.
- Acquiescence can inflate endorsement rates in agree/disagree items.
- Bias can both change effect sizes and, in some tests, reverse signs.
- Balanced scales and measurement models alter estimated relationships, but sensitivity depends on sample and construct.
Practical implications and recommendations
Based on these results, the authors offer survey-design advice and specific recommendations for redesigning several popular measures of political and psychological dispositions, arguing researchers should explicitly consider acquiescence when constructing and analyzing agree/disagree scales.
