Conjoint experiments are popular in political science surveys, but researchers lacked guidance on how many attributes could be included before survey satisficing became problematic. This paper addresses that gap by conducting two pre-registered studies examining choices among hypothetical US Senate candidates or hotel rooms.
Methodology
We employed a two-stage experimental design with Amazon Mechanical Turk and Survey Sampling International respondents:
- First stage identified core attributes uncorrelated with the primary variable of interest (Senate candidate appeal).
- Second stage randomly assigned filler attributes to conjoint profiles, varying their number.
Key Findings
Our results indicate that survey satisficing does occur but remains manageable even when respondents face large numbers of attributes:
- Core quantities of interest remain stable across conditions.
- Response quality improves with higher attribute loads (more options).
Why It Matters
These findings offer practical guidance for designing political science surveys using conjoint experiments. Researchers can now confidently include numerous attributes without crippling their survey's validity.
