Better Policy Measurement: Log Odds-Ratio Scaling vs. Current Word Count Approaches

This article introduces an improved method for measuring policy preferences from political texts by using log odds-ratios instead of simple word frequencies.

New Approach Proposed

The authors suggest a more theoretically grounded and linguistically sound alternative scaling technique based on the logarithm of odds-ratios. This approach offers several advantages over traditional methods:

• More accurate representation due to weighting schemes that account for semantic salience
• Better theoretical alignment with political science concepts
• Fewer statistical artifacts in constructed scales

Comparison With Existing Methods

They directly contrast their proposed Logit Scale with the established methodology used by the Comparative Manifesto Project (CMP), highlighting key flaws in CMP's current approach:

• Over-reliance on word counts without accounting for contextual meaning and importance
• Potential to amplify measurement errors over time or across texts
• Inability to effectively capture nuanced policy shifts

Validation Process

The proposed scale was rigorously tested through independent expert surveys, confirming its validity.

Data Expansion Potential

Applying this new methodology to CMP's existing dataset demonstrates:

• Ability to identify more distinct and meaningful policy dimensions
• Measurability across additional years of data previously unavailable

Call for Future Research

The authors conclude by advocating for the adoption of log odds-ratio scaling in political text analysis, suggesting it represents a significant advancement beyond current coding practices.