Small Coding Changes, Big Consequences for CoW MID Findings

🔎 What Palmer et al. Claimed

Palmer, D’Orazio, Kenwick, and McManus (PDKM) evaluated a minority of revisions made to the Correlates of War (CoW) Militarized Interstate Disputes (MID) dataset. PDKM agreed with most of the specific changes they reviewed but argued that none of those changes — nor the thousands of other revisions not reviewed — would alter scientific findings.

🧾 Why some dispute cases should be excluded

Principles of sound measurement and dataset construction require excluding dispute cases that cannot be substantiated by the historical record or that fail to meet the dataset’s coding rules. This response documents that many disputed cases fall into those categories and therefore should be removed from analysis rather than retained.

📊 How many cases remain unexamined

• More than ten thousand changed values remain unexamined by PDKM.
• The scope of these unreviewed edits matters because they collectively shape variable distributions and sample composition in studies that rely on CoW MIDs.

📈 How measurement differences change empirical conclusions

Contrary to PDKM’s claim, differences between the original and revised CoW MID data materially affect prior empirical results. Returning to prior replication exercises shows that using the corrected coding changes substantive inference in several published analyses.

⚠️ Replication errors in PDKM’s critique

PDKM’s replications contained substantial errors in implementation and interpretation. Those replication errors led PDKM to incorrect inferences about the sensitivity of past results to the coding changes. Correcting the replication mistakes demonstrates that data differences do, in fact, alter empirical conclusions.

🧭 Why it matters

Accurate measurement and rigorous replication are essential to cumulative knowledge in international conflict research. Discarding unsubstantiated cases, fully examining the thousands of revised values, and fixing replication mistakes change both data quality and substantive conclusions drawn from the CoW MID dataset.