Replication and Transparency in Empirical Legal Studies

Replication and transparency are fundamental to the advancement of scientific knowledge, including in the field of empirical legal studies. To that end, many political science journals maintain replication archives for their published articles as do many authors. However, the extent to which replication materials are available in this field remains limited. My own experience, supported by observations within the academic community, suggests that few authors or law reviews maintain or provide access to replication materials for their studies. This lack of accessibility poses a significant challenge for scholars who wish to verify results or build upon previous research.

The situation in empirical legal studies is complicated by the unique structure of law reviews, which are often edited by law students rather than permanent faculty. These student-run editorial boards turn over annually, which can lead to a lack of consistent leadership and long-term vision for maintaining archives of empirical research. Unlike traditional academic journals with stable editorial policies, law reviews may not prioritize or have the resources to support the archiving of replication materials.

To address these challenges, it would be beneficial to conduct a comprehensive survey and/or audit of law reviews that publish empirical legal studies to determine the availability of replication materials. Such a study could involve a random selection of law review articles with empirical analysis and determine whether the journal or authors have replication data available and if they are willing to share it. This approach would provide valuable insights into the current state of replication transparency and highlight potential areas for improvement.

Ensuring the availability of replication materials is crucial for the progress of empirical legal studies. Science, including social sciences, relies on cumulative progress, where new studies build on the foundations of previous research. There are now good models for journals and authors to emulate and archival tools, like Dataverse, available to make replication and transparency possible. Of course, law reviews are different than political science journals and it would be helpful to take stock of the current situation to understand the state of transparency and replication in empirical legal research.