Foundations of Open Science, Reproducibility and Replicability

Why This Course Exists¶

Modern research increasingly depends on code, data pipelines, and computational analyses. Yet a large fraction of published results cannot be reproduced or replicated, even by the original authors. This lecture introduces the vocabulary, motivation, and guiding principles that the rest of the course builds on: open science, reproducibility, and replicability.

The Reproducibility and Replicability Crisis¶

Concerns about the reliability of published research are not new. Ioannidis argued that, under common statistical and publication practices, a surprising share of published findings are likely to be false Ioannidis, 2005. A decade later, a large-scale effort to replicate 100 studies in psychology succeeded in reproducing the original results for well under half of them Open Science Collaboration, 2015. These findings, echoed across other fields, motivated a 2019 consensus report from the U.S. National Academies of Sciences, Engineering, and Medicine that examined the scope of the problem and proposed concrete recommendations for researchers, institutions, and funders National Academies of Sciences, Engineering, and Medicine, 2019.

Reproducibility vs. Replicability¶

These two terms are often used interchangeably, but this course follows the distinction adopted by the National Academies report National Academies of Sciences, Engineering, and Medicine, 2019:

In other words, reproducibility is about computational transparency — can someone else re-run your pipeline and get your numbers? Replicability is about scientific robustness — does the underlying effect hold up when the study is repeated independently?

Open Science¶

Open science is the broader movement to make the entire research lifecycle — data, code, materials, and publications — transparent and accessible. Munafò et al. propose a manifesto of concrete practices that address the reproducibility crisis directly, including pre-registration, sharing of data and code, and reporting guidelines Munafò et al., 2017.

A widely adopted framework for making research outputs usable by others is the FAIR principles Wilkinson et al., 2016:

Note that FAIR does not require data to be open — it is possible for data to be FAIR yet access-restricted (e.g., for privacy reasons). This is an important nuance: open science and open data are related but distinct from FAIRness.

What This Course Will Cover¶

Over the following lectures and labs, we will move from these foundational concepts to hands-on practice:

1. Version control and collaborative workflows (Git/GitHub)

2. Structuring reproducible computational environments

3. Data and code sharing practices, licensing, and persistent identifiers

4. Reporting standards and pre-registration

5. Building a fully reproducible research artifact from end to end

Further Resources¶

• The Turing Way — Reproducible Research — The canonical community handbook.

• Open Science Training Handbook — 12 chapters, funded by the EU.

• NASA Open Science 101 — Free 5-module course, ~12 hours.

• awesome-reproducible-research — Curated meta-list of tools, papers, and courses.

• INCF Training Space — Standards and project management.