Title: Dataset of A Large-scale Study about Quality and Reproducibility of Jupyter Notebooks / Understanding and Improving the Quality and Reproducibility of Jupyter Notebooks

Type Dataset João Felipe, Leonardo, Vanessa, Juliana (2019): Dataset of A Large-scale Study about Quality and Reproducibility of Jupyter Notebooks / Understanding and Improving the Quality and Reproducibility of Jupyter Notebooks. Zenodo. Dataset. https://zenodo.org/record/3519618

Authors: João Felipe (Pimentel) ; Leonardo (Murta) ; Vanessa (Braganholo) ; Juliana (Freire) ;

Links

• Item record in Zenodo
• Digital object URL

Summary

The self-documenting aspects and the ability to reproduce results have been touted as significant benefits of Jupyter Notebooks. At the same time, there has been growing criticism that the way notebooks are being used leads to unexpected behavior, encourages poor coding practices and that their results can be hard to reproduce. To understand good and bad practices used in the development of real notebooks, we analyzed 1.4 million notebooks from GitHub. Based on the results, we proposed and evaluated Julynter, a linting tool for Jupyter Notebooks.

Papers:

This repository contains three files:

Reproducing the Notebook Study

The db2020-09-22.dump.gz file contains a PostgreSQL dump of the database, with all the data we extracted from notebooks. For loading it, run:

Note that this file contains only the database with the extracted data. The actual repositories are available in a google drive folder, which also contains the docker images we used in the reproducibility study. The repositories are stored as content/{hash_dir1}/{hash_dir2}.tar.bz2, where hash_dir1 and hash_dir2 are columns of repositories in the database.

For scripts, notebooks, and detailed instructions on how to analyze or reproduce the data collection, please check the instructions on the Jupyter Archaeology repository (tag 1.0.0)

The sample.tar.gz file contains the repositories obtained during the manual sampling.

Reproducing the Julynter Experiment

The julynter_reproducility.tar.gz file contains all the data collected in the Julynter experiment and the analysis notebooks. Reproducing the analysis is straightforward:

The collected data is stored in the julynter/data folder.

Changelog

2019/01/14 - Version 1 - Initial version 2019/01/22 - Version 2 - Update N8.Execution.ipynb to calculate the rate of failure for each reason 2019/03/13 - Version 3 - Update package for camera ready. Add columns to db to detect duplicates, change notebooks to consider them, and add N1.Skip.Notebook.ipynb and N11.Repository.With.Notebook.Restriction.ipynb. 2021/03/15 - Version 4 - Add Julynter experiment; Update database dump to include new data collected for the second paper; remove scripts and analysis notebooks from this package (moved to GitHub), add a link to Google Drive with collected repository files

More information

• DOI: 10.5281/zenodo.3519618
• Language: en

Subjects

• jupyter notebook, github, reproducibility

Dates

• Publication date: 2019
• Issued: March 13, 2019

Rights

• https://creativecommons.org/licenses/by/4.0/legalcode Creative Commons Attribution 4.0 International
• info:eu-repo/semantics/openAccess Open Access

Format

electronic resource

Relateditems