More Rigorous Software Engineering Would Improve Reproducibility in Machine Learning Research

Source code for our position paper on software engineering in machine learning. An example template repository for most concepts discussed in the paper is available here.

Getting Started

First of all, we have to clone this repository,

git clone [email protected]:BonnBytes/position_we_need_more_tests_in_ml.git

In the next step, you need to configure an environment to use the code in this project. To do that, create a .env-file with the following content.

PYTHONPATH=.
OPENREVIEW_USERNAME=YOUR_OPENREVIEW_ACCOUNT_NAME
OPENREVIEW_PASSWORD=YOUR_PASSWORD

Reusability

After cloning and navigating into this repository, you can install the code in this repository via pip.

pip install .

Download the data run

To aggregate the statistical data we used for the paper, run the command below.

./run_all.sh

Funding

The Bundesministerium für Bildung und Forschung (BMBF) supported research through its "BNTrAInee" (16DHBK1022) and "WestAI" (01IS22094A) projects.