Forecasting the Maintained Score from the OpenSSF Scorecard: A Study of GitHub Repositories Linked to PyPI Packages

It contains all code, configuration files, input datasets, and generated outputs required to reproduce the study’s experiments on forecasting the OpenSSF Maintained score for GitHub Repositories linked to PyPI libraries.</p><p dir="ltr"><b>Note:</b> The replication package is distributed in ZIP-compressed form, as the total size of the uncompressed contents is approximately 10.5 GB.</p><h2 dir="ltr">Contents</h2><ul><li><code><strong>00_testing/</strong></code><br>End-to-end experimental pipeline covering repository filtering, maintenance score computation, data processing and sampling, time-series analysis, and plotting.

Experiments are driven by configurable JSON setups.</li><li><code><strong>01_input/</strong></code><br>Input datasets, including:</li><li><ul><li>Aggregated PyPI and GitHub ecosystem metrics (JSON)</li><li>Preprocessed relational dataset with computed maintenance scores (Parquet)</li></ul></li><li><code><strong>05_implementation/</strong></code><br>Model implementations and orchestration logic, including:</li><li><ul><li>Random Forest</li><li>Long Short-Term Memory (LSTM)<br>This directory also contains shared utilities and training/evaluation workflows.</li></ul></li><li><code><strong>06_results/</strong></code><br>Reproducible outputs such as JSON metrics, averaged and iterated confusion matrices, visualizations, and analysis notebooks.</li></ul><h2 dir="ltr">Reproduction Instructions</h2><ol><li>Create a virtual environment and install dependencies from <code>requirements.txt</code>.</li><li>Run experiments via <code>00_testing/main.py</code> using a selected configuration file, for example:<pre><pre>python 00_testing/main.py --config 00_testing/config/complete_lstm.json</pre></pre></li><li>Results are written to <code>06_results/</code> as JSON files and plots.</li></ol><p dir="ltr">Configuration files allow control over:</p><ul><li>Sampling strategies (unbalanced, balanced, hybrid)</li><li>Forecasting type (single-step, multi-step iterative, multi-step direct)</li><li>Sequence lengths</li><li>Time windows</li></ul><h2 dir="ltr">Models and Evaluation</h2><ul><li><b>Models</b></li><li><ul><li>Random Forest</li><li>LSTM</li></ul></li><li><b>Prediction Targets</b></li><li><ul><li><i>Daily mode</i>: absolute maintenance score (0–10)</li><li><i>Slope mode</i>: month-to-month change in maintenance score (−10 to +10)</li></ul></li><li><b>Evaluation Metrics</b></li><li><ul><li>Regression: MSE, RMSE, MAE, R²</li><li>Classification: accuracy, precision, recall, F1</li><li>Confusion matrices aggregated across iterations and parameter grids</li></ul></li></ul><h2 dir="ltr">Data Notes</h2><ul><li>Input JSON files aggregate ecosystem activity and repository-level metrics.</li><li>Parquet files provide the prepared relational dataset with computed maintenance scores.</li><li>Notebooks in <code>06_results/</code> document analyses (e.g., boxplots, bucketed confusion matrices and heatmaps) corresponding to figures presented in the paper.</li></ul><p><br></p>