Table · dataset · 2026
replication package
Listed in figshare and Loughborough Research Repository — shown once because both records carry DOI 10.6084/m9.figshare.33959791.v1
Description
<pre># Replication package<br><br>## Intelligent Agile Adoption Framework — archival demonstration (Section 5)<br><br>Khaled Ismail · ORCID 0000-0002-0499-5805<br><br>### What this tests<br><br>Proposition 2 of the manuscript is separated into two claims and both are tested here.<br><br>* **P2a** — the dominant component of squared forecast error in human Agile estimation is<br>dispersion, not bias. **Supported.**<br>* **P2b** — predictive inference reduces that dispersion component. **Not supported** on this<br>corpus, in a result consistent with independent replications (Tawosi et al., 2023, 2024).<br><br>Pooled human forecast error: SD 2.358 log-units, mean +0.071; the central 80% of issues are<br>realized at 0.05x to 13.5x their planned duration.<br><br>A third analysis (`p2\_conformal.py`) compares interval calibration methods on identical<br>predictions.
Quantile regression attains **64.3%** coverage on a nominal 80% interval;<br>locally adaptive split conformal attains **76.6%**, within 5 points of nominal in 21 of 24<br>projects (Wilcoxon p = 5.2e-6, rank-biserial 0.91), at a 28% cost in interval width. The<br>severe under-coverage is therefore largely an estimator artifact rather than an irreducible<br>property of the data; the residual 3.4-point shortfall (p = 0.039) indicates mild<br>non-exchangeability, i.e. process drift.<br><br>### Data<br><br>TAWOS — a versatile dataset of agile open source software projects (Tawosi, Al-Subaihin,<br>Moussa \& Sarro, MSR 2022), DOI 10.1145/3524842.3528029.
Read the rest (4 more)
Dataset DOI 10.5522/04/21308124,<br>Apache License 2.0. 458,232 issues, 39 projects, 12 public Jira repositories.<br><br>The dataset ships as a MySQL dump. `extract.py` stream-parses it directly and requires no<br>database server.<br><br>### Pipeline<br><br>```<br>python3 extract.py # TAWOS.sql -> issues.csv (458,232 rows)<br>python3 p2\_analysis.py primary # total-effort outcome, revised estimates excluded<br>python3 p2\_analysis.py sens\_resolution # resolution-time sensitivity<br>python3 p2\_conformal.py # conformal vs quantile interval calibration<br>python3 delegation\_rule.py # regenerates Table 3, Table 4 and Table S6<br>python3 figs.py # regenerates Figures 1, 2, 3 and S1<br>```<br><br>### Analysed sample<br><br>|Specification|Projects|Predictions|<br>|-|-|-|<br>|primary (Total\_Effort\_Minutes)|25|31,356|<br>|sensitivity (Resolution\_Time\_Minutes)|32|43,536|<br><br>Inclusion: story point > 0; outcome > 0; story point not revised after estimation;<br>at least 150 prior issues before the first prediction; at least 60 evaluable predictions<br>per project.<br><br>### Method<br><br>Error is taken in log space, e = ln(actual) − ln(predicted), giving the exact decomposition<br>MSLE = mean(e)² + var(e) into a bias and a dispersion term.
Three forecasters are evaluated<br>strictly out of sample under a time-ordered walk-forward within each project:<br><br>* **H** human — story point × historical minutes-per-point rate (velocity planning)<br>* **R** reference class — historical median duration of prior same-type issues<br>* **M** model — gradient-boosted quantile regression at the 0.10 / 0.50 / 0.90 quantiles<br><br>Inference across projects uses Wilcoxon signed-rank tests, 10,000-sample bootstrap<br>confidence intervals, and matched-pairs rank-biserial effect sizes.
The unit of analysis is<br>the project (n = 25), not the issue.<br><br>### Known limitations<br><br>* The outcome is a **duration proxy** derived from issue state transitions, not logged<br>effort. Only 900 issues in the corpus carry logged work, too few to analyse.<br>* The corpus contains **none of the Layer 1 telemetry** the framework specifies — no CI<br>outcomes, work-in-progress levels, dependency depth, review latency or team composition<br>history.
The P2b test is therefore a joint test of P1 and P2b, and a null is what P1<br>predicts under thin telemetry.<br>* The model arm is one model class with one feature set.<br><br>### Environment<br><br>Python 3.13 · numpy 2.4 · pandas 3.0 · scipy 1.17 · scikit-learn 1.8 · matplotlib.<br>Random seed 20260921 for all bootstrap resampling.<br><br>### Figure palette<br><br>The two-colour categorical palette (#2E6FA8, #C2681A) was validated for colour-vision<br>deficiency separation, lightness band, chroma floor and surface contrast before use.<br><br><br><br></pre><p></p>
Links
Where it is published
- DOI doi.org/10.6084/m9.figshare.33959791.v1 ↗
DOI / persistent id · from figshare com
Catalogue records · 1
- OAI-PMH record api.figshare.com/v2/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai%3Af… ↗
metadata API · from figshare com
Topics
- From keywords
- Astronomy & Astrophysics · Chemistry · Chemistry · Computer Science & AI · Computer Science & AI · Earth & Environmental Science · Earth & Environmental Science · Economics & Finance · Economics & Finance · Engineering · Engineering · Engineering practice · Engineering practice · Humanities · Humanities · Life Sciences · Life Sciences · Materials Science · Mathematics & Statistics · Medicine & Health · Medicine & Health · Ocean & Atmospheric Science · Psychology & Behavioral Science · Social Science · Social Science · Software architecture · Software architecture · Systems engineering · Systems engineering
- Inferred from text
- Tabular 65%
Provenance · 2 source records, 35 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| figshare | oai:figshare.com:article/33959791 | 5 d ago | JSON v1 |
| Loughborough Research Repository | oai:figshare.com:article/33959791 | 5 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].anzsrc:field:401003 | mapping · figshare com | vocabulary-mapper@1.0.0 | keywords['Engineering practice'] |
| concepts[field].anzsrc:field:401003 | mapping · repository lboro ac uk | vocabulary-mapper@1.0.0 | keywords['Engineering practice'] |
| concepts[field].anzsrc:field:401006 | mapping · figshare com | vocabulary-mapper@1.0.0 | keywords['Systems engineering'] |
| concepts[field].anzsrc:field:401006 | mapping · repository lboro ac uk | vocabulary-mapper@1.0.0 | keywords['Systems engineering'] |
| concepts[field].anzsrc:field:461206 | mapping · figshare com | vocabulary-mapper@1.0.0 | keywords['Software architecture'] |
| concepts[field].anzsrc:field:461206 | mapping · repository lboro ac uk | vocabulary-mapper@1.0.0 | keywords['Software architecture'] |
| concepts[field].local:field:astronomy | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:chemistry | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:chemistry | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:computer-science-ai | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:computer-science-ai | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:earth-environmental | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:earth-environmental | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:economics-finance | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:economics-finance | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:engineering | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:engineering | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:humanities | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:humanities | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:life-sciences | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:life-sciences | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:materials-science | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:mathematics-statistics | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:medicine-health | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:medicine-health | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:ocean-atmospheric | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:psychology-behavioral | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:social-science | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:social-science | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[modality].local:modality:tabular | enrichment · figshare com | keyword-concept-rules@1.0.0 | title+description (65%) |
| description | source · figshare com | connector:figshare_com@1.0.0 | /metadata/dc/description |
| license | source · figshare com | connector:figshare_com@1.0.0 | /metadata/dc/rights |
| publication_date | source · figshare com | connector:figshare_com@1.0.0 | |
| title | source · figshare com | connector:figshare_com@1.0.0 | /metadata/dc/title |