Data · dataset · 2026
Replication package for "When Does LLM Test Repair Destroy Bug-Revealing Tests? A Cost-Accounted Study on Real Repository Defects"
Listed in NCL Data
<p dir="ltr">Replication package for an empirical study of how iterative LLM test repair destroys verified bug-revealing (fail-to-pass) tests on real repository defects (SWT-bench Verified, 229 instances, three LLMs).
Description
It contains the study protocol with its dated amendments, the Docker-free execution harness, every LLM request and response with provider-reported cost, every recorded test version with its outcomes on the buggy and the fixed program version, the test-execution logs, the model-based audit samples, the official-harness cross-check, and the scripts that regenerate every table, figure and in-text number of the paper.
Benchmark metadata are not redistributed: src/fetch_data.py downloads them from Hugging Face, and work/data_checksums.json verifies them. Code: MIT. Data: CC BY 4.0.</p>
Links
Where it is published
- DOI doi.org/10.6084/m9.figshare.34068783.v1 ↗
DOI / persistent id · from data ncl ac uk
Catalogue records · 1
- OAI-PMH record api.figshare.com/v2/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai%3Af… ↗
metadata API · from data ncl ac uk
Topics
- From keywords
- Computer Science & AI · Earth & Environmental Science · Engineering · Humanities · Software testing, verification and validation
- Inferred from text
- Tabular 65%
Provenance · 1 source records, 11 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| NCL Data | oai:figshare.com:article/34068783 | 3 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · data ncl ac uk | connector:data_ncl_ac_uk@1.0.0 | |
| concepts[field].anzsrc:field:461208 | mapping · data ncl ac uk | vocabulary-mapper@1.0.0 | keywords['Software testing, verification and validation'] |
| concepts[field].local:field:computer-science-ai | mapping · data ncl ac uk | connector:data_ncl_ac_uk@1.0.0 | |
| concepts[field].local:field:earth-environmental | mapping · data ncl ac uk | connector:data_ncl_ac_uk@1.0.0 | |
| concepts[field].local:field:engineering | mapping · data ncl ac uk | connector:data_ncl_ac_uk@1.0.0 | |
| concepts[field].local:field:humanities | mapping · data ncl ac uk | connector:data_ncl_ac_uk@1.0.0 | |
| concepts[modality].local:modality:tabular | enrichment · data ncl ac uk | keyword-concept-rules@1.0.0 | title+description (65%) |
| description | source · data ncl ac uk | connector:data_ncl_ac_uk@1.0.0 | /metadata/dc/description |
| license | source · data ncl ac uk | connector:data_ncl_ac_uk@1.0.0 | /metadata/dc/rights |
| publication_date | source · data ncl ac uk | connector:data_ncl_ac_uk@1.0.0 | |
| title | source · data ncl ac uk | connector:data_ncl_ac_uk@1.0.0 | /metadata/dc/title |