Data · dataset · 2026
Evaluation Dataset for Bounded LLM Adjudication in Hybrid Malware Triage
Listed in Teesside University Research Data Repository
Supplementary evaluation data for the paper "When Should the LLM Decide?
Description
Bounded Adjudication and Claim-Level Verification for Hybrid Malware Triage" (Shaw, 2026). Contains the frozen evidence bundles (deterministic tool findings, extracted IOCs, EMBER classifier scores, and gate decisions) that three LLM adjudicators — two local models (qwen3:8b, gemma4:12b) and one frontier model (Gemini 3.1 Pro) — were shown during evaluation, the dataset manifest (sha256/label/source/split, no filesystem paths), and the raw per-sample experiment outputs across all four evaluation axes (main accuracy, adversarial robustness, calibration/reliability, claim-level verification), plus the aggregated statistics report the paper's tables and figures are drawn from.
No malware executables are included — only derived metadata, extracted structural/behavioral findings, and model output. All indicators of compromise (URLs, IP addresses) that were copied verbatim out of real malware samples during analysis have been defanged (hxxp://, bracketed dots) before deposit; see DATASET_CARD.md for the exact methodology. Full schema and provenance documentation is in DATASET_CARD.md at the root of this deposit.
Read the rest (1 more)
Code, evaluation harness, and reproduction instructions: github.com/AnimeshShaw/mal-agent
Links
Where it is published
- DOI doi.org/10.17632/9ttn7c2nph.1 ↗
DOI / persistent id · from researchdata tees ac uk
Catalogue records · 1
- OAI-PMH record data.mendeley.com/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai%3Adata… ↗
metadata API · from researchdata tees ac uk
Topics
Provenance · 1 source records, 9 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Teesside University Research Data Repository | oai:data.mendeley.com/9ttn7c2nph.1 | 2 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | |
| concepts[field].anzsrc:group:4602 | mapping · researchdata tees ac uk | vocabulary-mapper@1.0.0 | keywords['Artificial Intelligence'] |
| concepts[field].anzsrc:group:4611 | mapping · researchdata tees ac uk | vocabulary-mapper@1.0.0 | keywords['Machine Learning'] |
| concepts[field].local:field:computer-science-ai | mapping · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | |
| concepts[field].local:field:earth-environmental | mapping · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | |
| description | source · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | /metadata/dc/description |
| license | source · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | /metadata/dc/rights |
| publication_date | source · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | |
| title | source · researchdata tees ac uk | connector:researchdata_tees_ac_uk@1.0.0 | /metadata/dc/title |