← Search

Data · dataset · 2026

Evaluation Dataset for Bounded LLM Adjudication in Hybrid Malware Triage

Listed in Teesside University Research Data Repository

Supplementary evaluation data for the paper "When Should the LLM Decide?

Description

Bounded Adjudication and Claim-Level Verification for Hybrid Malware Triage" (Shaw, 2026). Contains the frozen evidence bundles (deterministic tool findings, extracted IOCs, EMBER classifier scores, and gate decisions) that three LLM adjudicators — two local models (qwen3:8b, gemma4:12b) and one frontier model (Gemini 3.1 Pro) — were shown during evaluation, the dataset manifest (sha256/label/source/split, no filesystem paths), and the raw per-sample experiment outputs across all four evaluation axes (main accuracy, adversarial robustness, calibration/reliability, claim-level verification), plus the aggregated statistics report the paper's tables and figures are drawn from.

No malware executables are included — only derived metadata, extracted structural/behavioral findings, and model output. All indicators of compromise (URLs, IP addresses) that were copied verbatim out of real malware samples during analysis have been defanged (hxxp://, bracketed dots) before deposit; see DATASET_CARD.md for the exact methodology. Full schema and provenance documentation is in DATASET_CARD.md at the root of this deposit.

Read the rest (1 more)

Code, evaluation harness, and reproduction instructions: github.com/AnimeshShaw/mal-agent

Links

Where it is published

Catalogue records · 1

Topics

Provenance · 1 source records, 9 field assertions
SourceKeyLast seenRaw
Teesside University Research Data Repositoryoai:data.mendeley.com/9ttn7c2nph.12 d agoJSON v1
FieldAssertionExtractorEvidence
access_levelsource · researchdata tees ac ukconnector:researchdata_tees_ac_uk@1.0.0
concepts[field].anzsrc:group:4602mapping · researchdata tees ac ukvocabulary-mapper@1.0.0keywords['Artificial Intelligence']
concepts[field].anzsrc:group:4611mapping · researchdata tees ac ukvocabulary-mapper@1.0.0keywords['Machine Learning']
concepts[field].local:field:computer-science-aimapping · researchdata tees ac ukconnector:researchdata_tees_ac_uk@1.0.0
concepts[field].local:field:earth-environmentalmapping · researchdata tees ac ukconnector:researchdata_tees_ac_uk@1.0.0
descriptionsource · researchdata tees ac ukconnector:researchdata_tees_ac_uk@1.0.0/metadata/dc/description
licensesource · researchdata tees ac ukconnector:researchdata_tees_ac_uk@1.0.0/metadata/dc/rights
publication_datesource · researchdata tees ac ukconnector:researchdata_tees_ac_uk@1.0.0
titlesource · researchdata tees ac ukconnector:researchdata_tees_ac_uk@1.0.0/metadata/dc/title