BinFuncGT

It provides function-level ground truth, static feature extraction, candidate retrieval, call-graph-aware structural matching, and evaluation of component and version identification.</p><p><br></p><p dir="ltr">The project, benchmark, and construction utilities are named BinFuncGT.

Python modules and filesystem paths use the lowercase spelling binfuncgt.</p><p><br></p><p dir="ltr">Features</p><p dir="ltr">Binary analysis: Parse ELF metadata, symbol tables, and DWARF information, and recover functions and call graphs with angr CFGFast.</p><p dir="ltr">Function representation: Extract instruction counts, basic-block counts, control-flow edges, call sites, immediate-constant statistics, cyclomatic complexity, and mnemonic histograms.</p><p dir="ltr">Candidate retrieval: Generate candidate function pairs using cosine similarity, a top-r selection rule, and a similarity threshold.</p><p dir="ltr">Structural matching: Combine local similarity, preserved call relationships, and one-to-one matching constraints in a unified optimization objective.</p><p dir="ltr">Reduction and classical solvers: Apply safe reductions and interaction-graph decomposition, with greedy, Hungarian, CP-SAT, simulated annealing, and small-instance exhaustive solvers.</p><p dir="ltr">Function-level evaluation: Measure precision, recall, and F1 against raw and strict ground-truth correspondences.</p><p dir="ltr">Version identification: Explore constant features, string features, differential weighting, and version-discriminative string scoring.</p><p dir="ltr">Statistical analysis: Provide paired comparisons, ranking diagnostics, and Benjamini–Hochberg false discovery rate correction.</p><p dir="ltr">Repository Structure</p><p dir="ltr">The main directories and modules are organized as follows:</p><p><br></p><p>.</p><p dir="ltr">├── configs/ # Benchmark and experiment configurations</p><p dir="ltr">├── data/ # Binary data, intermediate features, and instances</p><p dir="ltr">├── experiments/</p><p dir="ltr">│ ├── pilot/ # Matching, baseline, ablation, and ranking results</p><p dir="ltr">│ │ └── score_cache_v2/ # Per-instance scoring caches</p><p dir="ltr">│ ├── case_study/ # Debian case-study results and reports</p><p dir="ltr">│ └── versiongrid_v3/ # Extended version-grid design and build records</p><p dir="ltr">├── results/ # Aggregate results, statistics, and logs</p><p dir="ltr">├── scripts/ # Experiment and report-generation entry points</p><p dir="ltr">├── src/binfuncgt/</p><p dir="ltr">│ ├── binary/ # ELF parsing, call graphs, and ground truth</p><p dir="ltr">│ ├── data/ # BinFuncGT schemas and instance I/O</p><p dir="ltr">│ ├── features/ # Static features and vectorization</p><p dir="ltr">│ ├── retrieval/ # Candidate function retrieval</p><p dir="ltr">│ ├── matching/ # Matching instances and objectives</p><p dir="ltr">│ ├── reduction/ # Safe and heuristic reductions</p><p dir="ltr">│ ├── solvers/ # Classical optimization methods</p><p dir="ltr">│ ├── scoring/ # Downstream structural scores</p><p dir="ltr">│ ├── evaluation/ # Evaluation metrics</p><p dir="ltr">│ └── feasibility/ # Classical matching and solver statistics</p><p dir="ltr">└── requirements.txt # Pinned Python dependencies</p><p></p><p dir="ltr">Configuration</p><p dir="ltr">The principal benchmark configurations are:</p><p><br></p><p dir="ltr">ConfigurationExperiment setting</p><p dir="ltr">configs/binkit_x86_v2.yamlSame-architecture matching across build variants</p><p dir="ltr">configs/binkit_xarch_v2.yamlCross-architecture matching with the same compiler and optimization level</p><p dir="ltr">configs/pilot_versiongrid_v2.yamlComponent and version comparisons on VersionGrid v2</p><p dir="ltr">configs/pilot_versiongrid_v3.yamlExtended version-grid comparisons</p><p dir="ltr">The x86 configuration name is historical: its pairing rule is arch_relation: same, covering matching architectures represented in the input corpus.</p><p><br></p><p dir="ltr">Common settings include:</p><p><br></p><p dir="ltr">ParameterMain protocol setting</p><p dir="ltr">Candidate count per target functionr = 5</p><p dir="ltr">Cosine-similarity threshold0.5</p><p dir="ltr">Same-architecture feature setfull</p><p dir="ltr">Cross-architecture feature setstructural</p><p dir="ltr">Matching objective weightsalpha = beta = 1</p><p dir="ltr">CP-SAT time budget60 seconds by default in the main scoring runners</p><p dir="ltr">Structural score weights(0.5, 0.3, 0.2)</p><p dir="ltr">Experiment Scripts</p><p dir="ltr">ScriptPurpose</p><p dir="ltr">scripts/rq1_function_gt.pyFunction-level matching evaluation</p><p dir="ltr">scripts/rq1_function_gt_report.pyFunction-level result summaries and paired comparisons</p><p dir="ltr">scripts/run_feasibility.pyStructural matching and per-instance feasibility statistics</p><p dir="ltr">scripts/analyze_results.pyPilot result aggregation</p><p dir="ltr">scripts/score_v2.pyMulti-method benchmark scoring</p><p dir="ltr">scripts/report_v2.pyScoring-cache summaries and statistical comparisons</p><p dir="ltr">scripts/run_baselines.pyClassical baselines through run and summarize subcommands</p><p dir="ltr">scripts/rq5_feature_attribution.pyFeature attribution analysis</p><p dir="ltr">scripts/rq6_stripped.pyEvaluation with stripped target binaries</p><p dir="ltr">scripts/rq7_diff_weighted.pyDifferential feature-weighting experiments</p><p dir="ltr">scripts/rq7_tie_audit.pyTie and candidate-set diagnostics</p><p dir="ltr">scripts/rq7_str_scorer.pyVersion-discriminative string scoring</p><p dir="ltr">scripts/rq7_stats.pyStatistical comparisons of string-based methods</p><p dir="ltr">scripts/fdr_correction.pyBenjamini–Hochberg FDR correction</p><p dir="ltr">scripts/debian_case_study.pyDebian case-study evaluation</p><p dir="ltr">scripts/debian_str_idf.pyString-IDF evaluation on Debian binaries</p></th></tr></table></th></tr></table></pre></th></tr></table></pre><p></p>