Supplementary file 1_Explainable AI for phishing URL detection: a Bayesian-optimized stacking ensemble framework with SHAP-guided feature learning.zip

A four-layer statistical validation protocol — comprising a Friedman omnibus test, Wilcoxon signed-rank tests, paired t-tests, and Cohen's d effect sizes — was applied to five-fold cross-validation accuracy distributions to assess directional consistency, with the limited inferential resolution of five folds explicitly acknowledged.</p>Results<p>SHAP-driven selection reduced the feature space by 41.5% while retaining 95% of predictive signal.

The stacking ensemble achieved 96.75% accuracy, 96.74% F1-score, and AUC of 0.9947, attaining the lowest Brier score among all 13 models (0.0246), indicating superior probability calibration. The Friedman omnibus test confirmed significant performance differences across models (χ²F = 59.84, p < 0.0001), and all 12 Wilcoxon pairwise comparisons yielded the minimum attainable p-value (p = 0.0313), confirming the ensemble never lost a cross-validation fold against any baseline.

Post-hoc SHAP analysis identified subdomain structure, URL length, and URL entropy as the dominant phishing indicators at both ensemble and base-learner levels.</p>Discussion<p>The co-leaders — the stacking ensemble and Extra Trees — demonstrate that rigorous, interpretable AI pipelines can advance phishing detection accuracy and transparency simultaneously. The framework's calibrated risk scores, threshold flexibility, and multi-level SHAP explainability support analyst-facing decision-making in security operations, while its leakage-free</p>