Validation Strategy Dominates Model Choice in Chronological QSAR Evaluation

Four standard QSAR model classesrandom forest, support vector machine, XGBoost, and a feed-forward neural networkwere evaluated under the same training-only model-selection workflow. Random validation produced closely aligned internal and external estimates, with mean MCC values of 0.75 and 0.72, respectively, while random thinning gave essentially unchanged validation performance (MCC 0.73). Structural filtering reduced validation MCC to 0.48 without changing the corresponding training-only CV estimate.

Chronological validation produced a much larger generalization gap, with mean CV MCC of 0.73 and validation MCC of 0.19; additional structural filtering reduced validation MCC to 0.15. Model-class comparisons showed substantial overlap among classifiers, and no standard algorithm consistently overcame the chronological validation penalty. These results indicate that validation-set definition has a dominant influence on estimated QSAR generalization and that chronological data set shift is not reproduced by random or structure-only validation.