Data Sheet 1_Predicting disease progression under a “treatment-naive” counterfactual scenario in a competing risks framework.docx

Using the g-formula, time to the two competing events was generated under a counterfactual scenario in which no patients received prostatectomy. The performance of the MSKCC model developed in the TPCP cohort and applied to the counterfactual cohort (applied model) and the model updated under this counterfactual scenario (updated model) were compared.</p>Results<p>At 5 years post-diagnosis, the updated model had an Area Under the Curve (AUC) of 0.84 (95% CI: 0.82; 0.85) and the applied model had an AUC of 0.83 (95% CI: 0.81, 0.85).

The calibration slope for the updated model was 1.02 (95% CI: 0.99, 1.12) compared to 1.07 (95% CI: 0.97, 1.17) for the applied model. Calibration intercepts were 0.00 (95% CI: −0.10, 0.10) and 0.05 (95% CI: −0.04, 0.15) respectively. The predicted probabilities of the outcome of interest under counterfactual treatment strategy varied not only because of the treatment itself, but also due to its impact on competing risks.</p>Conclusion<p>When a prediction model cannot be validated in a new population due to treatment shifts, counterfactual risks under hypothetical regimens can still be cautiously estimated.

In particular, counterfactual modeling should be considered as a useful complementary assessment of model transportability and calibration under clinically relevant treatment scenarios.</p>