Transportability and model updating decisions

Separate evaluation from model modification

Preserve evaluation of the original model. Recalibration, coefficient revision and extension are distinct changes; all produce an updated model whose performance needs a separate validation strategy.

Flow diagram

Transportability and model updating decisionsF1.S1Review original performanceF1.S2Identify the source of mismatchF1.S3Define an updating decisionF1.S4Develop and validate updated modelseparatelyF1.S5Report both model versionsF1.S6Retain original modelF1.S7RecalibrateF1.S8Broader updateF1.N1Updating decision registerConcernEvidencePermittedupdateValidation planNew versionBaseline riskCalibrationslopePredictoreffectsNew predictorsF1.N2Assumptions and rationaleF1.N3Method and protocol evidenceF1.S0 Start flowTransportability and model updatingdecisionsF1.S999 End flowEnd flow

Steps

Review original performance

Use untouched validation outputs and applicability evidence.

Identify the source of mismatch

Assess case mix, measurement, baseline risk and predictor effects.

Define an updating decision

Choose no change, recalibration or a justified broader update.

Develop and validate updated model separately

Record new version and evaluate optimism from updating.

Report both model versions

Report original-model performance separately from recalibration, revision or extension. Identify the data used to update the model and the separate validation of the updated procedure.

Retain original model

Document applicability and unresolved limitations.

Recalibrate

Specify the allowed recalibration and new version.

Broader update

Justify refitting or new predictors and fresh evaluation.

Transportability and model updating decisions

End flow

Tables and supporting information

Updating decision register

Updating decision registerConcernEvidencePermittedupdateValidation planNew versionBaseline riskCalibrationslopePredictoreffectsNew predictors

Updating is a new model-development activity; do not overwrite original results.

Assumptions and rationale

An updated model requires its own evaluation. Reusing validation data to update makes subsequent apparent performance optimistic.

Method and protocol evidence

Attach the protocol, methodological sources and versioned software specification supporting these decisions. Record citations and provenance; do not enter participant identifiers.

Sources