Prediction internal validation

Estimate optimism using appropriate resampling

Repeat imputation, transformations, feature selection and tuning in the appropriate resampling loops. Respect repeated patients, hospitals and chronology. Internal validation estimates optimism; it does not demonstrate performance in a new setting.

Flow diagram

Prediction internal validationF1.S1Choose resampling designF1.S2Respect clustering and chronologyF1.S3Repeat the full modeling pipelineF1.S4Estimate optimism and uncertaintyF1.S5Report validation limitationsF1.N1Resampling specificationElementProcedureUnitRepetitionsrationaleOutputOuterevaluationInner tuningPreprocessingOptimismassessmentF1.N2Assumptions and rationaleF1.N3Method and protocol evidenceF1.S0 Start flowPrediction internal validationF1.S999 End flowEnd flow

Steps

Choose resampling design

Justify bootstrap, cross-validation or other internal validation for the data structure.

Respect clustering and chronology

Keep related records together and preserve temporal separation when needed.

Repeat the full modeling pipeline

Repeat imputation, transformations, predictor selection and tuning within appropriate training or nested resampling loops. Evaluate predictions without reusing their outcomes for fitting decisions.

Estimate optimism and uncertainty

Define performance measures, correction and interval methods.

Report validation limitations

Distinguish internally validated performance from external transportability.

Prediction internal validation

End flow

Tables and supporting information

Resampling specification

Resampling specificationElementProcedureUnitRepetitionsrationaleOutputOuterevaluationInner tuningPreprocessingOptimismassessment

Specify repetitions and seeds as study inputs; avoid fixed defaults.

Assumptions and rationale

Internal validation does not establish performance in new institutions, populations or later periods.

Method and protocol evidence

Attach the protocol, methodological sources and versioned software specification supporting these decisions. Record citations and provenance; do not enter participant identifiers.

Sources