Data linkage and harmonization

Source datasets → identifiers and linkage → matched and unmatched records → harmonized data

Flow diagram

Data linkage and harmonizationData linkage and harmonizationF1.S1Source datasets and identifiersF1.S2Normalize and deduplicateidentifiersF1.S3Apply linkage rulesF1.S4Accepted matchesF1.S5Ambiguous matches for reviewF1.S6Unmatched recordsF1.S7Harmonize linked variablesF1.S8Validate linkage and coverageF1.N1Linkage diagnosticsSourcepairRuleCandidateunitAcceptedAmbiguousUnmatchedValidationSourcepairF1.N2Linkage specification and provenance

Steps

Source datasets and identifiers

Specify versions, linkage permissions and identifier availability.

Normalize and deduplicate identifiers

Define uniqueness within each source before matching.

Apply linkage rules

Specify deterministic or probabilistic matching and decision thresholds.

Accepted matches

Record one-to-one or other allowed relationships.

Ambiguous matches for review

Resolve or retain ambiguity with documented rules.

Unmatched records

Preserve provenance and assess differential linkage failure.

Harmonize linked variables

Resolve conflicting measures, codes and units.

Validate linkage and coverage

Assess false matches, missed matches and differences between linked and unlinked records.

Tables and supporting information

Linkage diagnostics

Linkage diagnosticsSourcepairRuleCandidateunitAcceptedAmbiguousUnmatchedValidationSourcepair

Keep record and person counts distinct; many-to-many linkage is not a simple population split.

Linkage specification and provenance

Record linkage algorithm/version, permissions and reference definitions.

Sources