Interrater reliability
Descriptive statistics · Method and input reference
Rows are subjects, columns are numeric variables/items/raters. Partial correlation controls for columns 3 onward. Reliability reports nominal agreement for two raters or Fleiss kappa for multiple raters. Synthetic example: 180 cases independently classified by three reviewers; category codes 0–2, including disagreements.
Open calculator · Compare methods in the decision matrix
When to consider it
- Estimate categorical agreement using Cohen’s or Fleiss’ kappa.
Assumptions and scope
- Observations must be independent between subjects; repeated rows from one subject are not independent samples.
- Use the outcome and predictor coding required by the calculator; review missing data and model diagnostics.
Required inputs
- Data matrix (one observation per line; comma-separated columns) — Numeric observations; preserve the row and column structure described above.
Review missing values and data coding before running. Sample inputs are synthetic teaching examples.
Implementation
Lifescio validates inputs and formats results; listed functions are the principal calculation routines, not an exhaustive transitive dependency list.
- statsmodels 0.14.4: statsmodels.stats.inter_rater.fleiss_kappa, statsmodels.stats.inter_rater.cohens_kappa
These are pinned library versions; saved results record the versions actually used at runtime.