Rater experience influences reliability and validity of the brief international classification of functioning, disability, and health core set for stroke

Objective: To investigate how clinical experience and access to patient information regarding functional capability influence inter-rater reliability and validity of the Brief International Classification of Functioning, Disability, and Health Core Set for Stroke (ICF) assessment. Methods: Study 1 involved expert (clinical experience ≥5 years) and novice (clinical experience < 2 years) rater-pairs, each evaluating the same post-stroke patients using the ICF assessment (n = 149). Study 2 involved novice raters separately evaluating a different cohort of post-stroke patients with the ICF assessment (n = 78). The novice raters had prior knowledge of patient functioning through conducting 6 clinical tests. Results: For Study 1, the expert rater-pairs (kappa = 0.50. 0.85 for categories; intra-class correlation (ICC) = 0.76.0.96 for components) had higher reliability coefficients than novice rater-pairs (kappa = 0.18.0.69 for categories; ICC = 0.63. 0.88 for components). For Study 2, the novice raters with prior knowledge of patient's functioning yielded significantly higher ICF component scores than those without prior knowledge. The former raters' component scores were comparable to those of the expert rater-pairs. Conclusion: Clinical experience in post-stroke rehabilitation enhances inter-rater reliability of ICF assessment. Knowledge of patient's functional capability, such as conducting common clinical tests in post-stroke rehabilitation, is useful for improving assessment validity.