Evaluating the Global Rating scale's psychometric properties to assess communication skills of undergraduate medical students in video-recorded simulated patient encounters

OBJECTIVES: To evaluate the psychometric properties of the Global Rating scale (GR) as an observer-based tool to assess communication skills of undergraduate medical students in video-recorded patient encounters.

METHODS: Seventy advanced undergraduate medical students participated in a simulation-based assessment including patient consultations. Simulated patients rated these encounters with the Consultation and Relational Empathy (CARE) scale. Two independent, blinded raters assessed the videos of the encounters with the GR and another blinded rater with the Clinical Reasoning Indicators Scale (CRI-HT-S). To assess the GR's psychometric properties, we analysed reliability by means of a G-study, interrater reliability by ICC, convergent validity (correlation of GR and CARE), and divergent validity (correlation of GR and CRI-HT-S).

RESULTS: We analysed 325 videos of 65 students (56.9% female, mean age 26.1 ± 2.2 years). The G-coefficient was.90. Interrater reliability of the GR was ICC = .95, 95% CI [.91,.97]. CARE and GR correlated moderately (ρ = .47, 95% CI [.25,.65]). GR and CRI-HT-S did not correlate (ρ = .09, 95% CI [-.16,.34]).

CONCLUSIONS: With excellent reliability and adequate validity, the quality of the GR as assessment instrument for communication skills could be demonstrated.

PRACTICE IMPLICATIONS: The GR is a suitable instrument for video-based rating of communication skills.