Estimating learning outcomes from pre- and posttest student self-assessments: a longitudinal study

TY - JOUR

T1 - Estimating learning outcomes from pre- and posttest student self-assessments: a longitudinal study

AU - Schiekirka, Sarah

AU - Reinhardt, Deborah

AU - Beißbarth, Tim

AU - Anders, Sven

AU - Pukrop, Tobias

AU - Raupach, Tobias

PY - 2013/3/1

Y1 - 2013/3/1

N2 - PURPOSE: Learning outcome is an important measure for overall teaching quality and should be addressed by comprehensive evaluation tools. The authors evaluated the validity of a novel evaluation tool based on student self-assessments, which may help identify specific strengths and weaknesses of a particular course.METHOD: In 2011, the authors asked 145 fourth-year students at Göttingen Medical School to self-assess their knowledge on 33 specific learning objectives in a pretest and posttest as part of a cardiorespiratory module. The authors compared performance gain calculated from self-assessments with performance gain derived from formative examinations that were closely matched to these 33 learning objectives.RESULTS: Eighty-three students (57.2%) completed the assessment. There was good agreement between performance gain derived from subjective data and performance gain derived from objective examinations (Pearson r=0.78; P<.0001) on the group level. The association between the two measures was much weaker when data were analyzed on the individual level. Further analysis determined a quality cutoff for performance gain derived from aggregated student self-assessments. When using this cutoff, the evaluation tool was highly sensitive in identifying specific learning objectives with favorable or suboptimal objective performance gains.CONCLUSIONS: The tool is easy to implement, takes initial performance levels into account, and does not require extensive pre-post testing. By providing valid estimates of actual performance gain obtained during a teaching module, it may assist medical teachers in identifying strengths and weaknesses of a particular course on the level of specific learning objectives.

AB - PURPOSE: Learning outcome is an important measure for overall teaching quality and should be addressed by comprehensive evaluation tools. The authors evaluated the validity of a novel evaluation tool based on student self-assessments, which may help identify specific strengths and weaknesses of a particular course.METHOD: In 2011, the authors asked 145 fourth-year students at Göttingen Medical School to self-assess their knowledge on 33 specific learning objectives in a pretest and posttest as part of a cardiorespiratory module. The authors compared performance gain calculated from self-assessments with performance gain derived from formative examinations that were closely matched to these 33 learning objectives.RESULTS: Eighty-three students (57.2%) completed the assessment. There was good agreement between performance gain derived from subjective data and performance gain derived from objective examinations (Pearson r=0.78; P<.0001) on the group level. The association between the two measures was much weaker when data were analyzed on the individual level. Further analysis determined a quality cutoff for performance gain derived from aggregated student self-assessments. When using this cutoff, the evaluation tool was highly sensitive in identifying specific learning objectives with favorable or suboptimal objective performance gains.CONCLUSIONS: The tool is easy to implement, takes initial performance levels into account, and does not require extensive pre-post testing. By providing valid estimates of actual performance gain obtained during a teaching module, it may assist medical teachers in identifying strengths and weaknesses of a particular course on the level of specific learning objectives.

KW - Adult

KW - Education, Medical, Undergraduate

KW - Educational Measurement

KW - Female

KW - Germany

KW - Humans

KW - Longitudinal Studies

KW - Male

KW - ROC Curve

KW - Reproducibility of Results

KW - Self-Assessment

U2 - 10.1097/ACM.0b013e318280a6f6

DO - 10.1097/ACM.0b013e318280a6f6

M3 - SCORING: Journal article

C2 - 23348083

VL - 88

SP - 369

EP - 375

JO - ACAD MED

JF - ACAD MED

SN - 1040-2446

IS - 3

ER -