Estimating learning outcomes from pre- and posttest student self-assessments: a longitudinal study
Standard
Estimating learning outcomes from pre- and posttest student self-assessments: a longitudinal study. / Schiekirka, Sarah; Reinhardt, Deborah; Beißbarth, Tim; Anders, Sven; Pukrop, Tobias; Raupach, Tobias.
In: ACAD MED, Vol. 88, No. 3, 01.03.2013, p. 369-75.Research output: SCORING: Contribution to journal › SCORING: Journal article › Research › peer-review
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Estimating learning outcomes from pre- and posttest student self-assessments: a longitudinal study
AU - Schiekirka, Sarah
AU - Reinhardt, Deborah
AU - Beißbarth, Tim
AU - Anders, Sven
AU - Pukrop, Tobias
AU - Raupach, Tobias
PY - 2013/3/1
Y1 - 2013/3/1
N2 - PURPOSE: Learning outcome is an important measure for overall teaching quality and should be addressed by comprehensive evaluation tools. The authors evaluated the validity of a novel evaluation tool based on student self-assessments, which may help identify specific strengths and weaknesses of a particular course.METHOD: In 2011, the authors asked 145 fourth-year students at Göttingen Medical School to self-assess their knowledge on 33 specific learning objectives in a pretest and posttest as part of a cardiorespiratory module. The authors compared performance gain calculated from self-assessments with performance gain derived from formative examinations that were closely matched to these 33 learning objectives.RESULTS: Eighty-three students (57.2%) completed the assessment. There was good agreement between performance gain derived from subjective data and performance gain derived from objective examinations (Pearson r=0.78; P<.0001) on the group level. The association between the two measures was much weaker when data were analyzed on the individual level. Further analysis determined a quality cutoff for performance gain derived from aggregated student self-assessments. When using this cutoff, the evaluation tool was highly sensitive in identifying specific learning objectives with favorable or suboptimal objective performance gains.CONCLUSIONS: The tool is easy to implement, takes initial performance levels into account, and does not require extensive pre-post testing. By providing valid estimates of actual performance gain obtained during a teaching module, it may assist medical teachers in identifying strengths and weaknesses of a particular course on the level of specific learning objectives.
AB - PURPOSE: Learning outcome is an important measure for overall teaching quality and should be addressed by comprehensive evaluation tools. The authors evaluated the validity of a novel evaluation tool based on student self-assessments, which may help identify specific strengths and weaknesses of a particular course.METHOD: In 2011, the authors asked 145 fourth-year students at Göttingen Medical School to self-assess their knowledge on 33 specific learning objectives in a pretest and posttest as part of a cardiorespiratory module. The authors compared performance gain calculated from self-assessments with performance gain derived from formative examinations that were closely matched to these 33 learning objectives.RESULTS: Eighty-three students (57.2%) completed the assessment. There was good agreement between performance gain derived from subjective data and performance gain derived from objective examinations (Pearson r=0.78; P<.0001) on the group level. The association between the two measures was much weaker when data were analyzed on the individual level. Further analysis determined a quality cutoff for performance gain derived from aggregated student self-assessments. When using this cutoff, the evaluation tool was highly sensitive in identifying specific learning objectives with favorable or suboptimal objective performance gains.CONCLUSIONS: The tool is easy to implement, takes initial performance levels into account, and does not require extensive pre-post testing. By providing valid estimates of actual performance gain obtained during a teaching module, it may assist medical teachers in identifying strengths and weaknesses of a particular course on the level of specific learning objectives.
KW - Adult
KW - Education, Medical, Undergraduate
KW - Educational Measurement
KW - Female
KW - Germany
KW - Humans
KW - Longitudinal Studies
KW - Male
KW - ROC Curve
KW - Reproducibility of Results
KW - Self-Assessment
U2 - 10.1097/ACM.0b013e318280a6f6
DO - 10.1097/ACM.0b013e318280a6f6
M3 - SCORING: Journal article
C2 - 23348083
VL - 88
SP - 369
EP - 375
JO - ACAD MED
JF - ACAD MED
SN - 1040-2446
IS - 3
ER -