Statistical inference for diagnostic test accuracy studies with multiple comparisons
Standard
Statistical inference for diagnostic test accuracy studies with multiple comparisons. / Westphal, Max; Zapf, Antonia.
in: STAT METHODS MED RES, Jahrgang 33, Nr. 4, 04.2024, S. 669-680.Publikationen: SCORING: Beitrag in Fachzeitschrift/Zeitung › SCORING: Zeitschriftenaufsatz › Forschung › Begutachtung
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Statistical inference for diagnostic test accuracy studies with multiple comparisons
AU - Westphal, Max
AU - Zapf, Antonia
PY - 2024/4
Y1 - 2024/4
N2 - Diagnostic accuracy studies assess the sensitivity and specificity of a new index test in relation to an established comparator or the reference standard. The development and selection of the index test are usually assumed to be conducted prior to the accuracy study. In practice, this is often violated, for instance, if the choice of the (apparently) best biomarker, model or cutpoint is based on the same data that is used later for validation purposes. In this work, we investigate several multiple comparison procedures which provide family-wise error rate control for the emerging multiple testing problem. Due to the nature of the co-primary hypothesis problem, conventional approaches for multiplicity adjustment are too conservative for the specific problem and thus need to be adapted. In an extensive simulation study, five multiple comparison procedures are compared with regard to statistical error rates in least-favourable and realistic scenarios. This covers parametric and non-parametric methods and one Bayesian approach. All methods have been implemented in the new open-source R package cases which allows us to reproduce all simulation results. Based on our numerical results, we conclude that the parametric approaches (maxT and Bonferroni) are easy to apply but can have inflated type I error rates for small sample sizes. The two investigated Bootstrap procedures, in particular the so-called pairs Bootstrap, allow for a family-wise error rate control in finite samples and in addition have a competitive statistical power.Keywords: Diagnosis, medical testing, multiple testing, model selection, prediction, prognosis
AB - Diagnostic accuracy studies assess the sensitivity and specificity of a new index test in relation to an established comparator or the reference standard. The development and selection of the index test are usually assumed to be conducted prior to the accuracy study. In practice, this is often violated, for instance, if the choice of the (apparently) best biomarker, model or cutpoint is based on the same data that is used later for validation purposes. In this work, we investigate several multiple comparison procedures which provide family-wise error rate control for the emerging multiple testing problem. Due to the nature of the co-primary hypothesis problem, conventional approaches for multiplicity adjustment are too conservative for the specific problem and thus need to be adapted. In an extensive simulation study, five multiple comparison procedures are compared with regard to statistical error rates in least-favourable and realistic scenarios. This covers parametric and non-parametric methods and one Bayesian approach. All methods have been implemented in the new open-source R package cases which allows us to reproduce all simulation results. Based on our numerical results, we conclude that the parametric approaches (maxT and Bonferroni) are easy to apply but can have inflated type I error rates for small sample sizes. The two investigated Bootstrap procedures, in particular the so-called pairs Bootstrap, allow for a family-wise error rate control in finite samples and in addition have a competitive statistical power.Keywords: Diagnosis, medical testing, multiple testing, model selection, prediction, prognosis
KW - Bayes Theorem
KW - Data Interpretation, Statistical
KW - Computer Simulation
KW - Sample Size
KW - Diagnostic Tests, Routine
U2 - 10.1177/09622802241236933
DO - 10.1177/09622802241236933
M3 - SCORING: Journal article
C2 - 38490184
VL - 33
SP - 669
EP - 680
JO - STAT METHODS MED RES
JF - STAT METHODS MED RES
SN - 0962-2802
IS - 4
ER -