Selective cutoff reporting in studies of the accuracy of the Patient Health Questionnaire-9 and Edinburgh Postnatal Depression Scale: Comparison of results based on published cutoffs versus all cutoffs using individual participant data meta-analysis

Dipika Neupane; Brooke Levis; Parash M. Bhandari; Brett D. Thombs; Andrea Benedetti; DEPRESsion Screening Data (DEPRESSD) Collaboration

doi:10.1002/mpr.1873

Selective cutoff reporting in studies of the accuracy of the Patient Health Questionnaire-9 and Edinburgh Postnatal Depression Scale: Comparison of results based on published cutoffs versus all cutoffs using individual participant data meta-analysis

Standard

Selective cutoff reporting in studies of the accuracy of the Patient Health Questionnaire-9 and Edinburgh Postnatal Depression Scale: Comparison of results based on published cutoffs versus all cutoffs using individual participant data meta-analysis. / Neupane, Dipika; Levis, Brooke; Bhandari, Parash M.; Thombs, Brett D.; Benedetti, Andrea; DEPRESsion Screening Data (DEPRESSD) Collaboration.

In: INT J METH PSYCH RES, Vol. 30, No. 3, e1873, 09.2021, p. e1873.

Research output: SCORING: Contribution to journal › SCORING: Journal article › Research › peer-review

Harvard

Neupane, D, Levis, B, Bhandari, PM, Thombs, BD, Benedetti, A & DEPRESsion Screening Data (DEPRESSD) Collaboration 2021, 'Selective cutoff reporting in studies of the accuracy of the Patient Health Questionnaire-9 and Edinburgh Postnatal Depression Scale: Comparison of results based on published cutoffs versus all cutoffs using individual participant data meta-analysis', INT J METH PSYCH RES, vol. 30, no. 3, e1873, pp. e1873. https://doi.org/10.1002/mpr.1873

APA

Neupane, D., Levis, B., Bhandari, P. M., Thombs, B. D., Benedetti, A., & DEPRESsion Screening Data (DEPRESSD) Collaboration (2021). Selective cutoff reporting in studies of the accuracy of the Patient Health Questionnaire-9 and Edinburgh Postnatal Depression Scale: Comparison of results based on published cutoffs versus all cutoffs using individual participant data meta-analysis. INT J METH PSYCH RES, 30(3), e1873. [e1873]. https://doi.org/10.1002/mpr.1873

Vancouver

Neupane D, Levis B, Bhandari PM, Thombs BD, Benedetti A, DEPRESsion Screening Data (DEPRESSD) Collaboration. Selective cutoff reporting in studies of the accuracy of the Patient Health Questionnaire-9 and Edinburgh Postnatal Depression Scale: Comparison of results based on published cutoffs versus all cutoffs using individual participant data meta-analysis. INT J METH PSYCH RES. 2021 Sep;30(3):e1873. e1873. https://doi.org/10.1002/mpr.1873

Bibtex

@article{38cfa28da71c462e9ba37bd9f09d9411,

title = "Selective cutoff reporting in studies of the accuracy of the Patient Health Questionnaire-9 and Edinburgh Postnatal Depression Scale: Comparison of results based on published cutoffs versus all cutoffs using individual participant data meta-analysis",

abstract = "Objectives: Selectively reported results from only well-performing cutoffs in diagnostic accuracy studies may bias estimates in meta-analyses. We investigated cutoff reporting patterns for the Patient Health Questionnaire-9 (PHQ-9; standard cutoff 10) and Edinburgh Postnatal Depression Scale (EPDS; no standard cutoff, commonly used 10–13) and compared accuracy estimates based on published cutoffs versus all cutoffs. Methods: We conducted bivariate random effects meta-analyses using individual participant data to compare accuracy from published versus all cutoffs. Results: For the PHQ-9 (30 studies, N = 11,773), published results underestimated sensitivity for cutoffs below 10 (median difference: −0.06) and overestimated for cutoffs above 10 (median difference: 0.07). EPDS (19 studies, N = 3637) sensitivity estimates from published results were similar for cutoffs below 10 (median difference: 0.00) but higher for cutoffs above 13 (median difference: 0.14). Specificity estimates from published and all cutoffs were similar for both tools. The mean cutoff of all reported cutoffs in PHQ-9 studies with optimal cutoff below 10 was 8.8 compared to 11.8 for those with optimal cutoffs above 10. Mean for EPDS studies with optimal cutoffs below 10 was 9.9 compared to 11.8 for those with optimal cutoffs greater than 10. Conclusion: Selective cutoff reporting was more pronounced for the PHQ-9 than EPDS.",

keywords = "diagnostic test accuracy, individual participant data meta-analysis, meta-analysis, publication bias, selective cutoff reporting",

author = "Dipika Neupane and Brooke Levis and Bhandari, {Parash M.} and Thombs, {Brett D.} and Andrea Benedetti and {DEPRESsion Screening Data (DEPRESSD) Collaboration} and Ying Sun and Chen He and Yin Wu and Ankur Krishnan and Zelalem Negeri and Mahrukh Imran and Rice, {Danielle B.} and Riehm, {Kira E.} and Nazanin Saadat and Marleine Azar and Sanchez, {Tatiana A.} and Chiovitti, {Matthew J.} and Levis, {Alexander W.} and Boruff, {Jill T.} and Pim Cuijpers and Simon Gilbody and Ioannidis, {John P.A.} and Kloda, {Lorie A.} and Patten, {Scott B.} and Ian Shrier and Ziegelstein, {Roy C.} and Liane Comeau and Mitchell, {Nicholas D.} and Marcello Tonelli and Vigod, {Simone N.} and Akena, {Dickens H.} and Rub{\'e}n Alvarado and Bruce Arroll and Bakare, {Muideen O.} and Baradaran, {Hamid R.} and Beck, {Cheryl Tatano} and Bombardier, {Charles H.} and Adomas Bunevicius and Gregory Carter and Chagas, {Marcos H.} and Chaudron, {Linda H.} and Rushina Cholera and Kerrie Clover and Yeates Conwell and {Castro e Couto}, Tiago and {de Man-van Ginkel}, {Janneke M.} and Jaime Delgadillo and Fann, {Jesse R.} and Nicolas Favez and Daniel Fung and Llu{\"i}sa Garcia-Esteve and Bizu Gelaye and Felicity Goodyear-Smith and Thomas Hyphantis and Masatoshi Inagaki and Khalida Ismail and Nathalie Jett{\'e} and Khalifa, {Dina Sami} and Khamseh, {Mohammad E.} and Jane Kohlhoff and Zolt{\'a}n Kozinszky and Laima Kusminskas and Liu, {Shen Ing} and Manote Lotrakul and Loureiro, {Sonia R.} and Bernd L{\"o}we and Sidik, {Sherina Mohd} and {Naki{\'c} Rado{\v s}}, Sandra and Os{\'o}rio, {Fl{\'a}via L.} and Pawlby, {Susan J.} and Pence, {Brian W.} and Rochat, {Tamsen J.} and Rooney, {Alasdair G.} and Sharp, {Deborah J.} and Lesley Stafford and Su, {Kuan Pin} and Sung, {Sharon C.} and Meri Tadinac and {Darius Tandon}, S. and Pavaani Thiagayson and Annam{\'a}ria T{\"o}reki and Anna Torres-Gim{\'e}nez and Alyna Turner and {van der Feltz-Cornelis}, {Christina M.} and Vega-Dienstmaier, {Johann M.} and V{\"o}hringer, {Paul A.} and Jennifer White and Whooley, {Mary A.} and Kirsty Winkley and Mitsuhiko Yamada",

note = "Publisher Copyright: {\textcopyright} 2021 The Authors. International Journal of Methods in Psychiatric Research published by John Wiley & Sons Ltd.",

year = "2021",

month = sep,

doi = "10.1002/mpr.1873",

language = "English",

volume = "30",

pages = "e1873",

journal = "INT J METH PSYCH RES",

issn = "1049-8931",

publisher = "John Wiley and Sons Ltd",

number = "3",

}

RIS

TY - JOUR

T1 - Selective cutoff reporting in studies of the accuracy of the Patient Health Questionnaire-9 and Edinburgh Postnatal Depression Scale: Comparison of results based on published cutoffs versus all cutoffs using individual participant data meta-analysis

AU - Neupane, Dipika

AU - Levis, Brooke

AU - Bhandari, Parash M.

AU - Thombs, Brett D.

AU - Benedetti, Andrea

AU - DEPRESsion Screening Data (DEPRESSD) Collaboration

AU - Sun, Ying

AU - He, Chen

AU - Wu, Yin

AU - Krishnan, Ankur

AU - Negeri, Zelalem

AU - Imran, Mahrukh

AU - Rice, Danielle B.

AU - Riehm, Kira E.

AU - Saadat, Nazanin

AU - Azar, Marleine

AU - Sanchez, Tatiana A.

AU - Chiovitti, Matthew J.

AU - Levis, Alexander W.

AU - Boruff, Jill T.

AU - Cuijpers, Pim

AU - Gilbody, Simon

AU - Ioannidis, John P.A.

AU - Kloda, Lorie A.

AU - Patten, Scott B.

AU - Shrier, Ian

AU - Ziegelstein, Roy C.

AU - Comeau, Liane

AU - Mitchell, Nicholas D.

AU - Tonelli, Marcello

AU - Vigod, Simone N.

AU - Akena, Dickens H.

AU - Alvarado, Rubén

AU - Arroll, Bruce

AU - Bakare, Muideen O.

AU - Baradaran, Hamid R.

AU - Beck, Cheryl Tatano

AU - Bombardier, Charles H.

AU - Bunevicius, Adomas

AU - Carter, Gregory

AU - Chagas, Marcos H.

AU - Chaudron, Linda H.

AU - Cholera, Rushina

AU - Clover, Kerrie

AU - Conwell, Yeates

AU - Castro e Couto, Tiago

AU - de Man-van Ginkel, Janneke M.

AU - Delgadillo, Jaime

AU - Fann, Jesse R.

AU - Favez, Nicolas

AU - Fung, Daniel

AU - Garcia-Esteve, Lluïsa

AU - Gelaye, Bizu

AU - Goodyear-Smith, Felicity

AU - Hyphantis, Thomas

AU - Inagaki, Masatoshi

AU - Ismail, Khalida

AU - Jetté, Nathalie

AU - Khalifa, Dina Sami

AU - Khamseh, Mohammad E.

AU - Kohlhoff, Jane

AU - Kozinszky, Zoltán

AU - Kusminskas, Laima

AU - Liu, Shen Ing

AU - Lotrakul, Manote

AU - Loureiro, Sonia R.

AU - Löwe, Bernd

AU - Sidik, Sherina Mohd

AU - Nakić Radoš, Sandra

AU - Osório, Flávia L.

AU - Pawlby, Susan J.

AU - Pence, Brian W.

AU - Rochat, Tamsen J.

AU - Rooney, Alasdair G.

AU - Sharp, Deborah J.

AU - Stafford, Lesley

AU - Su, Kuan Pin

AU - Sung, Sharon C.

AU - Tadinac, Meri

AU - Darius Tandon, S.

AU - Thiagayson, Pavaani

AU - Töreki, Annamária

AU - Torres-Giménez, Anna

AU - Turner, Alyna

AU - van der Feltz-Cornelis, Christina M.

AU - Vega-Dienstmaier, Johann M.

AU - Vöhringer, Paul A.

AU - White, Jennifer

AU - Whooley, Mary A.

AU - Winkley, Kirsty

AU - Yamada, Mitsuhiko

PY - 2021/9

Y1 - 2021/9

N2 - Objectives: Selectively reported results from only well-performing cutoffs in diagnostic accuracy studies may bias estimates in meta-analyses. We investigated cutoff reporting patterns for the Patient Health Questionnaire-9 (PHQ-9; standard cutoff 10) and Edinburgh Postnatal Depression Scale (EPDS; no standard cutoff, commonly used 10–13) and compared accuracy estimates based on published cutoffs versus all cutoffs. Methods: We conducted bivariate random effects meta-analyses using individual participant data to compare accuracy from published versus all cutoffs. Results: For the PHQ-9 (30 studies, N = 11,773), published results underestimated sensitivity for cutoffs below 10 (median difference: −0.06) and overestimated for cutoffs above 10 (median difference: 0.07). EPDS (19 studies, N = 3637) sensitivity estimates from published results were similar for cutoffs below 10 (median difference: 0.00) but higher for cutoffs above 13 (median difference: 0.14). Specificity estimates from published and all cutoffs were similar for both tools. The mean cutoff of all reported cutoffs in PHQ-9 studies with optimal cutoff below 10 was 8.8 compared to 11.8 for those with optimal cutoffs above 10. Mean for EPDS studies with optimal cutoffs below 10 was 9.9 compared to 11.8 for those with optimal cutoffs greater than 10. Conclusion: Selective cutoff reporting was more pronounced for the PHQ-9 than EPDS.

AB - Objectives: Selectively reported results from only well-performing cutoffs in diagnostic accuracy studies may bias estimates in meta-analyses. We investigated cutoff reporting patterns for the Patient Health Questionnaire-9 (PHQ-9; standard cutoff 10) and Edinburgh Postnatal Depression Scale (EPDS; no standard cutoff, commonly used 10–13) and compared accuracy estimates based on published cutoffs versus all cutoffs. Methods: We conducted bivariate random effects meta-analyses using individual participant data to compare accuracy from published versus all cutoffs. Results: For the PHQ-9 (30 studies, N = 11,773), published results underestimated sensitivity for cutoffs below 10 (median difference: −0.06) and overestimated for cutoffs above 10 (median difference: 0.07). EPDS (19 studies, N = 3637) sensitivity estimates from published results were similar for cutoffs below 10 (median difference: 0.00) but higher for cutoffs above 13 (median difference: 0.14). Specificity estimates from published and all cutoffs were similar for both tools. The mean cutoff of all reported cutoffs in PHQ-9 studies with optimal cutoff below 10 was 8.8 compared to 11.8 for those with optimal cutoffs above 10. Mean for EPDS studies with optimal cutoffs below 10 was 9.9 compared to 11.8 for those with optimal cutoffs greater than 10. Conclusion: Selective cutoff reporting was more pronounced for the PHQ-9 than EPDS.

KW - diagnostic test accuracy

KW - individual participant data meta-analysis

KW - meta-analysis

KW - publication bias

KW - selective cutoff reporting

UR - http://www.scopus.com/inward/record.url?scp=85115048600&partnerID=8YFLogxK

U2 - 10.1002/mpr.1873

DO - 10.1002/mpr.1873

M3 - SCORING: Journal article

C2 - 33978306

AN - SCOPUS:85115048600

VL - 30

SP - e1873

JO - INT J METH PSYCH RES

JF - INT J METH PSYCH RES

SN - 1049-8931

IS - 3

M1 - e1873

ER -