TY - JOUR
T1 - A multi-source feedback tool for measuring a subset of Pediatrics Milestones
AU - The APPD LEARN—NBME Pediatrics Milestones Assessment Group
AU - Schwartz, Alan
AU - Margolis, Melissa J.
AU - Multerer, Sara
AU - Haftel, Hilary M.
AU - Schumacher, Daniel J.
AU - Wright, Kristen
AU - Moffitt, Rosalind
AU - Hicks, Patricia J.
AU - Ronan, Jeanine
AU - Tenney-Soeiro, Rebecca
AU - Young, Dawn
AU - Turner, Teri
AU - Allison, Melodie
AU - Waggoner-Fountain, Linda
AU - Morris, Lisa
AU - Caputo, Grace
AU - Barker, Sandra
AU - Bhavaraju, Vasu
AU - Velazquez, Ana
AU - del Rey, Javier Gonzales
AU - Poynter, Sue
AU - Berry, Jackie
AU - Guillot, Ann
AU - Leonard, Karen
AU - Boland, Kimberly
AU - Mittel, Olivia
AU - Osta, Amanda
AU - Barnes, Michelle
AU - Tas, Emri
AU - McDonnell, Jen
AU - Powell, Cynthia
AU - Black, Nicole Paradise
AU - Chang, Lilly
AU - Leavens-Maurer, Jill
AU - Kothari, Ulka
AU - Lee, Robert
AU - Buchter, Susie
AU - Carter, Meredith
AU - Blair, Robyn
AU - Arnold, Sandra
AU - Bugnitz, Mark C.
AU - Burke, Ann
AU - Clyman, Stephen G.
AU - Rebbecchi, Thomas
AU - Canavan, Colleen
AU - Chaffinch, Christa
AU - Richmond, Margaret
AU - Rose, Kathleen M.
AU - Rose, Leta
AU - Spector, Yelena
N1 - Publisher Copyright:
© 2016 Informa UK Limited, trading as Taylor & Francis Group.
PY - 2016/10/2
Y1 - 2016/10/2
N2 - Background: The Pediatrics Milestones Assessment Pilot employed a new multisource feedback (MSF) instrument to assess nine Pediatrics Milestones among interns and subinterns in the inpatient context. Objective: To report validity evidence for the MSF tool for informing milestone classification decisions. Methods: We obtained MSF instruments by different raters per learner per rotation. We present evidence for validity based on the unified validity framework. Results: One hundred and ninety two interns and 41 subinterns at 18 Pediatrics residency programs received a total of 1084 MSF forms from faculty (40%), senior residents (34%), nurses (22%), and other staff (4%). Variance in ratings was associated primarily with rater (32%) and learner (22%). The milestone factor structure fit data better than simpler structures. In domains except professionalism, ratings by nurses were significantly lower than those by faculty and ratings by other staff were significantly higher. Ratings were higher when the rater observed the learner for longer periods and had a positive global opinion of the learner. Ratings of interns and subinterns did not differ, except for ratings by senior residents. MSF-based scales correlated with summative milestone scores. Conclusion: We obtain moderately reliable MSF ratings of interns and subinterns in the inpatient context to inform some milestone assignments.
AB - Background: The Pediatrics Milestones Assessment Pilot employed a new multisource feedback (MSF) instrument to assess nine Pediatrics Milestones among interns and subinterns in the inpatient context. Objective: To report validity evidence for the MSF tool for informing milestone classification decisions. Methods: We obtained MSF instruments by different raters per learner per rotation. We present evidence for validity based on the unified validity framework. Results: One hundred and ninety two interns and 41 subinterns at 18 Pediatrics residency programs received a total of 1084 MSF forms from faculty (40%), senior residents (34%), nurses (22%), and other staff (4%). Variance in ratings was associated primarily with rater (32%) and learner (22%). The milestone factor structure fit data better than simpler structures. In domains except professionalism, ratings by nurses were significantly lower than those by faculty and ratings by other staff were significantly higher. Ratings were higher when the rater observed the learner for longer periods and had a positive global opinion of the learner. Ratings of interns and subinterns did not differ, except for ratings by senior residents. MSF-based scales correlated with summative milestone scores. Conclusion: We obtain moderately reliable MSF ratings of interns and subinterns in the inpatient context to inform some milestone assignments.
UR - https://www.scopus.com/pages/publications/84961909209
U2 - 10.3109/0142159X.2016.1147646
DO - 10.3109/0142159X.2016.1147646
M3 - Article
C2 - 27027428
AN - SCOPUS:84961909209
SN - 0142-159X
VL - 38
SP - 995
EP - 1002
JO - Medical Teacher
JF - Medical Teacher
IS - 10
ER -