Reliability of Verbal Handoff Assessment and Handoff Quality Before and After Implementation of a Resident Handoff Bundle - Feraco, Angela M. (Acad Pediatr, 2016)
OBJECTIVE: 1) To develop validity evidence for the use of the Verbal Handoff Assessment Tool (VHAT) and examine the reliability of VHAT scores, and 2) to determine whether implementation of a resident handoff bundle (RHB) was associated with improved verbal patient handoffs among pediatric resident physicians. METHOD: In a pre-post design, prospectively audio recorded verbal patient handoffs conducted at Boston Children’s Hospital before and after implementation of the RHB were rated using the VHAT, which was developed for this study (primary outcome). Using Generalizability theory, we evaluated the reliability of VHAT scores. RESULTS: Overall, VHAT scores increased following RHB implementation (mean 142 vs. 191, possible score 0–500, p<0.0001). When accounting for clustering by resident physician, hospital unit, unit census, and patient complexity, implementation of the RHB was associated with a 63-point increase in VHAT score. Using Generalizability theory, we determined that a resident’s mean VHAT score based on handoff of 15 patients assessed by a single observer was sufficiently reliable for relative ranking decisions (i.e. norm-based; G-coefficient 0.81), whereas a VHAT score based on handoff of 21 patients would be sufficiently reliable for high-stakes, standard-based decisions (Phi 0.80). CONCLUSIONS: Verbal handoffs improved following implementation of a resident handoff bundle, though gains were variable across the two clinical units. The VHAT shows promise as an assessment tool for resident handoff skills. If used for competency or entrustment decisions, a resident’s mean VHAT score should be based on observation of verbal handoff of ≥21 patients.