Predicting Success: Does Performance on the Anesthesia Knowledge Test - 6 (AKT-6) correlate with the American Board of Anesthesiology (ABA) Licensing Exam first-time pass rate?

Abstract

We sought to determine the relationship between residents' Anesthesia Knowledge Test 6 (AKT-6) scores and their first-time success/failure on the American Board of Anesthesiology written licensing examination. Reliable early identification of residents at risk for failing the ABA exam would be an invaluable screening tool for program leadership and facilitate timely remediation for struggling residents. Program directors were invited to submit anonymous data regarding their residents' performance on the AKT-6 and their subsequent first-time success/failure on the American Board of Anesthesiology written licensing examination. Eight residency programs responded with AKT6 percentile scores and ABA part 1 first-time pass/fail status from 306 residents spanning 2004-2011. Of these, 292 also included AKT6% correct scores. AKT-6 performance was significantly better for trainees who went on to pass the ABA exam on their first attempt compared to those who failed. Trainees who scored at or below the 4(th) percentile (or answered ≤42% of questions correctly) failed the ABA exam while all those scoring above the 84(th) percentile (or answered >68% of questions correctly) passed. A Mantel-Haenszel common odds ratio estimate revealed significantly increased odds of failure below the thresholds of AKT-6 scores ≤ 36(th) percentile (≤56% correct). Observations from this work help to validate educators' use of AKT-6 exam performance as a marker for likelihood of success/failure on the ABA written licensing exam. Our analysis, based on data from eight training programs, yielded definitive cut points for ABA exam failure and passing. ROC analysis of our data supports a recommendation for educators to intervene with trainees scoring at or below the 36(th) percentile or 56% correct on AKT-6 testing. Our results likely require confirmation in a larger subset of anesthesiology residency programs.