On the Cover

Abstract

Educational Measurement: Issues and PracticeVolume 36, Issue 2 p. 4-4 Data VisualizationFree Access On the Cover First published: 22 June 2017 https://doi.org/10.1111/emip.12157AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat The cover features the work of Richard Feinberg and Daniel Jurich of the National Board of Medical Examiners. Their graphic entitled Data Visualization for Incomplete Test Administrations was one of four winning entries in the 2017 EM:IP Cover Graphic/Data Visualization competition. These designers describe their work as follows: Occasionally, testing programs, especially those administering long computer-based tests, may encounter unplanned interruptions during an otherwise standard administration. Such disruptions may result from a driver failure, power outage, or problem at the testing site; but, regardless of their cause, they prevent examinees from completing all portions of the test. In these cases, examinees often need to retest in order to receive a valid score. However, for classification tests, particularly those in which the primary purpose is to make a pass/fail decision, partial results may provide enough information to make defensible classification decisions.Given a fixed-length exam with a predetermined cut-score, a Data Visualization for Incomplete Test Administrations graphic characterizes various levels of decision confidence based on partial results. Illustrated in the cover graphic for a 100-item test with a passing score of 65, score users can see at a glance the classification confidence associated with a given raw score (x-axis) and a given (partial) test length (y-axis). In the example, any examinee scoring 65 or higher will pass (Guaranteed Pass) regardless of performance on the remaining items; and, likewise, any examinee responding incorrectly to 36 or more items will fail (Guaranteed Fail) regardless of when the interruption occurred. More interesting are the regions that cannot be classified with certainty. In these regions, model-based confidence thresholds can be used to articulate more nuanced policy information: Probable Pass, Probable Fail, and Indeterminate. Here, for example, it cannot be said with certainty that an examinee responding correctly to 58 items prior to an interruption at the 67th item would pass, but policy makers may decide that for the purpose of the exam such a performance demonstrates enough proficiency to be designated a Probable Pass. Decision maps like this reduce ambiguity, eliminate case-by-case ad hoc responses to unplanned interruptions, and can prevent examinee-related costs associated with unnecessary retesting. Interested in exploring this visualization approach? The design team has made available the requisite R code. (See the Online Supplemental Information for the R (R Core Team, 2016) for details at http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1745-3992). Supporting Information Filename Description emip12157-sup-0001-SuppMat.pdf118.5 KB R Code for creating the Data Visualization for Incomplete Test Administrations Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article. Reference R Core Team. (2016). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Available at https://www.R-project.org/. Suggested citation Feinberg, R., & Jurich, D. (2017). Data Visualization for Incomplete Test Administrations. On the Cover, Educational Measurement: Issues and Practice, 36(2). Available at https://doi.org/10.1111/emip12159. Volume36, Issue2Summer 2017Pages 4-4 ReferencesRelatedInformation