A Test of the Validity of the LCSW Examination: Quis Custodiet Ipsos Custodes?

Abstract

The authors administered to 59 first-year MSW students the practice test for the clinical-level examination developed by the Association of Social Work Boards (ASWB); the latter is used by most jurisdictions to qualify individuals to practice as licensed clinical social workers. To assess the practice test's validity, the authors blanked out actual questions, leaving only four choices (one correct answer and three incorrect ones) and instructed students to select correct answers. On the basis of chance alone, students should have answered an average of 25% of items correct. In reality, they correctly guessed 52% of the items, a statistically significant difference. This suggests that the practice test (said by ASWB to contain items like those on actual examinations) is excessively transparent or guessable and cannot justifiably be claimed to be a valid assessment of competence to practice social work. The authors encourage ASWB to publish in independent peer-reviewed .journals a series of psychometric studies on the reliability and validity of the licensing examinations they have developed. The multimillion dollar social work testing industry is big business; both the profession and the public, which relies heavily on the gate-keeping function of these tests, deserve greater transparency and accountability with respect to their legitimacy. KEY WORDS: assessment; Association of Social Work Boards; LCSW; replication; validity ********** In 2007, over 8,000 social workers paid for and took the clinical social work examination for the first time, and of these, fewer than 75% passed the test (Association of Social Work Boards [ASWB], 2008a, 2008b, 2008c). Over 12,000 additional social workers completed other ASWB examinations (associate, bachelor's, master's, or advanced generalist). With a fee of $175 per individual examination, the social work testing industry is a multimillion dollar business. ASWB (formerly the American Association of State Social Work Boards), the independent organization that prepares, markets, and administers the tests, was incorporated in 1979 under the auspices of state credentialing bodies to promote consumer protection and to assist social work regulatory bodies in carrying out their legislated mandates, and to encourage jurisdictional efforts to protect a diverse public served by social workers who are regulated through common values, ethics, and practice standards. The Association will help foster public and professional understanding of the value, competency, and accountability of regulated social workers. (ASWB, 2007, p. 4) The ASWB developed and began administering social work examinations in 1983, and it claims that its tests continue to be one of the most important assurances that a social worker possesses the competence to practice responsibly (ASWB, 2007, p. 1). Prerequisites for being allowed to take the clinical examination typically are that the applicant have earned his or her MSW from a program accredited by the Council on Social Work Education and have had at least two years of postdegree supervised clinical practice experience. Each examination is composed of a total of 170 multiple choice questions, with each item consisting of a stem (or question), one correct response (the key), and three distractor (incorrect) answers. Only 150 of the 170 questions are used in determining the candidate's actual score; the remaining 20 questions are being pilot tested for future use. The ASWB conducted its last practice analysis of social work in 2003.A new one will be released shortly. The practice analysis is a survey of licensed social workers that asks them to list their common professional tasks, rate how often they perform each task, the importance of the task regardless of the frequency of performance, and whether or not it is a necessary skill. The periodically updated practice analysis survey is developed and piloted, sent to participants, and the results are used to update examination content. …