Computer-based Test Interpretation Research Articles

Computer utilization and clinical judgment in psychological assessment reports

The process of assessment report writing is a complex one, involving both the statistical evaluation of data and clinical methods of data interpretation to appropriately answer referral questions. Today, a computer often analyzes data generated in a psychological assessment, at least in part. In this article, the author focuses on the interaction between the decision-making processes of human clinicians and the test interpretations that are computer-based. The benefits and problems with computers in assessment are highlighted and are presented alongside the research on the validity of automated assessment, as well as research comparing clinicians and computers in the decision-making process. The author concludes that clinical judgment and computer-based test interpretation each have weaknesses. However, by using certain strategies to reduce clinicians' susceptibility to errors in decision making and to ensure that only valid computer-based test interpretations are used, clinicians can optimize the accuracy of conclusions that they draw in their assessment report.

Review of Computer-Based Test Interpretation Software for the MMPI-2

(2004). Review of Computer-Based Test Interpretation Software for the MMPI-2. Journal of Personality Assessment: Vol. 83, No. 1, pp. 78-83.

Computer-Based Test Interpretation in Career Assessment: Ethical and Professional Issues

The use of computer-based test interpretation (CBTI) in career assessment raises numerous ethical and professional issues. For example, how do current testing standards apply to tests on the Internet? What are appropriate and inappropriate uses of CBTI in career assessment? How can CBTI be successfully integrated into the career counseling process? Who is qualified to use CBTI? What are the potential benefits and problems with CBTI? In an attempt to answer some of these questions, this article examines the nature of CBTI and the use of CBTI in career assessment. Strategies for establishing and communicating the validity of CBTI are also presented. By proactively addressing potential ethical and professional issues, it is more likely that this technology will contribute to positive outcomes for individuals, clients, and practitioners.

The utility of computer versus clinician-authored assessments in aiding the prediction of patient symptomatology

The present study assessed the differences between clinician-authored and computer-generated personality assessments. The participants, 155 psychologists in independent practice, received one of four packets of client information which contained: a) Minnesota multiphasic personality inventory (MMPI)–2 profile; b) MMPI–2 profile and demographic information; c) MMPI–2 profile, demographics, and a computer based test interpretation report; or d) MMPI–2 profile, demographics, and a clinician-authored report. Participants were asked to assess a patient based on the information they received. These judgments were compared to symptom ratings made by staff where the patient was hospitalized. Results demonstrated that participants who received clinician-authored reports were most able to match hospital staff's ratings of patient symptomatology. Those who received computer-based test interpretation (CBTI) reports, however, were typically not able to make accurate assessments of the patients. In fact, participants who received a CBTI report of an ambiguous MMPI–2 profile (one which did not appear to match the patient's diagnosis) were actually less accurate at assessing symptomatology than were participants who received no report.

Validity and utility of computer-based test interpretation.

Computers have been important to applied psychology since their introduction, and the application of computerized methods has expanded in recent decades. The application of computerized methods has broadened in both scope and depth. This article explores the most recent uses of computer-based assessment methods and examines their validity. The comparability between computer-administered tests and their pencil-and-paper counterparts is discussed. Basic decision making in psychiatric screening, personality assessment, neuropsychology, and personnel psychology is also investigated. Studies on the accuracy of computerized narrative reports in personality assessment and psychiatric screening are then summarized. Research thus far appears to indicate that computer-generated reports should be viewed as valuable adjuncts to, rather than substitutes for, clinical judgment. Additional studies are needed to support broadened computer-based test usage.

Validity and utility of computer-based test interpretation.

Validity and utility of computer-based test interpretation.

Introduction to the special section on the use of computers for making judgements and decisions.

Computers have the potential to greatly increase the accuracy of judgments and decisions that are made in the area of psychological assessment. The validity of present-day computer programs are described in the first 2 articles of this Special Section. Both computer-based test interpretation programs and mechanical prediction rules (e.g., statistical prediction rules) are described. Limitations of present-day computer programs are described in the 3rd article along with methodological recommendations for building more powerful rules. In the 4th article, a statistical analysis (neural network modeling) is described that may allow us to build better rules.

Introduction to the special section on the use of computers for making judgments and decisions.

Computers have the potential to greatly increase the accuracy of judgments and decisions that are made in the area of psychological assessment. The validity of present-day computer programs are described in the first 2 articles of this Special Section. Both computer-based test interpretation programs and mechanical prediction rules (e.g., statistical prediction rules) are described. Limitations of present-day computer programs are described in the 3rd article along with methodological recommendations for building more powerful rules. In the 4th article, a statistical analysis (neural network modeling) is described that may allow us to build better rules.

Ethical perspectives and practice behaviors involving computer-based test interpretation.

The debates of the 1980s regarding responsible use of computer-based test interpretation (CBTI) software have mostly disappeared, as CBTI use has become common practice. We surveyed 364 members of the Society for Personality Assessment to determine how they use CBTI software in their work and their perspectives on the ethics of using CBTI in various ways. Psychologists commonly use CBTI software for test scoring and to provide a complementary source of input for case formulations. Most do not use CBTI software as the primary way to formulate a case, nor as an alternative to a written report. Controversy and uncertainty were expressed about importing sections of CBTI narratives into psychological reports. We distinguish between support and replacement functions of CBTI use, arguing that adequate research evidence should be present before using CBTI as a replacement for established assessment procedures.

The Role of Computer‐Based Test Interpretation (CBTI) in Occupational Assessment

The paper explores a range of issues relating to the design and use of computer‐based test interpretation (CBTI) systems. It describes a number of dimensions along which CBTI systems vary and considers both the positive and negative implications of these for test users, for their clients and for test candidates. A distinction is drawn between systems that generate finished reports and those that generate questions or guidance for the user. The question of how the validity of narrative reports can be assessed is considered. A study on the discriminative validity of reports generated by the PREVUE ICES CBTI system is reported which shows clear differences between judgements made about true and bogus reports. It is concluded that there are two main issues to consider when evaluating a CBTI system: first, the validity of the information generated by it and, second, the ways in which that information is likely to be used.

Computerized offender assessment: Validation study.

The validity of a computer-based test interpretation (CBTI) system for mental health screening in correctional settings was evaluated. First, 100 adult male felons were administered a brief mental status interview and a series of psychological tests, including the MMPI, the revised Beta IQ Examination, and the Buss-Durkee Hostility Inventory. Psychiatric diagnoses and CBTI ratings of victimization, violence, suicide, and substance abuse potentials were compared with clinicians' evaluations. Second, CBTI diagnoses of 109 inmates were compared with Diagnostic Interview Schedule (DIS) diagnoses. Last, CBTI risk ratings were compared with institutional infraction records of 1,718 inmates

The effects of human versus computer authorship on consumers' perceptions of psychological reports

This study examined the impact on school personnel of a psychological report purported to have been authored by a computer-based test interpretation (CBTI) system versus a psychologist. Identical reports, differing only in terms of authorship, were rated by school personnel in terms of perceived overall quality, credibility, and diagnostic interpretations. Subjects also indicated how much confidence they had in their evaluations of the psychological report they read. Data analyses indicated that authorship had virtually no effect on subjects' appraisal of the psychological report. The results of this study do not support the concerns of numerous authors who have theorized that report readers might inaccurately attribute an “aura” of scientific accuracy to computer- versus clinician-generated psychological reports. Suggestions for future research are provided.

Issues in computer-based test interpretive systems

This article presents issues sorrounding the review of computer based test interpretive systems. The Buros Institute of Mental Measurements is now producing reviews dedicated to the evaluation of computer-based test interpretation (CBTI) systems.

Assessment: What Role the Assessor?

The advent of automated assessment contains the seeds of both threat and promise for professional assessors. Placing the problem in the historical and sociological contexts, this article examines computer-based test interpretation (CBTI) and the frequently misunderstood role of actuarial data in relation to it. I argue that sound clinicianship is central not only to the development of adequate CBTI systems, but also to the systems' appropriate and skillful use in practice. When properly utilized by otherwise well-qualified clinicians, CBTI can substantially enhance practice.

Computer-Assisted Testing and the Goals of Counseling Psychology

This article explores how the use of computer-assisted testing and assessment can potentially contribute to the goals of counseling psychology. The following five trends in computer applications in testing and assessment are analyzed in relation to five goals of counseling psychology: (1) computer-based test interpretation (goals of remediation and rehabilitation, and serving diverse populations); (2) computer-assisted instruction (goals of remediation and rehabilitation, and education and development); (3) integration of audio-visual media (goals of prevention and serving diverse populations); (4) adaptive devices (goal of serving diverse populations); and (5) computer-assisted research (goal of research). The necessary commitment of human and financial resources, research priorities, and ethical issues are then discussed as issues that will influence the extent to which computer-assisted testing can contribute to the goals of counseling psychology.

The Barnum Effect in a Computerized Rorschach Interpretation System

Twelve psychiatric outpatients were administered the Rorschach test, and results were interpreted using the Exner (1983, 1986) Report for the Comprehensive System computer-based test interpretation (CBTI) program. Four psychiatrists made accuracy ratings for both real and bogus reports for each of their patients. Data were analyzed using two-way analyses of covariance (ANCOVAs), where report type was a repeated main effect, psychiatrist was a random main effect, and the number of statements in the report was the covariate. Results indicated that this CBTI provided only 5% discriminating power for any one patient, with 60% of the interpretive statements merely describing typical characteristics of the outpatient population. No significant psychiatrist, interaction, or covariate effects were encountered.

Ethical Issues in the Use of Computer-Based Assessment

In the past several years, increasing concern has been raised about the ethical issues involved in computer applications in mental health, particularly regarding computerized psychological testing and interpretation. The ethical concerns are reviewed relative to the various activities computers may be asked to perform. The authors argue that in some cases the use of computer-assisted assessment may be viewed as at least as, if not more, ethical than the clinician. In others, particularly regarding computer-based test interpretation, the authors recognize the existence of problems, but feel solutions for many of these exist. Even where problems remain, these should not preclude the use of technolgoies, but rather signal the need for careful use. Even where problems remain, these should not preclude the use of these technologies, but rather signal the need for careful use. Some specific concerns covered are dehumanization, nonvalidated computer test interpretation, the use of computer assessment by untraine...

Computer-based test interpretation in psychoeducational assessment: An initial appraisal

The development of computer-based interpretive systems has generated much enthusiasm and a fair amount of concern. The focus herein is on the use of computer-based test interpretation (CBTI) in psychoeducational assessment. Development influences affecting the use of CBTI in psychoeducational assessment are reviewed, the current status of CBTI systems is explored, and strategies for the evaluation of CBTI products by consumers and professionals are advanced.

Millon Clinical Multiaxial Inventory (MCMI) computer-generated diagnoses: How do they compare to clinician judgment?

During the past several decades, computers have achieved increasing prominence in psychological assessment procedures. This is particularly true for computer-based test interpretation and diagnosis. This study reports on a study designed to compare the accuracy of computer-based diagnoses with clinician-generated diagnoses. The Millon Clinical Multiaxial Inventory (MCMI) was administered to 151 consecutively admitted inpatients at a large private psychiatric hospital. The computer-generated diagnoses were compared with those generated by admitting psychiatrists. The results indicated that the MCMI diagnostic impressions underestimated the severity of depressive disorders when compared with clinician diagnoses on Axis I. Specifically, clinicians diagnosed major depression much more frequently than did the MCMI. In addition, clinicians diagnosed anxiety disorders much less frequently than did the MCMI.

"On the question of professional standards for computer-based test interpretation": Response to Kramer on Matarazzo.

"On the question of professional standards for computer-based test interpretation": Response to Kramer on Matarazzo.