Criminal Justice Evaluation Research Articles

Criminalization and Criminal Justice Evaluation of Mental Disorder Infliction from the Standpoint of the Principle of Justice

Criminalization and Criminal Justice Evaluation of Mental Disorder Infliction from the Standpoint of the Principle of Justice

An evaluator’s reflections and lessons learned about gang intervention strategies: an agenda for research

PurposeThis paper is designed to critically review and analyze the body of research on a popular gang reduction strategy, implemented widely in the United States and a number of other countries, to: (1) assess whether researchers designed their evaluations to align with the theorized causal mechanisms that bring about reductions in violence; and (2) discuss how evidence on gang programs is generated and consumed. That review and assessment is then used to frame a research agenda for studying gang interventions.Design/methodology/approachA case study design is used to generate a multi-faceted understanding of the possible avenues for evaluation research on the law enforcement-based strategy known as the Group Violence Intervention. The paper discusses questions that remain to be answered about the strategy, such as “what type of deterrence is operating?” and if the model actually works by the threat of deterrence, and not by removing high-risk offenders and shootings from the street, what activities are needed to maintain the effect?FindingsAcross roughly two dozen impact evaluations of GVI, none have examined the likely cause and effect components of this multi-partner strategy in reducing the violence. Furthermore, there are many issues related to the production and generation of criminal justice evaluation research that have adversely pushed the balance of evidence on what works in gang reduction toward law enforcement programming. However, there are many strategies that researchers can use to think broadly about appropriate and holistic research and evaluation on gangs and gang programming.Practical implicationsThe recommendations for research, if implemented, can help build a body of knowledge to move toward community-based and restorative models of gang violence reduction.Originality/valueThis original piece is one of the first essays to contextualize and discuss how aspects of the production of social science research on gangs may directly impact what programs and strategies are implemented on the ground.

Evaluating evaluation

AbstractThis study examines a participant’s narrative in a focus group interview dealing with the evaluation of criminal justice policy – the impact of community policing training. However, rather than look at the narrative solely in the speech of the interviewee, I analyze the integration of speech and embodied conduct like gesture, gaze, and posture in the production and negotiation of professional identities. I demonstrate the applied merits of a multimodal approach to criminal justice evaluation in the mapping between denotational text and interactional positioning, a mapping that inheres in embodied stance and broader sociocultural context.

Validating Peer Review in Criminal Justice Evaluation Research: Evidence from CrimeSolutions.gov

Peer review is the bedrock of the scientific enterprise, yet it enjoys scant validation. The federal government’s CrimeSolutions.gov initiative provides a unique opportunity to address this limitation. As part of the initiative, trained experts evaluate criminal justice evaluation research on several of the same criteria editors use to make publication decisions. Data from a sample of articles published in Social Sciences Citation Index journals were obtained from the CrimeSolutions.gov database, then used to model publication quality, operationalized as the product of the journal’s five-year impact factor and article citations per year (an article-level measure). The model explained only five percent of the variation in publication quality, raising several questions about the validity of peer review in criminal justice evaluation research.

Vegetables, Not Dessert: Teaching Sociological Research Principles in an Evaluation Course

The author recounts her journey from graduate school researcher to full time evaluator to criminal justice professor teaching research and evaluation. With some humor, she describes the disconnect between her enjoyment of research and evaluation and her students’ disdain for the same disciplines, eventually arriving at a happy medium whereby she has learned how to teach criminal justice graduate students some basic evaluation concepts.

Regression Discontinuity Design in Criminal Justice Evaluation

Corrections agencies frequently place offenders into risk categories, within which offenders receive different levels of supervision and programming. This supervision strategy is seldom evaluated but often can be through routine use of a regression discontinuity design (RDD). This article argues that RDD provides a rigorous and cost-effective method for correctional agencies to evaluate and improve supervision strategies and advocates for using RDD routinely in corrections administration. The objective is to better employ correctional resources. This article uses a Neyman-Pearson counterfactual framework to introduce readers to RDD, to provide intuition for why RDD should be used broadly, and to motivate a deeper reading into the methodology. The article also illustrates an application of RDD to evaluate an intensive supervision program for probationers. Application of the RDD, which requires basic knowledge of regressions and some special diagnostic tools, is within the competencies of many criminal justice evaluators. RDD is shown to be an effective strategy to identify the treatment effect in a community corrections agency using supervision that meets the necessary conditions for RDD. The article concludes with a critical review of how RDD compares to experimental methods to answer policy questions. The article recommends using RDD to evaluate whether differing levels of control and correction reduce criminal recidivism. It also advocates for routine use of RDD as an administrative tool to determine cut points used to assign offenders into different risk categories based on the offenders' risk scores.

Theorizing Criminal Justice Evaluation and Research

Theories of the criminal justice process continue to develop and mature, as do theories of crime; however, less need has been perceived for theory in evidence-based policy evaluations. The authors ...

Connecting the dots: Crime rates and criminal justice evaluation research

The National Research Council’s report on evaluating anticrime programs contains sensible suggestions for improving evaluation research in criminal justice but neglects the important role of substantive theory in linking evaluations of anticrime initiatives to variation in crime rates across time and place. A working knowledge of crime rates is essential for designing and evaluating anticrime programs. This essay calls for the development of a policy evaluation infrastructure that would support the continuous monitoring of crime rates, generate knowledge of crime-producing conditions, and link evaluation research findings to one another and to expected policy outcomes, notably crime reduction.

Critical Criminology and Its Discontents: A Response to Travers' Critique of Criminal Justice Evaluation

In a recent article in this journal Max Travers advanced the view that criminal justice evaluation is boring, biased and beset with methodological problems. As evidence of this he cited (among other things) an evaluation of the NSW Drug Court carried out by the present author and several of his colleagues. This article offers a brief response to Travers' claims.

Critical Criminology and Its Discontents: A Response to Travers' Critique of Criminal Justice Evaluation

Critical Criminology and Its Discontents: A Response to Travers' Critique of Criminal Justice Evaluation

Why do evaluation researchers in crime and justice choose non-experimental methods?

Despite the general theoretical support for the value and use of randomized controlled experiments in determining ‘what works’ in criminal justice interventions, they are infrequently used in practice. Reasons often given for their rare use include that experiments present practical difficulties and ethical challenges or tend to over-simplify complex social processes. However, there may be other reasons why experiments are not chosen when studying criminal justice-related programs. This study reports the findings of a survey of criminal justice evaluation researchers as to their methodological choices for research studies they were involved in. The results suggest that traditional objections to experiments may not be as salient as initially believed and that funding agency pressure as well as academic mentorship may have important influences on the use of randomized controlled designs.

When can we Conclude that Treatments or Programs “Don’t Work”?

In this article, the authors examine common practices of reporting statistically nonsignificant findings in criminal justice evaluation studies. They find that criminal justice evaluators often make formal errors in the reporting of statistically nonsignificant results. Instead of simply concluding that the results were not statistically significant, or that there is not enough evidence to support an effect of treatment, they often mistakenly accept the null hypothesis and state that the intervention had no impact or did not work. The authors propose that researchers define a second null hypothesis that sets a minimal threshold for program effectiveness. In an illustration of this approach, they find that more than half of the studies that had no statistically significant finding for a traditional, no difference null hypothesis evidenced a statistically significant result in the case of a minimal worthwhile treatment effect null hypothesis.

Breaking the routine: Assessing the effectiveness of a multi-neighborhood anticrime initiative through qualitative interviewing

This paper reports on the evaluation of a key component of the comprehensive anticrime initiative currently undertaken in Milwaukee, Wisconsin. The Community Partners Program is the programmatic approach designed by the HIDTA/Safe and Sound Initiative to deal with crime and disorder in twenty of the highest crime, lowest income areas of the city. Quantitative data collection constraints favored the use of qualitative research for assessing the efficacy of the Community Partners' strategy in achieving desired results. Focus group interviewing, in particular, proved to be an effective method for conducting process-focused criminal justice evaluation research. Through systematic analysis of transcribed focus group interviews, the researcher was able to identify significant trends and patterns in the way participating residents related to the block-by-block organizing efforts and other crime prevention activities of the Community Partners. The evaluation indicates tangible benefits that neighborhood residents attribute to the program, the activities implemented, and the changes desired and achieved.

Efficiency Evaluation of Michigan Prisons Using Data Envelopment Analysis

The purpose of this study is to demonstrate the utility of data envelopment analysis (DEA) in the evaluation of the administration of justice. The study incorporates a set of inputs and outputs that state and federal agencies as well as prison managers can utilize to assess the relative efficiency of prison operations and to set targets for improvement. DEA is a linear programming technique that constructs a frontier of relatively efficient "decision making units" (DMUs) by optimally allocating weights among the multiple inputs and multiple outputs of the DMUs. DMUs that fall behind that frontier are classified as inefficient. This analysis presents DEA to criminal justice evaluation, demonstrates the general applicability of DEA to prison administration, and serves as a starting point for making comparative evaluations of prisons. An application of DEA to Michigan prisons is used to identify prisons that are inefficient, provide some insight into the source of the inefficiency, and identify potential remedies and target performance levels.

Toward a synthesis of criminal justice planning and evaluation

Abstract The traditional social engineering approach to planning is essentially an allocative planning based on the distribution of limited means among a number of established users. We need instead a transactive planning strategy based on substantive rationality. Such a strategy involves innovative planning which avoids narrow objectivism through a perspec‐tivistic approach. Evaluation is regarded as strategy for system learning. Both planning and evaluation are to be seen within a general systems framework. In this framework, evaluation is applied not only at the outcome stage but also at the stage of problem definition, in the various phases of policy development, and during policy implementation.

New methods in criminal justice evaluation

AbstractThis chapter chronicles the history of a program designed to develop methods for criminal justice evaluation.

Book Reviews: Handbook of Criminal Justice Evaluation. Edited by Malcolm W. Klein and Katherine S. Teilman. Sage Publications, Beverly Hills, CA (694 pages; $39.95 hardcover), 1980

Book Reviews: Handbook of Criminal Justice Evaluation. Edited by Malcolm W. Klein and Katherine S. Teilman. Sage Publications, Beverly Hills, CA (694 pages; $39.95 hardcover), 1980

The Randomized Response Approach

A considerable amount of attention has been afforded the randomized response approach over the past decade in an attempt to refine this method of reducing evasive answer bias in surveys of sensitive information. In this article we review these developments in an introductory fashion and address various concerns in designing a randomized response instrument. Last, the potential applicability of this approach to criminal justice research and evaluation is discussed.

Framing the Questions in Criminal Justice Evaluation

Framing the Questions in Criminal Justice Evaluation

Remedies for the Key Deficiency in Criminal Justice Evaluation Research

More useful criminal justice evaluation research would differentiate offenses and offenders on the basis of causal theory and would interrelate several levels of abstraction. This is illus trated in correctional practice evaluation by a linkage of be havior modification, symbolic interactionist, and sociocultural diffusion theory, from which three propositions on the effective ness of specific treatment methods for particular types of offenders are derived. Research thus far supports the validity of these propositions. Boards of autonomous criminologists and public representatives supervising criminal justice statistics and research agencies would foster more grounding of inquiries in policy- relevant theory.