Barriers to Effective Performance Review: The Seduction of Raw Data

Abstract

As the staff in public agencies have become more educated and professional, their operations have also come under more public scrutiny and criticism (Cigler, 1990). A management or performance review system is, therefore, instrumental in its ability to assist administrators in making informed decisions and to plan for activities that will be preventive in nature. Review processes are needed that will evaluate current operations against accepted goals and project future accountability needs. Performance review itself is a broad concept that encompasses a variety of measures that provide organizational authorities with information about operations. Assessments would be sensitive to statutory requirements, regulations, and the goals and objectives articulated by the organization. The purpose of such a review is to improve the quality of management internally and to meet the information needs of outside authorities. Obviously, management review systems today are invariably tied to research and automation. Both research and performance review systems must use available technology in a wide variety of mainframe and personal computer data analysis packages that have offered managers many ways to track operations, solve problems, and answer questions about efficiency and effectiveness (Hy, Waugh, and Nelson, 1987). These automated systems, however, are only superficial tools. The real core of an organization's review system is a commitment to performance-based management principles and to scientific inquiry and rational decision making.(1) The Controversy over Useful Information and How It is Used Although all government agencies have developed their own management information systems, there is still debate over information is most useful. As Hummel (1991) suggests, for many managers, experience, subjectivity, and the application of a good story, still seem preferable to scientific methodology. Although methods may vary between states and agencies within states, several common themes pervade the gathering of information for performance review. First, performance review is an active process of maintaining control over the organization's planning and operations. Secondly, performance information is accumulated in a cycle of input and evaluation. Across the country, there may be differences in who gathers, processes, and uses the information in each government agency; however, the needs and goals of these activities are more alike from institution to institution and from state to state than one may initially believe. Finally, the success of performance review hinges on management's recognition of its value. Top administrators must remain committed to improving the quality and timeliness of the information acquired. Whether operations are large or small scale, the need exists for continued long-range and short-range planning, development, refinement of the review process, and the allocation of resources toward that end. Although the placement of auditors, monitors, and researchers in a system will facilitate the review process, it does not insure that performance review will take place. In this article, we examine criminal justice systems to highlight the types of problems that develop between researchers and administrators in the creation of performance review. Program Evaluation More and more, today's managers are becoming consumers of many types of research. As a result, die field is finding that many of our research procedures do not work as well in evaluation settings as they have in traditional situations. In addition, we are learning that the question of what works is not a simplistic one. In criminal justice, for example, the notion of reduction in crime as the primary measure of effectiveness has been supplemented by cost-benefit, quality-assurance, time-savings, and difficult-to-measure humanistic goals. Fortunately, new techniques are now being developed to measure these alternative goals. …