Social Work Evaluation Research Articles

The Federal GPRA Evaluation Mandate

Abstract The Government Performance and Results Act (GPRA) mandates inclusion of evaluation in federal agency strategic plans, impacting human service programs receiving federal support. Recommended evaluation methods include flexible ongoing and participatory approaches. Yet social work may not be prepared to appropriately respond to the government's demands for accountability. This paper reviews GPRA and its implications for social work practice, evaluation, and research, offering response strategies. The paper suggests that the creation of a National Center for Social Work Research (S. 70/H.R. 663) could help bridge evaluation gaps, offering a centralized mechanism for monitoring evaluation activities and maximizing federal research resources.

Can “Palliative Care Reports” Improve End-of-Life Care for Hospitalized Patients?

Improvement in the care of dying hospitalized patients was sought by providing evaluative feedback to individual physicians (n = 46) and nurses in three biannual “Palliative Care Reports.” Hospitalized adult patients (n = 194) for whom “death was probable” were prospectively identified from the critical care or geriatric services, and a “palliative care” social work evaluation provided. Educational sessions on palliative care were held for physicians and nurses. Medical record review and family interviews were used to generate 10 scores per patient, which evaluated satisfaction with care, relief of symptoms (pain, dyspnea, gastrointestinal, psychological), and the timeliness of care planning. Subjective comments from a quality improvement committee and focussed educational material was also included. Despite these efforts, no change in the cohort's median report scores occurred over the 18 months, but several institutional policies were examined and altered, and interest and support for a palliative care consultation service was obtained.

Book Review: Social Work Evaluation: Principles and Methods

Book Review: Social Work Evaluation: Principles and Methods

Book Review: Social Work Evaluation: Principles and Methods

Book Review: Social Work Evaluation: Principles and Methods

Economic grand rounds: what is psychiatric "medical necessity?.

Economic grand rounds: what is psychiatric "medical necessity?.

Behavioral Problems in Community‐Dwelling People with Dementia

Meredeth A. Rowe, RN, PhD, Zeta Iota & Zeta Upsilon, Associate Professor and Clinical Nurse Specialist at the Alzheimer’s Disease Assistance Center, Decker School of Nursing, Binghamton University, Binghamton, NY; Jo A. Straneva, RN, DN, CS, Zeta Iota, Director, Lourdes Breast Care Center, Binghamton, NY; Kathleen Byrne Colling, RN, PhD, Zeta Iota, Research Scientist and Lecturer, School of Nursing, University of Michigan, Ann Arbor, MI; Theresa Grabo, RN, PhD, CS, CRNP, Zeta Iota, Associate Professor and Family Nurse Practitioner, Decker School of Nursing, Binghamton University, Binghamton, NY. Correspondence to Dr. Rowe, Decker School of Nursing, Binghamton University, Binghamton, NY 13902-6000. E-mail: mrowe@binghamton.edu Accepted for publication March 17, 1999. Increasing evidence shows that behavioral problems place a significant burden on home caregivers of those with dementia (Hamel et al., 1990; Swearer, Drachman, O’Donnell, & Mitchell, 1988; Teri, Borson, Kiyak, & Yamagishi, 1989). The purpose of our small study was to describe the incidence of burdensome behaviors using a newly developed classification scheme. Currently no widely accepted typology for behavioral problems that occur with dementia exists (Taft & CroninStubbs, 1995). After a review of the literature, particularly work by Cohen-Mansfield, Werner, and Marx (1989), behavioral problems were classified as verbally agitated, physically agitated, verbally aggressive, and physically aggressive as shown in the Table. Verbally agitated behaviors are inappropriate vocalizations that disturb self or others. Physical agitation is inappropriate motor activity that disturbs self or others. Forceful vocalizations that are potentially harmful to self or others are classified as verbally aggressive. Physically aggressive behaviors are forceful motor activities that are potentially harmful to self or others. Behaviors were considered problematic only if they were observed by the caregiver and required supervision or intervention. Data were collected from a retrospective review of records for the years 1992-1996 conducted in the northeastern United States. Subjects were persons with dementia who had been evaluated at a community-based clinic for the diagnosis of dementia. Data were obtained from records of extensive medical, nursing, and social work interviews and evaluations performed in the home and the clinic by staff specialized in gerontology. The form for data collection included information about behavioral problems, past and present medical conditions, and the results of standardized cognitive and psychological tests. Clinical records based on caregiver interviews and patient evaluations were reviewed to determine the presence of any problematic behaviors. Behavioral categories were dichotomously scored as present or absent, depending on whether any of the typifying behaviors were exhibited. Twenty-three of the original 81 records did not meet inclusion criteria because of incomplete data or diagnoses other than dementia; 58 subjects with confirmed diagnoses of dementia were studied. Dementia subjects ranged in age from 52 to 93 years (mean = 78), and were predominately Caucasians (>90%), women (65%), and married (57%); 19% lived alone. Behavioral Problems in Community-Dwelling People with Dementia

Analyzing longitudinal rating data: A three-level hierarchical linear model

Using generalizability theory as a guide, this study discusses statistical problems and strategies of analyzing rating data involving multiple raters--a common type of data issue frequently encountered in social work evaluations. To disentangle raters' bias from clients' true change, the study shows the importance of looking into the multifaceted structure of measurement error. To analyze data containing nonnegligible variability associated with raters, this study proposes using a three-level hierarchical linear model. It demonstrates that the three-level model produces a better model fit to the data, smaller sample residual, and more accurate significance testing than the popular two-level model when analyzing rating data with nonnegligible raters' influences. Key words: generalizability; hierarchial linear model; data; multiple raters; seriously emotionally disturbed children Monitoring client progress and evaluating the effectiveness of a program intervention are of central interest to social work research. Studies with these objectives often involve designs and collection of subjective rating data. Analyzing such data, however, is not as straightforward as investigators may wish. In a inquiry, research questions often address the change of client outcomes over time. Because change is inevitably a function of measurement, disentangling raters' bias from the clients' true change is crucial to most data analyses. This article discusses statistical problems and strategies in analyzing one type of rating data, namely, subjective ratings made by multiple raters at two or more time points. By this definition, we exclude the methodologically less complicated case of self-administered data from consideration. Examples of multiple-rater data include teachers' and parents' ratings of children's behavioral functioning, case workers' and service consumers' ratings on a program outcome, and so forth. Longitudinal rating data are likely to be collected by more than one rater. In a study, keeping track of study participants over time is expensive. By the same token, using a group of same raters (interviewers) requires additional resources that may be practically infeasible. Just like the attrition of study participants, raters may change jobs and make themselves no longer a part of the data-collection team. During the past two decades, the number of studies has grown at a fast pace, primarily because more social worker-researchers have recognized the importance and advantages of a dynamic inquiry. (We searched on the keyword longitudinal from the database Social Work Abstracts between 1977 and 1997, and found 542 journal articles in total. Among these articles, 191 [or more than one third] were published during the past seven years. Longitudinal studies often were used in the areas of program evaluations, such as assessment of programs preventing drug and alcohol abuse, mental health treatment services, welfare, foster care, and adoption programs; and correlates studies, such as factors associated with poverty, homelessness, children's aggressive behaviors, teenage pregnancy, living arrangement and service use for elderly people, HIV epidemic, and so forth.) However, the design of such studies has not received equally important attention. A typical scenario is the application of an instrument that was primarily designed for a cross-sectional study to other time points. Although the instrument has proven to be valid and reliable at a static time point, its usefulness to dynamic inquiry remains questionable. By doing so, the researcher may underestimate the importance of choosing the optimal number of raters and time points to achieve maximum generalizability. In this study we addressed the following questions: How can researchers analyze rating data containing multiple sources of measurement error? …

Social Work Assessment of Adaptive Functioning Using the Vineland Adaptive Behavior Scales

The assessment of client adaptive functioning is often an important component of a comprehensive social work evaluation. The Vineland Adaptive Behavior Scales (VABS) are the most commonly used quantitative measures of adaptive functioning for clients meeting the criteria for a wide range of disorders. We review the development of the VABS and current knowledge pertaining to the instrument's reliability and validity. We conclude that the ability to administer and interpret the VABS is an important skill for clinical social workers to acquire.

Responding to child sexual abuse: an evaluation of social work by children and their carers

This paper presents the views and perceptions of 35 children and their current carers regarding the social work response to child sexual abuse. Interviews with the children and carers, including foster carers and residential carers, were carried out 2–4 years following disclosure, thus the nature of social work support in the longer term, as well as the initial response, is assessed and described. Three evaluations of social work support are presented: the child‐ren’s evaluations of social workers known since disclosure, and two evaluations from the carers, one focusing on the child’s needs, and one focusing on their own as carers. All three evaluations perceived the majority of social workers as having helped. Aspects of social work considered essential included providing emotional support and reassurance, providing information and explanation and arrang‐ing and coordinating services, including counselling or therapy. Carers of children who were abused by someone outside the family were less likely to receive social work support. The frustra‐tion and disappointment expressed by some of these carers reaffirms the importance of assessing needs even when the child is considered protected from further abuse. The findings of this study support the need to refocus responses to abused children and their families.

The Use of Experimental Design in Social Work Evaluation

The evaluation of social work practice is distinguished from most other research, in a way that its purposes and “treatment” are different from others. This paper illustrates why experimentation does not often serve the purposes of social work evaluation and refutes the conventional viewpoint that programme evaluation has the same general design as an experiment in the natural science.

Ethics in program evaluation

As is the general practice, the evaluation of social work is fitted with ethical structures, some of them encoded, most of them implicit in the activities, rationales, and compassions of the profession. It is useful to have multiple codes, expressing ethical concern from different perspectives, for each reaches into different shadows. Ethical behavior is not so much a matter of following principles as of balancing competing principles. Realization and resolution of ethical conflict come largely from within the individual social worker and evaluator. We describe three program evaluation situations to illustrate these points.

Internal and external evaluation of social work

In the UK a number of factors are bringing the idea of evaluation of social work into a new prominence. This paper considers the different meanings which may be attached to the terms internal and external in the context of social work evaluation, and outlines the differences between scientific, developmental and managerial approaches. It is argued that while some suspicion of the purpose of current demands for evaluation may be justified, the calls for greater accountability have some justice and are unlikely to diminish. If the domination of bureaucratic and financial considerations over professional judgements is to be avoided, social workers must overcome their alleged anti‐evaluative professional culture and consider how to systematically evaluate their own practice, both individually and collectively.

Linking the university and the social agency in collaborative evaluation research: principles and examples

This paper presents an analysis of components of successful social work evaluation research practice. Among the key obstacles to successful evaluations of social work intervention are the impoverished knowledge base on which many social work interventions are based, the isolation of researchers and practitioners in the conduct of evaluations, and during research implementation failure to cope with the changing field context. To remedy this situation social work needs to invest in university and social agency partnerships focussed on knowledge building for the profession. In such partnerships the traditional roles of the researcher and the practitioner are altered so that shared responsibility for knowledge development is possible. In addition, to deal with changing field contexts the research team needs to be continuously engaged in problem solving and redesign. An initial example of such a partnership is described together with a case study illustrating implementation of a multi‐site field experiment evaluation.

The Importance of Context in Understanding Homelessness and Mental Illness: Lessons Learned From a Research Demonstration Project

Research reports on the housing outcomes for persons who are homeless and mentally ill have focused on client characteristics, program type, and services as independent variables, with mixed results. From social work practice, evaluation theory, and public policy perspectives, context is an important variable. Yet, it has received scant research attention in studies of the outcomes of persons who are mentally ill and homeless. This article summarizes research results from a demonstration project providing outreach or linkage services to this target population, illustrating the significant impact of context variables (site and recruitment source) on client characteristics, implementation, qualitative and quantitative service assessments, and housing outcomes. The discussion suggests how these contextual factors may operate, and it goes on to make recommendations to improve social work research and practice concerning the important dimensions of context that should be assessed.

Beyond Retroduction? -- Hermeneutics, Reflexivity and Social Work Practice

Over the last three decades qualitative research methodologies have been in the ascent within social science. Yet social work evaluation studies have tended to be quantitative in nature, conventially relying upon the generation of criteria against which interventions are retrospectively judged. The generation of such criteria inevitably depends upon pre-suppositions, which in themselves go unresearched. As a consequence the .sense making activities on which social work interventions depend are rendered immune from critical analysis. This reflects a broader tendency for social work to cling to naive realist epistemologies, which are arguably obsolete within the interpretive paradigm in which its activity is properly located. By examining the debates within interpretive social science, this paper argues for an approach to social work assessment which avoids the pitfalls of naive objectivism and the nihilism of anarchic relativism, whilst retaining creativity, imagination and hope.

Evaluating Social Work: Progress and Prospects

Within the past decade, evaluative research has achieved an established and well-regarded place within the repertoire of social work inquiry. Although increasingly wide-ranging and methodologically sophisticated, social work evaluation still has to contend with important challenges. These include taking greater account of the several audiences of research, namely, service users, practitioners, and policy makers; developing, through a variety of research methods, the range and quality of evidence of impact; and drawing research, policy, and practice into closer union. To reach its full potential, social work research must focus both on individuals and their social worlds and must therefore be firmly rooted in the social sciences.

Evaluating Employee Assistance Programs

This article provides a review and conceptual framework for the evaluation of social work and other services provided by employee assistance programs (EAPs). A discussion of the unique challenges and opportunities afforded in the evaluation of BAP services is also presented.

Applying Single-Case Evaluation in Social Work

Mansoor A. F. Kazi is a Senior Lecturer in Social Work and Applied Social Studies at the University of Huddersfield. He is also Director of the Centre for Evaluation Studies based at the University. Following graduation from the London School of Economics, he went on to complete an MA in Social Work at the University of Hull. His previous position was Service Manager of Rochdale Education Welfare Service and Team Leader of the local education authority's Performance Review Team. He is currently undertaking Ph.D. research in the application of single-case evaluation in social work and health services.

Forthcoming Issues of APJSW Call for Papers and Reports on Research and Evaluation in Social Work and Management of Social Services

Forthcoming Issues of APJSW Call for Papers and Reports on Research and Evaluation in Social Work and Management of Social Services

A Human Rights Approach to Social Work Research and Evaluation

Although the advancement of human rights and research are considered important aims for social work, they are viewed as relatively independent activities. This paper presents an argument for the integation of human rights and social work research and proposes some ways this objective might be accomplished. Suggestions for teaching a human rights approach to research are also described.