Program Evaluation Process Research Articles

Community‐based organizations and state initiatives: The negotiation process of program evaluation

AbstractEvaluators must operate in a multicontextual environment with multiple stakeholders. As the hierarchy of stakeholders becomes more complex, evaluators must react to and work within the ecological environment in which their evaluated programs are conducted. Recognizing the factors that influence the evaluation plan and knowing the best strategies in dealing with different authorizing environments can reduce the difficulty of negotiating with multiple stakeholders and increase the evaluation's validity.

Évaluation de l'implantation d'un programme d'hébergement en alternance de personnes âgées en perte d'autonomie et de ses effets sur les aidants naturels

ABSTRACTWith a view to reducing the exhaustion experienced by informal caregivers and thus encourage the frail elderly to remain at home, a new free respite care program, which offers to house the frail elderly one week per month, was evaluated using an array of multiple cases. Ajzen and Madden's (1986) theory of reasoned action was used as a model to explain the effect of the program on the caregivers' intention to resort to permanent accommodation. Fourteen informal caregivers took part in the program evaluation process. An analysis of the reports of the meetings of the program's Implementation Committee revealed that human and material resources had been used as established in the initial plan. Seven subjects completed the 12 months planned for evaluating the program and were able to continue with the program thereafter. An analysis of the data, using graphic representations and pre-experimental and experimental phase medians, did not reveal any decrease in the perceived burden and in the intention to resort to accommodation in subjects who had completed the program evaluation process. Those who dropped out did so in part because of the high level of perceived burden and the caregivers' intention to resort to accommodation prior to entering the program. Unstructured telephone interviews with the caregivers suggest, moreover, that an elderly person's loss of autonomy, a good financial situation, and the caregiver's decreased feeling of guilt at the idea of housing his/her relative are important determining factors in the caregiver's decision to use permanent accommodation.

ENHANCING INSTRUCTIONAL PROGRAMS THROUGH EVALUATION: TRANSLATING THEORY INTO PRACTICE

This article describes an instructional program evaluation process used at Fox Valley Technical College, Appleton, Wisconsin. Evidence indicates that this is a valuable process which can be used to enhance instructional programs. This evidence was uncovered during a recent evaluation designed to determine appropriate changes to enhance the process. Surveys, interviews, and focus groups were conducted with various stakeholders to obtain feedback on the relevance for each instructional program evaluation component. In addition, an analysis of the instructional program evaluation recommendations was reviewed to determine the level to which the recommendations were implemented. When the assessment results were compared to the effectiveness criteria, the evidence supported the conclusion that the instructional program evaluation process is effective in achieving the intended outcomes. The assessment information demonstrated that stakeholder perceptions were positive.

Performing a program evaluation in a family case management program: determining outcomes for low birthweight deliveries.

Program evaluations have become an integral part of public health practice. To illustrate the program evaluation process, an evaluation of clinical outcomes for clients of the Family Case Management (FCM) program administered by the nursing division of the DuPage County Health Department, Wheaton, Illinois, is presented. Birth outcomes for 1996 clients of the program who had a previous low-birthweight (LBW) infant and a subsequent birth using the prenatal services of the FCM program were analyzed. A purposive sample of clients was selected from the FCM prenatal program. Birth outcomes including infant gestational age and birthweight, maternal weight gain, and other key variables were compared to provision of health department services. Descriptive statistical methods were used to explore the relationship between birth outcomes and prenatal care between the first pregnancy and the second pregnancy. Analysis of the data showed improvements in maternal weight gain, infant birthweight and gestational age, prenatal care access, public health nurse (PHN) contacts, and physician visits. A positive association was shown between the early entry and increased use of prenatal services to improved birth outcomes in the subsequent pregnancy.

An Application of a Basic Quality Management Approach in a Higher Education Environment

ABSTRACT The Grand Valley State University Engineering Cooperative Education Program (ECEP) was able to utilize simple quality principles and tools for improving the program process. The main goal was to utilize a basic quality management approach to improve the efficiency of the engineering cooperative education program placement and evaluation process. After two years of redesign and implementation the efficiency of the program was considerably improved. All students participating in the program received competent training before the first cooperative experience and all were placed in cooperative positions on time. Furthermore, proactive measures were instituted as a continuous improvement measure to insure the efficiency of the process with the expected growth of the Grand Valley State University Padnos School of Engineering program.

Evaluation Research in Context: A Community Application for Youth and Family Programs

Despite extensive literature on evaluation strategies and models, few published reports exist relative to practical applications of such strategies. This article follows one community as it mobilized around youth issues, resulting in a pregnancy prevention program. The intent of this article is to outline the application of evaluation concepts and strategies and to examine the effect of the larger program development and evaluation processes rather than to present specific results of the evaluation data collected over the course of this work. The utility and merit of evaluation concepts will be discussed as the authors follow a small southwestern rural community over nearly 2 years.

Using Hypermedia in Response-Based Literature Classrooms

AbstractResponse-based literature teaching and learning regards readers as active meaning-makers, and therefore emphasizes the importance of teaching and learning the processes of literary understanding, which are viewed as both socially and personally mediated. Many of the qualities of hypermedia suggest that it may be well suited for supporting such approaches. The National Center for Research on Literature Teaching and Learning’s ongoing Multimedia and Literature Teaching and Learning Project is concerned with systematically exploring this idea. The project’s first stage, detailed in this article, involved reviewing commercial multimedia literature applications from a response-based perspective. This article describes criteria developed for evaluating applications from that perspective, as well as the program acquisition and evaluation process. Findings suggest that the hypermedia literature applications currently commercially available, although technologically quite good and perhaps supportive of tex...

Accreditation in Rehabilitation Counseling: Historical and Current Context and Process

The early history of the Council on Rehabilitation Education (CORE) is described through discussion of CORE's development, recognition, and foundation research at the University of Wisconsin, Madison. CORE's multi-stakeholder program evaluation process is described. Current trends and issues in accreditation of rehabilitation counselor education are discussed.

Evaluation practices of model early childhood programs

During the last half of the twentieth century, increasing numbers of young children have been placed in out‐of‐home care. The need to assure quality experiences for young children in such settings is a pressing social and parental need. One important method of insuring program quality lies in the program evaluation process. This article summarizes literature on early childhood program evaluation, presents an overview of the national standards addressing quality in early childhood programs and presents the results of a study of evaluation practices of model early childhood programs. A questionnaire was devised based on the program evaluation literature. The questionnaire asked programs to identify the methods and staff used to evaluate nine program components. Programs were identified in the literature which were cited for exemplary practices. The questionnaire was sent to these programs. The study and analyzes the results of the information returned by the model programs.

Needs Assessment Experience Among Area Agencies on Aging

Assessing needs is a responsibility of area agencies on aging (AAAs), as explicated in the Older Americans Act. A national survey on AAAs' needs assessment experience indicates that all 308 participating AAAs have developed methods to assess needs and prioritize services. AAAs perceive that needs assessment is an important part of planning and an on-going process of program monitoring and evaluation. With reference to the multi-dimensional aspect of needs assessment methods, a framework with review procedures, service allocation processes, and output references is developed.

Focusing Agricultural Education Research: Strategies For The Professor

No abstract available

Professionalism and School-Based Program Evaluation

Abstract: Participating in program evaluations is a new part of the classroom teacher's role. If teachers are to embrace the opportunities provided by this participation, they need to understand the process of program evaluation and to appreciate its potential to empower them in their professional lives. This article discusses these issues in the context of Saskatchewan's current curriculum change and offers an illustration of one instance where participating in a proper evaluation study did enhance the professionalism of a teacher.

A National Study of Nursing Education's Use of a Model of Program Evaluation

This study indicates nursing educators have developed a sensitivity to the issues of program evaluation, are interested and committed to demonstrating accountability to their various audiences through the process of program evaluation, and recognize the contribution a model of program evaluation can make to the evaluative procedures they must implement. At this time, however, nursing education could benefit from the introduction of a new or revised model of program evaluation that is easily understood by the nursing faculty and adapts well to the specific concerns of the nurse educator.

Energy conservation programmes: The analysis and measurement of their effects

Over the past decade, energy conservation programmes have been used extensively by US utilities. However, during much of this period, their popularity has been uncritical — insufficient programme evaluation was available to indicate whether the programmes were economically reasonable or not. In the past three years, however, a substantial array of evaluation results have become available. Much of this evaluation has been stimulated by the Electric Power Research Institute (EPRI). EPRI's initial contribution to the process was a watershed workshop in 1982 on the effects of utility residential conservation programmes. Since then, EPRI and a variety of utilities, public agencies and private consulting groups have continued to refine and extend the evaluation process. This paper reviews the programme evaluation process begun with the 1982 EPRI workshop and discusses the extent and validity of its methods and findings.

Cost/effectiveness analysis of alcoholism and drug abuse treatment programs: The relevance of recidivism and resource absorption

Cost/effectiveness measures are increasingly being applied to alcoholism and drug abuse treatment programs. Program evaluators usually regard readmissions as inadequate for assessing outcome, and resource absorption is rarely examined; this paper presents the argument that increased attention to recidivism and resource absorption is integral to the program evaluation process. Readmissions formed 53% of admissions to U.S. drug abuse programs, and in alcoholism treatment centers, 4% of the patients accounted for 24% of the detoxications. Problems with assessing cost/effectiveness are associated with duration of treatment, average cost, time span of analyses, quasi-experimental designs, multiple episodes of treatment, “silting up, ” and extrapolation. Five approaches to dealing with these problems are suggested: analyzing existing program data bases for recidivism and resource absorption, considering recidivism when evaluating programs, tempering clinical enthusiasm with skepticism, matching patients with levels of care appropriate to previous treatment history, and assessing additional benefit derived from increasing amounts of care per patient as part of cost/effectiveness analysis.

Evaluating programs in applied settings through behavioral consultation

A behavioral consultation school psychology service model is presented which provides a vehicle for services and communication of process and outcome measures in program evaluation research. Within the context of the problem-solving perspective of applied behavior analysis, the consultation model (a) promotes process and outcome evaluation across a wide variety of settings, situations, and time; (b) leads to use of an applied research technology with wide applicability in educational settings; and (c) provides a communication technology for the program evaluation process through a four-stage problem-solving strategy. It is argued that the professional behavioral school psychologist can provide technical expertise to function as a system change agent within this evaluation model.

Impact Assessment: A Technique for Evaluating Criminal Justice Programs

The systems perspective focuses on the importance of considering the linkage between criminal justice institutions in the program evaluation process. This paper reports on a study which employed the technique of social impact assessment to examine the effect of a change instituted by a juvenile justice probation and court agency on police operations. The change was the introduction of a citation procedure for making juvenile misdemeanor arrests and an arbitration session for carrying out the initial screening of cases. The authors used this study to illustrate how impact assessment can be employed to identify a wide range of inter-system effects. The findings that confirm the existence of impacts can be used to modify the initial program innovation or to insure support for successful innovations. The conclusion drawn from the discussion of the utility of such research is that impact assessment is a technique particularly well suited to identifying unexpected and/or unwanted effects of a program innovation on other parts of the justice system.