Program Evaluation Data Research Articles

The Academic Dental Careers Fellowship Program: A Pilot Program to Introduce Dental Students to Academia

The Academic Dental Careers Fellowship Program (ADCFP) was established in 2006 by the American Dental Education Association (ADEA) and the American Association for Dental Research (AADR) with the financial support of the ADA Foundation to encourage dental students to consider careers in dental education and to provide participating fellows with insights into academic life. The ADA Foundation provided funding during the 2006-07 academic year for eleven dental student fellows, who were paired with faculty mentors at their respective schools. Fellows and mentors attended a two-day retreat in the summer of 2006, and over the course of the subsequent year in dental school, the fellows with guidance from their mentors participated in preclinical laboratory, classroom, small-group, and clinical teaching experiences; designed and implemented a research project; developed a philosophy of education; completed career reflection essays; assembled a portfolio to represent their ADCFP activities and projects; conducted a series of interviews with faculty designed to expose students to roles, issues, and career paths in academic dentistry; and presented a synopsis of their experiences at the ADEA Annual Session in New Orleans in March 2007. The fellows and mentors completed midyear and end-of-year evaluations of the ADCFP in which feedback and recommendations were collected by telephone interviews and questionnaires. Fellows reported positive experiences and an increased interest in and understanding of academic careers. Mentors also evaluated the ADCFP positively and reported enhancements in their mentoring skills. This article describes the goals and format of the ADCFP, summarizes program evaluation data elicited from the fellows and mentors, and proposes recommendations for future fellowship classes.

Applying a Response-to-Intervention Model for Early Literacy Development in Low-Income Children

This article describes the design and implementation of a program that incorporates a response-to-intervention (RTI) framework for promoting the development of early literacy and language skills among low-income minority children. The early literacy program, called the Exemplary Model of Early Reading Growth and Excellence, or EMERGE, combines classroom practices that are grounded in empirical research, a multitiered intervention hierarchy, high-quality professional development, and continuous progress monitoring to help children in Head Start classrooms acquire early literacy competencies to prepare them for later success in school. Preliminary program evaluation data are presented, and challenges associated with applying an RTI model in early childhood education are addressed.

Uncovering potential agency: assessing orphaned and vulnerable children (OVe) empowerment via the ‘instrumentalisation of disorder’ as informed by Bourdieu

From a macro-viewpoint, the devastating impact of adult HIV/AIDS mortality upon 1.2 million South African minors cannot be denied. One could be forgiven for excessive pessimism due to the lack of concentrated interventions to assist these orphaned and vulnerable children (OVC). Nevertheless. a foray into the world of a grass-roots programme lacated in Eldorado Park. Gauteng. namely the Elton john Masibamisane Centre (EJMC). provides room for qualified optimism. Using Bourdieu's theory of practice as a lens to analyse programme evaluation data. we came to believe that the provision of economic. social. cultural and symbolic copitals could transform the lives of the Eldorado Park OVCS. Nevertheless. the provision of copitals was imperfect and often characterised by shortages that were remedied by ‘instrumental disorder’. We interpreted this to be indicative of agency within a blended Western-African OVC empowerment habitus. In reflecting on the findings we commended EJMC for surviving despite constraints. This is. however. a short term view. To ensure longer term sustainability we recommend that EJMC staff extend their agency to pressuring the State for greater assistance as it is unrealistic to allow this community organisation to bear the brunt of Eldorado Pork OVC empowerment indefinitely. In deepening understandings of OVC work in South Africa. especially in terms of its operation within a skewed power framework, we recommend that more studies of this nature take place.

Cost analysis of the National Breast and Cervical Cancer Early Detection Program

AbstractBACKGROUND.The National Breast and Cervical Cancer Early Detection Program (NBCCEDP) was established by the U.S. Congress in 1990. In recent years, there has been an emphasis on ascertaining the NBCCEDP's costs of delivering screening and diagnostic services to medically underserved, low‐income women. The objective of this report was to address 3 economic questions: What is the cost per woman served in the program, what is the cost per woman served by program component, and what is the cost per cancer detected through the program?METHODS.The authors developed a questionnaire to systematically collect activity‐based costs on screening for breast and cervical cancer from 9 participating programs. The questionnaire was developed based on well established methods of collecting cost data for program evaluation. Data were collected from July 2003 through June 2004.RESULTS.With in‐kind contributions, the cost of screening services to women in 9 programs was estimated at $555 per woman served. Without in‐kind contributions, this cost was $519. Among the program components, screening and coalitions/partnerships accounted for the highest and lowest cost per woman served, respectively. The median cost of screening a woman for breast cancer was $94, and the cost per breast cancer detected was $10,566. For cervical cancer, these costs were $56 and $13,340, respectively.CONCLUSIONS.Costs per woman served, screened, and cancers detected are needed for programs to accurately determine the resources required to reach and screen eligible women. With limited program resources, these cost estimates can provide useful information to assist programs in planning and implementing cost‐effective activities that could maximize the allocation of program resources. Cancer 2008. Published 2007 by the American Cancer Society

A Practical Framework for Evaluating a Culturally Tailored Adolescent Substance Abuse Treatment Programme in Molokai, Hawaii

Objective. Successful substance abuse treatment requires many changes in behavior, attitude and skills. Culturally tailored approaches to substance abuse treatment have shown initial success, but are not yet accepted as best practice models. In order to document programme effectiveness of a new culturally tailored substance abuse treatment programme on the rural island of Molokai, Hawaii, the authors worked to develop a multi-level evaluation plan to measure behavior changes occurring after participation in activities targeting identified causes of substance abuse in the population of interest. Methods. The authors compiled interview results to develop a map of identified causes of substance abuse in the community studied. Strategic planning then identified the specific activities aimed at impacting identified root causes. A literature review was performed to document the effectiveness of such activities. An evaluation plan was developed to measure programme impact on antecedent conditions contributing to substance use in this community. Results. Prioritized causes of substance abuse in the target group included low self esteem, lack of self identity and life plan, and limited communication and conflict resolution skills. Activities targeting these conditions included cultural activities, group counseling, and individual counseling. Literature to support the benefit of addressing these factors was uncovered, and evaluation methodology was developed to measure changes in behaviors, attitudes, and practices, as a measure of programme success. Discussion. While programme evaluation data is still being collected, the authors have demonstrated a sound foundation for programme activities, and designed methodology for collecting meaningful data to measure programme effectiveness at changing important root causes of substance abuse in a rural Native Hawaiian community.

The estimation of Unwanted Fertility

The estimation of unwanted fertility is a major objective of demographic surveys, including DHS surveys conducted in Asia, Africa, and Latin America. Levels and trends in unwanted fertility are important input to the formulation of population policy and the evaluation of family planning programs. Yet existing methods for estimating unwanted fertility are known to be defective, among other reasons because they rely on subjective data whose validity and reliability are questionable. In this article, we propose a new estimator of unwanted fertility-the "aggregate prospective estimator"--so named because it depends on the stated preference for another child at the time of the survey, the fertility-desires item consistently shown to possess the highest validity and reliability. Under reasonable assumptions, the aggregate prospective estimator produces less biased estimates of unwanted fertility than the most widely used existing methods. The new estimator has the limitation of generating only aggregate-level estimates, but such estimates are the primary data for policy formulation and program evaluation. The new estimator is presented in this article, along with an evaluation of its underlying assumptions and its sensitivity to several sources of error. In an illustrative application to recent DHS data from six countries, the new estimator yields substantially higher estimates of unwanted fertility than existing methods in all six countries.

Student-Led Stress Management Program for First-Year Medical Students

Background: The medical education community has emphasized repeatedly the importance of teaching stress management and self-care skills to medical students. However, descriptions and evaluations of intervention programs are infrequent. This article describes a student-led stress management program for 1st-year medical students and summarizes program evaluation data from 1,111 participants. Description: The Stress Management Program is a voluntary activity that involves small groups of 1st-year medical students meeting with 2nd-year student coleaders. At the beginning of the fall semester, each group meets 1 hr per week for 7 consecutive weeks. Two psychologist faculty members serve as program coordinators. Evaluation: Mean annual participation rate for 1st-year students was 94% over 16 years. Program evaluation results are strongly positive. Conclusions: Based on longevity, participation, and student feedback, the program has been successful. Recommendations for creating and maintaining similar programs are offered.

Domestic Violence and Chemical Dependency Co-Morbidity: Promoting Eclectic Responses to Concomitant Mental Health Concerns

Chemical dependency and a history of physical or sexual violence are problems commonly experienced concomitantly among women, yet barriers persist to implementing programs that deal with them together. This article describes an eclectic, cross-problem group intervention designed to acknowledge the specific needs of women who have experienced problems of both chemical dependency and domestic violence. Program evaluation data suggest a positive effect for the proposed intervention, providing support for creating and maintaining policies among mental health provider organizations to recognize and address common co-morbidity.

Testing Conditional Independence via Rosenblatt Transforms

This paper investigates the problem of testing conditional independence of Y and Z given λθ(X) for some unknown θ ∈ Θ ⊂ Rd, for a parametric function λθ(·). For instance, such a problem is relevant in recent literatures of heterogeneous treatment effects and contract theory. First, this paper finds that using Rosenblatt transforms in a certain way, we can construct a class of tests that are asymptotically pivotal and asymptotically unbiased against √n-converging Pitman local alternatives. The asymptotic pivotalness is convenient especially because the asymptotic critical values remain invariant over different estimators of the unknown parameter θ. Even when tests are asymptotically pivotal, however, it is often the case that simulation methods to obtain asymptotic critical values are yet unavailable or complicated, and hence this paper suggests a simple wild bootstrap procedure. A special case of the proposed testing framework is to test the presence of quantile treatment effects in a program evaluation data set. Using the JTPA training data set, we investigate the validity of nonexperimental procedures for inferences about quantile treatment effects of the job training program.

Developing bed managers: the why and the how

This paper describes and analyses the role of bed managers within hospitals, and how this can be supported through a development programme. Bed management has increased in importance as access targets have developed, and appreciation of the importance of patient flow has grown. The bed management role is usually performed by nurse managers, and can be particularly stressful and managerially complex. Despite calls for training support, little has been available. In response we have developed and delivered a training programme to over 70 staff in the past 2 years. The programme evaluation data is used to show that it has had an impact on the knowledge and skills of the bed managers involved, but also on their subsequent behaviours. These changes have translated into results for the organization and the individual, including staff retention, promotion and fewer cancelled operations. We conclude that the programme is an effective way to enhance the knowledge and skills of bed managers, and that the enhanced profile of bed management raises challenges for nurse managers in many areas of the organization.

The River Street Autism Program

An urgent demand from Connecticut parents for behavioral intervention resulted in the development of the River Street Autism Program (RSAP). This research-to-practice program implements intervention service based on empirical research findings conducted with children diagnosed with autism and pervasive developmental disorders. RSAP is provided through a regional service center and provides services for children entering the program at 2 to 5 years old. Because of the diverse nature of the districts served by RSAP, the delivered services varied according to the needs of the districts, available funding, and family preferences. Program evaluation data were therefore examined with regard to outcomes for children who received programs with differing numbers of treatment components. Treatment components that varied across children were treatment intensity, duration, extent of family participation, staff training, and supervision. Outcome data revealed that families reported greater gains in child functioning and quality of life when children received programs with more treatment components.

Optimizing Evaluation of Nursing Education Programs

Program evaluation data must be meaningful, credible, objective, and responsive. The content, inputs, process, and product model provides an effective and useful framework for organizing the evaluation process. The model is a decision/accountability-oriented approach that emphasizes both proactive and retroactive evaluation to assess program improvement and quality. The authors describe the steps used in developing an evaluation plan to address undergraduate, graduate, and specialty standards and to serve as a framework for data collection, decision-making, continuous quality improvement, and demonstration of outcomes.

Birth Characteristics Associated With Early Intervention Referral, Evaluation for Eligibility, and Program Eligibility in the First Year of Life

The Individual with Disabilities Education Act mandates Early Intervention (EI) services for infants and toddlers with developmental delay. We assessed the percentage of infants at risk for developmental delay due to characteristics present at birth who were referred to Massachusetts EI within 1 year of birth, evaluated for eligibility, and eligible for services. In addition, we identified birth characteristics that independently predicted 0-1 year program referral, evaluation, and eligibility. The Pregnancy to Early Life Longitudinal (PELL) data system linked birth certificate, hospital discharge, and EI program data of 219,037 infants born in Massachusetts, 1998-2000. Multivariate logistic regressions identified independent infant predictors of referral, evaluation, and eligibility. Of 219,037 births, 14,852 (6.8%) were referred to EI within 1 year. Birthweight<1200 g (OR=9.7, 95% CI 3.3-12.9) and birthweight 1200-1499 g (OR=7.4, 95% CI 5.8-9.5) strongly predicted referral. Referral was high (88%) among infants with two or more birth risks. Among referrals, 88% were evaluated for eligibility. The strongest predictor of evaluation was triplet birth (OR=4.3, 95% CI 1.6-11.8). Among infants evaluated, 85% were determined to be eligible. EI referral and evaluation are high among infants born at risk for delay in Massachusetts. Some characteristics not included in the state's high-risk definition (e.g. birthweight 1200-1499 g) were identified that predicted referral. Most referrals were eligible for services. Results demonstrate the value to states of using linked population and program data for program evaluation.

Development of the Diabetes Education, Counseling, Information Delivery and Evaluation (DECIDE) Program: A Health Promotion Intervention For Preadolescents with Type 1 Diabetes

Careful attention to self-care behaviors is crucial to achieving good health outcomes in children and adolescents with type 1 diabetes. Despite the unequivocal research findings that adolescence is a time of poor metabolic control among affected individuals, there have been few prevention efforts geared toward health promotion in this age group. The diabetes education, counseling and information delivery, and evaluation (DECIDE) Program is intended to prevent the deterioration of self-care behaviors often evident during adolescence by intervening during the earlier, more stable time period of preadolescence. This paper describes the foundation for the DECIDE Program and outlines intervention development, current study recruitment, and preliminary program evaluation data. Findings indicate that enrollment into the program is feasible among children and their parents; early feedback suggests that parents and children are also satisfied with their participation. The challenges of conducting randomized controlled trials in health promotion for childhood diabetes are discussed, and ideas are offered for future research directions to improve the integration of child health psychology with public health approaches in this population.

Doctoral Gero-Psychology Training in Primary Care: Preliminary Findings from a Clinical Training Project

This article provides descriptive information and preliminary program evaluation data regarding a University based doctoral training program with a special focus on gero-psychology training in a primary care setting. The program sought to combine the resources of a University, an urban public hospital, and an academic medical center to develop geropsychology didactic and clinical training experiences, and specifically direct services to minority elders. Preliminary program findings suggest that educational and service objectives may be simultaneously achieved. A relatively modest increment in didactic training resulted in measurable gains in knowledge and interest. While the integration of geropsychology into primary care medicine presents some formidable challenges, preliminary data suggest great potential for further expansion of training and geropsychological services in primary care to meet the burgeoning mental health needs of older adults.

Updates and Five-year Evaluation of the S.A.F.E. Program: A Family Psychoeducational Program for Serious Mental Illness

This brief report reviews recent updates to the S.A.F.E. Program, a family psychoeducational intervention for serious mental illness created in the Veterans Affairs (VA) system. The improvements and significant content additions to the curriculum are outlined. Further, positive five-year program evaluation data are described, including high levels of participant retention and satisfaction. Program attendance is positively correlated with understanding of mental illness, awareness of VA resources, and ability to engage in self-care activities-and inversely correlated with caregiver distress. This data lays the groundwork for a randomized clinical trial and raises questions about the necessity of diagnostic-specific family programming.

Program Evaluation in Graduate Nursing Education: Hardiness as a Predictor of Success Among Nursing Administration Students

In an effort to develop meaningful program evaluation data, this exploratory study examined hardiness and stress among a select group of graduate nursing administration students at The University of Texas at Arlington in Arlington, TX. Students completed a questionnaire, which included the Hardiness Scale and Perceived Stress Scale, at both beginning and end of their program. Six to 12 months after graduation, each participant completed a telephone survey with items related to hardiness and American Association of Colleges of Nursing and American Organization of Nurse Executives core competencies. Mean scores for both hardiness and stress did improve from beginning to end of program; however, alpha coefficient reliabilities were disappointing, so no further quantitative analyses were completed. Most enlightening were the telephone interviews, which revealed that graduates were engaged in behaviors indicative of high hardiness. Use of hardiness as a guiding construct may prove useful to nursing administration program faculty in developing and refining a quality program; use of both quantitative and qualitative methods are recommended for obtaining meaningful program evaluation data.

Healthcare and Early Intervention Collaborative Supports for Families and Young Children

Children who had a prolonged hospital stay are often at risk for developmental disabilities or delays by nature of their illnesses and diagnoses. The hospitalization also can significantly disrupt the developing infant-parent relationship. This article describes an innovative intervention model providing interagency and interdisciplinary supports for preterm infants and sick young children between birth and 3 years of age and their families in a hospital-based setting. Model components are outlined according to the sources of support provided to families and children such as social-emotional, resource-material, and informational supports. Program evaluation data are shared on the basis of interviews with the models' major stakeholders—parents, healthcare providers, and early interventionists. Suggestions for replication and improvement of the model toward the realization of a seamless system of supports for infants and toddlers in hospital-based settings and during their transition to community-based early intervention services are provided.

Strengthening community-based programming for juvenile sexual offenders: key concepts and paradigm shifts.

The past decade has been witness to a sharp increase in residential placement of adjudicated delinquent youth, including juvenile sexual offenders. It is argued that this trend has fiscal implications and may be clinically contraindicated for less characterologically disturbed and dangerous youth. The authors advocate greater investment of public funds in the development and refinement of community-based intervention programs. It is believed that clinically and legally integrated programming, using newer social-ecological methodologies and supports, offers promise of reducing the number of youth who require residential placement, shortening residential lengths of stay and improving the transition of residentially treated youth back into community settings. Key concepts relevant to bolstering community-based programming for juvenile sexual offenders are identified and discussed. Two programs are described, and program evaluation data reviewed, in support of the viability of innovative community-based approaches to the management of this population.

Heterogeneity within the Asian American community.

BackgroundEducational interventions are grounded on scientific data and assumptions about the community to be served. While the Pan Asian community is composed of multiple, ethnic subgroups, it is often treated as a single group for which one health promotion program will be applicable for all of its cultural subgroups. Compounding this stereotypical view of the Pan Asian community, there is sparse data about the cultural subgroups' similarities and dissimilarities. The Asian Grocery Store based cancer education program evaluation data provided an opportunity to compare data collected under identical circumstances from members of six Asian American cultural groups.MethodsA convenience sample of 1,202 Asian American women evaluated the cultural alignment of a cancer education program, completing baseline and follow-up surveys that included questions about their breast cancer knowledge, attitudes, and screening behaviors. Participants took part in a brief education program that facilitated adherence to recommended screening guidelines.ResultsUnique recruitment methods were needed to attract participants from each ethnic group. Impressions gained from the aggregate data revealed different insights than the disaggregate data. Statistically significant variations existed among the subgroups' breast cancer knowledge, attitudes, and screening behaviors that could contribute to health disparities among the subgroups and within the aggregate Pan Asian community.ConclusionHealth promotion efforts of providers, educators, and policy makers can be enhanced if cultural differences are identified and taken into account when developing strategies to reduce health disparities and promote health equity.