Process Evaluation Components Research Articles

Why do preconception and pregnancy lifestyle interventions demonstrate limited success in preventing overweight and obesity in children? A scoping review investigating intervention complexity, process evaluation components, and author interpretations.

Preventing childhood obesity from early life is considered essential. However, evidence from recent systematic reviews has highlighted inconsistent results and limited effectiveness of preconception and pregnancy lifestyle interventions regarding offspring weight outcomes and adiposity. Therefore, to improve our understanding regarding the mixed success of these early life interventions, we conducted a scoping review examining intervention complexity, process evaluation components, and authors' statements. Eligible articles (preconception or pregnancy lifestyle trials with offspring data beyond 1month of age) were identified by searching databases (PubMed, Embase, and CENTRAL), previous reviews, and performing CLUSTER searches. The Intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR) was used to evaluate intervention complexity. A thematic analysis provided insight into process evaluation components and authors' interpretations. Finally, an expert consultation on the results was conducted. We identified 40 eligible publications corresponding to 27 trials. Only two trials started before conception. Potential reasons for interventions' limited success included the late intervention initiation, short intervention duration, and insufficient sample size. Few studies reported process evaluations and included stakeholder involvement, which are essential according to the expert group. We discuss current limitations and outline suggestions for future interventions in this field of research.

Navigating process evaluation in co-creation: a Health CASCADE scoping review of used frameworks and assessed components

BackgroundCo-creation is seen as a way to ensure all relevant needs and perspectives are included and to increase its potential for beneficial effects and uptake process evaluation is crucial. However,...

Multicomponent Behavioural Intervention during Pregnancy to Reduce Home Exposure to Second-Hand Smoke: A Pilot Randomised Controlled Trial in Bangladesh and India.

Pregnant women exposed to second-hand smoke (SHS) are at increased risk of poor birth outcomes. We piloted multicomponent behavioural intervention and trial methods in Bangalore, India, and Comilla, Bangladesh. A pilot individual randomised controlled trial with economic and process evaluation components was conducted. Non-tobacco-using pregnant women exposed to SHS were recruited from clinics and randomly allocated to intervention or control (educational leaflet) arms. The process evaluation captured feedback on the trial methods and intervention components. The economic component piloted a service use questionnaire. The primary outcome was saliva cotinine 3 months post-intervention. Most pregnant women and many husbands engaged with the intervention and rated the components highly, although the cotinine report elicited some anxiety. Forty-eight (Comilla) and fifty-four (Bangalore) women were recruited. The retention at 3 months was 100% (Comilla) and 78% (Bangalore). Primary outcome data were available for 98% (Comilla) and 77% (Bangalore). The multicomponent behavioural intervention was feasible to deliver and was acceptable to the interventionists, pregnant women, and husbands. With the intervention, it was possible to recruit, randomise, and retain pregnant women in Bangladesh and India. The cotinine data will inform sample size calculations for a future definitive trial.

A qualitative evaluation of a co-design process involving young people at risk of suicide.

Co-design is becoming common practice in the development of mental health services, however, little is known about the experience of such practices, particularly when young people are involved. The aim of this study was to conduct a process evaluation of the co-design which was undertaken for the development of an intervention for youth and adolescents at risk of suicide. This paper briefly outlines the co-design process undertaken during a COVID-19 lockdown and then focuses on a qualitative evaluation of the experience of taking part in a co-design process. The evaluation involved young consumers of a public youth mental health service, their carers/parents and service delivery staff who had taken part in the co-design process. This study used follow-up semistructured interviews with the co-design participants to explore their experience of the co-design process. Inductive thematic analysis was used to draw out common themes from the qualitative data. It was found that despite the practical efforts of the project team to minimise known issues in co-design, challenges centred around perceptions regarding power imbalance, the need for extensive consultation and time constraints still arose. Despite these challenges, the study found that the co-design provided a human-centred, accessible and rewarding process for young people, parents and staff members, leaving them with the feeling that they had made a worthwhile contribution to the design of the new service, as well as contributing to changing practice in service design. With sensitivity and adaptation to usual practice, it is possible to include young people with suicidal ideation, their parents/carers and professional staff in a safe and effective co-design process. The authors would like to thank and acknowledge the young people with a lived experience and their carers who participated in the co-design process and research evaluation component of this study. We also wish to thank the clinical staff, peer workers and family peer workers who participated in this research.

Are codesigned programmes more difficult to implement? A qualitative study of staff perceptions on the implementation of a new youth mental health programme.

Codesigned interventions are becoming more common in health services and, in particular, in the design and development of mental health programmes and interventions. However, previous research has established that the transition from codesign to implementation can experience several challenges and that this transition process has received little research attention. The aim of this study was to explore the experience of staff members charged with the implementation of a codesigned intervention for young people and adolescents at risk of suicide. Five staff members involved in the implementation of the new codesigned programme took part in semi-structured interviews. The study involved qualitative evaluation of staff experiences during the implementation of a new child and youth suicide intervention. Interviews were analysed using reflexive thematic analysis. The analysis identified four themes of 'disconnect', 'operational challenges, 'service user' and 'being authentic'. 'Disconnect' captures the difficulties of implementing a codesigned programme which leads to 'operational challenges' in meeting broader expectations while ensuring the feasibility of the programme. The third theme, 'service user', captures the realisation that the young people accessing the new service were different to those involved in the codesign process. The final theme, 'being authentic', highlights how staff needed to be responsive and flexible while remaining true to the principles proposed in the codesign. This study yielded some valuable insights into the challenges around the implementation of a codesigned intervention, an under-researched area. The findings suggest that adaption of the design may be necessary, if it is not informed by implementation constraints, making it necessary for the implementation team to be well-briefed on the initial design and given plenty of time to make the necessary adjustments in a coproduction process. Limitations for the generalisation of the results include a small sample of staff and particular challenges that may be unique to this study. The present study highlights that for health services undertaking codesign approaches, appropriate time and resources need to be considered for the implementation phase of an initiative, to ensure that there is effective translation from design to implementation and that new codesigned services can be effective within operational constraints. The authors would like to thank and acknowledge the young people with a lived-experience and their carers who participated in the codesign process and research evaluation component of this study. We also wish to thank the clinical staff, peer workers and family peer workers who participated in the evaluation.

Promotion of training course on ICD-10 Poisoning coding : necessity to adopt preventive strategies

BackgroundPoisoning is considered the most common cause of referral to emergency departments and hospitalization in the intensive care unit (ICU). Training or retraining of coders and ensuring the positive impact of these trainings in assigning accurate codes to poisoning cases is necessary to adopt practical health measures for optimal management of this disease. The present study aimed to evaluate the impact of holding a training course on poisoning coding rules based on ICD-10 in clinical coders.MethodsThis study is descriptive and analytical. With the target population included the coders of hospitals affiliated with Shahid Beheshti University of Medical Sciences (N = 45). In order to evaluate the training course on poisoning coding rules, the Conex Input Process Product (CIPP) evaluation model was used. This model was the first goal-oriented approach evaluation model. According to the CIPP model, evaluation of the training course held in four components, including Context factors (course objectives and priority of objectives), Input factors (instructor, curriculum, facilities, equipment, and training location), Process factors (teaching process, learning, management, and support), and Product factors (feedback, knowledge, and skills). A researcher-made questionnaire containing 39 questions with a 5-point Likert scale was used to collect data. The validity of the questionnaire was calculated through content validity, and its reliability was calculated using Cronbach’s alpha coefficient (alpha = 90% in all components). In order to analyze the data, descriptive statistics (frequency percentage distribution) and inferential statistics (one-sample t-test) were used.ResultsThe findings of this study were presented in four components of context, input, process, and product evaluation. The average criterion for all questions in the questionnaire was considered 3. As a result, the significance level obtained from the one sample t-test was equal to P = 0. 0001.The training course had a favorable effect in terms of context, input, process and products.ConclusionThe knowledge and skills of clinical coders can be enhanced by updating medical knowledge, holding training courses, workshops, seminars, and conducting clinical coder accreditation. Extensive and continuous training for clinical coders is essential due to the impact of code quality on financial forecasting, electronic health records, and conducting research.

ANALISIS PROGRAM UNGGULAN RODANYA MAS BAGIA KOTA MAGELANG BERBASIS CONTEXT, INPUT, PROCESS, PRODUCT (CIPP)

The City Government of Magelang has implemented the Rodanya Mas Bagia program to reduce the poverty rate. There is no information showing the evaluation, success, constraints, obstacles or carrying capacity of the Wheel of Mas Bagia flagship program. it is necessary to analyze the program objectively to monitor the progress and success of the Rodanya Mas Bagia program, to fulfill the answer to the transparency of the Magelang City Government's flagship program. The research objective is to describe; 1) results of evaluation of the context, input, process, product regarding the wheel program, bro; 2) assess feasibility, 3) prepare sustainability recommendations. Evaluative research with the CIPP evaluation model approach. The primary data source in this study is community satisfaction mapping data obtained through the response of the program target community, the performance of the actors, the program implementing committee. The population of all RTs is 1,201 RTs. Samples with the slovin formula obtained as many as 93 RT. The sampling technique in this study was a proportional random sampling technique. Data collection methods with questionnaires, interviews, observation, documentation. The data analysis used is descriptive percentage to determine the feasibility level of the CIPP evaluation aspect. Analysis of recommendations and sustainability with qualitative descriptions, with saturated data from triangulation of data, sources, methods, and time. Efforts to increase the credibility of the data are carried out by extending the research time and re-examining the research data obtained. The results of the study, the description of the CIPP evaluation explained in quantitative data that the aspect with the lowest quantification is context evaluation with a percentage of 58%, input evaluation with a percentage of 69%. Process evaluation component with a percentage of 66%. product evaluation with a percentage of 69%. The feasibility of the program, the legal aspects fulfill the legal legality aspects, do not violate legal and statutory aspects and are worthy of being continued. A positive culture that strengthens empowerment: a culture of mutual cooperation in solving problems, reaching consensus, identifying problems, and discussing and agreeing on solutions. Sustainability of the program is continued and continued with socialization, understanding of the community regarding the objectives and essence of the program. Community participation can be optimized so as to increase the degree of welfare. Scheme of technical assistance for program proposals in accordance with the professional field.

Why do preconception and pregnancy lifestyle interventions show little evidence of success in preventing overweight and obesity in children? A scoping review

BackgroundAdverse lifestyle factors in the periconception and early life period, including high maternal prepregnancy BMI and excessive gestational weight gain, are important risk factors for childhood obesity. Early prevention is key, but results from systematic reviews of preconception and pregnancy lifestyle interventions have shown mixed success in improving children's weight outcomes and adiposity. We aimed to investigate the complexity of these early interventions and process evaluation components and authors’ statements to improve our understanding regarding their limited success. MethodsWe did a scoping review, guided by frameworks of the Joanna Briggs Institute and Arksey and O’Malley. Eligible articles (with no language restriction) were identified between July 11 and Sept 12, 2022, by searching PubMed, Embase, and CENTRAL; consulting previous reviews; and conducting CLUSTER searches. A thematic analysis was conducted with NVivo in which process evaluation components and authors’ interpretations were coded as reasons. Intervention complexity was evaluated with the Complexity Assessment Tool for Systematic Reviews. Findings40 publications corresponding to 27 eligible preconception or pregnancy lifestyle trials with child data beyond age 1 month were included. Most interventions started during pregnancy (n=25) and focused on multiple lifestyle factors (eg, diet and exercise). The preliminary results show that almost no interventions involved the participants’ partner or social network. Potential reasons for limited success in the interventions preventing overweight or obesity in children included the start time of the intervention, duration and intensity, and sample size or dropout rates. The results will be discussed with an expert group as part of a consultation stage. InterpretationThe results and discussions with an expert group are expected to uncover gaps and inform the design or adaptation of future interventions and approaches to potentially increase success rates in preventing childhood obesity. FundingFunded by the Irish Health Research Board through the transnational JPI HDHL ERA-NET HDHL-INTIMIC-2020 call (PREPHOBES): EU Cofund action (number 727565; EndObesity project).

EVALUASI PROGRAM PRAKTEK BAHASA ARAB DI PONDOK PESANTREN ISLAM INTERNASION AL AL-ANDALUS

This study aims to identify and analyze the evaluation of the Arabic language practice program at the Al-Andalus International Islamic Boarding School. This study uses a qualitative descriptive method with the CIPP evaluation approach. In this study the data collection techniques carried out by researchers were observation, interviews and documentation. After the data has been collected, an analysis is carried out with data reduction, data display and verification to obtain an overview of the implementation of the evaluation of the Arabic language practice program at the Al-Andalus International Islamic Boarding School. The results of the study show that: 1) the context evaluation component of the Arabic language training program is in accordance with the background of the program, the objectives of the program, the interests of the participants and the targets of the participants in participating in the program; 2) the input evaluation component in the Islamic boarding school environment is in accordance with the criteria, namely teachers who have and master 4 Arabic language competencies, namely; speaking, writing, reading and listening as well as having teaching experience. Meanwhile, in terms of facilities and infrastructure, more support is needed; 3) the process evaluation component is in accordance with the program design, namely it already has a foundation of objectives, timeframe, methods, strategies, in the implementation of the program students are actively involved in this practicum activity but not all strategies can be carried out, and 4) the product evaluation component of the practicum program implementation has went well and in line with expectations so as to be able to produce alumni who master Arabic both orally and in writing.

A mixed-methods program evaluation of the Alda Healthcare Experience- a program to improve healthcare team communication

BackgroundCommunication among interprofessional healthcare worker teams is critical to ensure a thriving and resilient workforce. We will evaluate the implementation and effectiveness of the Alda Healthcare Experience (AHE), a novel medical improvisation (improv) workshop designed to improve interprofessional communication skills among healthcare professionals. The AHE workshop includes a two-hour experiential training workshop led by an improv specialist and a clinical co-facilitator. In July 2022 we began implementing the AHE workshop by training 18 clinical co-facilitators who will co-facilitate the workshops for 550 healthcare workers from five hospital departments at Stony Brook University Hospital over the course of a year and a half. Using mixed-methods, we will conduct an Effectiveness-Implementation Hybrid Design project that includes an outcome evaluation (effectiveness) and a process evaluation (implementation).MethodsOur outcome evaluation will assess the impact of the AHE workshop on short- and long-term improvement in interprofessional communication, stress, and professional fulfillment. The process evaluation component will examine programmatic, organizational, and individual facilitators or barriers to effective implementation of the AHE workshop. Qualitative methods will include dimensional analysis employing individual interviews of 20–40 AHE Project Participants, 5–10 Selected Informants, and all the clinical co-facilitators. Quantitative methods will use a quasi-experimental longitudinal design with an intervention group and surveillance of a control group (wait-list) and repeated assessments using validated instruments measuring communications skills, professional fulfillment, stress, burnout, uncertainty tolerance, and teamwork.DiscussionEffective and efficient communication within healthcare teams is fundamental to building team cohesion that, in turn, supports individual resilience and builds positive organizational culture. The AHE program is an innovative approach to improve interprofessional healthcare communication and reduce healthcare worker burnout. In addition to institutional buy-in, rigorous evaluations of medical improv programs are necessary as a critical step in making such programs scalable.Trial registrationN/A

Why do preconception and pregnancy lifestyle interventions demonstrate limited success in preventing overweight and obesity in children? A scoping review protocol.

Adverse family-based lifestyle factors in the preconception period, pregnancy and early-childhood are major risk factors for childhood obesity and there is a growing consensus that early life interventions to prevent overweight and obesity in children are required. However, results from recent systematic reviews of preconception and pregnancy interventions have demonstrated mixed success. Therefore, this protocol presents a study aiming to summarise and evaluate complex preconception and pregnancy intervention components, process evaluation components, and authors' statements, with a view to improving our understanding regarding their success and informing design or adaptation of more effective interventions to prevent childhood obesity. A scoping review will be conducted, using the frameworks of the JBI and Arksey and O'Malley. A two-step approach will be used to identify relevant literature: (1) systematic searches will be conducted in the databases PubMed, Embase and CENTRAL to identify all eligible preconception and pregnancy trials with offspring data; and (2) CLUSTER searches will be conducted to find linked publications to eligible trials (follow-ups, process evaluation publications). Two researchers will independently select studies, chart, and synthesise data. A qualitative thematic analysis will be performed in which statements related to process evaluation components and authors' interpretations will be coded as "reasons". A descriptive analysis will be performed to evaluate intervention complexity using a complex intervention framework (AHRQ series; Medical Research Council guidance). The results of this study, which will be discussed with an expert group as part of a consultation stage, aim to identify gaps and inform the design or adaptation of future preconception and pregnancy interventions and approaches to potentially increase success rates. We expect that our results, which will be submitted for publication in a peer-reviewed journal, will be of interest to researchers, families, and practitioners concerned with good preconception and prenatal care, and healthy child outcomes.

Examining the impact of a Volunteer-Delivered counselling service for people experiencing sight loss: a mixed methods evaluation

Purpose This article describes a mixed methods evaluation of a volunteer-delivered counselling service for people experiencing sight loss, to identify the outcomes, benefits, and challenges of this model of service provision within rehabilitation services. Materials and method A mixed methods approach was used, with both outcome and process evaluation components. Outcomes for 817 service users were assessed pre-and-post service use, using standardised (CORE10) measures. Semi-structured interviews and surveys with 22 volunteer counsellors and 4 senior counsellors were conducted, to understand their experiences of the format, content, and delivery of the service. Results Clients presenting for counselling experienced significant levels of self-reported depression, anxiety, and feelings of bereavement. Analysis showed a significant positive change in clinical outcomes for service users who received volunteer-delivered counselling services. Volunteer counsellors saw the unique and specialist nature of the service as critical for its impact on clients, and reported benefits of volunteering for this service, including developing their skills with this client group. Conclusions Volunteer-delivered counselling can be effective for people affected by sight loss, providing accessible and much-needed psychological support, which is relevant given current pressures on health and social care services. There are key lessons for future models of this type. Implications for Rehabilitation Vision loss can have a profound negative impact on people’s emotional and mental well-being, with many advocating for counselling and emotional support (CESS) to be included in low vision rehabilitative services. Despite this, access to psychological support is often overlooked in low vision rehabilitation, and access to CESS services is often poor. This study evaluated a volunteer-delivered CESS service, demonstrating that this model of service delivery is effective for this group. There are clear advantages to a volunteer-delivered model for those with sight loss, in terms of widening access, supporting integrated rehabilitative care, and reducing health inequalities.

Process Evaluation in Health Communication Media Campaigns: A Systematic Review.

Objective. The objective is to examine the scope of health communication media campaign process evaluation methods, findings, and dissemination practices. Data Source. A systematic review of peer-reviewed literature was conducted using database searches. Study Inclusion and Exclusion Criteria. Published studies on process and implementation evaluation of health campaigns with a media component were included. Exclusion criteria included not health, non-empirical, no media campaign, or a focus on other evaluation types. Data Extraction. Articles were assessed for general campaign information, theory use, and details about process evaluation plan and procedures. Data Synthesis. A coding scheme based on 9 process evaluation best practice elements (e.g., fidelity and context) was applied. Process evaluation methods, measures, and reporting themes were synthesized. Results. Among 691 unique records, 46 articles were included. Process evaluation was the main focus for 71.7% of articles, yet only 39.1% reported how process evaluation informed campaign implementation strategy. Articles reported 4.39 elements on average (SD = 1.99; range 1-9), with reach (87.0%) and recruitment (73.9%) described most frequently, yet reporting was inconsistent. Further, the level of detail in reporting methods, theory, and analysis varied. Conclusions. Process evaluation provides insight about mechanisms and intervening variables that could meaningfully impact interpretations of outcome evaluations; however, process evaluations are less often included in literature. Recommendations for evidence-based process evaluation components to guide evaluation are discussed.

Instructional Materials and Alternative Teaching Practices in Physical Education

In physical education (PE) learning, one of the main problems faced by PE teachers is the lack of facilities and equipment. Therefore, it is necessary to find a better solution to solve this problem. This study used a descriptive evaluative and comparative research design to determine the level of achievement of the context, input, process, and product evaluation components in the learning program. This research was conducted in 3 selected schools in the Maguindanao Division-1, Philippines. The results show that books are used "widely" by sports teachers and journals are used for printed materials while electronic journals are never used for non-print references. Furthermore, overhead projectors, fitness centers, lockers, kick pads, jumping pits, punching bags, megaphones, and signage are teaching materials that have never been used in the delivery of lessons by sports teachers, local material development and alternative games are rarely used by PE teachers. Sports teachers invite alumni to help with sporting activities, and they also tap old players to help in the basketball game even though they never use beeps but only whistle. The level of use of printed materials by schools differed significantly between schools. So are results with non-printed or electronic material. In addition, researchers suggest that schools provide additional and up-to-date sports books for teachers and students, and, schools should provide other teaching materials that will assist teachers in teaching in the classroom.

Process Evaluation of a Faith-Based Multicomponent Behavioral Intervention to Reduce Stroke Risk in Mexican Americans in a Catholic Church Setting: The SHARE (Stroke Health and Risk Education) Project.

Church-based stroke prevention programs for Hispanics are underutilized. The Stroke Health and Risk Education (SHARE) project, a multicomponent cluster-randomized trial, addressed key stroke risk factors among predominantly Mexican Americans in a Catholic Church setting. Process evaluation components (implementation, mechanisms of impact, and context) are described. Partner support promoted positive health behavior change. Motivational interviewing calls were perceived as helpful, however, barriers with telephone delivery were encountered. Intervention exposure was associated with theory constructs for targeted behaviors. We conclude that health behavior interventions to prevent stroke can be successfully implemented for Mexican Americans within a Catholic Church setting, with parish priest support.

Effectiveness and process evaluation in obesity and type 2 diabetes prevention programs in children: a systematic review and meta-analysis

BackgroundObesity in children is one of the most severe public health challenges of the current century and Type 2 Diabetes Mellitus (T2DM) frequency is also escalating. More so, the importance of process evaluation (PE) in complex interventions is increasingly recognized. The present review, aims to identify the effectiveness in terms of body composition parameters in a generation of articles to prevent obesity and T2DM in children. We hypothesise that those studies reporting PE applying the latest implementation guidelines suggested by the researchers would potentially show positive changes in body composition compared to those not reporting it. Additionally, we will evaluate the implementation degree of PE in those articles considering it and describe the PE subcomponents. Lastly, we aim to assess the intervention target used and its results.MethodsA literature review was performed in parallel by 2 independent reviewers. A final number of 41 studies were selected for inclusion criteria.ResultsMeta-analysis of BMI and zBMI found non-significant effects of the proposed interventions. Sub-group analysis revealed only a significant effect in studies which performed PE. Moreover, PE was reported in 42% effective studies and 57% non-effective studies. Fidelity and satisfaction were the most implemented PE subcomponents, although there was a generally low grade of PE use (7/41). The highest proportion of effectiveness (83%) was shown in interventions of physical activity alone while the intervention most used was 3-arm target (diet, PA and BS).ConclusionsOverall, obesity and T2DM prevention studies included in this review are not effective in terms of BMI and zBMI. Those studies performing PE reported to be effective in terms of BMI, while studies not reporting PE did not have positive results in terms of BMI and zBMI. In addition, none of the intervention studies included all PE indicators and most studies, which included PE in their interventions, did not provide full report of the PE components, according to the guidelines used for the present review. PROSPERO registration number: CRD42018093667.

Process Evaluation of a National Elderly Nutrition-Care Program in Iran: Perspectives of Clients and Providers.

ObjectiveAs aging populations increase, many countries have begun implementation of policies to improve elder health and nutrition. This study evaluated key process components of nutrition sections of a national elderly integrated–care program in health-care centers in Iran.MethodsWith stratified three-stage random cluster sampling, a total of 256 elderly subjects (clients) and 76 staff members of health-care centers (providers) participated in the study. Quantitative and qualitative data were collected using two self-administrated questionnaires to evaluate various components of process evaluation for clients and providers. Program reach was measured by the ratio of the number in the target group who underwent the program to the number of eligible individuals. Exposure rate was measured as clients’ awareness of program services. Delivery and fidelity were assessed from providers' reports for each service. Satisfaction rates were assessed for the whole program and for each service of the program. All reasons behind strengths and weaknesses in each of the process-evaluation components were examined and are reported in detail.ResultsThe clients reported low-reach (20.0%) and moderate-exposure rates of 77.5%, with a program target of 90%. Primary training sessions were delivered to the providers as intended (100%), but most planned services for clients, especially vitamin and mineral supplementation, follow-up, and physical activity sessions, were poorly implemented, as they were delivered correctly to 24.7% (n=63, vitamin and mineral supplementation), 24% (n=62, follow-up), and 40.3% (n=103, physical activity sessions) of the clients, with a set program goal of 60%. An overall low level of implementation fidelity was observed, and 39.3% (n=30) of the providers believed that most clients did not benefit from the nutritional services. Overall, less than half (42.8%, n=101) the clients were highly satisfied with the program.ConclusionThe process evaluation showed insufficient reach, exposure, and fidelity of the program, as well as imprecise delivery of some services, which resulted in low levels of client satisfaction. The findings may have implications for further reinforcement of the program, and indicate the importance of continuous monitoring and evaluation of such programs.

Process evaluation of the ‘Singapore Physical Activity and Nutrition Study’

IntroductionThe Singapore Physical Activity and Nutrition Study (SPANS) aimed to improve the physical activity (PA) and nutrition behaviours of Singaporean women aged 50 years and over. The SPANS program consisted of PA classes, nutrition workshops, telephone dietary counselling, health booklets, a health calendar and program ambassadors. This study aimed to assess and understand the implementation of the program strategies and gain insight into process evaluation components to inform future programs. MethodsThe evaluation was guided by a process evaluation framework and collected data via questionnaires (n = 209), program ambassador documentation and exit interviews with program completers (n = 13) and non-completers (n = 12). ResultsIn total, 295 participants completed the program (response rate = 84 %). Participants reported high levels of satisfaction with the overall program (99.5 %) and program activities (96.7 %), and also rated program ambassadors highly. Participation rates were highest for telephone dietary counselling sessions. The main reason for not attending program activities was having a ‘busy schedule’ (n = 158). Participants cited a need for improved recreational centre facilities and increased flexibility around program delivery. ConclusionsThe process evaluation showed that the program strategies were implemented as planned and were deemed suitable for supporting behaviour change among Singaporean women aged 50 years and over. The program reached and involved the majority of participants throughout the six months. The combination of practical educational resources and supportive program ambassadors were key strategies that facilitated positive PA and dietary behaviours. However, there needs to be some flexibility in the delivery of programs. The findings of this research may inform other programs in the region.

Pilot study of a new road test to assess cognitive fitness to drive

The Traffic Injury Research Foundation (TIRF) developed and evaluated a new road test, called the Enhanced Road Test (ERT), to assess a driver’s cognitive fitness to drive for the Ministry of Transportation in Ontario, Canada (MTO). The ERT was designed to flag individuals for potential cognitive impairment, but was not meant to indicate, or assess the level of impairment. Practical feasibility for implementation of the ERT was determined in a pilot study as well as its ability to differentiate drivers with and without cognitive impairment. The research design included both process and outcome evaluation components. The qualitative process evaluation was conducted using surveys administered to participant drivers (N = 70) and feedback obtained from driver examiners involved in the pilot (N = 3). The quantitative outcome evaluation used an experimental design to administer the ERT to a sample of drivers with (N = 42), and without (N = 28), mild cognitive impairment. Regression analysis was conducted and scoring functions of the ERT were compared through a Receiver Operating Characteristic (ROC) curve analysis.Survey results revealed that drivers did not raise specific concerns about the ERT and that examiners believed more training on driving behaviours associated with cognitive impairment and some scoresheet formatting changes would be beneficial. ROC Area Under the Curve (AUC) results revealed that some components of the ERT were better at identifying individuals whose driving skills declined. These components are the adjust-control, route finding and divided attention tasks. Two final scoring schemes for the ERT are proposed as well as several recommendations for its possible implementation.

Abdominal massage plus advice, compared with advice only, for neurogenic bowel dysfunction in MS: a RCT.

Between 50% and 80% of people with multiple sclerosis (PwMS) experience neurogenic bowel dysfunction (NBD) (i.e. constipation and faecal incontinence) that affects quality of life and can lead to hospitalisation. To determine the clinical effectiveness and cost-effectiveness of abdominal massage plus advice on bowel symptoms on PwMS compared with advice only. A process evaluation investigated the factors that affected the clinical effectiveness and possible implementation of the different treatments. A randomised controlled trial with process evaluation and health economic components. Outcome analysis was undertaken blind. The trial took place in 12 UK hospitals. PwMS who had 'bothersome' NBD. Following individualised training, abdominal massage was undertaken daily for 6 weeks (intervention group). Advice on good bowel management as per the Multiple Sclerosis Society advice booklet was provided to both groups. All participants received weekly telephone calls from the research nurse. The primary outcome was the difference between the intervention and control groups in change in the NBD score from baseline to week 24. Secondary outcomes were measured via a bowel diary, adherence diary, the Constipation Scoring System, patient resource questionnaire and the EuroQol-5 Dimensions, five-level version (EQ-5D-5L). A total of 191 participants were finalised, 189 of whom were randomised (two participants were finalised in error) (control group, n = 99; intervention group, n = 90) and an intention-to-treat analysis was performed. The mean age was 52 years (standard deviation 10.83 years), 81% (n = 154) were female and 11% (n = 21) were wheelchair dependent. Fifteen participants from the intervention group and five from the control group were lost to follow-up. The change in NBD score by week 24 demonstrated no significant difference between groups [mean difference total score -1.64, 95% confidence interval (CI) -3.32 to 0.04; p = 0.0558]; there was a significant difference between groups in the change in the frequency of stool evacuation per week (mean difference 0.62, 95% CI 0.03 to 1.21; p = 0.039) and in the number of times per week that participants felt that they emptied their bowels completely (mean difference 1.08, 95% CI 0.41 to 1.76; p = 0.002), in favour of the intervention group. Of participant interviewees, 75% reported benefits, for example less difficulty passing stool, more complete evacuations, less bloated, improved appetite, and 85% continued with the massage. A cost-utility analysis conducted from a NHS and patient cost perspective found in the imputed sample with bootstrapping a mean incremental outcome effect of the intervention relative to usual care of -0.002 quality-adjusted life-years (QALYs) (95% CI -0.029 to 0.027 QALYs). In the same imputed sample with bootstrapping, the mean incremental cost effect of the intervention relative to usual care was £56.50 (95% CI -£372.62 to £415.68). No adverse events were reported. Limitations include unequal randomisation, dropout and the possibility of ineffective massage technique. The increment in the primary outcome favoured the intervention group, but it was small and not statistically significant. The economic analysis identified that the intervention was dominated by the control group. Given the small improvement in the primary outcome, but not in terms of QALYs, a low-cost version of the intervention might be considered worthwhile by some patients. Research is required to establish possible mechanisms of action and modes of massage delivery. Current Controlled Trials ISRCTN85007023 and NCT03166007. This project was funded by the National Institute for Health Research Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 22, No. 58. See the NIHR Journals Library website for further project information.