Process Evaluation Methodology Research Articles

Representing the experiences of charity peer leaders facilitating well-being interventions to their peers using process evaluation methodology (Part 2)

Representing the experiences of charity peer leaders facilitating well-being interventions to their peers using process evaluation methodology (Part 2)

Understanding the what, how, and why in advocacy: Assessing the applicability of participatory process evaluation methodology in an advocacy context

While evaluations are critical for non-governmental organizations to strengthen their advocacy strategies, evaluators and advocates encounter many difficulties evaluating such efforts. This article discusses the contribution of the participatory process evaluation methodology to advocacy evaluation, using a Dutch global health advocacy program as a case study. As participatory process evaluation is a novel methodology in the field of advocacy, the article’s primary focus concerns the application and utility of the methodology. Findings suggest that participatory process evaluation in an advocacy context can provide insights into the implementation of advocacy tools and activities, encouraging reflection and leading to ideas and practical tools to strengthen advocacy efforts. While participatory process evaluation can help overcome some of the often-experienced barriers in advocacy evaluation, further research is needed to consolidate advocacy evaluation theory and practice.

Real-time process operation evaluation and model reliability assessment for chemi-thermomechanical pulping process

In this paper, a real-time chemi-thermomechanical pulping (CTMP) process was investigated, where opportunities were explored to increase energy consumption efficiency with a guarantee of expected pulp properties as a priority. Due to the process instrumentation limitations, such as the lack of advanced regulatory controllers and optimization units, the advanced process controllers and real-time optimization strategies can hardly be implemented in the existing CTMP processes. Therefore, a novel visualization-based process evaluation methodology is proposed to aid the mill operators in making further decisions to improve the process operations. Additionally, to determine the desirable pulp properties for production, an inferential sensor model is developed that estimates the handsheet properties from pulp properties. Also, to assess the model prediction accuracy in real-time, a reliability index model is designed. The proposed methodologies have been validated on a practical CTMP process, and the results show the satisfactory performance of the system.

Impact of Modern Technologies on the Organization of the Cadastral Data Modernization Process

Land surface and environmental data (cadastral data) are extremely important in the functioning of the country and society. Upgrading the data is fundamental. Methods of traditional surveying (TM) or using modern remote data acquisition methods (PhM) are used for this purpose. The aim of the study is to compare the process of upgrading space and environmental data made by traditional methods and using modern remote data collection methods. The study established the following research hypotheses: (1) the election of the method of performing the cadastral data modernization process to consider effectiveness, productivity, profitability, quality (accuracy), reliability, and efficiency; (2) technical factors, as well as employee well-being and commitment, are equivalent motivators for the election of the cadastral data modernization method; (3) modern survey technologies using photogrammetric images are more efficient than traditional survey methods. The process evaluation methodology was tested on two objects located in Poland. The analyses considered both technical aspects and the comfort of the process contractors. The results showed that despite the higher unit price per cadastral plot (TM 180 PLN/cadastral plot, PhM 190 PLN/cadastral plot), the remote methods require less time commitment (TM-86 days; PhM-50 days) and involve reduced business travel (TM-65 days; PhM-29 days). The comfort of working with modern methods (PhM) is higher than with traditional measurement methods. In total, considering all the parameters studied, traditional methods required about 33% more commitment than modern remote methods of collecting surface and environmental data collection. Modern data acquisition methods are friendly to process contractors but gain less public acceptance than traditional methods (the level of border non-acceptance is higher in PhM methods than in TM (TM-3, Phm-8).

An Aboriginal Community-Controlled Health Organization model of service delivery: qualitative process evaluation of the Tulku wan Wininn mobile clinic

BackgroundMobile clinics have been implemented in diverse clinical and geographical settings to provide proximal health care for specific populations. Primary health care mobile clinics have been implemented widely for Indigenous populations, with a paucity of research evaluations around service delivery models internationally. To redress factors impeding service accessibility for Aboriginal and Torres Strait Islander Peoples, Budja Budja Aboriginal Cooperative (Aboriginal Community Controlled Health Organisation located in a small rural town in Victoria, Australia), developed and implemented the Tulku wan Wininn primary health mobile clinic.MethodsA qualitative process evaluation methodology was used to explore contextual factors mediating the implementation of the mobile clinic, including the acceptability of the service to health service personnel, external key informants, and Aboriginal and/or Torres Strait Islander clients. A synthesis of international ethical guidelines, (Consolidated Criteria for strengthening reporting of health research involving Indigenous peoples (CONSIDER statement), was prospectively applied to shape the study design and research process. Semi-structured interviews were conducted with participants. Data collection occurred from July 2019 to October 2021. Inductive thematic data analysis was undertaken concurrently with data collection.ResultsData was collected from 19 participants which included 12 health service personnel and key informants, and 7 Aboriginal clients. In total, data from 22 interviews were included as interviews with three clients were undertaken twice. Four themes were developed: considerations for early implementation, maintaining face-to-face services during COVID-19, acceptability as a model of service delivery, and maintaining the mobile clinic as a service delivery model.ConclusionEvidence supporting the acceptability of a primary health care mobile clinic for Aboriginal Peoples residing in rural Victoria is provided. Despite the experience of early implementation challenges and adaptations, the mobile clinic addressed known transport and cultural barriers to accessing primary health care services. In the context of COVID-19 lockdowns, the mobile clinic was valued for the provision of face-to-face care for Aboriginal clients. Key issues for maintaining the mobile clinic include health workforce and funding. Findings are of value to other organizations seeking to implement a primary health mobile clinic service delivery model to redress barriers to accessibility experienced by the communities they serve.

Protocol for a process evaluation: face-to-face physiotherapy compared with a supported home exercise programme for the management of musculoskeletal conditions: the REFORM trial

IntroductionThe REFORM (REhabilitation FOR Musculoskeletal conditions) trial is a non-inferiority randomised controlled trial (n=210) designed to determine whether a supported home exercise programme is as good or better than a...

Water alternating gas (WAG) recovery process evaluation through simulation for a giant middle-eastern carbonate reservoir

The objectives of this article it to present a water alternating gas (WAG) process evaluation methodology for a giant middle eastern carbonate reservoir. The methodology involves enhanced oil recovery through an immiscible hydrocarbon water alternating gas (WAG) injection method. Although water alternating gas injection is a proven enhance oil recovery technique and has been around for some time now, only a limited published literature is available on complete evaluation of immiscible WAG process in field application. This article aims to bridge the gap by presenting a detailed study of immiscible water alternating gas evaluation process. In this article a complete methodology for evaluating a water alternating gas injection recovery technique is presented. It includes optimization of WAG cycle length, WAG ratio, production constraints, injected gas compositions, WAG duration, and WAG slug size. The work presented in this article could aid oil field operators to understand the impact of different parameters on immiscible WAG injection process and potentially help them to optimize production from their fields.

Bringing Automated Fault Detection and Diagnostics Tools for HVAC&R Into the Mainstream

Abstract Heating, ventilation, and air-conditioning (HVAC) systems consume over 5 quads of energy annually, representing 30% of energy consumption in the U.S. commercial buildings. Additionally, commercial refrigeration (R) systems add about 1 quad to commercial buildings energy consumption. Most HVAC systems operate with one or more faults that result in increased energy consumption. Fault detection and diagnostics (FDD) tools have been developed to address this national issue, and many tools are commercially available. FDD tools have the potential to save considerable energy for an existing commercial rooftop unit (RTU) and refrigeration systems. These devices can be used for both retro commissioning and, when faults are addressed, continuous commissioning. However, there appears to be multiple market barriers for this technology. Although there are efforts to develop FDD tool standards, currently there are no standards and methods to define functions, capabilities, accuracy, and reliability of FDD tools in the field. Moreover, most of the commercial FDD tools have not been verified in the field independently. This paper presents a comprehensive approach for bringing HVAC FDD tools into the mainstream. The approach involves demonstrating ten commercially available FDD tools installed at ten different sites, independent testing and evaluation of the FDD tools, communication with various stakeholders, identifying and assessing market barriers, creating a process evaluation methodology, and assisting utility companies in developing incentive programs. The preliminary baseline results from the case study demonstrate how the use of an independent monitoring system (IMS) can be used for ground-truth in evaluating FDD tools in the field.

Working with women who use force: a feasibility study protocol of the Positive (+)SHIFT group work programme in Australia

IntroductionThis study assesses the feasibility of the Positive Shift (+SHIFT) programme in the context of legal responses and social welfare provision in the state of Victoria, Australia.The +SHIFT programme, adapted...

Implementing Same Day Discharge Following Percutaneous Coronary Intervention: A Process Evaluation.

The safety and effectiveness of same day discharge (SDD) following percutaneous coronary intervention are well demonstrated; however, the uptake of this model of care is low. The aim was to examine the effectiveness of implementing SDD using a process evaluation methodology. This study was undertaken in a cardiac services department of a tertiary teaching hospital in southeast Queensland, Australia. It was anticipated before the implementation that 120 patients could be discharged the same day in a 6 months' time period. Patient selection process and guideline adherence were assessed along with patients' and relatives' satisfaction. During implementation, 22 patients were discharged home the same day. It was found that staff did not follow the guideline consistently, with an overall adherence of 77.3%. The uptake of SDD was low in this implementation. The study is important as it provides direction for future improvement both in the criteria and the implementation process.

The development and implementation of the Art of Happiness intervention for community-dwelling older adults

ABSTRACTResearch has shown that positive psychology interventions can enhance subjective well-being and reduce depression, however, efficacy on older adult populations has not been widely examined. Specifically, subjective well-being, life satisfaction, optimism, and positive emotions have been shown to improve health and longevity. The present study soughts to develop and evaluate an intervention with the goals of enhancing positive mental health among older adults. The Art of Happiness is a theory-based, eight-week intervention that was conducted at three senior centers in the state of Delaware. Each 90-min class examined a different topic including: (1) defining happiness, (2) stress management, (3) reflecting on happiness, (4) compassion and human connection, (5) forgiveness, (6) transforming suffering, (7) mindfulness, and (8) humor. Pre and post questionnaires assessed participant subjective happiness, stress, gratitude, life satisfaction, depression, mindfulness, arousal states, and general demographic and health information. This paper outlines the formative and process evaluation methodology related to the Art of Happiness project.

Community dialogues for child health: results from a qualitative process evaluation in three countries

BackgroundAcross the developing world, countries are increasingly adopting the integrated community case management of childhood illnesses (iCCM) strategy in efforts to reduce child mortality. This intervention’s effectiveness is dependent on community adoption and changes in care-seeking practices. We assessed the implementation process of a theory-driven community dialogue (CD) intervention specifically designed to strengthen the support and uptake of the newly introduced iCCM services and related behaviours in three African countries.MethodsA qualitative process evaluation methodology was chosen and used secondary project data and primary data collected in two districts of each of the three countries, in purposefully sampled communities. The final data set included 67 focus group discussions and 57 key informant interviews, totalling 642 respondents, including caregivers, CD facilitators community leaders, and trainers. Thematic analysis of the data followed the ‘Framework Approach’ utilising both a deduction and induction process.ResultsResults show that CDs contribute to triggering community uptake of and support for iCCM services through filling health information gaps and building cooperation within communities. We found it to be an effective approach for addressing social norms around child care practices. This approach was embraced by communities for its flexibility and value in planning individual and collective change.ConclusionsRegular CDs can contribute to the formation of new habits, particularly in relation to seeking timely care in case of child sickness. This study also confirms the value of process evaluation to unwrap the mechanisms of community mobilisation approaches in context and provides key insights for improving the CD approach.

Process evaluations in neurological rehabilitation: a mixed-evidence systematic review and recommendations for future research

ObjectiveTo systematically review how process evaluations are currently designed, what methodologies are used and how are they developed alongside or within neurological rehabilitation trials.MethodsThis mixed-methods systematic review had two evidence...

Rehabilitation professionals’ perceptions of the use of new visualisation software tools with people with stroke

Purpose: The envisage programme of research was funded to explore and evaluate the use of visualisation software tools using biomechanical data within rehabilitation. Three work packages were developed to evaluate the impact of the tools within stroke rehabilitation. The research presented here aimed at exploring the perceptions of rehabilitation therapists about the use of the visualisation software tools in the context of future randomised controlled trials and stroke rehabilitation practice. Methods: Sixteen therapists working in a range of stroke rehabilitation contexts participated in semi-structured interviews. Interview questions explored their current practice, and the perceived impact of the new visualisation technologies on their workplace environment and practice. Framework analysis was used to analyse the textual data. Results: In general, the stroke therapists were enthusiastic about the potential application of the visualisation software tools. Three themes were identified through qualitative framework analysis: potential uses of the visualisation tools; integration within current service provision; and trial involvement. Conclusions: The study highlights important contextual considerations which may impact significantly on the success of novel technologies in stroke rehabilitation. Normalisation process theory was proposed as a useful process evaluation methodology to optimise both trial evaluation and future service implementation.Implications for RehabilitationThere is limited research exploring the use of visual software technologies featuring biomechanical data within stroke rehabilitation.The perspectives of stroke rehabilitation therapists about the potential of such tools are useful both in terms of planning trial evaluations, and implementation.Therapists were generally positive about the contribution of visual software tools in stroke rehabilitation, but highlighted a number of practical constraints which required addressing.Normalisation process theory provides a useful process evaluation methodology which can support both trial evaluation and implementation of such novel technologies within stroke rehabilitation.

A study on the quantitative method for choosing an appropriate skill developing strategy (A simulation based manufacturing process redesign for skill developing)

Planning human resource procurement, as well as planning cost, quality and capacity of production, is one of the crucial issues for enterprises in order to achieve the target sales and profit. For the planning, two issues should be addressed. First is how to transfer the necessary skills and knowledge, which is in the educational perspective and many studies and methodologies have been achieved in respect to the issue. Second is how to change the production processes where the human resources are allocated considering the effect of the design on the production capacity, cost and quality of the target processes along the human resource procurement schedule, which is in the management perspective. This paper focuses on the second issue. For instance, procuring and training many resources can minimized the human resource procurement schedule. It, however, might increase the quality and cost risks of the target production processes. Contrarily, gradually procuring and training human resources might reduces the quality and cost risks. It, however, increases the procurement schedule and risks of the opportunity loss. The strategies for procuring the necessary human resources are varied by the requirements to the management based on the enterprise strategy and surrounding market demands. In this paper, we applied the simulation based process evaluation methodology for quantitatively choosing the human resource procurement strategy appropriate to the management requirements. Firstly, we predicted the process performances in respect to the available human resources with our developing simulation tool. Secondary, we estimated the changes of process performances along the various human procurement strategies. Finally, we demonstrated how to choose a human resource procurement strategy appropriate to the management requirements.

An economic, sustainability, and energetic model of biodiesel production from microalgae

A new process evaluation methodology of microalgae biodiesel has been developed. Based on four evaluation criteria, i.e. the net energy ratio (NER), biodiesel production costs, greenhouse gases (GHG) emission rate and water footprint, the model compares various technologies for each step of the process, from cultivation to oil upgrading. An innovative pathway (hybrid raceway/PBR cultivation system, belt filter press for dewatering, wet lipid extraction, oil hydrotreating and anaerobic digestion of residues) shows good results in comparison to a reference pathway (doubled NER, lower GHG emission rate and water footprint). The production costs are still unfavourable (between 1.94 and 3.35€/L of biodiesel). The most influential parameters have been targeted through a global sensitivity analysis and classified: (i) lipid productivity, (ii) the cultivation step, and (iii) the downstream processes. The use of low-carbon energy sources is required to achieve significant reductions of the biodiesel GHG emission rate compared to petroleum diesel.

A process evaluation of a comprehensive school based injury prevention programme

A process evaluation enables understanding of critical issues that can inform the improved, ongoing implementation of an intervention programme. This study describes the process evaluation of a comprehensive, multi-level injury...

Hydrothermal Gasification of Waste Biomass: Process Design and Life Cycle Asessment

A process evaluation methodology is presented that incorporates flowsheet mass and energy balance modeling, heat and power integration, and life cycle assessment Environmental impacts are determined by characterizing and weighting (using CO2 equivalents, Eco-indicator 99, and Eco-scarcity) the flowsheet and inventory modeling results. The methodology is applied to a waste biomass to synthetic natural gas (SNG) conversion process involving a catalytic hydrothermal gasification step. Several scenarios are constructed for different Swiss biomass feedstocks and different scales depending on logistical choices: large-scale (155 MW(SNG)) and small-scale (5.2 MW(SNG)) scenarios for a manure feedstock and one scenario (35.6 MW(SNG))for a wood feedstock. Process modeling shows that 62% of the manure's lower heating value (LHV) is converted to SNG and 71% of wood's LHV is converted to SNG. Life cycle modeling shows that, for all processes, about 10% of fossil energy use is imbedded in the produced renewable SNG. Converting manure and replacing it, as a fertilizer, with the process mineral byproduct leads to reduced N20 emissions and an improved environmental performance such as global warming potential: -0.6 kg(CO2eq)/MJ(SNG) vs. -0.02 kg(CO2eq)/MJ(SNG) for wood scenarios.

Plugging a hole and lightening the burden: a process evaluation of a practice education team

To investigate the perceptions of clinical and senior managers about the role of practice educators (PEs) employed in one acute hospital in the UK. Producing nurses who are fit for practice, purpose and academic award is a key issue for nurse education partnership providers in the UK. Various new models for practice learning support structures and new roles within health care institutions have been established. To sustain funding and policy support for these models, there is a need for evaluation research. A process evaluation methodology was employed to determine the current value of a practice education team and to provide information to guide future direction. Data were collected through semi-structured telephone interviews using a previously designed schedule. All senior nurse managers (n = 5) and a purposive sample of Clinical Managers (n = 13) who had personal experience of and perceptions about the role of PEs provided the data. Interview notes were transcribed, coded and a thematic framework devised to present the results. A number of key themes emerged including: qualities needed for being a successful practice educator; visibility and presence of PEs; providing a link with the university; 'plugging a hole' in supporting learning needs; providing relief to practitioners in dealing with 'the burden of students;' alleviating the 'plight of students;' and effects on student attrition. Findings provided evidence for the continued funding of the practice educator role with improvements to be made in dealing with stakeholder expectations and outcomes. In the UK, there still remain concerns about the fitness for practice of newly Registered Nurses, prompting a recent national consultation by the professional regulating body. Despite fiscal pressures, recommendations for further strengthening of all systems that will support the quality of practice learning may continue to sustain practice learning support roles.

Developing an interprofessional learning culture in primary care

SummaryThis paper discusses the concept of interprofessional learning based on empirical data from an evaluation of an interprofessional learning project that was set in a British primary health care centre. A process evaluation methodology was chosen to collect the data using semi-structured interviews and focus groups with stakeholders and staff to gather their experiences and views of the project and documentary data from records written over the lifetime of the project. The paper argues that an interprofessional learning culture requires time to become embedded in everyday practice and to achieve such a culture, shared values, aims and clear communication are essential. The data suggest that there is a need to recognize responsibility for one's own learning as individuals as well as learning as teams of work colleagues if interprofessional learning is to be successful. However, even when these pre-requisites of interprofessional learning are agreed and acknowledged openly in the workplace, participants in the development of a learning culture need to recognize that there are structural controls which influence and constrain such developments which are external to participants and beyond their immediate control.