Project Evaluation Research Articles

Evaluation of Urban Quality Improvement Based on the MABAC Method and VIKOR Method: A Case Study of Shandong Province, China

In the current stage of urbanization in China, urban construction has gradually shifted from incremental construction dominated by real estate to stock transformation based on improving urban quality. The evaluation of urban renewal projects is crucial for decision-making and resource allocation. However, there is no consensus on the evaluation indicators and methods. This study evaluates the current status of urban quality using the MABAC method based on numerical indicators. It evaluates the improvement of urban quality using the VIKOR method based on public satisfaction indicators. It is found that the comprehensive weights and subjective weights of the five first-level indicators in the evaluation index system of urban quality improvement in Shandong Province, namely blue–green space improvement, air cleanliness improvement, road traffic improvement, life service improvement, and governance capacity improvement, are quite different. The weight distribution of these indicators needs to be adjusted and optimized, especially for improving living services, as well as secondary indicators such as black and odorous water treatment, urban clean heating, the number of public vehicles per 10,000 people in the city, the 15 min living circle, the renovation of old residential areas, and the construction of urban intelligent management platforms. Based on the comprehensive evaluation results, an attempt is made to propose a more scientific evaluation index system, providing valuable references for urban renewal management.

EXAMINING THE FINANCIAL RISK MANAGEMENT OF DONOR FUNDS: CASE OF MINISTRY OF HEALTH IN ZAMBIA

<p>This study examines the financial risk management of donor funds using the Ministry of Health (MOH) in Zambia as a case study. The study employs an exploratory case study methodology on a sample of 224 MOH employees. Zambia, being a developing country, significantly relies on donor funds to support its healthcare sector. Prudent management of donor funds is crucial for the sustained delivery of quality healthcare services. However, there has been concerns regarding the financial risk management practices at the MOH in the efficient utilization and accountability of donor funds. From the literature review, it seems no study has been done regarding the financial risk management of donor funds focusing on the MOH in Zambia. Therefore, this study fills this gap. The study findings seem to show evidence of the misappropriation of donor funds exhibited by misallocation of donor funds to unplanned projects or activities. The misappropriations of donor funds seem to be amplified by a lack of strategic planning, lack of coordination, and the absence of effective monitoring and evaluation systems. Additionally, fraud and corruption posed significant risks to donor funds at MOH. Therefore, to mitigate the risks, the study recommends that robust anti-misappropriation measures, enforcement of due diligence on implementing partners, a thorough review of procurement processes, and regular monitoring and evaluation of donor-funded projects be strengthened at the MOH in Zambia. Whistleblower protection and reporting mechanisms should be promoted to help detect and deter fraudulent activities at this Ministry.</p><p><strong> </strong></p><p><strong>JEL:</strong> H51, I18, O23</p><p> </p><p><strong> Article visualizations:</strong></p><p><img src="/-counters-/soc/0787/a.php" alt="Hit counter" /></p>

Estimating multinomial logit models with endogenous variables: Control function versus two adapted approaches

It is shown that the control function (CF) method’s estimates of the modal constants in a multinomial logit model (MNL) with endogenous explanatory variables are biased. This has not previously been reported in the literature, and has consequences in demand analysis, transportation policy design and project evaluation. Two adaptations of existing approaches are proposed as alternatives to CF for deriving estimators of parameters in MNL models with endogenous explanatory variables that evidence good consistency properties. The first approach is based on moment conditions while the second one combines parameters obtained in two consecutive estimation stages. Both approaches employ instrumental variables. These two adapted approaches are implemented using simulated data from a transport mode choice problem. The results are compared with those obtained using the classic control-function method, typically used by practitioners for estimating transport demand models with endogenous variables and making quantitative evaluations of transport policies and projects. All three approaches generate similar estimates for the parameters of the explanatory variables, but the two proposed adaptations produce considerably more accurate estimates of the modal constants. This greater accuracy has potentially significant consequences for multinomial logit models’ predictive ability and estimates of marginal effects, elasticities and the social benefits of projects based on consumer surplus calculations.

Evaluation of renewable energy projects based on sustainability goals using a hybrid pythagorean fuzzy-based decision approach

Due to the growth in energy demand and the need to pay attention to sustainable development, many developing countries face the challenge of making policies for growing sustainable energy. With attention to industrial expansion and population growth, Iran, a developing country, is confronting an energy shortage problem. Despite the appropriate capacity of renewable resources in Iran, due to the lack of proper planning for adopting renewable resources in Iran, these resources have a small share of the energy supply. This study aims to present a decision framework for assessing renewable energies in developing countries, in the case of Iran. In addition to technical aspects, sustainability aspects such as economic, social, and environmental factors are proposed to assess renewable sources in this work. The Best-Worst Method (BWM) and the Interval-Valued Pythagorean Fuzzy Weighted Aggregated Sum Product Assessment (IVPF-WASPAS) techniques are suggested to prioritize renewable sources. The findings indicated that solar energy was Iran's most suitable renewable resource. Also, the validation and sensitivity analysis confirmed the findings. This research provides a comprehensive perspective for policy-making and planning of sustainability in the energy industry.

Analysis of Risk Factors Affecting Road Work Construction Failure in Sigi District

One of the causes of project failure is work risks that often occur at work locations. The aim of this research is what risk factors influence road work in Sigi Regency and what indicators have an important role in influencing roads in Sigi Regency. The 50 respondents in this study included expert staff and service supervisors, consultants and contractors. Data analysis uses Factor Analysis. The Planning and Conflict Management factor is the factor that has the highest variance value when compared to other factors, namely 26.323%, which makes this factor a risk factor with a greater influence on road work in Sigi Regency compared to other factors. Indicators that have an important role in the risk factors that influence road work in Sigi Regency are changes in the contents of documents while the contract is running, inaccurate project evaluations in terms of time and duration, changes in government policy, increases in tax prices while the work is in progress, and By not following the provisions according to the contract.

Maclaurin Symmetric Mean Aggregation Operators Based on Novel Frank T-Norm and T-Conorm for Picture Fuzzy Multiple-Attribute Group Decision-Making

The proposed study introduces innovative Maclaurin symmetrical mean aggregation operators (MSMAO) in picture fuzzy multiple-attribute group decision-making. These operators are built upon a newly developed Frank t-norm (FTN) and t-conorm (TCN). Utilizing these novel operators aims to enhance the effectiveness of decision-making processes in scenarios characterized by picture-fuzzy information. The study explores these aggregation operators' potential applications and advantages within the context of multiple-attribute group decision-making under uncertainty. As a novel method to handle MAGDM difficulties, we build these aggregation operators (AOs) on PFs in this paper by utilizing the FTN and TCN under PFs information. In addition, PF values (PFVs) and their fundamental operations are created and shown. Picture fuzzy Frank weighted Maclaurin symmetrical mean (MFFWMSM) and Picture fuzzy Frank Maclaurin symmetrical mean (MFFMSM) operators are two AOs introduced and studied based on these operations. The induction approach theoretically and quantitatively verifies the newly created AOs' accuracy and dependability. The problem of project evaluation utilizing these proposed operators is thoroughly examined to provide further applications and investigate the sensitivity of these PS Frank (PFF) operators. The outcomes of employing these PFF operators are contrasted with a few AOs of PFVs that were previously in existence. The suggested strategy is dependable and effective based on comparison outcomes.

The moderating role of gender and acculturative status in the socialization of cultural values in U.S. Mexican families.

Familismo and respeto are traditional Latine values often taught to U.S. Latine children by their caregivers. Deference to authority figures and restraint from defiance to such figures are also taught. Despite the central role of these as markers of Latine ethnic identity and their relevance to children's health and well-being, little is known on how these traditional cultural values are socialized and whether differences exist in these values between English-speaking and Spanish-speaking U.S. Latine mothers and between boys and girls. We examined expressions of familismo, respeto, deference, and defiance in conversations of low-income, U.S. Mexican mothers and their children as a function of mothers' acculturative status and children's gender. One hundred thirty-eight video-recorded U.S. Mexican mother-child conversations (65 girls; age: M = 10.5 years, SD = 0.53) were transcribed and coded to obtain indices of respeto, familismo, deference, and defiance. There were 70 Spanish-speaking mothers and 68 English-speaking mothers (age: M = 34.9 years, SD = 5.8) from the Early Head Start Research and Evaluation project. Spanish-speaking children were more deferent than English-speaking children. English-speaking children were more defiant. There were no group differences, however, in familism and respect. The findings advance a more nuanced conceptual framework of cultural values socialization that is sensitive to acculturative status in U.S. Latine populations and can inform the development of effective intervention efforts (e.g., parenting education programs) aimed at supporting enculturative practices that can enhance U.S. Latine children's health and well-being. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Equitable access to quality injury care; Equi-Injury project protocol for prioritizing interventions in four low- or middle-income countries: a mixed method study

BackgroundEquitable access to quality care after injury is an essential step for improved health outcomes in low- and middle-income countries (LMICs). We introduce the Equi-Injury project, in which we will use integrated frameworks to understand how to improve equitable access to quality care after injury in four LMICs: Ghana, Pakistan, Rwanda and South Africa.MethodsThis project has 5 work packages (WPs) as well as essential cross-cutting pillars of community engagement, capacity building and cross-country learning. In WP1, we will identify needs, barriers, and facilitators to impactful stakeholder engagement in developing and prioritising policy solutions. In WP2, we will collect data on patient care and outcomes after injuries. In WP3, we will develop an injury pathway model to understand which elements in the pathway of injury response, care and treatment have the biggest impact on health and economic outcomes. In WP4, we will work with stakeholders to gain consensus on solutions to address identified issues; these solutions will be implemented and tested in future research. In WP5, in order to ascertain where learning is transferable across contexts, we will identify which outcomes are shared across countries. The study has received approval from ethical review boards (ERBs) of all partner countries in South Africa, Rwanda, Ghana, Pakistan and the University of Birmingham.DiscussionThis health system evaluation project aims to provide a deeper understanding of injury care and develop evidence-based interventions within and across partner countries in four diverse LMICs. Strong partnership with multiple stakeholders will facilitate utilisation of the results for the co-development of sustainable interventions.

Implementation of Step-Down Intermediate Care (IC) in Buckinghamshire, UK: A Qualitative Evaluation Study of Healthcare Professionals’ Experiences and Perspectives

Step-down intermediate care aims to offer short-term care for people who are medically optimised for discharge but needing a period for further assessment and/or rehabilitation. The aim of this study, which was nested in a larger evaluation project, was to explore the experiences and perspectives of healthcare professionals to understand the implementation of a step-down IC service in Buckinghamshire, UK. The evidence is used to inform the service providers of what elements worked well and what areas require improvement. A qualitative study using semistructured interviews was conducted in May 2022. Ten healthcare professionals participated. Interviews were transcribed verbatim and reflexive thematic analysis was used to analyse the data. The following five core themes were developed: (1) the developmental period of step-down IC, (2) providing care together, (3) perceived functions of the integrated hub, (4) communication, and (5) resources. Findings from the interviews highlighted the excellent working relationships among healthcare professionals from different disciplines, which contributed to their job satisfaction and the efficiency of the service. In addition, healthcare professionals stressed the importance of the integrated hub, as it facilitated the communication between the teams and speeded up the decision-making. Several organisational challenges, such as communication issues, healthcare professionals’ capacity, and the need for further funding were also voiced. Involving staff in the evaluation of a step-down intermediate care service has provided useful information on the service’s implementation process and will inform the development of a long-term strategy for intermediate care.

Using science communication research to practice iterative engagement in collaborative nutrient management.

Thoughtful science communication is essential for the success of collaborative, transdisciplinary environmental research. We present an innovative evaluation of a four-year pilot project that took a highly engaged and collaborative approach to managing excess nutrients in the Cape Cod region of Massachusetts, USA. The evaluation approach included mid- and end-of-project interviews with researchers and project partners and a reflection from the lead science communication researcher. We found that an effective science communication evaluation needs to be (1) adaptive, (2) multistage, (3) holistic and objective-based, and (4) democratic and reflexive. Results demonstrate that formative and end-of-project science communication evaluation of research projects lead to improved engagement that better meets all collaborators' needs.

Evaluating the Feasibility of Low-Carbon Technology in the Sustainable Economic Development of Electricity Using Generative Adversarial Networks

Abstract Electricity occupies a central position in energy strategy, and the low-carbon development of the power industry is an important guarantee for achieving sustainable development and carbon dioxide reduction goals. Orderly carrying out low-carbon technology transformation and innovation in electricity, ultimately achieving the goal of sustainable development of the power industry, has strong practical significance. Therefore, this article conducts a feasibility analysis of using generative adversarial networks to evaluate low-carbon technologies in the sustainable economic development of electricity. Firstly, the background of the theory of technological and economic evaluation was analysed and used as a theoretical basis. Secondly, an adversarial network was generated for the study of low-carbon emissions in the power industry, and an evaluation system for low-carbon technology in the sustainable economic development of electricity was constructed. The study showed that the preliminary economic evaluation index system and comprehensive benefit evaluation model for low-carbon technology transformation projects in electricity established in this article have feasibility and reliability for systematic analysis and evaluation of actual projects.

Assessing a pharmacist provided mental health screening service in a rural community to address anxiety and depression

BackgroundThe current state of the mental health crisis has been a topic of discussion around the nation, with those in rural communities being at a particularly higher risk. Community pharmacists are uniquely positioned to screen patients for mental health disorders and provide education and patient monitoring following medication changes by prescribers. ObjectiveThe objective of this study is to assess mental health screening outcomes provided by rural community pharmacists. MethodsThis retrospective evaluation of a pilot project describes a behavioral health service. Eligible patients were 18 years of age or older, taking one or more mental health medications and covered by the specific Medicaid health plan. Participating pharmacies were those that had a considerable patient population participating with that plan, as well as credentialed and trained pharmacists. Pharmacists performed telehealth visits with eligible patients where they confirmed their mental health diagnosis and medications. Based on patient confirmed diagnosis, a Generalized Anxiety Disorder Questionnaire-7 (GAD-7) or Patient Health Questionnaire-9 (PHQ-9) test was administered. The pharmacists then made recommendations to the patients or their prescribers and created a plan with the patient for implementation based on their score.Follow-up calls were conducted to readminister the appropriate test to identify changes in mental health scores. ResultsThere were 61 patients who participated in the services across 4 pharmacies. Mean GAD-7 scores were 8.1 at initial appointment and 6.4 at follow-up (n = 24). Mean PHQ-9 scores were 11.2 initially and 10 at follow-up (n = 37). ConclusionThis implementation project supports the creation of additional mental health services in the community pharmacy setting to reinforce and encourage follow-up provider visits in areas where there are mental health provider shortages and may possibly improve patient outcomes.

An evaluation of the xenobotic cognitive project: Towards Stage 1 of xenobotic cognition

Xenobot, the world’s first biological robot, puts numerous philosophical riddles before us. One among them pertains to the cognitive status of these entities. Are these biological robots cognitive? To evaluate the cognitive status of xenobots and to resolve the puzzle of a single mind emerging from smaller sub-units, in this article, I juxtapose the cognitive capacities of xenobots with that of two other minimal models of cognition, i.e., basal cognition and nonliving active matter cognition. Further, the article underlines the essential cognitive capabilities that xenobots need to achieve to enter what I call stage 1 of xenobotic cognition. Stage 1 is characterized by numerous cognitive mechanisms, which are integral for the survival and cognition of basal organisms. Finally, I suggest that developing xenobots that can reach Stage 1 can help us achieve sophistication in the areas of evolution of the human mind, robotics, biology and medicine, and artificial intelligence (AI).

Impact of Projects with Future Potential on the Global Competitiveness Index of Countries

The concept of project success has evolved from the perspective of conforming to the project triangle to that of benefiting the environment, and then from the perspective of the following generation. Scientists increasingly assert that successful projects require a set of criteria that include such item(s) as future potential. The meaning of project success varies depending on where it is executed. The purpose of this study is to identify whether projects with future potential have a certain effect on indicators of the Global Competitiveness Index (GCI) of the Republic of Kazakhstan (RK) and what other success criteria are inherent in such projects. By using the method of descriptive analysis of data collected from 107 experts and analyzing 19 influential projects, the study revealed that projects oriented towards the future have a significant impact on the indicators of the GCI in the RK. This finding confirms the necessity of considering the long-term sustainability and social significance of projects when assessing their successes. Additionally, a specific combination of success criteria that contributes most to this impact was identified. This research provides a brand-new understanding of project success criteria in the context of their impact on the GCI and emphasizes the importance of considering future potential in project planning and evaluation. Doi: 10.28991/ESJ-2024-08-02-012 Full Text: PDF

Care for Staff: A Novel Technical Assistance Approach to Promote Occupational Mental Health Among Healthcare Workers in Lower and Middle Income Country Settings.

Healthcare workers (HCWs) in lower- and middle-income countries (LMICs) face unique, intersectional threats to their mental health at work. Despite the existence of recommendations for multi-tiered interventions to promote and protect occupational mental health for HCWs, there remain significant challenges to implementation worldwide. FHI 360, a global development organization, developed a novel technical assistance framework to accompany partners, including government and healthcare leaders to design, implement, improve, or evaluate any mental health and psychosocial support intervention. The EpiC Project, implemented by FHI 360, has utilized this framework in four countries (Vietnam, Philippines, Paraguay and Sri Lanka) specifically to guide the development of locally adapted occupational mental health interventions for HCWs. Each country applied this framework in various project cycle phases and in their unique local contexts; all countries reported positive developments in the advancement of their chosen interventions. With the application of an adaptable, evidence-based technical assistance framework to guide collaborative consultation for project design, implementation, improvement, and/or evaluation, locally led teams pivoted from a solely "mental health" approach to more comprehensive, evidence-based interventions that framed mental health for HCWs as an occupational health priority. This allowed for teams advising interventions in LMICs to consider unique workplace, structural and policy-level factors rather than focusing solely on individual mental health strategies.

Collaboration about a floating signifier. An analysis of a case of project evaluation committee

El objetivo de este artículo es proponer una lectura de la colaboración como posible gracias a la existencia de ambigüedades organizacionales y comunicacionales resueltas por la operación de un significante flotante. Para ilustrar la propuesta teórica se utiliza la observación participante de un comité de evaluación de proyectos cuyas decisiones van más allá de su cometido principal. El significante flotante se presenta como objeto de transacciones organizacionales. Su flexibilidad semiótica permite la colaboración al organizar intereses divergentes mediante la suspensión de impulsos críticos y la movilización de mecanismos normativos.

135 A Year Later: A multi-institutional QI project to enhance leadership conversations about retention

OBJECTIVES/GOALS: Optimize an Individual Retention Conversation (IRC) toolkit aimed at enhancing trust amongst CRPs and leadership via a 2-phase project wherein 9 academic medical centers (AMCs) with significant CRP workforces developed and assessed a 16 question IRC guide and accompanying manager/leader instructional guide. #_msoanchor_1 METHODS/STUDY POPULATION: Significant interest in adapting the Stay Interview concept for the CRP workforce led to a 2-phase pilot to optimize the re-envisioned IRC toolkit. Representatives from nine AMCs and research sites volunteered to navigate their respective institutional IRB processes to initiate the assessment. Additional sites, such as Frontiers Clinical and Translational Institute (Frontiers) launched variations of the IRCs outside of the structured QI project to meet the needs of their institutional environments and reported feedback to the larger group. Feedback on both the standardized IRC, as well as Frontiers’ tailored version, will be presented. This will serve as an entryway into Phase 2, a multi-institutional mixed methods evaluation project open to all AMC members of ACTS and the CRPT SIG. RESULTS/ANTICIPATED RESULTS: To date, 7 institutions have initiated IRCs with test groups at their institutions. Each institution had unique requirements, but all IRBs deemed Phase 1 to be exempt/not human research. Preliminary data suggest not only that the IRC process is valuable to both employee and their manager/unit leadership, but also that the simple act of conducting IRCs was found to be unique and meaningful to employees. For example, in their tailored IRC process, Frontiers found that the 90% of their team found the process to be beneficial (n=9). DISCUSSION/SIGNIFICANCE: By acknowledging issues, understanding motivations, and increasing engagement, IRCs foster positive change, allowing team leaders to take immediate action on important issues. By doing so, retention and engagement of team members, and the CRP workforce as a whole, is likely to grow and strengthen, as supported by results from our initial test pilots.

Performance Evaluation of Deformable Image Registration Systems - SmartAdapt® and Velocity™.

To commission and validate commercial deformable image registration (DIR) systems (SmartAdapt® and Velocity™) using task group 132 (TG-132) digital phantom datasets. Additionally, the study compares and verifies the DIR algorithms of the two systems. TG-132 digital phantoms were obtained from the American Association of Physicists in Medicine website and imported into SmartAdapt® and Velocity™ systems for commissioning and validation. The registration results were compared with known shifts using rigid registrations and deformable registrations. Virtual head and neck phantoms obtained online (DIR Evaluation Project) and some selected clinical data sets from the department were imported into the two DIR systems. For both of these datasets, DIR was carried out between the source and target images, and the contours were then propagated from the source to the target image data set. The dice similarity coefficient (DSC), mean distance to agreement (MDA), and Jacobian determinant measures were utilised to evaluate the registration results. The recommended criteria for commissioning and validation of DIR system from TG-132 was error <0.5*voxel dimension (vd). Translation only registration: Both systems met TG-132 recommendations except computed tomography (CT)-positron emission tomography registration in both systems (Velocity ~1.1*vd, SmartAdapt ~1.6*vd). Translational and rotational registration: Both systems failed the criteria for all modalities (For velocity, error ranged from 0.6*vd [CT-CT registration] to 3.4*vd [CT-cone-beam CT (CBCT) registration]. For SmartAdapt® the range was 0.6*vd [CT-CBCT] to 3.6*vd [CT-CT]). Mean ± standard deviation for DSC, MDA and Jacobian metrics were used to compare the DIR results between SmartAdapt® and Velocity™. The DIR algorithms of SmartAdapt® and Velocity™ were commissioned and their deformation results were compared. Both systems can be used for clinical purpose. While there were only minimal differences between the two systems, Velocity™ provided lower values for parotids, bladder, rectum, and prostate (soft tissue) compared to SmartAdapt. However, for mandible, spinal cord, and femoral heads (rigid structures), both systems showed nearly identical results.

Experimental Covariance Determination for Critical Integral Experiments

Integral benchmarks for criticality safety and nuclear data validation require expensive uncertainty quantification studies. In general, uncertainty quantification techniques ignore correlations between experiments and shared components. Experiments, such as the Thermal/Epithermal eXperiments (TEX) campaigns, consist of many shared components, such as the Jemima highly enriched uranium (HEU) fuel plates, which create a strong correlation in their uncertainties. While these correlations are known to exist, they are often not estimated because of the complexity of such calculations. This paper describes an intuitive method of determining the covariance for each of the experimental components, providing a correlation for each family of components across the multiple cases examined within a benchmark. A proof-of-principle study using the TEX-HEU experimental campaign was performed and verified that the covariance and correlation matrices can be calculated with information commonly found in the International Criticality Safety Benchmark Evaluation Project benchmarks. This study showed that the introduction of model and experimental covariances reduces the χ2 per degree of freedom from 2.203 to 1.179, indicating that the omission causes overly pessimistic bias quantifications. This technique can be seamlessly integrated to current benchmark evaluations as well as reevaluations of legacy benchmarks.

Generating trajectory data without behavior modeling: An online virtual reality method in wayfinding performance evaluation for buildings

Wayfinding performance is one of the key issues in pre- or post-occupancy evaluation for large-scale public buildings and such evaluation typically revolves around participants’ wayfinding trajectories. Traditional evaluation methods, including laboratory virtual reality (VR) experiments and computer simulations, have their respective advantages in accuracy or cost-efficiency. However, they either suffer from high costs and low efficiency or sacrifice the diversity of behavioral data for higher efficiency and lower costs. Therefore, this paper proposes a novel method named “sampling as simulation” (SaS) via an online VR wayfinding experiment. This method allows for the rapid collection of behavioral data at a lower cost to generate highly accurate wayfinding trajectory data for evaluation purposes. It utilizes a model-image integrated approach to create realistic online virtual environments and employs a spatiotemporal replay algorithm to generate a crowd of true-trajectory-driven avatars, significantly enhancing the immersion and realism of the online virtual scenes. This method was demonstrated and validated with a wayfinding performance post-occupancy evaluation (POE) project at the International Departure of Satellite Terminal One (S1) in Shanghai Pudong International Airport (PVG). In this example, a total of 13,712 original trajectories were collected within a 72-h time window, serving as the original dataset for evaluation. Based on this dataset, the study effectively simulated wayfinding behavior in three different scenarios and evaluated wayfinding performance. Compared to existing methods, the SaS method not only improves data collection efficiency and reduces costs but also maintains the diversity of behavior by generating transformed trajectories directly from human data without behavior modeling.