Policy Evaluation Research Articles

An inter-regional input-output table series of China from 1987–2017 with integrated carbon emission data

Inter-regional input-output (IRIO) tables are essential for socioeconomic and environmental analysis. This paper compiled a continuous time series of Chinese IRIO tables with a detailed regional and sectoral classification, covering a longer period from 1987 to 2017 than existing Chinese IRIO tables. Additionally, we integrated the China provincial CO2 emission inventory data (1987–2017) to analyze trends and identify the main driving factors behind regional economic linkages and carbon emissions changes. Specifically, we adjusted the initial values of the international and interprovincial imports and exports using customs data. Subsequently, we employed the minimum cross-entropy model to balance the adjusted input-output tables (IOTs) from both row and column perspectives. Following this, we estimated the interprovincial trade flow for each sector using the gravity model. Then, we balanced these estimates with the minimum cross-entropy model to form the multi-regional input-output (MRIO) table. Finally, we constructed the IRIO table based on the proportion assumption. The compiled IRIO tables, integrated with carbon emission data, have significant applications in regional economic analysis, policy evaluation, and environmental studies.

Quantitative evaluation of solid waste management policies: the application of the policy modeling consistency index model in Pakistan

Quantitative evaluation of solid waste management policies: the application of the policy modeling consistency index model in Pakistan

Key informant perspectives of suicide prevention in Australia

BackgroundSuicide prevention policy in Australia is in a period of reform. The National Suicide Prevention Office is leading the development of a new National Suicide Prevention Strategy (Strategy). Stakeholder input is a critical element in the development of the new Strategy. This article describes key informant views about government-led suicide prevention efforts in Australia obtained as part of an environmental scan conducted as one input to inform the Strategy development process.MethodsWe interviewed 24 key informants in November and December 2022. Key informants were purposively recruited to ensure representation from cross-jurisdiction government departments/agencies, peak bodies and leaders in the suicide prevention sector, people with lived experience of suicide, and suicide prevention researchers. We enquired about successes, challenges, and opportunities. NVivo was used to conduct thematic analysis.ResultsKey themes identified as successes in Australia’s suicide prevention efforts included: leadership and funding for programs, services, and research; valuing the collective lived experience voice; moving towards a whole-of-government/system approach; and high community and political suicide (prevention) awareness. Key themes emerging as challenges in the sector were: defining the suicide prevention sector, limitations in the service system, workforce issues, and building the evidence base. Key themes mentioned as opportunities for improving suicide prevention efforts were: leveraging the current unprecedented awareness and desire for collaboration among multiple stakeholder groups; adopting wellness rather than crisis-driven models of care; including lived experience and co-design in all stages and aspects of policy planning, service development, and evaluation; and investing in data, research, and evaluation.ConclusionsKey informants from across the suicide prevention sector in Australia identified a range of issues for consideration in the development of Australia’s new National Suicide Prevention Strategy which are also relevant for suicide prevention policy and program development in other high-income countries. Key issues include the need for concerted efforts to define and build the capacity of the suicide prevention sector, implement and monitor a whole-of-government approach that includes wellness models of care and lived experience, and bolster the evidence base. These efforts require effective leadership and resourcing.

Impacts of the US CDC recommendation on human papillomavirus vaccine uptake, 2010–2015

ObjectivesAs one type of vaccine policy, the effectiveness and spillover effects of the US CDC vaccine recommendations are inadequately evaluated. This study aims to fully evaluate its impacts on male adults, in addition to children, using better data.DesignA before-after study design to examine the CDC’s 2011 HPV vaccine recommendation for men aged 11–21.Data analysisIndividual-level data included the 2010–2015 US National Health Interview Survey full sample of 7,000 male children aged 11–18, younger adults aged 19–21 and 22–25, and older adults aged 26–60. Pooled cross-sectional surveys contained individual-level vaccination, socioeconomic, and demographic information. Outcome variable is an individual HPV vaccination status, measured as individual probability of HPV vaccination. Dummy regressions were estimated by a Linear Probability Model (LPM) with fixed effects for target and non-target age groups.ResultsThe policy was significantly associated with a 14.8% (p < 0.001) increased individual likelihood of HPV vaccination for men aged 11–21. It was also associated with a modest spillover effect, a 5.6% (p < 0.001) increased individual likelihood for men aged 22–25 and marginally for men aged 26–60. African American men and men with poor health were 2.7 and 15.4% less likely to uptake HPV vaccines than white men and men with good or fair health, respectively.ConclusionThis study complements the existing policy evaluation literature on HPV vaccine recommendation among male children by including adults and using better data. Findings offer comprehensive evidence of the effectiveness and spillover effects of this recommendation type of federal-level policy, provide policy lessons for other vaccines, and identify vulnerable subpopulations as targets for future policies.

Multi-agent Reinforcement Learning for the Control of Three-Dimensional Rayleigh–Bénard Convection

AbstractDeep reinforcement learning (DRL) has found application in numerous use-cases pertaining to flow control. Multi-agent RL (MARL), a variant of DRL, has shown to be more effective than single-agent RL in controlling flows exhibiting locality and translational invariance. We present, for the first time, an implementation of MARL-based control of three-dimensional Rayleigh–Bénard convection (RBC). Control is executed by modifying the temperature distribution along the bottom wall divided into multiple control segments, each of which acts as an independent agent. Two regimes of RBC are considered at Rayleigh numbers $$\textrm{Ra}=500$$ Ra = 500 and 750. Evaluation of the learned control policy reveals a reduction in convection intensity by $$23.5\%$$ 23.5 % and $$8.7\%$$ 8.7 % at $$\textrm{Ra}=500$$ Ra = 500 and 750, respectively. The MARL controller converts irregularly shaped convective patterns to regular straight rolls with lower convection that resemble flow in a relatively more stable regime. We draw comparisons with proportional control at both $$\textrm{Ra}$$ Ra and show that MARL is able to outperform the proportional controller. The learned control strategy is complex, featuring different non-linear segment-wise actuator delays and actuation magnitudes. We also perform successful evaluations on a larger domain than used for training, demonstrating that the invariant property of MARL allows direct transfer of the learnt policy.

Stochastic First-Order Methods for Average-Reward Markov Decision Processes

We study average-reward Markov decision processes (AMDPs) and develop novel first-order methods with strong theoretical guarantees for both policy optimization and policy evaluation. Compared with intensive research efforts in finite sample analysis of policy gradient methods for discounted MDPs, existing studies on policy gradient methods for AMDPs mostly focus on regret bounds under restrictive assumptions, and they often lack guarantees on the overall sample complexities. Toward this end, we develop an average-reward stochastic policy mirror descent method for solving AMDPs with and without regularizers and provide convergence guarantees in terms of the long-term average reward. For policy evaluation, existing on-policy methods suffer from suboptimal convergence rates as well as failure in handling insufficiently random policies due to the lack of exploration in the action space. To remedy these issues, we develop a variance-reduced temporal difference (VRTD) method with linear function approximation for randomized policies along with optimal convergence guarantees, and design an exploratory VRTD method that resolves the exploration issue and provides comparable convergence guarantees. By combining the policy evaluation and policy optimization parts, we establish sample complexity results for solving AMDPs under both generative and Markovian noise models. It is worth noting that when linear function approximation is utilized, our algorithm only needs to update in the low-dimensional parameter space, and thus can handle MDPs with large state and action spaces. Funding: T. Li and G. Lan were supported in part by the National Science Foundation [Grant CCF-1909298] and the Office of Navel Research [Grant N00014-20-1-2089].

Examining the Perceived Roles of Educational Planners in Shaping Pupils' Academic Performance in Dodoma Municipality, Tanzania

This study examines the roles of educational planners in shaping pupils' academic performance in Dodoma Municipality, Tanzania. The study aims to address two main specific objectives namely identifying strategies employed for enhancing pupils’ academic performance and assessing the challenges faced by educational planners in achieving the goals. An interpretive paradigm guided the study, using a qualitative approach with an exploratory case study design. The target population included 2600 teachers and 37 educational officers in Dodoma municipality. However, from the total population of 2637 only 15 participants were purposively sampled to participate in the study. Data were analyzed using content analysis with the aid of Atlas.ti software; and ethical approval was granted by the Directorate of Research, Publications, and Innovation at the Open University of Tanzania. The findings revealed that the strategies employed by educational planners to improve pupils' academic performance include curriculum development and implementation, teacher training and professional development, resource allocation and infrastructure development, monitoring and evaluation, stakeholder engagement, policy evaluation and implementation, and promoting equity and inclusion. Despite these efforts, several challenges hinder the effectiveness of these strategies. These challenges include economic constraints, inadequate resources, demographic factors, socio-cultural influences, technological gaps, and the need for enhanced teacher capacity building. In conclusion, while educational planners in Dodoma Municipality have implemented strategies aimed at improving academic outcomes, significant challenges remain. Addressing these barriers is essential for sustainable improvement. The study recommends aligning the curriculum with local contexts, expanding teacher training opportunities, particularly in modern pedagogies, and increasing community involvement in educational planning. Additionally, it is crucial to address resource allocation and infrastructure disparities, particularly in rural areas, to ensure that all pupils benefit from improved educational opportunities.

Modeling the impacts of governmental and human responses on COVID-19 spread using statistical machine learning

ABSTRACT Understanding the impacts of governmental and human responses on the pandemic control is imperative for forecasting pandemic spread under various responsive scenarios and guiding localized interventions before pharmaceutical interventions are available. This study analyzed multiple data sets, including social media, mobility, policy evaluations, and COVID-19 infection reports, to delineate the interactions between governmental and human responses and COVID-19 spread in the United States in 2020 when vaccinations were unavailable. The contributions are (1) uncovering the spatiotemporal variations in governmental and human responses during COVID-19; (2) developing a statistical machine learning algorithm that incorporates spatiotemporal dependencies and temporal lag effects to model the relationships between governmental and human responses and the pandemic spread; (3) dissecting the impacts of human responses on the pandemic across space and time. Results reveal that the determinants of COVID-19 health impacts transitioned from human mobility during the initial outbreak phase to both human mobility and stay-at-home policies during the rapid spread phase, and ultimately to the compound of human mobility, stay-at-home policies and the public awareness in the full-blown phase. These findings furnish guidance for policymakers in implementing adaptive and phased strategies.

Evaluation of sports tourism development policy in Sumedang

The local government is making serious efforts to develop tourism in several destinations in Sumedang. Various policies regarding tourism were created to facilitate the implementation of regional tourism. To ensure the level of success of tourism policies, researchers evaluated sports tourism policies. Materials and Methods. The research method used is an evaluation method with a logic model with keys including context, input, activity, output, outcome, and impact. Research subjects include local governments, sports players, tourists, and the community. The research instruments used included interviews with 32 questions, observation, and documentation. Results. The results of this research include a context (policy, organization and program objectives), input (human resources, number of sport tourism, budget, facility time, and activities (training, material development, program implementation, and event promotion), output (participants, total income, increased employment opportunities, and improved regional image), outcome (knowledge, behaviour, social conditions, and economic quality), impact (social change, improved quality of life) Conclusions. This research provides implications for policymakers to maximise local governments' role in developing regional tourism, especially sports tourism. Evaluation of sports tourism policies in this research provide suggestions for local governments to review and make sports tourism a priority tourism.

Productive in disagreement: stakeholder deliberation insights on carbon dioxide removal in Germany

Attention to carbon dioxide removal (CDR) in climate policy is growing, and many CDR methods such as direct air carbon capture and storage (DACCS) are controversial. As such, broadening knowledge creation to include stakeholder perspectives upstream of policy is important. This exploratory study provides insights into the stakeholder engagement process of a transdisciplinary research project and its findings regarding co-creative CDR policy design and evaluation. It analyzes the views of participants in a stakeholder engagement workshop on CDR and an online survey. In all instances, experts highlighted the importance of the context in which these technologies are deployed. Workshop participants’ views of DACCS, including its risks and opportunities, evolved throughout the process, indicating that learning took place. We also present stakeholders’ reflections on their own role in knowledge creation and policy design. The qualitative experience and joint exploration of topics reaffirm the relevance of proper engagement on controversial, wicked problems such as the scaling of CDR as exemplified for DACCS. A nuanced discussion of the deployment context matters for stakeholders’ perception of DACCS. Our results underscore the importance of deliberative and adaptable policymaking in the current formative phase of CDR policy in Europe. Additionally, they highlight the need for policies to proactively address tradeoffs between climate mitigation efficiency and other goals. They advocate for government-backed CDR research and development (R&D) as a basis for future deployment alongside a fossil fuel phaseout to maintain a clear carbon budget and avert mitigation deterrence.

Building a community of practice to address alcohol harms in Canada: Experience from the Canadian Alcohol Policy Evaluation project.

Alcohol is a major cause of health and social costs and harms in Canada. While research and awareness of harms caused by alcohol are on the rise, few transdisciplinary platforms exist that are committed to facilitating bold alcohol policy change to reduce health inequities and improve lives. In response to feedback heard during engagement for the Canadian Alcohol Policy Evaluation project, an alcohol policy-focused community of practice (CoP) was launched in January 2022. Webinars, roundtable discussions, working group meetings, networking events, and a digital platform allow practitioners from various sectors (e.g. public health, alcohol regulation and distribution, public safety, justice, non-governmental organizations), researchers, and people with lived/living experience (PWLLE) to connect, share experiences and resources, and build capacity. More than 500 members have joined the CoP from all Canadian provinces and territories, and international jurisdictions. CoP members engage in learning opportunities, contribute to letter-writing campaigns in support of alcohol policy initiatives, and lead a working group focused on alcohol warning labels. Through the CoP, members report gaining and applying new knowledge in their work, while also establishing valuable connections and collaborations that have supported positive change. This cross-jurisdictional, intersectoral alcohol policy CoP facilitates knowledge sharing, networking, and collaboration among practitioners, policymakers, advocates, and PWLLE, while contributing to public health efforts to prevent alcohol harms. Furthermore, as transdisciplinary approaches continue to be prioritized in research and practice, this CoP offers an example that could be applied to other public health initiatives.

Policy Oversimplification in the First 100 Days Post-Regional Head Elections

The 100-day work period concept is often used to evaluate the initial performance of elected officials, including in Indonesia. While useful as an early indicator, this concept tends to oversimplify the complex policymaking process into campaign promises that must be fulfilled in a short timeframe. This study highlights the phenomenon of oversimplification during this period, where policies often focus on visible short-term outcomes at the expense of sustainability and long-term impact. Using semi-structured interviews, policy document analysis, and data triangulation, the research identifies that policy evaluations during the 100-day period frequently emphasize administrative and statistical achievements without considering deeper social changes. The study recommends a more sustainable approach to policy formulation and evidence-based evaluation involving various stakeholders. The findings suggest that policy success should be measured based on its ability to foster inclusive change, reduce social inequality, and improve community quality of life in the long term. Further research is needed to develop ongoing evaluation mechanisms to assess the impact of policies beyond the 100-day work period.

An examination of food insecurity among Canadian Aboriginal people

Background Food insecurity is a significant problem affecting many Indigenous people in Canada. This paper examines the prevalence, causes, and effects of food insecurity among Aboriginal populations. Methods Using a review of peer-reviewed articles, government reports, policy evaluations, and data from Statistics Canada, it highlights how factors such as remoteness, mental illness, traditional food consumption patterns, and socioeconomic conditions contribute to food insecurity. The paper also discusses existing policies, such as the Canada Child Benefit (CCB) and Nutrition North Canada (NNC), which are designed to address this issue. Additionally, it offers policy recommendations, including improving supply chain efficiency, monitoring subsidy programs, reevaluating eligibility issues associated with the NNC, providing support to improve access to existing government policies, and dismantling racist structures through initiatives such as Canada’s Anti-Racism Strategy 2024-2028. Results One important link made in the study is how educational attainment and income levels among Indigenous people are a reflection of structural injustices that lead to greater vulnerability to food insecurity, as well as the effectiveness of government policies designed to mitigate food insecurity. Conclusions The study emphasizes the need for a multi-pronged approach that combines modern strategies with traditional Indigenous values to build resilience against food insecurity. Addressing these factors can enhance the impact of policies targeted at effectively reducing vulnerability and improving food security among Indigenous communities in Canada.

Empowering emerging evaluators in evaluation project phases: Perspectives and recommendations

Background: Engaging emerging evaluators (EEs) in evaluation projects offers substantial benefits for both evaluators and organisations; however, effective guidance on empowering them is often limited.Objectives: This article provides actionable recommendations for empowering EEs across all phases of evaluation projects, drawing primarily on insights from EEs and their mentors. Leveraging the Eval4Action campaign’s standards for meaningful youth engagement, this article aims to enrich EEs’ learning experiences, foster their professional growth, and enhance evaluation quality and impact.Method: A qualitative and participatory approach was used, incorporating the author’s mentorship experiences, discussions with EEs and mentors, and consultations with established programmes such as the South African Monitoring and Evaluation Association’s (SAMEA) EE initiative and the EvalYouth Global Network. A comprehensive literature review on youth engagement and partnerships further informs this study.Results: Findings indicate that effective EE engagement relies on well-defined roles, open communication, and genuine appreciation of EEs’ contributions. Despite these needs, many EEs reported being limited to data collection roles, which restricts their opportunities for growth and limits their developmental impact within evaluations.Conclusion: Adopting these recommendations can foster a culture of empowerment and inclusion, enhancing both the quality and effectiveness of evaluation projects.Contribution: This study underscores the importance of engaging EEs in meaningful roles throughout projects, advancing inclusivity and effectiveness in evaluations. Empowering EEs leads to more comprehensive assessments, providing deeper insights for policy and programme evaluation.

Quantitative evaluation study of lithium sustainable development policies in China based on the PMC index model

The sustainable development of lithium is essential for facilitating the low-carbon energy transition and ensuring sustainable economic growth. Understanding the effectiveness of China’s policies on lithium development is vital for advancing these goals. This study constructed a comprehensive policy evaluation system using content analysis and the PMC (Policy Modeling Consistency) index model. A total of 12 representative policies were analyzed, and their performance was visualized through PMC surface diagrams. The evaluation revealed that among the 12 policies, 2 were rated as “excellent,” 9 as “good,” 1 as “acceptable,” and none as “poor.” These results indicate that China’s policies for lithium’s sustainable development are generally well-designed and scientifically grounded. Despite their strengths, the policies exhibit areas for improvement, including low timeliness, insufficient long-term planning, and limited dynamic adjustment mechanisms. Regional policy coordination is also lacking, especially in non-major lithium-producing areas where policy quality tends to be lower. To address these issues, we recommend optimizing future policies by enhancing timeliness and flexibility, improving regional coordination, refining policy tools, and increasing public participation.

Labor Engine or Welfare Trap: New Evidence from the Impact of China's Long-Term Care Insurance Policies on Labor Market Performance

The long-term care insurance system is a crucial social service policy for disabled and elderly individuals with dementia in the context of the unfeasible traditional family old-age care model. Through the support of the social insurance model for old-age care security, it effectively addresses and meets the actual needs of long-term care recipients. This has become a significant way to alleviate the burden of disabled and demented elderly individuals, mitigate the impacts of ageing, and create employment opportunities for family caregivers. Using the long-term care insurance pilot policy implemented in Qingdao in 2012 as a quasi-natural experiment, this paper estimates the influence of long-term care insurance on the labor market performance of the pilot city using the synthetic control method of project policy evaluation. Several robustness tests of the labor market effects of long-term care insurance are conducted, utilizing two placebo methods based on time and space, and the ranked test method. The study's findings demonstrate that the implementation of the long-term care insurance policy pilot has a constructive influence on the employment choices of workers in the labor market. Specifically, it encourages participation in the workforce and expands the scale of labor force participation in the pilot cities. Furthermore, the policy's effect strengthens over time and has a marked positive effect on average employee wages, thereby spurring labor supply growth. Through further analysis, it can be determined that the pilot cities have contributed to an average annual growth of around 1 million people in total labor force participation between 2013-2017, surpassing the potential growth rate by approximately 10.5%. Additionally, during this period, the average employee wage displays a linear upward trend with a maximum increase of over 5,000 pounds per capita per year. This suggests a substantial effect on the overall performance of the labor market due to its growth-promoting influence. In addition, the analysis of the heterogeneity of its impact mechanism demonstrates that the long-term care insurance policy aims to enhance the willingness of tertiary industry workers to work and considerably increase the likelihood of their entry into the labor market. It also improves labor income expectations of urban private and self-employed workers, resulting in a virtuous circle of enhanced labor and employment driven by the increased value of labor and the strengthened willingness to participate.

Costos de transacción en los proyectos ambientales

It is based on the need to analyze the scope of transaction costs (TC) in the development and use of instruments in environmental policies, as well as their effects on social welfare. Investigating the causes of TK and their dimensions is essential for the design, selection and evaluation of environmental policies. Studying the influence of TK throughout the development, implementation and administration of environmental policies, as well as their distribution among different stakeholders, provides a deeper understanding to environmental policymakers, enabling them to more efficiently manage environmental resources and public goods. Lack of knowledge about TK limits the information available to environmental policy decision-makers, affecting their ability to address ex ante and ex post issues in relation to environmental projects.

Causal Inference in Transport Policy Evaluation: Principle, Method and Application of Regression Discontinuity Design

The methods of causal inference have swept the social sciences in recent years, and regression discontinuity design is one of the typical methods in causal inference. In the field of estimation and impact of traffic policy in developing countries, the research on cause-and-effect relationships is extremely limited. This paper introduces the principles, methods of regression discontinuity design, research application in the field of transport policy, and compares regression discontinuity design with methods of causal inference on the advantages and disadvantages in the field of transport policy. The purpose of this paper is to introduce the regression discontinuity design method into the field of transport policy evaluation, to look forward to its application prospects and to provide basic support for the application of regression discontinuity design in the field of developing countries' transport policy.

Evaluating heavy metals-related risk in staple crops and making financing strategy for corresponding soil remediation across China

China's staple crops face heavy metal (HMs) contamination, a widespread issue lacking a national assessment. We used machine learning (ML) to assess risks of 8 HMs in rice, wheat, and maize, and estimated a financing strategy for soil remediation via linear optimization and computable general equilibrium (CGE). The accumulation of HMs in crops depends on Soil-HMs, climate, soil properties, and crop types. Cd and Hg pose major soil pollution risks, while Cr, Pb, and Cd are the most threatening in crops. High-risk zones are located at the warm temperature and subtropical zones, with wheat most vulnerable. Over a quarter (26.77%) of the nation's croplands are classified as high-risk, with a significant 60.89% falling into the medium-risk category, leaving merely 12.34% of the agricultural land in a safe condition. The estimated remediation cost is 58596.73 billion RMB and the crop loss is 808.03 billion RMB in a ten-year remediation period at the context of secure crop supply. The reallocation of social investment rather than raising new taxation for the remediation is beneficial to the GDP increase and social welfare despite some loss in the household income and enterprise income. This study provides a comprehensive evaluation for Crop-HMs risk and remediation policy, crucial for national crop security.

사회정책과정 단계별 관련 괘의 사회정책적 해석과 정책적 과제

The purpose of this study is to interpret the social policy situation of the relevant hexagrams of the stages in social policy process from the standpoint of the social policy by using the framework which is sought through modifying and supplementing the existing framework and to identify its policy tasks. The policy tasks of the revevant hexagrams related to each stages in social policy process identified through this study can be summarized as follows. First, the policy task of hexagram in the social problem forming stage is to seek new approach or theoretical perspective on social problems Second, the policy task of hexagram in the social policy agenda forming stage is about which of the various social policy agenda forming factors should be prioritized and applied. Third, the policy task of the hexagrm in social policy decision-making stage is about what should be reflected as the core content when setting social policy goals. Fourth, the policy task of hexagram in social policy implementing stage is that communication between persons concerned social policy should be emphasized in order to achieve social policy goals. Fifth. The policy task of hexagram in the social policy evaluating stage is that among the various types of policy evaluation, performance evaluation or program evaluation should be given the highest priority. The policy tasks of of relevant hexagrams in the stages in social policy process through interpreting the social policy situation from the standpoint of social policy have been identified. If so, it can be said that The Book of Change is a human heritage that should be used in various ways in the field of social welfare and social policy.