Impact Evaluation Methodologies Research Articles

Application of a climate impact evaluation methodology to compare turboprop and jet aircraft

Application of a climate impact evaluation methodology to compare turboprop and jet aircraft

Corporate impact of the COVID-19 pandemic: An approximation based on business data from Ecuador

This study analyzes the impact of the COVID-19 pandemic and the economic reactivation carried out in Ecuador in 2021 on the corporate performance of Ecuadorian companies; in this country, most of the business fabric is made up of micro and small companies, which is why it could provide new insights to the literature. The empirical analysis is based on the DID impact evaluation methodology from a company database obtained from the financial statements of the Superintendency of Companies of Ecuador. The results reveal that there is great heterogeneity in the corporate impact of the pandemic and that the affected ones were mainly micro and small companies; additionally, they indicate that the negative effects of the pandemic on financial performance extended into 2021, thus signaling a weak economic recovery. These results are key for designing policies aimed at economic reactivation in the business segments and industries most impacted by the pandemic.

Life cycle assessment of liquid hydrogen fuel for vehicles with different production routes in China

Liquid hydrogen has advantages in terms of energy density, refueling speed, driving range and emission performance compared to electric and gasoline in road vehicle applications. However, the disadvantage is that high energy losses occur during production and utilization. Therefore, it is necessary to consider the energy consumption and emissions of different hydrogen production options and to choose the best development option among the various pathways. In this study, a fuel life cycle analysis model was constructed for six hydrogen production pathways and two comparison pathways from the Chinese reality by using the assessment methodology of life cycle assessment (LCA). The life cycle environmental impacts of these pathways were accounted by the GREET software, which yielded the energy consumption, greenhouse gas and pollutant emissions at each phase of these fuel pathways. An evaluation of the environmental impacts of each of these pathways was also completed by introducing the Environmental Toxicity Impact Evaluation (ETIE) methodology. The results showed that natural gas (production plant and refueling station) and solar photovoltaic pathway could effectively reduce energy consumption. In the future, it will be necessary to optimize the technical structure of hydrogen production and storage to accelerate the achievement of energy savings and emission reductions.

Harnessing impact evaluation to build evidence in upstream conservation initiatives

The conservation community needs to improve the quality and quantity of evidence to address urgent and complex environmental challenges. The growth of Upstream Conservation Initiatives (UCIs) such as those designed to eliminate deforestation by shifting consumer or corporate behavior, exemplify the tension between the urgency to act and the need to build evidence to enable progressive improvements in implementation. These initiatives—often led by non-governmental organizations (NGOs)—are rarely evaluated rigorously in terms of their causal effect on intended outcomes, contributing to the evidence deficit. Impact evaluations have been useful tools to analyze effectiveness and build evidence in other fields. However, they are seldom used in the conservation field broadly or in UCIs specifically. To help practitioners involved in UCIs make better use of these tools, we introduce the field of impact evaluation, its core concepts, and how it contrasts with and complements traditional performance measurement approaches. Using these concepts, we propose a guidance process which can be used by practitioners at the project design phase to weigh tradeoffs and consider the most appropriate impact evaluation methodologies. We then use this guidance process to analyze two UCIs implemented by an NGO in Indonesia. Through these case studies we show how impact evaluation concepts could have been employed in project design to improve implementation and promote progressive learning, even if actual impact evaluations were infeasible. We conclude by proposing recommendations on how implementers, donors, and academia can build off these ideas to improve learning and strengthen the evidence base.

Income diversification patterns and their impact on bank risk

AbstractThis study focuses on the crucial topic of income diversification and its implications for bank risk, using the context of India's public and private Sector banks. The research spans the years 2005–2021, investigating the influence of varying degrees of non‐interest income on the risk profiles of Indian banks. Employing a program impact evaluation methodology, the study employs the generalised propensity score‐fractional dose–response function technique to ascertain the impact of different levels of non‐interest income on bank risk. The outcomes underscore the diverse effects of diversification contingent upon factors like bank size and ownership type, including large and small banks as well as public and private sector institutions. Notably, the aftermath of the Global Financial Crisis introduced distinctive dynamics to income diversification's impact. By segmenting the study timeline into two periods—2005–2013 and 2014–2021—it emerges that income diversification lowered bank risk in the earlier period, while conversely amplifying risk in the latter phase. These insights bear significance as an early warning mechanism for bank executives, policymakers, and regulatory authorities, enabling them to navigate strategic transitions associated with non‐interest income. Employing a novel approach, this research contributes to the literature by evaluating the interplay between income diversification and bank risk across diverse levels of non‐interest income mix. In contrast to conventional econometric methods, the use of the generalised propensity score method stands out as a robust technique to mitigate confounding biases inherent in non‐experimental inquiries.

Estimating the environmental impact of Generative-AI services using an LCA-based methodology

As digital services are increasingly being deployed and used in a variety of domains, the environmental impact of Information and Communication Technologies (ICTs) is a matter of concern. Artificial intelligence is driving some of this growth but its environmental cost remains scarcely studied. A recent trend in large-scale generative models such as ChatGPT has especially drawn attention since their training requires intensive use of a massive number of specialized computing resources. The inference of those models is made accessible on the web as services, and using them additionally mobilizes end-user terminals, networks, and data centers. Therefore, those services contribute to global warming, worsen metal scarcity, and increase energy consumption. This work proposes an LCA-based methodology for a multi-criteria evaluation of the environmental impact of generative AI services, considering embodied and usage costs of all the resources required for training models, inferring from them, and hosting them online. We illustrate our methodology with Stable Diffusion as a service, an open-source text-to-image generative deep-learning model accessible online. This use case is based on an experimental observation of Stable Diffusion training and inference energy consumption. Through a sensitivity analysis, various scenarios estimating the influence of usage intensity on the impact sources are explored.

Assessing Impact in Europe: A Systematic Review of Evaluation Methodologies in Homeless Interventions

Homelessness presents a complex societal challenge, necessitating evidence-based interventions. This paper conducts a PRISMA systematic review of impact evaluation methodologies in homeless interventions, examining the existence of standardized methodologies, and the role of theoretical frameworks, consensus on evaluation designs, and reliable outcome variables. Drawing from diverse studies, the research comprehensively analyzes impact evaluations with the goal of yielding valuable insights for practitioners and policymakers responding to the challenges and dynamics of the European context. The findings reveal a lack of standardized methodologies validated by regulatory agencies, particularly within Europe. Theoretical foundations guiding the evaluations vary widely, highlighting the need for a context-sensitive framework that considers the complexities of homelessness and socio-political factors across welfare states. While randomized controlled trials offer rigor, they are underutilized in Europe. The review advocates a mixed-methods approach for comprehensive insights to capture the multifaceted nature of homelessness interventions. Furthermore, the identification of suitable outcome variables emerges as a challenge, with inconsistent definitions hindering cross-study comparisons. The analysis underscores the significance of adopting standardized outcome variables, such as the ETHOS definition, to facilitate robust impact assessments. This review emphasizes the need for methodological refinement and collaboration, enabling comparability between programs and the generation of reliable evidence. Advocating standardized methodologies, robust frameworks, and comprehensive designs, it guides future research, evidence-based policymaking, and effective homeless interventions.

Impact of a Spatial and Regional Planning Policy for Sustainable Food Agricultural Land Protection in Banten Province

The development strategy within the regulations on Spatial and Regional Planning (Bahasa Indonesia: Rencana Tata Ruang dan Wilayah - RTRW) intersects with social, cultural, and environmental conditions. One impact is the conversion of agricultural land, plantations and forestry into industrial and residential areas. In the context of Banten Province, the regional regulation on Spatial and Regional Planning (Bahasa Indonesia: Peraturan Daerah Rencana Tata Ruang dan Wilayah - Perda RTRW) is considered to have played a major role in the conversion of agricultural land on a large scale in several regions in Banten Province. This study aimed to determine the impact of spatial and regional planning policies in Banten Province on efforts to protect sustainable food agricultural land (Bahasa Indonesia: Lahan Pertanian Pangan Berkelanjutan – LP2B), which is a national priority. The impact evaluation methodology was used to compare the condition of agricultural land before and after the implementation of this regional and spatial planning regulation. This study found a lack of strong commitment to the protection of LP2B, and that the conversion of agricultural land into industrial and residential areas largely occurred in areas that were included in the regional regulation as agricultural designation areas.
 Keywords: policy impact, spatial and regional planning (RTRW), land conversion

Embodied Carbon and Embodied Energy Scenarios in the Built Environment. Computational Design Meets EPDs

This article aims to study the political, environmental and economic factors in contemporary society that influence new approaches and decision making in design in terms of carbon emissions and energy employment. These issues are increasingly influencing political decision making and public policy throughout every aspect of society, including the design practice. Managing this kind of complexity means adopting new forms of collaboration, methodologies and tools, knowledge and technology sharing. The article aims to narrate a PhD research experience grounded in academy–industry collaboration and aimed at creating a digital methodology for impact evaluation and investment planning. In particular, the digital methodology focuses on responding to international public policy for the sustainable growth of cities, in terms of footprint and energy demand, by including a holistic view of the design process made possible by the use of life-cycle assessment (LCA) procedures. To simplify the calculation, the methodology focuses on the Environmental Product Declaration (EPD) data rather than the entire LCA. The EPD is a document that describes the environmental impacts linked to the production of a specific quantity of product or service. The objective was not to create another evaluation method but to employ the EPD results in combination with parametric and computational procedures. The integration of those procedures by using visual programming and scripting allowed the calculation of Embodied Carbon and Embodied Energy and created a user-friendly interface to query the results. The output obtained included automatic and dynamic diagrams able to identify impact scenarios in terms of CO2 emissions and MJ of embodied energy after the conceptual design stage. The strategic use of the charts lies in their potential to simulate impact conditions and, therefore, in the chance to create sustainable transformation scenarios in the early stages of design. At this point, the influence on choices is at its highest, and the costs are low. Moreover, the methodology represents a platform of collaboration that potentially increases the level of interaction between the actors of the construction process with the consequent improvement in design quality. In conclusion, building the design methodology and testing its performance within a specific sociotechnical context was important in critically evaluating certain topics, for example, the recent European strategies on new technology to reach sustainable objectives, the role of digital tools in proposing solutions towards contemporary social issues, the birth of new forms of partnership and collaboration and the new possibilities coming from digital evaluation approaches.

Lean techniques impact evaluation methodology based on a co-simulation framework for manufacturing systems

ABSTRACT Lean implementation plays a major role in optimizing productivity and reducing waste. Applying the adequate integration of Lean Techniques (LT) can ensure a higher profitable benefit. Many companies face difficulties in choosing the LT that best suit their situations to reach their objectives. In this study, we propose the simulation of specific modeled industrial contexts and check the impact of implementing LT simultaneously. Market fluctuation, demand diversification, and uncertainty of resources contexts are studied to perceive how LT behaves accordingly. Four KPIs (Key Performance Indicators) are retained for the analysis: Work in Progress, Lead-time, Production Throughput, and Defect Rate. An aeronautical company is modeled and experiments are performed to demonstrate the usefulness of a developed co-simulation framework to perceive the sensitivity of LT to some industrial contexts. The results showed that Poka Yoke and 5S are context-free LT valid in any industrial context. Pull, SMED, and Cross training are contextual and deserve careful applicability regarding the simulated context. Cross training, suitable for uncertainty of resources, does not show any significant improvements when the company was exposed to market fluctuations and demand diversification contexts.

Nature documentaries as catalysts for change: Mapping out the ‘Blackfish Effect’

Abstract It is essential for us to understand what drives human behaviour if we want to tackle anthropogenic damage to the environment. Popular media can play an important role in shaping public attitudes, behaviours and norms towards wildlife, and documentaries in particular have become an increasingly prominent tool for social change. There is, however, a need for robust impact evaluation both in documentary‐making and in conservation, to refine future interventions. The 2013 documentary Blackfish portrayed human–orca interactions at the US‐based marine park, SeaWorld. Following its release, SeaWorld suffered financial difficulties and the company underwent structural changes, including a cessation of its orca breeding programme. These impacts have often been attributed to the Blackfish documentary, but little evidence has been provided to justify these claims. We combined an analysis of stock market data and semi‐structured interviews with 26 key informants to build an in‐depth contribution analysis. We used General Elimination Methodology, a qualitative impact evaluation methodology to build an understanding of the impact of Blackfish. We found a consensus among stakeholder groups that Blackfish induced negative publicity for SeaWorld and a change in people's perceptions of captivity. As a result, attendance at the park decreased and the market value of the company dropped. Blackfish catalysed a whole movement against marine mammal captivity. There were three key factors that led to its impact: the support from major distribution channels which allowed it to reach major audiences, emotional impact of the content and timing of its release. Blackfish benefitted from a perfect storm, building upon decades of activism to create an appropriate cultural climate for its release in 2013. A free Plain Language Summary can be found within the Supporting Information of this article.

Decarbonization of industry: Guidelines towards a harmonized energy efficiency policy program impact evaluation methodology

The decarbonization of EU energy system is under way, but manufacturing industry is still using approximately 25% of the EU total final energy use. To maintain long-term competitiveness while contributing to the EU goal of carbon neutrality by 2050, manufacturing industry needs to improve energy efficiency in a cost-effective way. One important way to achieve this is through energy audits. The Energy Efficiency Directive promotes member states’ development of energy efficiency programs to encourage industry to undergo energy audits. Previous studies have reviewed industrial energy efficiency policy program evaluations and argued that there is no harmonized way to conduct them. This leads to difficulties in: i) comparing energy efficiency and cost saving potentials throughout different programs, and ii) providing necessary information that supports the improvement of the policy program. Therefore, we argue that a harmonized methodology for industrial energy efficiency policy program evaluation is of great importance, and, we have developed a set of five-steps guidelines that lay the foundation for an ex-ante energy efficiency policy program evaluation methodology. The guidelines are to be be conducted during the lifetime of the program, in five steps, as follows: (s1) define key issues, (s2) set the objectives for each key issue, (s3) identify the options for each key issue, (s4) analyze options from an energy and environmental perspective, and (s5) compare options and select the recommended one. Our proposed methodology will support policymakers and evaluators answer questions such as: i) how can the objectives of the policy program be achieved? ii) is there any need to change the policy program? Furthermore, a comparison in terms of relevance, efficiency, effectiveness, and sustainability of all major policy options developed, including the status quo option is proposed in the methodology. This paper can be seen an important step towards the goal of creating a harmonized policy evaluation methodology.

AN EVALUATION METHODOLOGY FOR MEASURING THE LONG-TERM IMPACT OF FAMILY STRENGTHENING AND ALTERNATIVE CHILD CARE SERVICES: THE CASE OF SOS CHILDREN’S VILLAGES

Until recently, SOS Children’s Villages International, like many organisations in the social sector, lacked a rigorous and systematic approach to gauging the long-term impact of their services. With this in mind, SOS Children’s Villages International developed a social impact evaluation methodology in 2014 to measure the long-term effects of its services on children and their families and communities, as well as the social return on investment. This evaluation methodology has been tested and applied to similar service types across 15 low-, middle-, and high-income countries worldwide. The findings are regularly consolidated, in order to derive trends and learnings for the global organisation and to inform strategy and policy. The present article will discuss the evaluation methodology and the related limitations. Conclusions regarding the validity of the methodology will be offered in terms of the measurement of social service impact and the way forward.

Learning How to Learn in Sustainability Transitions Projects: The Potential Contribution of Developmental Evaluation

Background: Community-based sustainability transitions projects are increasingly being considered for their potential as policy delivery vehicles for the UK government Climate Change Act commitments. At the same time, project funders seek reassurances that their investments are relevant in helping communities mitigate, and adapt to, the effects of climate change. Despite this increased pressure, recent research suggests that evaluations of such sustainability transitions projects have, on the one hand variable impacts, or impacts that are of an inadequate scale, duration, or type, or on the other, that project staff lack the capacity or resources to undertake monitoring and evaluation to the degree of rigour expected by policy makers and funders.
 Purpose: This article reports on an extended case study of a fully-funded five year community-based sustainability transitions project in Leicestershire, England. In particular, it reviews the deployment of developmental evaluation (DE) methods in an attempt to capture the project team’s learning about doing community-based sustainability work.
 Setting: A funded community-based sustainability transitions project in a south Leicestershire market town.
 Intervention: Developmental evaluation methods were used to capture project-based learning as a resource for project innovation and adaptation.
 Research Design: Phronetic case study.
 Data Collection and Analysis: Participant-observation, action research, focus and special issue group facilitation.
 Findings: Use of a developmental evaluation method identified key learning points for the project actors; these focused on how the project had adapted to the complexities of the operating environment through innovations in second-order learning or learning how to learn. The paper makes recommendations for the design and funding arrangements of community-based sustainability transitions initiatives and developmental evaluation is endorsed as a viable and promising adjunct to more traditional impact, economic, and process evaluation methodologies.
 Keywords: developmental evaluation; second-order learning; community-based sustainability; phronesis; action research; project design.

Endline Report for the Self-Supporting Rural Development Project with Saemaul Undong's Participatory Approach in Myanmar

The Self-Supporting Rural Development Project with Saemaul Undong's Participatory Approach (hereafter SMU) in Myanmar, implemented jointly by Korea International Cooperation Agency (KOICA) and the Ministry of Agriculture, Livestock, and Irrigation (MOALI) from 2014 to 2019, is the largest rural development project in Southeast Asia implemented by KOICA. Excluding the budget for the construction of the Saemaul Undong Training Institute in Myanmar, the project is worth more than 10 billion Korean won (equivalent to USD 9 million), which includes the budget for the implementation of rural community development projects in 100 pilot villages across nine regions in Myanmar over three years. As the scale of KOICA's development projects has expanded dramatically, and projects with large budgets of over 10 billion won, including Myanmar's rural community development projects, have increased significantly, there have been increasing discussions on how to evaluate various projects implemented by KOICA. The evaluation method used by KOICA can be described as Process Evaluation using Project Design Matrix(PDM), which summarizes the logical framework based on the Theory of Change. While the PDM evaluated in the order of inputs, activities, outputs, outcomes, and impacts has the advantage of being particularly useful for process evaluation, it assumes a causal relationship between inputs/activities and outcomes/impacts and thus cannot be scientifically proven. In an effort to address these limitations of process evaluation, impact evaluation method focused on identifying causal relationships has received increasing attention. Abhijit Banerjee (University of MIT), Esther Duflo (University of MIT), and Michael Kremer (University of Harvard) have conducted impact evaluations applying a randomized controlled trial (RCT) to various development projects since the mid-1990s. They have also received the 2019 Nobel Prize in Economics for their contributions to poverty alleviation and development policies based on scientific evidence derived from impact evaluation findings. National aid agencies such as USAID and DFID, and inter-governmental organizations such as the World Bank and the Asian Development Bank have been actively conducting impact evaluations of their development projects. In particular, the World Bank has established the Development Impact Evaluation Initiative (DIME) in order to evaluate their major projects. KOICA also carried out an impact evaluation, along with process evaluation, when implementing the Myanmar rural community development project in line with these international evaluation trends. The Myanmar Rural Development Project Impact Evaluation Study is meaningful in that it is the first large-scale impact evaluation research conducted by KOICA.3) The rest of the report is as follows. Chapter 2 summarizes the background of the impact evaluation of the KOICA Myanmar Rural Development Project, Chapter 3 discusses the impact evaluation methodology, and Chapter 4 analyzes the main results of the impact evaluation. Finally, in Chapter 5, we provide recommendations for policymakers who are planning to implement a large-scale impact evaluation study such as the Myanmar rural development project. The summary of the impact evaluation results of the Myanmar rural development project is as follows. There was no statistically significant impact on social capital, which was set as one of the main outcome variables, but the Myanmar rural development project improved the living environment in the village, increased access to microfinance, and increased income. Since a randomized controlled trial, comparing randomly selected treatment villages to control villages, was not possible for this impact evaluation study, two behavioral experiments were implemented. KOICA's rural development project in Southeast Asia reflects the experience of Saemaul Undong in Korea, and includes a differential incentive structure that pays incentives to villages with better performance through competition among project villages. The behavioral experiments attempted to analyze whether the inter-village competition, a key mechanism of the KOICA rural development projects, affected the increase of social capital in the villages. In Myanmar, where the level of social capital was remarkably high from the baseline, we could not observe any effect of the inter-village competition from the behavioral experiments.

Environmental Performance Analysis and Economic Perspectives of Concrete Girders

To reduce the environmental loads of construction projects, a methodology needs to be established that reasonably assesses the environmental impact. Currently, life cycle assessment (LCA) is used as a methodology for evaluation of the environmental impact in various fields. However, this approach is not employed for the assessment of the environmental impact at the planning or design stages of construction projects. The reason is that the features of construction projects cannot entirely provide the full information required for performing LCA until the design has been complete. This study performed LCA of concrete girders and estimated the properties and basic unit quantity of the environmental loads. Our findings support that environmental impact assessment based on LCA should be performed at the planning or design stage when no full information is still available. Moreover, we propose an environmental impact index based on the input cost and the occurrence rate of environmental loads as an environmental performance indicator of concrete girders. This methodology is expected to induce the paradigm to transit from the existing environmental impact assessment methodology based on input resource amounts to the environmental impact assessment methodology in economic perspective considering the characteristics of construction projects.

Assessing the impacts of governance reforms on health services delivery: a quasi-experimental, multi-method, and participatory approach

Despite considerable advances in developing new and more sophisticated impact evaluation methodologies and toolkits, policy research continues to suffer from persistent challenges in achieving the evaluation trifecta: identifying effects, isolating mechanisms, and influencing policy. For example, evaluation studies are routinely hampered by problems of establishing valid counterfactuals due to endogeneity and selection effects with respect to policy reform. Additionally, robust evaluation studies often must contend with heterogeneity in treatment, staggered timing, and variation in uptake. And finally, on practical grounds, researchers frequently struggle to involve policymakers and practitioners throughout the research process in order to engender the type of trust needed for policy influence. While it can be difficult to generalize about appropriate evaluation methodologies across contexts, prominent policy interventions like governance reforms for improving health services delivery nonetheless demand rigorous and comprehensive evaluation strategies that can produce valid results and engage policymakers. Drawing on illustrations from our research on health sector decentralization in Honduras, in this paper we present a quasi-experimental, multi-method, and participatory approach that addresses these persistent challenges to policy evaluation.

Analysis Methodology for Evaluation of Time-Delay Impact on Network-Based System for Droop-Controlled AC Microgrid

Potential performance improvements can be obtained by introducing communication techniques in the power electronic systems. However, network-induced time-delay could bring negative consequences of degrading performance or destabilizing the system in many cases. To investigate and handle the impacts of time-delay, a suit of analytical methodology is proposed, where both delay-insensitive and delay-sensitive control strategies of network-based system have their theoretical methods and different problem-solving paths. The former is to predict the maximum allowable boundaries of time-delay for releasing more network resources, and the latter is to use the controller-altering method for changing its original instability, respectively. The proposed methodology is concretely applied in network-based system of droop-controlled AC (Alternating Current) microgrid in islanded mode. Two different current-sharing strategies are mathematically analyzed and given to verify their validity. Experimental results also show the effectiveness of the proposed methodology in droop-controlled AC microgrid system, which provides theoretical guidance on how to use network-based control for the other power electronic systems.

Assessment of the Acoustic Impact of Wind Farm Projects: Methodology and Case

Wind energy is a clean renewable and its benefits are considered greater than the disadvantages. However, even though wind farms represent environmentally friendly projects, they frequently generate public resistance. Wind turbines are socially controversial because of their visual and acoustic impacts. The aim of this work was to design a methodology for the preliminary evaluation of potential acoustic impact of wind farms. Three steps have been proposed: identification, characterization and assessment of the acoustic impact during the construction, operation and decommissioning phase. Obtained results in the studied area suggested that at a maximum power, at 500 meters of wind turbines, noise level is lower than the reference level established in the legislation. In summary, proposed methodology could help to determine and to evaluate the best location for new wind farm projects. Proposed methodology could help to determine and to evaluate the best location for new wind farm projects. Moreover, this method could be used as a consulting tool for public organizations and private institutions, being quick, concise and clear. Obtained results for the studied wind farm project suggested that almost all the area showed noise levels lower than established background noise level.

Achieving impact: impact evaluations and narrative simplification

ABSTRACT This study is concerned with how impact from research and innovation (R&I) programmes is accounted for in impact evaluation reports. Establishing causal links between a research funding instrument and different effects, poses well known methodological difficulties. In the light of such challenges, textual accounts about causal links ought to be carefully written. Nevertheless, impact evaluation reports have a tendency towards unwarranted simplification as far as impact inferences are concerned. In this study, we illustrate how such simplifications – versions of the narrative device ellipsis – are accomplished. Using examples from three Swedish impact evaluation reports, we focus on the constituent components of longer impact accounts, that of the impact argument, to analyze the various ways that impact is narratively achieved through simplification. We believe this analysis can contribute to the methodology of impact evaluation, as well as spread light on some the difficulties in the historiography of innovation in general.