National Evaluation System Research Articles

Evaluation systems and the pace of change – The example of Swedish higher education1

The aim of this article is to illuminate and discuss evaluation and evaluation systems in relation to the pace of change. It is argued that evaluation promotes and accelerates change. The article will thus contribute to a critical scrutiny of evaluation as a societal phenomenon and as a widespread practice in education. To accomplish this aim, the inherent purpose of evaluation and evaluation systems is brought forward. National evaluation systems for Swedish higher education are used as an empirical example. An analysis using Rosa’s three aspects of social acceleration (technical acceleration, acceleration of social change, and acceleration of the pace of life) is offered to demonstrate how the evaluation systems are related to, sustain, and promote an increase of the pace of change (acceleration) in educational practice in higher education.

Publication counting methods for a national research evaluation exercise

Abstract In this paper, we investigate the effects of using four methods of publication counting (complete, whole, fractional, square root fractional) and limiting the number of publications (at researcher and institution levels) on the results of a national research evaluation exercise across fields using Polish data. We use bibliographic information on 0.58 million publications from the 2013–2016 period. Our analysis reveals that the largest effects are in those fields within which a variety publication and cooperation patterns can be observed (e.g. in Physical sciences or History and archeology). We argue that selecting the publication counting method for national evaluation purposes needs to take into account the current situation in the given country in terms of the excellence of research outcomes, level of internal, external and international collaboration, and publication patterns in the various fields of sciences. Our findings show that the social sciences and humanities are not significantly influenced by the different publication counting methods and limiting the number of publications included in the evaluation, as publication patterns in these fields are quite different from those observed in the so-called hard sciences. When discussing the goals of any national research evaluation system, we should be aware that the ways of achieving these goals are closely related to the publication counting method, which can serve as incentives for certain publication practices.

Percutaneous Closure of the Patent Ductus Arteriosus in Very Low Weight Infants: Considerations Following US Food and Drug Administration Approval of a Novel Device

Percutaneous Closure of the Patent Ductus Arteriosus in Very Low Weight Infants: Considerations Following US Food and Drug Administration Approval of a Novel Device

Establishing national monitoring and evaluation systems for foreign economic policy implementation

Introduction. The increasing demand in government regulation transparency creates a special focus on introduction of the monitoring and evaluation (M&E) system. The article analyzes the current practices of the national governments in development and implementation of the M&E systems for assisting international development. With the help of the M&E component, the governments, on the one hand, improve accountability to taxpayers, and, on the other hand, M&E increases program's overall efficiency. Methods of research . The game theory is used as a key method of the study. It allows identifying the logic of choosing a specific M&E system in different countries, as well as analyzing the key factors affecting its implementation. Research results . The main principles for the formation of national M&E systems were identified as a result of the analysis, and three levels (general, strategic and coalition) of actions were identified that determine the decision-making process on assessment systems. Conclusion. In conclusion several policy recommendations are proposed on how to use the M&E system to increase the effectiveness of state foreign economic programs.

The development of reproductive management practices in New Zealand: what will the future hold in a consumer-focused, environmentally-conscious, export-driven marketplace?

The New Zealand (NZ) economy and its dairy industry are sensitive to global consumer perception of farming practices used to generate milk products because milk exports account for >25% of national export earnings and >90% of milk produced is exported as products. Astute management of product image and market risk is, therefore, important for the viability of the industry and country. More than 95% of milk produced in NZ comes from strictly seasonal, pasture-based systems, with associated constraints on reproductive performance. Increasing herd sizes, operational changes and genetic selection priorities have further challenged dairy farmers to achieve optimal levels of herd fertility. Reproductive management practices have developed to address the need to maintain a 365-day inter-calving interval, essentially through maximizing the number of cyclic cows during the breeding period and minimizing the duration of the seasonal calving period. Aspects of the hormonal interventions developed and routinely used to achieve these objectives have been the subject of product quality and market risk concerns forcing the industry to explore alternative ways of achieving reproductive performance goals. One approach has been to exploit the inherently high level of fertility in NZ dairy herds. This approach has seen the inclusion of fertility-related traits in the national genetic evaluation system to prevent further decline in genetic fertility. More recently, a nationally coordinated extension program has been adopted to support farmers and their advisors to identify, prioritize and improve on key management areas for incremental gains in herd reproductive performance. Advances in automation and bio-sensing are yet to make a significant impact, but remain potentially valuable additions in supporting the dairy farmer to manage the areas having the largest effects on reproductive performance.

Comparison of the Assessment Systems of Public Libraries in the United States and China

ABSTRACTThis work aims to compare the evaluation and rating systems of public libraries from the United States and China with a view to understand the similarities and differences and to learn from both systems. First, the main evaluation and rating systems of public libraries from the US are discussed, including Hennen’s American Public Library Ratings and Library Journal Index. Second, we discuss the Sixth National Evaluation and Rating System (SNERS) of public libraries above the county level in China. Finally, we compare and analyse the similarities and differences between the evaluation systems from the US and China. Between the two evaluation systems, there is one similarity (i.e. people-orientedness) and many differences in terms of goals, properties, subjects, processes, object classification, cycle, data sources, indicators, weights, scoring and result treatment. The public libraries of the US (a developed country) pay more attention to library services, whereas the public libraries of China (a developing country) pay more attention to a library’s basic construction and infrastructure. By comparing and analysing the evaluation and rating systems of public libraries from the US and China, this paper helps gain some insights into these different systems. We believe this will be helpful for librarians and library evaluators interested in assessing the service quality of public libraries in any country.

시각장애학교 교원능력개발평가에 대한 타당도와 교사 만족도에 대한 관계 연구

시각장애학교 교원능력개발평가에 대한 타당도와 교사 만족도에 대한 관계 연구

Examining how country-level science policy shapes publication patterns: the case of Poland

This country case study describes how science policy instruments are designed to shape publication patterns and identifies the changes in researchers’ productivity that can be observed over the period 2009–2016 in Poland by analysing data on 452,277 publications submitted to the country’s national research evaluation system. Our analysis reveals that policy instruments used in the country’s national research evaluation system, academic promotion procedures and competitive grants have increased the number of articles with an impact factor without compromising publication quality, as measured by a bibliometric indicator. Our findings highlight that only clear and stable incentives have influenced researchers’ publications. Therefore, patterns in scholarly book publications—for which regulations were not clear and stable—have not been significantly shaped by science policy.

OntoSIDES: Ontology-based student progress monitoring on the national evaluation system of French Medical Schools.

We introduce OntoSIDES, the core of an ontology-based learning management system in Medicine, in which the educational content, the traces of students' activities and the correction of exams are linked and related to items of an official reference program in a unified RDF data model. OntoSIDES is an RDF knowledge base comprised of a lightweight domain ontology that serves as a pivot high-level vocabulary of the query interface with users, and of a dataset made of factual statements relating individual entities to classes and properties of the ontology. Thanks to an automatic mapping-based data materialization and rule-based data saturation, OntoSIDES contains around 8 millions triples to date, and provides an integrated access to useful information for student progress monitoring, using a powerful query language (namely SPARQL) allowing users to express their specific needs of data exploration and analysis. Since we do not expect end-users to master the raw syntax of SPARQL and to express directly complex queries in SPARQL, we have designed a set of parametrized queries that users can instantiate through a user-friendly interface.

Advances in the Use of Real-World Evidence for Medical Devices: An Update From the National Evaluation System for Health Technology.

Advances in the Use of Real-World Evidence for Medical Devices: An Update From the National Evaluation System for Health Technology.

HIV Partner Service Delivery Among Blacks or African Americans - United States, 2016.

Identifying persons with human immunodeficiency virus (HIV) infection who are unaware of their infection status, linking them to HIV care, and reducing racial/ethnic disparities are important national HIV prevention goals (1). Blacks/African Americans (blacks)* are disproportionately affected by HIV infection in the United States. Although blacks represent 13% of the U.S. population (2), in 2017, 44% of diagnoses of HIV infection were in blacks, and the rate of new diagnoses in blacks (41.1 per 100,000 persons) was approximately eight times that of non-Hispanic whites (5.1) (3). HIV partner services are offered by health officials to persons with diagnosed HIV infection (index patients) and their sex- or needle-sharing partners, who are notified of their potential HIV exposure and offered HIV testing and related services (4). CDC analyzed 2016 data from the National HIV Prevention Program Monitoring and Evaluation system submitted by 59 health departments.† Among 49,266 index patients identified as potential candidates for partner services, 21,191 (43%) were black. The percentage of black index patients interviewed for partner services (76%) was higher than that for all index patients combined (73%). Among the 11,088 black partners named by index patients, 78% were notified of their potential HIV exposure. Fewer than half (47%) of those notified were tested for HIV infection. Among those tested, one in six (17%) received a new HIV diagnosis. The prevalence of newly diagnosed HIV infection was particularly high among black partners who were gay, bisexual, and other men who have sex with men (MSM) (37%) and transgender persons (38%). Effective implementation of partner services is important to identify HIV infection, link patients to care or reengage them in care, and provide prevention services to reduce HIV transmission.

An Effective National Evaluation System of Schools for Sustainable Development: A Comparative European Analysis

An international context of increasing attention to the quality of educational systems in school establishments is becoming more important to develop a governance that is compatible with the principles of sound and prudent management, inspired by effectiveness, efficiency, and sustainable development, in the long run. The article initially investigates the evolution of national evaluation system of schools as an important tool for management and for school quality improvement. The analysis is carried out on the 43 education systems belonging to the Eurydice Network in the period July–September 2018. Mainly, we have noticed a wide diffusion of evaluation and self-evaluation mechanisms and therefore the development of a school management culture to improve the quality of the system and students’ performance. Subsequently, we aim to verify whether it is possible to identify a linear relationship between the assessment indicator expressed by the international statistical surveys on student performance and the overall quality of the school system. In this regard, the research data on the evolution of the national evaluation system and the statistical data of the OECD-PISA surveys are cross-referenced. Briefly, results on student performance show that the assessment of the quality improvement of the school system cannot be based exclusively on pupils’ performance. However, there is a generalized evolution in the adoption of national evaluation systems, confirming that the managerial culture in European schools is growing. As a further future study, the research will observe the above reflections taking into account the evolution of student performance results for the three-year period of 2015–2018 (OECD-PISA 2018 will be published in December 2019), considering a substantial evolution of national evaluation systems in schools during this three-years period.

国家公园管理评估体系构建与应用

国家公园管理评估体系构建与应用

Healthcare Access Among Cambodia’s Poor: An Econometric Examination of Rural Care-seeking and Out-of-Pocket Expenditure

To inform efforts to improve Cambodia’s social health protection system and advance universal health coverage, health care-seeking and out-of-pocket expenditure (OOPE) were assessed using the 2016 Cambodia Socioeconomic Survey data. This study focuses on the poorest wealth quintile who reside in rural areas- the primary target population of Cambodia’s largest social health protection scheme, the Health Equity Fund (HEF). The study also estimates the proportion of poor with an Equity card which provides access to HEF benefits at public facilities. Overall, 76% of people who sought healthcare in the past 30 days went to private providers, paying, on average, US$39.43 for treatment. About 18% of patients first sought care from public facilities, paying, on average, US$38.15. Though HEF aims to provide free healthcare for the rural poor, this analysis found that 67.2% of such patients seeking first care at public health facilities pay, on average, US$11.61 after controlling for confounding factors. However, treatment expenditure among the rural poor is about 52% less compared to third wealth quintile patients (p<0.01). About 36% of people under the national poverty line do not hold an Equity card to access HEF benefits. Thus, we conclude that HEF is not yet fully reaching its intended impact of removing OOPE as a barrier to access among the poor. Finally, free access to healthcare should incentivize utilization of public services; however, this study was unable to isolate such an effect among patients from the poorest wealth quintile. Access to healthcare can be strengthened with policy directives focused on further reducing OOPE and addressing other challenges to improve patient demand for public services such as quality of care. Enrollment exclusion errors should be corrected by relaxing the eligibility criteria with population coverage expansion. In addition, health service access should be systematically monitored by integrating service utilization, OOPE, and quality indicators into national monitoring and evaluation systems.

THE CONCEPT OF THE IMPLEMENTATION OF THE PERFORMANCE-BASED RESEARCH FUNDING TOOL

Topicality. The implementation of the Performance-based research funding (PBRF) in EU Member States has been a priority for the development of research and innovation in last time. This financial mechanism refers to the type of competitive organizational and institutional allocation of research funding.Aim and tasks. The aim of the article is to develop the theoretical basis of Performance-Based Research Funding (PBRF) assessment to increase the effectiveness of scientific research. Also the aim of the article to analyze the implementation of this mechanism in in Ukraine.Research results. Given the formation of Academic Freedom in the article, it will be clarified: the structure of the evaluation of the financing of research based on the results, the constellations of scientific activities, the category of research activities, and the hypotheses concerning the effectiveness of the national research evaluation systems (NRES). The taxonomy of the indicators of the efficiency of subsidiary activity in Ukraine, New Zealand, Sweden is considered.The following factors explain efficiency differences between science systems: The level of competition; Share of project funding; Performance based funding systems; National evaluation systems.The following activities are excluded from the definition of research except where they are used primarily for the support, or as part, of research and experimental development activities: preparation for teaching; the provision of advice or opinion, except where it is consistent with the PBRF’s Definition of research; scientific and technical information services; general purpose or routine data collection; standardisation and routine testing (but not including standards development); feasibility studies (except into research and experimental development projects); specialised routine medical care; the commercial, legal and administrative aspects of patenting, copyrighting or licensing activities.Conclusions. Theoretical basis of Performance-Based Research Funding (PBRF) assessment to increase the effectiveness of scientific research in Ukraine is developed. The Performance-based research funding is governed by the following set of principles: comprehensiveness; respect for academic traditions; consistency; continuity; differentiation; credibility; efficiency; transparency; complementarity. The Methodology for evaluating the effectiveness of scientific institutions of the National Academy of Sciences of Ukraine, in our opinion, can be supplemented by 3 elements: Moderation Panel. Comprehensive peer-review panel and PBRF audit.

The genre regime of research evaluation: contradictory systems of value around academics’ writing

This article addresses how academics navigate different kinds of prestige and different systems of value around what ‘counts’ in academic writing, focusing particularly on the impact of the genre regime associated with research evaluation in the UK Research Excellence Framework (REF). It draws on data from an Economic and Social Research Council (ESRC)-funded project working with academics across different disciplines and different institutions in England. We interviewed people about their writing practices several times, exploring their practices, life histories, institutional contexts, and the tools and resources they draw on as they write. Academics’ research writing is framed within explicit institutional and departmental strategies around the numbers and publication venues of research outputs, driven by institutions’ need to succeed in the national competitive research evaluation system. Such institutional strategies do not always map well onto other values systems in which academics have been trained and within which they locate themselves. The articles analyses the interviews we carried out, exploring how academics negotiate tensions between these systems of value and considering the implications of this for what is considered to be important in academic work and, therefore, what it means to be an academic.

Evaluating Teacher Evaluation – Evidence From Chile

This study applies a quasi-experimental research design to evaluate the effects of Chile’s national teacher evaluation system on teacher productivity (as measured by student learning), on teaching behaviors, and on teacher beliefs. The paper’s identification strategy exploits a recent change in the evaluation assignment mechanism, together with a discontinuity in the evaluation system’s scoring mechanism. Jointly, these two factors provide plausibly exogenous variation in which teachers get newly evaluated, after two years. My econometric analysis is motivated by, and overcomes, two limitations of a simple regression-discontinuity approach: systematic sorting of teachers close to the cut-off score, as well as additional programs that also use the same cut-off to determine teacher eligibility. The study’s main results suggest that student learning, teacher beliefs, and teaching behaviors remain unaffected by a teacher’s reevaluation, both in the year of the evaluation and in the year thereafter. The article confirms that these findings are not driven by a teacher’s level of work experience, by student sorting, or by systematic attrition.

Evaluating the quality of teaching: can there be valid differentiation in the middle of the performance distribution?

ABSTRACTAn important challenge in evaluating the quality of teaching is to arrive at valid distinctions between teachers, particularly in the middle of the performance distribution. This study addressed this issue for the Chilean national teacher evaluation system. Mathematics teachers (N = 51) from the 2 middle adjacent performance categories, and their students, were examined by videotaping classroom practice and coding the videos with the Classroom Assessment Scoring System for Secondary School, measuring students’ perceptions of their teachers’ practices and assessing student learning on standardized grade-level and unit-specific tests. Findings show that on almost all criterion variables, competent teachers outperformed basic teachers, but differences were usually small. Teachers evaluated as competent obtained consistently better results regarding their teaching practices, as assessed by trained observers, as well as based on reports by their own students. However, there was only weak indication of differences between the 2 groups in terms of student learning.

A study of city-level building energy efficiency benchmarking system for China

The city-level public building energy efficiency benchmarking system has an important guiding value for building energy-saving work, especially for Beijing, with its high energy-consumption density and both long heating and air-conditioning periods. This paper analyzed the characteristics of large-scale office buildings in Beijing and gives a detailed description of the process of establishing a public building energy efficiency benchmarking system for large-scale office buildings in Beijing, China, including data investigation and verification, Energy Usage Intensity(EUI) calculation, determining independent and dependent variables, developing and testing the benchmarking model, and deriving the benchmarking rating table. Then the first city-level public office building energy efficiency benchmarking system in China is established based on 88 large-scale office buildings in Beijing. The results showed that the model's performance index R2 is 0.667 and gives a good representation. The process has a strong potential for replication for other cities in China and even for a national energy-consumption evaluation system.

O estado e a avaliação: o nascimento da política publica em busca da equidade educacional

Nesta discussão o espaço das políticas públicas de planejamento destaca instrumentos regulatórios como foco das políticas estatais; sob a influência, primeiramente do liberalismo e, posteriormente, do neoliberalismo. Estados passaram a reformular seu sistema de plano econômico prevendo a alocação de recursos. Neste contexto, as avaliações institucionais são assumidas como estratégia de administrar o bem público, controlando resultados e transmitindo a imagem do Estado que se preocupa e respeita o cidadão. No Brasil, organiza-se o Sistema Nacional de Avaliação da Educação Básica (Saeb), a Avaliação Nacional da Educação Básica (Aneb). Influenciado por este modelo, a Educação compôs seu quadro, o que suscita apontamentos de sequencia dentro de um modelo que segrega a escola.