Interdisciplinary Research Evaluation Research Articles

Interdisciplinary knowledge combinations and emerging technological topics: Implications for reducing uncertainties in research evaluation

Abstract This article puts forth a new indicator of emerging technological topics as a tool for addressing challenges inherent in the evaluation of interdisciplinary research. We present this indicator and test its relationship with interdisciplinary and atypical research combinations. We perform this test by using metadata of scientific publications in three domains with different interdisciplinarity challenges: Nano-Enabled Drug Delivery, Synthetic Biology, and Autonomous Vehicles. Our analysis supports the connection between technological emergence and interdisciplinarity and atypicality in knowledge combinations. We further find that the contributions of interdisciplinary and atypical knowledge combinations to addressing emerging technological topics increase or stay constant over time. Implications for policymakers and contributions to the literature on interdisciplinarity and evaluation are provided.

Does faculty disciplinary background play a role in the publication pattern of an interdisciplinary research area? The case of science education in Brazil

Research in the Social Sciences and Humanities is published in a variety of formats and types of documents. Local journals are relevant for these fields, particularly in non-English-speaking countries, but coverage of large bibliographic databases is still limited, leading to a large percentage of non-source items. This is an acknowledged challenge for Performance-based Research Funding Systems and also for national evaluation exercises, such as the evaluation of the graduate degree programs performed by CAPES in Brazil. Our assumption is that the evaluation of interdisciplinary research may face similar challenge, particularly when faculty´s disciplinary background is predominantly from the Social Sciences and Humanities. The purpose of this article is to analyse an interdisciplinary research area in Brazil to calculate the percentage of non-source items in its scholarly output, in comparison to the 3 areas in which the majority of its faculty graduated. We extracted 11,000 publications reported by 140 degree programs from the Teaching area -also known as Science Education, and found 77.5% of non-source titles. We also analysed another 50,000 publications from the Education, Astronomy/Physics and Chemistry areas. Comparison showed that the Teaching Area had a similar publication pattern to the Education Area, and typical of the Humanities, which is the predominant Major Area of its faculty graduation. Our conclusion is that faculty disciplinary background may influence publication patterns and stakeholders involved in research evaluation should consider using alternative sources of data when assessing the scholarly output of interdisciplinary research, besides the main international bibliographic databases.

La construction des objets dans la recherche interdisciplinaire finalisée: de nouvelles exigences pour l'évaluation

Evaluation of interdisciplinary research is problematic owing in particular to its being generally viewed as a variant of usual research, which is simply ‘enriched’ through associating several disciplines. Based on work carried out in the Département Systèmes Agraires et Développement (Sad) of Inra, the authors suggest that evaluation procedures accord the same importance to the conditions in which the research results are being produced as they do to the results themselves, given that the research involves opting for a particular approach justified by particular conditions of carrying out research on concrete ‘objects’ and, in most cases, in partnership with other actors. This problemoriented research, which generally involves intervention research procedures, requires that a reflexivity process evolves between the people who formulate the problem, those it aims to produce results which will help the non-scientific partners participating in the research to renew their framework of action in order to deal with new challenges. Emphasis is laid on highlighting the process of methodological construction of research settings and objects, using examples of work on the grazing management of herds on rangelands, the management of cattle batches and the creation of a new type of gastronomic sausage in Corsica. To simultaneously evaluate the scientific rigour of the research and its relevance requires analysing and judging jointly the research outputs and the processes by which these outputs have been achieved. The former can be evaluated on the one hand by checking them against the scientific disciplines and testing their contents and limits, and on the other hand by testing the efficiency of the research outputs against the categories and rules for action. The latter are evaluated with reference to a stated logic of research object construction and treatment. In this perspective the authors propose a framework to analyse the elaboration process of the partnership research setting and to translate the problems into research questions and objects. They demonstrate the need to develop relevant evaluation procedures for this type of research, which is increasingly challenged by the questions posed to research bodies by the social actors. Evaluation exclusively based on academic criteria can no longer satisfy the research institutions.

Pour une approche sociotechnique de la « gestion de la qualité de l'eaupar l'agriculture

Agricultural research is under increasing pressure to deal with complex issues involving varying degrees of uncertainty and urgency. Such is in particular the case for environmental issues, product quality and conservation of natural resources. To address such issues, research needs to adopt a new approach to scientific investigation and get more extensively involved in real-life on-site situations. These issues consist of a mix of technical, social and organisational dimensions which need to be articulated. This can be achieved within the framework of a ‘socio-technical’ problematic. How do researchers stand in relation to this problematic? How is research organised to address the problematic? To what extent do researchers need to engage in the change processes which they analyse and what are the consequences of such an engagement? The authors have attempted to draw some guidelines for the management and evaluation of interdisciplinary research on environmental issues in agriculture from the analysis of four research projects on water quality and farming practices. They propose an analytical framework based on a model designed to investigate a socio-technical innovation process. The framework takes into consideration the context in which the environmental question has emerged, the reformulating of the question into a research problematic, the type of engagement of the actors and the way the research is operated (team composition, relations with the actors). Three types of results are also considered : knowledge produced, change in farming practices and effects on the natural and social systems. Developing a socio-technical problematic has shown itself to be a necessity in defining new farming practices and proposing individual as well as collective conditions for their implementation. Choosing this type of problematic entails several provisions, in particular the need to articulate laboratory and field investigations, to negotiate the research design with the actors, and to steer the research process by constantly taking into account achievements, failures and opportunities.