Evaluate Research Outputs Research Articles

Citation Review and Scientific Visualization of Articles Published in the Iranian Rehabilitation Journal (IRJ) 2003-2023 in the Scopus Database

Objectives: Accurate scientific planning and societal macro policies require reviewing and evaluating research output. Scientometrics offers a valuable approach for assessing the activity of journals that publish a majority of scientific productions. This study aims to analyze the scientific activity of the Iranian Rehabilitation Journal (IRJ) by examining its publication history in the Scopus database. Methods: This bibliometric analysis adopts a scientometrics approach and includes all documents published in IRJ indexed in the Scopus database. Data collection and analysis were performed using VOSviewer and Excel software. Results: IRJ has been active since 2003, initially publishing seven articles in the Scopus database. The journal boasts an H-index 11 and has produced 701 documents over 20 years. The citation score for the journal in 2022 was approximately 0.6. The University of Social Welfare and Rehabilitation Sciences had the highest number of affiliated articles (n=395). The most productive author was “Dadkhah” with 42 documents; the most cited author was “Masah”. Most documents are articles written in English, and the most frequently used keyword is “Rehabilitation”. Discussion: This study offers an overview of the status of IRJ, indicating an increasing trend in the number of documents published and citations received since its establishment. The research findings can serve as a road map for researchers in the field of rehabilitation, providing valuable insights for future studies.

Cumulative group metrics: a new and efficient method to measure the scientific impact of research groups.

The authors collected peer-reviewed output in 2020 from two academic orthopedic surgery departments [University of Michigan (UM) and Mayo Clinic Rochester (MC)] and one medical device research department [Arthrex Inc. (AI)]. The sites evaluated the Cumulative Group Number of Publications (CGNP), Cumulative Journal Impact Factor (CJIF), Cumulative CiteScore (CCS), Cumulative SCImago Journal Rank (CSJR), and Cumulative Source Normalized Impact per Paper (CSNIP) for the three institutions. In 2020, UM published 159 peer-reviewed studies, MC published 347 peer-reviewed studies, and AI supported 141 publications. The UM publications achieved a CJIF of 513, a CCS of 891, a CSJR of 255, and a CSNIP of 247. The MC publications achieved a CJIF of 956, a CCS of 1568, a CSJR of 485, and a CSNIP of 508. AI-supported publications achieved a CJIF of 314, a CCS of 598, a CSJR of 189, and a CSNIP of 189. The presented cumulative group metrics are an effective tool to assess the scientific impact of a research group. The cumulative submetrics can further evaluate research groups compared with other departments due to field normalization. Department leadership and funding agencies can utilize these metrics to evaluate research output quantitatively and qualitatively.

Global research trend on yoga intervention in educational systems

Yoga is being used as a mind-body complementary and alternative medicine intervention to help people with specific physical and mental health issues. In terms of the study area, yoga is an attractive field with various scopes and also contributes to scientific research. In this context, the current study is simply aimed at fully identifying the function of yoga intervention on students in academic institutions, as well as evaluating the available research data in this sector. A search query of relevant keywords was strategized and optimized to extract the information of research articles published from 1992 to 2021 from the “Dimention.ai” database. The obtained dataset was analyzed using PRISMA to evaluate the effect of interventions. The results indicate that the United States has published the maximum number of scholarly publications (553) among all countries. If we talk about organisation Swami Vivekananda Yoga Anusandhana Samsthana from India has published the most of the articles (39) in the field of yoga education. The journal “Mindfulness” (14) and author “Shirley Telles” (14) from the “Patanjali Research Foundation-India” have published the highest scholarly articles. In the bibliometric coupling of distinct clusters, the strongest link strengths among the authors, countries, and organizations were observed for “Khalsa, Satbir S.” (4915), “The United States” (47347), and “Harvard University” (8485), respectively. The current study indicated that yoga education studies have a huge spectrum of publications. Moreover, these bibliometric indicators provide significant endpoints to evaluate research output productivity and quality. The study's findings aid in developing a framework for additional research in the field of yoga and its usage in the academic environment.

The Hirsch Index and Self-Citation in Academic Physiatry Among Graduate Medical Education Program Directors.

The Hirsch Index is a measure of academic productivity which captures both the quantity and quality of an author's output. A well-accepted bibliometric, the Hirsch Index still may be influenced by self-citation, which has been assessed in other medical and surgical specialties. This study aims to evaluate research output and self-citation in physiatry, establishing a benchmark for the field, in addition to identifying differences between physical medicine and rehabilitation subspecialties. This study identified physical medicine and rehabilitation residency and fellowship program directors and analyzed the number of publications, citations, self-citations, and h-indices. A total of 169 program directors were identified, and the mean number ± SD of publications, citations, and Hirsch Index for the cohort were 16.7 ± 29.5, 348 ± 753, and 5.7 ± 6.7, respectively. When self-citation was excluded, less than 2% of program directors (3 of 169) had changes in Hirsch Index greater than one integer, and none greater than two integers. The Hirsch Index remained unchanged for 90% (152 of 169). Spinal cord injury fellowship directors had significantly higher mean number of publications (28, P = 0.04), mean number of citations (672, P = 0.03), and Hirsch Index (9.2, P < 0.01; 95% confidence interval). Overall, self-citation is infrequent in physical medicine and rehabilitation, and spinal cord injury directors had more robust academic profiles.

Four challenges when conducting bibliometric reviews and how to deal with them.

The evidence base in environmental sciences is increasing steadily. Environmental researchers have been challenged to handle massive volumes of data to support more comprehensive studies, assess the current status of science, and move research towards future progress. Bibliometrics can provide important insights into the research directions by providing summarized information for several end users. Here, we present an in-depth discussion on the use of bibliometric indicators to evaluate research outputs through four case studies comprising disciplines in environmental sciences. We discuss four big challenges researchers may face when conducting bibliometric reviews and how to deal with them. We also address some primary questions researchers may answer with bibliometric mapping, drawing lessons from the case studies. Lastly, we clarify some misuses of review concepts and suggest methodological principles of systematic reviews and maps to improve the overall quality of bibliometric studies.

Use of Benford's law on academic publishing networks

Benford's law, also known as the first-digit law, has been widely used to test for anomalies in various data ranging from accounting fraud detection, stock prices, and house prices to electricity bills, population numbers, and death rates. Scientific collaboration graphs have been studied extensively as data availability increased. Most research was oriented towards analysing patterns and typologies of citation graphs and co-authorship graphs. Most countries group publications into categories in an attempt to objectively measure research output. However, the scientific community is complex and heterogeneous. Additionally, scientific fields may have different publishing cultures, which make creating a unified metric for evaluating research output problematic. In complex systems like these, it is important to regularly observe potential anomalies and examine them more carefully in an attempt to either improve the evaluation model or find potential loopholes and misuses. In this paper, we examine the potential application of Benford's law on the official research database of Slovenia. We provide evidence that metrics such as number of papers per researcher conform to Benford's distribution, while the number of authors per paper does not. Additionally, we observe some anomalies and provide potential reasoning behind them.

Global research trends on bioflocculant potentials in wastewater remediation from 1990 to 2019 using a bibliometric approach.

The preference of biofloculants over chemical flocculants in water and wastewater remediation systems has gained wider attention due to their biodegradability, innocuousness, safety to human and environmental friendliness. The present study aimed to evaluate research outputs on bioflocculant potentials in wastewater remediation from 1990 to 2019 using bibliometric analyses. To the best of our knowledge, this is the first bibliometric report in bioflocculant research. The subject bibliometric dataset was extracted from the Web of Science Core Collection (WoSCC) and Scopus using the Boolean, 'bioflocculant* and waste*' and analysed for indicators such as a yearly trend, productivity (authors, articles, country, institution and journal source), conceptual framework and collaboration network. We found 119 documents with 347 authors from 78 journal sources on the subject, an annual growth rate of 12·1%, and average citations/document of 15·08. Guo J. and Wang Y. were the top researchers with 15 and 12 outputs respectively. China (42%) and South Africa (9·24%) ranked the top two dominant countries in the field. The top journals were Bioresource Technology (9 papers, 506 citations), Applied Microbiology and Biotechnology (5 papers, 268 citations), whereas, the top institution was Chengdu University of Information and Technology (n=9 documents) followed by Sichuan Univ. Sci. & Engn, China (n=8 documents). This study found that lack of intercountry collaboration and research funding adversely affects research participants in the field.

Why a National University Hospital should Prioritize and Fund Research

Why a National University Hospital should Prioritize and Fund Research

Using a rapid environmental scan methodology to map country-level global health research expertise in Canada

BackgroundMany countries are currently rethinking their global health research funding priorities. When resources are limited, it is important to understand and use information about existing research strengths to inform research strategies and investments and to drive impact. This study describes a method to rapidly assess a country’s global health research expertise and applies this method in the Canadian context.MethodsWe developed a three-pronged rapid environmental scan to evaluate Canadian global health research expertise that focused on research funding inputs, research activities and research outputs. We assessed research funding inputs from Canada’s national health research funding agency and identified the 30 Canadian universities that received the most global health research funding. We systematically searched university websites and secondary databases to identify research activities, including research centres, research chairs and research training programmes. To evaluate research outputs, we searched PubMed to identify global health research publications by Canadian university-affiliated researchers. We used these three perspectives to develop a more nuanced understanding of Canadian strengths in global health research from different perspectives.ResultsCanada’s main global health research funder, the Canadian Institutes of Health Research, invested a total of $314 M from 2000 to 2016 on global health research grants. This investment has contributed to Canada’s wealth of global health research expertise, including 12 training programmes, 27 Canada Research Chairs, 6 research centres and 30 WHO Collaborating Centres across 27 universities. Research activities were concentrated in Canada’s biggest cities and most commonly focused on health equity and globalisation issues. Canadian-affiliated researchers have contributed to a research output of 822 unique publications on PubMed. There is an opportunity to build global health expertise in regions not already concentrated with research activity, focusing on transnational risks and neglected conditions research.ConclusionsOur three-pronged approach allowed us to rapidly identify clear geographic and substantive areas of strength in Canadian global health research, including urban regions and research focused on health equity and globalisation topics. This information can be used to support research policy directives, including to inform a Canadian global health research strategy, and to allow relevant academic institutions and funding organisations to make more strategic decisions regarding their future investments.

The Accuracy of Newspaper Citation Count Reported and Actual Citation Found in Web of Science Citation Database

Bibliometric indicators are widely used in evaluating research output. Citation information about a publication can be obtained from various databases like Web of Science (WoS), SCOPUS and Google Scholar. Bibliometricans solely depend on the above databases to get citation data. WoS is considered a credible source for citation data. In this paper, we have tried to study the accuracy of the citation count reported and actually found with special reference to three newspapers namely Wall Street Journal, Washington Post, and Guardian. It was found that only 69% of citation data was accurate for above three newspapers.

Social media presence of scholarly journals

Recently, social media has become a potentially new way for scholarly journals to disseminate and evaluate research outputs. Scholarly journals have started promoting their research articles to a wide range of audiences via social media platforms. This article aims to investigate the social media presence of scholarly journals across disciplines. We extracted journals from Web of Science and searched for the social media presence of these journals on Facebook and Twitter. Relevant metrics and content relating to the journals' social media accounts were also crawled for data analysis. From our results, the social media presence of scholarly journals lies between 7.1% and 14.2% across disciplines; and it has shown a steady increase in the last decade. The popularity of scholarly journals on social media is distinct across disciplines. Further, we investigated whether social media metrics of journals can predict the Journal Impact Factor (JIF). We found that the number of followers and disciplines have significant effects on the JIF. In addition, a word co‐occurrence network analysis was also conducted to identify popular topics discussed by scholarly journals on social media platforms. Finally, we highlight challenges and issues faced in this study and discuss future research directions.

Web of Science: Showing a Bug Today That Can Mislead Scientific Research Output Prediction

As in all domains of human activity, economic issues and the increase of people working in scientific research have altered the way scientific production is evaluated as well as the objectives for performing the evaluation. The h index was introduced in 2005 by J. E. Hirsch as an indicator for the measurement of individual scientific output not only in terms of quantity but also in terms of quality, and its use has since spread throughout the world. In 2007, Hirsch proposed its adoption as the best way to predict future scientific achievement and, consequently, as a useful guide for investment in research and for institutions when hiring members for their scientific staff. Since then, several authors have also been using the Thomson ISI Web of Science database to develop their proposals for evaluating research output. Here, using a software we have developed, we analyze more than 100,000 articles and show that a subtle flaw in Web of Science can inflate the results of information collected, therefore compromising the exactness and, consequently, the effectiveness of Hirsch’s proposal and its variations.

Social impact assessment of scientist from mainstream news and weblogs

Research policy makers, funding agencies, universities and government organizations evaluate research output or impact based on the traditional citation count, peer review, h-index and journal impact factors. These impact measures also known as bibliometric indicators are limited to the academic community and cannot provide the broad perspective of research impact in public, government or business. The understanding that scholarly impact outside scientific and academic sphere has given rise to an area of scientometrics called alternative metrics or “altmetrics.” Moreover, researchers in this area incline to center around gauging scientific activity via social media, namely Twitter. However, these count-based measurements of impact are sensitive to gaming as they lack concrete references to the primary source. In this work, we expand a conventional citation graph to a heterogeneous graph of publications, scientists, venues, organizations based on more reliable social media sources such as mainstream news and weblogs. Our method is composed of two components: the first one is combining the bibliometric data with social media data like blogs and mainstream news. The second component investigates how standard graph-based metrics can be applied to a heterogeneous graph to predict the academic impact. Our result showed moderate correlations and positive associations between the computed graph-based metrics with academic impact and also reasonably predict the academic impact of researchers.

Using Google Scholar institutional level data to evaluate the quality of university research

In recent years, the extent of formal research evaluation, at all levels from the individual to the multiversity has increased dramatically. At the institutional level, there are world university rankings based on an ad hoc combination of different indicators. There are also national exercises, such as those in the UK and Australia that evaluate research outputs and environment through peer review panels. These are extremely costly and time consuming. This paper evaluates the possibility of using Google Scholar (GS) institutional level data to evaluate university research in a relatively automatic way. Several citation-based metrics are collected from GS for all 130 UK universities. These are used to evaluate performance and produce university rankings which are then compared with various rankings based on the 2014 UK Research Excellence Framework (REF). The rankings are shown to be credible and to avoid some of the obvious problems of the REF ranking, as well as being highly efficient and cost effective. We also investigate the possibility of normalizing the results for the university subject mix since science subjects generally produce significantly more citations than social science or humanities.

Evidence for Impact and Impact of Evidence

Researchers working in university settings have become used to constantly shifting goalposts when it comes to metrics used to evaluate research outputs. The productivity of individual researchers u...

Inefficient Allocation of Resources in Science

Researchers’ incentives to publish in top-tier journals are examined by a theoretical model. Using incentives to deliver optimal quality of research as a benchmark, the model concludes that the former incentives result in misallocation of resources and welfare loss. The main reasons behind it are lack of unpublishable but otherwise valuable research, misalignment between effort and value of research, and suboptimal choices of research topics. A bibliographic data set is used to empirically test predictions of the model. Indeed, while choosing a field of study, researchers tend to be more focused on their ability to publish rather than on their ability to deliver valuable research. Alternative methods of evaluating research output are suggested to mitigate these problems.

Scientists’ framing of the ocean science–policy interface

Scientists’ ideas, beliefs, and discourses form the frames that shape their choices about which research to pursue, their approaches to collaboration and communicating results, and how they evaluate research outputs and outcomes. To achieve ocean sustainability, there are increasing calls for new levels of engagement and collaboration between scientists and policy-makers; scientists’ willingness to engage depends on their current and evolving frames. Here, I present results about how scientists involved in diverse fields of ocean research perceived their role as scientists working at or near the ocean science–policy interface and how this related to their perceptions regarding ocean research priorities. The survey of 2187 physical, ecological and social scientists from 94 countries showed that scientists held different perspectives about their appropriate level of engagement at the ocean science–policy interface and the relative primacy of science versus politics in formulating ocean policy. Six clusters of scientists varied in their frames; three clusters accounted for 94% of the sample. Of 67 research questions identified from 22 research prioritization and horizon scanning exercises, the top eight were shared among all three clusters, showing consistency in research priorities across scientists with different framings of their role at the science–policy interface. Five focused on the mechanisms and effects of global change on oceans, two focused on data collection and management for long-term ocean monitoring, and one focused on the links between biodiversity and ecological function at different scales. The results from this survey demonstrated that scientists’ framings of the role of ocean science at the science–policy interface can be quantified in surveys, that framing varies among scientists, and that research priorities vary according to the framings.

Beyond the impact factor?

Over the past few years, increasing efforts have been made to evaluate research output using different markers of quality.1 This is most clearly reflected by the work of the Research...

A new ranking scheme for the institutional scientific performance

We propose a new performance indicator to evaluate the productivity of research institutions by their disseminated scientific papers. The new quality measure includes two principle components: The normalized impact factor (IF) of the journal in which paper was published, and the number of citations received per year since it was published. In both components, the scientific impacts are weighted by the contribution of authors from the evaluated institution. As a whole, our new metric, namely, the Institutional Performance Score (IPS) takes into account both journal based impact and articles specific impacts. We apply this new scheme to evaluate research output performance of Turkish institutions specialized in astronomy and astrophysics in the period of 1998–2012. We discuss the implications of the new metric, and emphasize the benefits of it along with a comparison to other proposed institutional performance indicators. This study contains a new bibliometric indicator were developed to measure the scientific research performance of the institutions.

The significance of citation impact indicators of research performance in the developing countries of sub-Saharan Africa

This paper argues that Sub-Saharan Africa needs to produce more journals indexed by ISI Web of Science (WoS). Researchers from the region should also publish in other ISI indexed, reputable and high impact journals such as Nature and Science. Inevitably, this will make Sub-Saharan African researchers visible and globally competitive. The Sub- Saharan African region has only about 40 journals out of over 12 000 journals that have been indexed by the ISI Web of Science (WoS). Arguably, ranking of universities across the globe and qualification for Nobel Prizes are determined by metrics-based evaluation of research performance. Sub-Saharan Africa is poorly represented on the world university rankings. The region has also produced only six Nobel Prize award winners from 1901 to 2010. In the same period, USA, UK and Germany produced 326, 116 and 102 recipients respectively. While there are some limitations on the use of citation indicators to evaluate research output, this researcher argues that citation impact indicators of research performance provide policymakers, researchers and funding agencies with an objective measure for assessing research performance and therefore are of great significance in the developing countries of Sub-Saharan Africa.