Measure Of Scientific Quality Research Articles

National Institutes of Health funding among vascular surgeons is rare

BackgroundThe National Institutes of Health (NIH) is an essential source of funding for vascular surgeons conducting research. NIH funding is frequently used to benchmark institutional and individual research productivity, help determine eligibility for academic promotion, and as a measure of scientific quality. We sought to appraise the current scope of NIH funding to vascular surgeons by appraising the characteristics of NIH-funded investigators and projects. In addition, we also sought to determine whether funded grants addressed recent Society for Vascular Surgery (SVS) research priorities. MethodsIn April 2022, we queried the NIH Research Portfolio Online Reporting Tools Expenditures and Results (RePORTER) database for active projects. We only included projects that had a vascular surgeon as a principal investigator. Grant characteristics were extracted from the NIH Research Portfolio Online Reporting Tools Expenditures and Results database. Principal investigator demographics and academic background information were identified by searching institution profiles. ResultsThere were 55 active NIH awards given to 41 vascular surgeons. Only 1% (41/4037) of all vascular surgeons in the United States receive NIH funding. Funded vascular surgeons are an average of 16.3 years out of training; 37% (n = 15) are women. The majority of awards (58%; n = 32) were R01 grants. Among the active NIH-funded projects, 75% (n = 41) are basic or translational research projects, and 25% (n = 14) are clinical or health services research projects. Abdominal aortic aneurysm and peripheral arterial disease are the most commonly funded disease areas and together accounted for 54% (n = 30) of projects. Three SVS research priorities are not addressed by any of the current NIH-funded projects. ConclusionsNIH funding of vascular surgeons is rare and predominantly consists of basic or translational science projects focused on abdominal aortic aneurysm and peripheral arterial disease research. Women are well-represented among funded vascular surgeons. Although the majority of SVS research priorities receive NIH funding, three SVS research priorities are yet to be addressed by NIH-funded projects. Future efforts should focus on increasing the number of vascular surgeons receiving NIH grants and ensuring all SVS research priorities receive NIH funding.

NIH Funding Among Vascular Surgeons Is Rare and Aligns Poorly With Society for Vascular Surgery Priorities

The National Institutes of Health (NIH) is an essential source of funding for vascular surgeon-scientists. NIH funding is frequently used to benchmark institutional and individual research productivity, help determine eligibility for academic promotion, and as a measure of scientific quality. We sought to determine NIH-funded vascular surgeon characteristics along with amounts, types, and content areas of NIH funding. In addition, we also sought to determine whether funded grants addressed recent Society for Vascular Surgery (SVS) research priorities.

Measuring the influence of non-scientific features on citations

Citations remain a prime, yet controversial, measure of academic performance. Ideally, how often a paper is cited should solely depend on the quality of the science reported therein. However, non-scientific factors, including structural elements (e.g., length of abstract, number of references) or attributes of authors (e.g., prestige and gender), may all influence citation outcomes. Knowing the predicted effect of these features on citations might make it possible to ‘game the system’ of citation counts when writing a paper. We conducted a meta-analysis to build a quantitative understanding of the effect of similar non-scientific features on the impact of scientific articles in terms of citations. We showed that article length, number of authors, author experience and their collaboration network, Impact Factors, availability as open access, online sharing, different referencing practice, and number of figures all exerted a positive influence on citations. These patterns were consistent across most disciplines. We also documented temporal trends towards a recent increase in the effect of journal Impact Factor and number of authors on citations. We suggest that our approach can be used as a benchmark to monitor the influence of these effects over time, minimising the influence of non-scientific features as a means to game the system of citation counts, and thus enhancing their usefulness as a measure of scientific quality.

Citation Metrics and Impact Factors Fail as Measures of Scientific Quality, in Particular in Taxonomy, and are Biased by Biological Discipline and by Geographic and Taxonomic Factors

Based on my experience and analyses of citation frequencies of papers published in various biological journals, I discuss the use of citation counts and journal Impact Factors as measures of scientific quality. I conclude that citation frequency differs not only among biological sub-disciplines such as ecology and taxonomy, but also among taxonomic papers on highly similar topics, and that these latter differences largely depend on the organismal group (cryptogams vs. vascular plants and species-rich vs small genera and families) and the geographic region concerned. As a result, journals with a broad biological scope face difficulties in competing in terms of Impact Factors, and thus in competing for the best manuscripts, with journals specialized on particular sub-disciplines that attract many citations. However, since papers on “hot” topics also tend to be short-lived, these differences, at least in part, would be much reduced if citations were counted for longer periods of time or were compensated based on the average age of cited papers in the particular field. In particular in taxonomy, factors such as originality and innovativeness, generally regarded as measures of scientific quality, appear to be inversely correlated with citation frequency. I, therefore, conclude that scientific editors would be able to significantly increase the Impact Factors of taxonomic journals, not by choosing for publication the most scientifically valuable and supreme articles, but by picking manuscripts based the taxonomic position and geographic distribution of the organisms concerned. Still, the author sincerely hopes that no editor will ever take such measures!

Title length

We document strong and robust negative correlations between the length of the title of an economics article and different measures of scientific quality. Analyzing all articles published between 1970 and 2011 and referenced in EconLit, we find that articles with shorter titles tend to be published in better journals, to be more cited and to be more innovative. These correlations hold controlling for unobserved time-invariant and observed time-varying characteristics of teams of authors.

Abuso de las medidas y medidas abusivas. Crítica al pensamiento bibliométrico hegemónico

.

Production Indicators of Mathematics Teachers in Public Universities of the Valencian Community on Web of Science

The current evaluation and comparison system of scientific production by impact factor has been criticised from different perspectives in recent years, and has ensured that publishing in high-impact journals does not necessarily imply that works are quality works. Many of these jobs are mostly not cited or, in the best of cases, only a very small number are, especially during the first years since their publication [4, 21]. In this context, the h-Index appeared in 2005 [19], and is currently one of the most bibliometric indicators used to estimate the success of research work and to predict the impact of future production. Following that proposed in [11], we used the h-index as a measure of scientific quality and productivity of mathematics teachers in Public Universities of the Valencian Community on the Web of Science. The results showed that the h-Index widely correlates citations and number of items.

The Interplay between Scientific Overlap and Cooperation and the Resulting Gain in Co-Authorship Interactions.

Considering the importance of scientific interactions, understanding the principles that govern fruitful scientific research is crucial to policy makers and scientists alike. The outcome of an interaction is to a large extent dependent on the balancing of contradicting motivations accompanying the establishment of collaborations. Here, we assembled a dataset of nearly 20,000 publications authored by researchers affiliated with ten top universities. Based on this data collection, we estimated the extent of different interaction types between pairwise combinations of researchers. We explored the interplay between the overlap in scientific interests and the tendency to collaborate, and associated these estimates with measures of scientific quality and social accessibility aiming at studying the typical resulting gain of different interaction patterns. Our results show that scientists tend to collaborate more often with colleagues with whom they share moderate to high levels of mutual interests and knowledge while cooperative tendency declines at higher levels of research-interest overlap, suggesting fierce competition, and at the lower levels, suggesting communication gaps. Whereas the relative number of alliances dramatically differs across a gradient of research overlap, the scientific impact of the resulting articles remains similar. When considering social accessibility, we find that though collaborations between remote researchers are relatively rare, their quality is significantly higher than studies produced by close-circle scientists. Since current collaboration patterns do not necessarily overlap with gaining optimal scientific quality, these findings should encourage scientists to reconsider current collaboration strategies.

Science Communication: Power of Community

J. Bohannon's News story “Who's afraid of peer review?” (special section on Communication in Science, 4 October, p. [60][1]) incriminates many open-access (OA) journals. Our journal, PLOS ONE , was not implicated. It rejected the fraudulent paper promptly and for the right reasons, as Bohannon acknowledges. Still, the “study” was disappointing: It was not controlled, which would have required seeking to entrap a matched set of closed-access journals, yet it claims that a source of the problem is open access. It then concludes that profitability for OA journals is driven by volume, without acknowledging that the same is true for closed-access journals. The issues raised by Bohannon's exercise are not about open-access journals; they are about science and technical publishing and the peer-review processes used throughout the industry. In the short term, all scientific publishers have a responsibility to reinforce and strengthen prepublication review. We must improve the efficiency of peer review and continue to perform checks that uncover conflicts of interest, identify financial disclosures, confirm author affiliations, and ensure compliance with international standards of animal and human testing. Even with these tools, peer review will never be flawless. As Science Editor-in-Chief Marcia McNutt points out, it is “timehonored” and the “gold standard” (“Improving scientific communication,” Editorial, 4 October, p. [13][2]), but that doesn't mean our methods of evaluation can't and shouldn't be improved. This is the real challenge. And this is why the Public Library of Science (PLOS) is working to transform scientific communication by developing better measures of scientific quality both before publication (currently traditional peer review) and after publication (currently the dreaded impact factor). To this end, PLOS is developing article-level metrics that enable the scientific community itself to confer on a research contribution its credibility, relevance, and importance, independent of the journal in which it is published. Peer review at its best is a continual process of critique and assessment. [1]: /lookup/doi/10.1126/science.342.6154.60 [2]: pending:yes

Deep impact: unintended consequences of journal rank

Most researchers acknowledge an intrinsic hierarchy in the scholarly journals (“journal rank”) that they submit their work to, and adjust not only their submission but also their reading strategies accordingly. On the other hand, much has been written about the negative effects of institutionalizing journal rank as an impact measure. So far, contributions to the debate concerning the limitations of journal rank as a scientific impact assessment tool have either lacked data, or relied on only a few studies. In this review, we present the most recent and pertinent data on the consequences of our current scholarly communication system with respect to various measures of scientific quality (such as utility/citations, methodological soundness, expert ratings or retractions). These data corroborate previous hypotheses: using journal rank as an assessment tool is bad scientific practice. Moreover, the data lead us to argue that any journal rank (not only the currently-favored Impact Factor) would have this negative impact. Therefore, we suggest that abandoning journals altogether, in favor of a library-based scholarly communication system, will ultimately be necessary. This new system will use modern information technology to vastly improve the filter, sort and discovery functions of the current journal system.

“3 . . 2 . . 1 . . Impact [Factor]: Target [Academic Career] Destroyed!”

"Publish or perish" is the time-honored "principle" for academicians who race to accumulate lines under the "publications" section of a curriculum vitae. The original intent of publication-to inform others of findings and further scientific knowledge-has been corrupted by factors including (1) exponential growth of journals and the journal industry, fueled in part by intrusion of the Internet into all aspects of academic life; and (2) adoption of journal metrics (rather than written content) as the measure of scientific quality. The proprietary Thomson Reuters Impact Factor is the most pernicious metric, having caused editors and publishers to change editorial practices to boost the number. At the same time, gullible administrators and government agencies have been persuaded that metrics for the journal in which materials are published can be used as a measure of the worth of individual investigators (and institutions) and their research efforts: simple numbers can be substituted for the burdensome effort required to read and assess research quality. Thus, granting of research funds, awarding of academic rank and tenure, and determination of salaries (including bonus payments) have become tied to manipulable journal metrics rather than the significance or quality of reported research. Therefore, it is no wonder that the integrity of science is more often being questioned. How should a young investigator approach the "publish or perish" dilemma? Performing sound research and preparing optimal materials for publication must remain the overriding goals: properly articulate the question addressed by the study; thoroughly document all methods and case information; carefully describe results including any conflicting or negative findings; discuss the importance of the findings along with how the results address the initial question and whether findings refute or confirm previous studies; prepare properly cited bibliographic references; list all author contributions, potential conflicts of interest, financial support, and required ethical approvals; and provide a catchy title and an abstract containing sufficient information that other investigators perusing scientific indices will be enticed to read the published article. Submit the completed manuscript to the most appropriate journal based on that journal's previously published content and relevance to the field of study regardless of journal metrics. On publication, notify investigators in the same field to ask for their comments on the work. Thus, an individual will become known for the quality of his or her work product and the worshiping of publication metrics will be unnecessary.

How well are journal and clinical article characteristics associated with the journal impact factor? a retrospective cohort study

Journal impact factor (JIF) is often used as a measure of journal quality. A retrospective cohort study determined the ability of clinical article and journal characteristics, including appraisal measures collected at the time of publication, to predict subsequent JIFs. Clinical research articles that passed methods quality criteria were included. Each article was rated for relevance and newsworthiness by 3 to 24 physicians from a panel of more than 4,000 practicing clinicians. The 1,267 articles (from 103 journals) were divided 60∶40 into derivation (760 articles) and validation sets (507 articles), representing 99 and 88 journals, respectively. A multiple regression model was produced determining the association of 10 journal and article measures with the 2007 JIF. Four of the 10 measures were significant in the regression model: number of authors, number of databases indexing the journal, proportion of articles passing methods criteria, and mean clinical newsworthiness scores. With the number of disciplines rating the article, the 5 variables accounted for 61% of the variation in JIF (R(2) = 0.607, 95% CI 0.444 to 0.706, P<0.001). For the clinical literature, measures of scientific quality and clinical newsworthiness available at the time of publication can predict JIFs with 60% accuracy.

The Hirsch index: more than a number?

The Hirsch index: more than a number?

The Use and Abuse of Citations as Indicators of Research Quality

Citation counts have been widely applied as indicators of performance in the evaluation of research. The supposition underlying such applications is that number of citations can be regarded as a measure of scientific quality or impact (Aksnes, 2005); that is, highly cited research can be considered meritorious or significant, since the extent to which research is used (i.e., cited) is a measure of its contribution to knowledge. This note briefly reviews the background of this approach and then lists the shortcomings of the approach.

The Impact Factor – a dubious measure of scientific quality

The Impact Factor – a dubious measure of scientific quality

Counting on citations: a flawed way to measure quality.

The journal Impact Factor and citation counts are misconstrued and misused as measures of scientific quality. Articles must be read in order to judge their quality. We have introduced a system, which may be easily replicated, to identify the best articles published in a journal.