Tool For Research Evaluation Research Articles

Evaluative altmetrics: is there evidence for its application to research evaluation?

Altmetrics have been demonstrated as a promising tool for analyzing scientific communication on social media. Nevertheless, its application for research evaluation remains underdeveloped, despite the advancement of research in the study of diverse scientific interactions. This paper develops a method for applying altmetrics in the evaluation of researchers, focusing on a case study of the Environment/Ecology ESI field publications by researchers at the University of Granada. We considered Twitter as a mirror of social attention, news outlets as media, and Wikipedia as educational, exploring mentions from these three sources and the associated actors in their respective media, contextualizing them using various metrics. Our analysis evaluated different dimensions such as the type of audience, local attention, engagement generated around the mention, and the profile of the actor. Our methodology effectively provided dashboards that gave a comprehensive view of the different instances of social attention at the author level. The use of altmetrics for research evaluation presents significant potential, as shown by our case study. While this is a novel method, our results suggest that altmetrics could provide valuable insights into the social attention that researchers garner. This can be an important tool for research evaluation, expanding our understanding beyond traditional metrics.

Influence of Work-Life Balance on Job Contentment and Organizational Loyalty in Manufacturing Companies

Work-life balance is accomplishing stability between non-public life and work. Work-existence coordination facilitated work prioritization and remained the right top reputation ideal final representation at work. Many former studies on charismatic management, work-lifestyle balance, job pleasure, and loyalty of organizations, but there are not many studies of charismatic management and work-existence balance. Research on charismatic leadership carried out within the manufacturing enterprise has not often been studied in the last 3 years. Some other distinctions that demarcate this examination from former research are the dimension of job pride variables and the usage of the weiss dimension. Primarily, the study’s objective delivers proof of the influence of charismatic management and work-existence balance on the loyalty of organizations on worker activity pride in manufacturing groups in Indonesia. The research evaluation tool used the Structural Equation version (SEM), with a pattern populace of 250 production corporation fellows in Indonesia. The outcomes that can be concluded from this look at our first, accurate charismatic management will grow worker job contentment, 2nd, a great work-existence stability will boom worker job pride, 1/3, suitable activity pleasure will escalate worker organizational dedication, fourth, charismatic. Management of an awesome work-lifestyle balance will maximize worker organizational loyalty and 5th, amazing work-lifestyle stability does not immediately escalate worker organizational dedication.

Further development of the Teacher Attitudes to Inclusion Scale: principal components and Rasch analysis

ABSTRACT Providing an effective and high-quality education for all children and young people remains a significant challenge throughout the world. Disputes and contradictions contribute to the prevailing debate as to the justification and merits of inclusive education. One of the reoccurring themes within the literature on inclusive education is the relationship between the successful application of teachers’ knowledge, skills, and abilities utilised to include inclusive strategies in their classrooms, and their attitude towards inclusion. A teacher with more positive attitudes will be more accepting of students, build more successful relationships with them, aid in a child’s sense of belonging, student academic success and social integration. To address this issue, the Teacher Attitudes to Inclusion Scale (TAIS) was developed by the first author to measure the attitudes of qualified (in-service) teachers towards inclusive education. Using an Exploratory Principal Components Analysis and Reliability, a revised version of the TAIS, the TAIS-R provides a psychometrically validated measure of two constructs – a global attitude and a personal attitude towards inclusive education that provides an accurate evaluation tool for research and practice.

The experience of patient dropout from eating disorders treatment: a systematic review and qualitative synthesis

AimsEating disorders are serious and debilitating mental health conditions associated with high relapse and mortality rates and significant psychiatric comorbidities. Research suggests approximately 50% of patients with an eating disorder dropout prematurely from treatment services, fostering poorer health outcomes and impacting significantly on patients, their families, health services and research quality. The aim of this review is to synthesise the current qualitative literature available on the patient experience of dropout from eating disorder services and understand the reasons motivating early treatment termination.MethodA systematic search was carried out and articles selected from MEDLINE, PsycINFO, EMBASE and CINAHL. Studies were included if they explored eating disorder treatment dropout using qualitative data collection or analysis methods. Study quality was critically appraised using the Critical Appraisal Skills Programme qualitative research evaluation tool. Thematic synthesis was used to interpret and synthesise themes from the primary studies.ResultTen studies met the inclusion criteria for the systematic review. Five studies were scored as high quality and five as medium quality. 13 descriptive sub-themes encompassing the dropout experience were identified under four overarching analytical themes: inner conflict, connection and communication with others, experience of the treatment service, and factors related to progress in treatment.ConclusionThe decision to drop out from eating disorder treatment is a complex, multi-faceted issue, involving an interplay between individual, social and service-level factors. This review highlights the need for further high quality qualitative investigation into dropout experiences, with an effort to increase representation across ethnic groups and gender identities. This review also identifies the need for a reconsideration of current clinical practice and services provision with an emphasis on the use of patient perspectives to guide decision making in eating disorder services delivery and research. Moreover, the findings emphasise the need for standardised dropout definitions, fostering a unified literature base.

Building criteria to evaluate qualitative research papers: a tool for peer reviewers.

To present some criteria, many of which were created in the form of a checklist of items to be considered throughout the review process. Furthermore, we present a study whose main objective was to build (and validate) a review instrument by specialists that is clear, comprehensive, concise and consistent with the highest standards of excellence. We use a survey to collect data (123 answers). To analyze the open answers we use the technique of content analysis through the Qualitative Data Analysis Software. Besides the creation of new criteria with a focus on methodological dimensions based on 12 items, the involvement of the Ibero-American Congress on Qualitative Research scientific committee in the validation of this instrument. This option, in line with the others described, will allow to implement improvements to the editions of the events using the Qualitative Research Evaluation Tool in 2018.

Towards prediction of paradigm shifts from scientific literature

Amid the enormous volume of knowledge generated due to knowledge explosion, to a great extent, scientific literature mining can play a crucial role in research evaluation and tracking of important developments. Linked through citation relations, scientific literature forms network of papers or citation networks. Citation network analysis is gaining recognition as effective tool for research evaluation. Paradigm shift detection is of great importance to a multitude of beneficiaries and prediction of such paradigm shifts is of much greater gravity towards policy making. Recently, a metric named Flow Vergence (FV) gradient is proposed to detect paradigm shift pivots in scientific literature using the property named as FV effect. In this paper, its predictive power is investigated and tested statically and dynamically using publications in the field Nanotechnology for Engineering for a period 1989–2014. Validation included a post analysis validation of the field in 2017. As predictive power of FV gradient is confirmed, it can be regarded as an effective method for prediction of likely paradigm shifts and added to the tool kit of research evaluators and policy makers.

The Critical Research Evaluation Tool (CRET): A Teaching and Learning Tool to Evaluate Research for Cultural Competence.

The aim of this study was to present the Critical Research Evaluation Tool (CRET) which teaches evaluation of the researchers' worldview, applicability to multicultural populations, and ethics surrounding potential harms to communities. To provide best cultural care nurses' need to understand how historical/social/political experiences impact health and also influence research. The Student using the CRET reported receiving a strong foundation in research fundamentals, gaining a better understanding of critical frameworks in research, and learning more about themselves and reflecting on their own privileges and biases. The CRET provides nursing students and nursing faculty with a tool for examining diversity and ultimately decreasing health disparity.

BIBLIOMETRIC ANALYSIS OF ESTONIAN FOLKLORE RESEARCH AND FOLKLORE: ELECTRONIC JOURNAL OF FOLKLORE

1. Background The research areas in Estonia that are most often considered to have a strong impact on the world based on their citations per publication are molecular biology and genetics (18th in the world on the Essential Science Indicator), environment/ ecology (15th on the ESI), and plant and animal science (12th on the ESI). Overall the level on Estonian research is a small miracle considering Estonia's size and history (Allik 2015). But what is not known is that there is an area of research where Estonia is at world level--and it is an area not visible in either ESI or Journal Citation Reports because it is an area in arts and humanities. The humanitarian researchers in Estonia are skeptical about bibliometrics and how or if it can display their impact on a local or on a larger scale, and for the most part this is true (Allik 2012). It is no secret that the humanities with their publishing and citation practices are largely different from most of the other research areas in science and for this they are not ideal for bibliometrical comparisons or analysis. The citation tradition in the humanities is not as strong as it is in science. An art historian may not formally cite such works as Guernica or Mona Lisa. A literary critic would not cite Shakespeare every time he mentions Hamlet (Garfield 1980a). The citations to articles are slow to grow and in many subfields of the humanities articles do not even have any great impact (because of the book-oriented nature of the fields) (Stern 1983). Also most of the research done in the humanities are with localized conceptions, meaning that linguistic studies on Estonian or Finnish will be published in Estonian or Finnish for Estonian or Finnish readers and researchers. But there is still a remarkable amount of documents on arts and humanities in the Web of Science Arts & Humanities Citation Index (A&HCI) and this data is bibliometrically analyzable and actually quite interesting (Ho et al. 2015, Konur 2012). The following paper tries to answer these questions: How does an Estonian folklore researcher compare with its neighbors and with the whole world? What kind of impact has the Estonian folklore journal The Electronic Journal of Folklore (EJoF) had and how has it changed the area in Estonia? 2. Methodology To measure, compare and visualize the area of folklore in Estonia, 100 documents published from 2005-2014 and 73 documents published from 2010-2014 indexed in the Web of Science (WoS) Arts & Humanities Citation Index (A&HCI) were analyzed. For comparison with Estonia, Finland was naturally chosen because of its neighboring location and its similar culture to Estonia. Also Latvia, Sweden, the United Kingdom (England, Scotland, Northern Ireland, and Wales in total) and the United States of America were chosen because of their location to Estonia and their history and their role in the area of folklore. The analysis on the journal the Estonian Electronic Journal of Folklore (EJoF) was made by using data from Thomson Reuters Web of Science Core Collection (WoS) Arts & Humanities Citation Index (A&HCI), Journal Citation Report (JCP), Scopus, and SCImago. To compare the impact of EJoF to a Finnish journal FF Communications was chosen since it has published the most cited and important work in folklore (Aarne, Stith 1928). Folk Life--Journal of Ethnological Studies from the United Kingdom, Folklore, the journal for the Folklore Society of England, which is one of the earliest English-language journals in the area of folkloristics, first published in 1879 and the Journal of Folklore Research from the Indiana University in the United States of America were chosen. The analysis was done in InCites (InCites is a customized, web-based research evaluation tool that allows users to analyze institutional productivity and benchmark research output against peers worldwide) during the last week (23-29) of November 2015. …

Assigning publications to multiple subject categories for bibliometric analysis

Purpose – This study is concerned with a problem in measuring citation impact with the aid of percentile data, which arises from the assignment of publications (or the journals in which the publications have appeared) by Thomson Reuters for the Web of Science to more than one subject category. If there is more than one subject category for a publication, it is initially unclear which category is to be used to create the reference set for the calculation of the percentile. This paper seeks to address these issues. Design/methodology/approach – In this study the author would like to look at whether the calculation of differences between the citation impact of research institutions is affected by whether the minimum (the maximum percentile), the maximum (the minimum percentile), the mean or the median impact (percentile) for the different subject categories is used. The study is based on a sample of percentile data for three research institutions (n=4,232). Findings – The result of the comparison of citation impact of the three institutions remains very similar for all the calculation methods, but on a different level. Originality/value – It is the first study, which investigates how far it makes a difference in the comparison of the citation impact of three different research institutes whether – with multiple assignments of subject categories to one publication – the minimum, the maximum, the mean or the median inverted percentile is used. An answer to the question is very relevant since different methods are used in practical application. For example, the web-based research evaluation tool InCites uses the minimum percentile.

F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations

F1000 is a postpublication peer review service for biological and medical research. F1000 recommends important publications in the biomedical literature, and from this perspective F1000 could be an interesting tool for research evaluation. By linking the complete database of F1000 recommendations to the Web of Science bibliographic database, we are able to make a comprehensive comparison between F1000 recommendations and citations. We find that about 2% of the publications in the biomedical literature receive at least one F1000 recommendation. Recommended publications on average receive 1.30 recommendations, and more than 90% of the recommendations are given within half a year after a publication has appeared. There turns out to be a clear correlation between F1000 recommendations and citations. However, the correlation is relatively weak, at least weaker than the correlation between journal impact and citations. More research is needed to identify the main reasons for differences between recommendations and citations in assessing the impact of publications.

A Century of Citation Indexing

It has been 60 years since I began my career in information science. Since that time there have been huge changes in technology that are reflected in the way that scientific and scholarly information is disseminated, retrieved, and evaluated. The younger generation of scientists in Turkey and elsewhere will not even remember Current Contents or the huge printed volumes of the Science Citation Index. In its first generation of life, Current Contents became the compulsory reading habit of scientists throughout the world. Then in its second generation of life, we began to publish in Current Contents the series called This week's Citation Classic. Over the next 15 years, we asked thousands of authors to write commentaries on their highly cited papers. Four thousand of these one-page commentaries can be found at (http://www.citationclas-sics.org/). Quite often authors would say that these were not necessarily their most important papers. So when I was asked to write a commentary about my 1955 paper in Science1 I had to point out that this primordial paper on Citation Indexing was neither my most cited work nor my most significant. My 1972 paper in Science, [2] on using citation analysis to evaluate journals, has attracted much more attention. My most-cited work is in fact my 1979 book, Citation Indexing. [3] Books, like review papers, are easy ways to refer to an author’s work in a general way. However, tracing the genealogy of citations to my 1955 paper reveals the evolution of the concept of citation indexing -- from a system for information retrieval to a tool for research evaluation. In a paper that I had prepared at the request of the then editor of Science in 1995, I suggested that the tail was now wagging the dog [4]. I will say more about this later. Indeed, this was the theme of my Keynote Address at the last COLLNET conference in Dalian, China. I feel honored to have been invited to COLLNET again.

An evaluation tool for research of user behavior in a realistic mobile environment

User behavior is significantly influenced by the surrounding environment. Especially complex and dynamically changing environments (like mobile environment) are represented by a wide variety of extraneous variables, which influence the user behavior in an unpredictable and mostly uncontrolled way. For researchers, it is challenging to measure and analyze the user behavior in such environments. We introduce a complex tool—the IVE tool—which provides a unique way of context visualization and synchronization of measured data of various kinds. Thanks to this tool it is possible to efficiently evaluate data acquired during complex usability tests in a mobile environment. The functionality of this tool is demonstrated on the use case “Navigation of visually impaired users in the building with support of a navigation system called NaviTerier.” During the experiment, we focused on collection and analysis of data that may show user stress and which may influence his/her ability to navigate. We analyzed objective data like Galvanic Skin Response parameter (GSR), Heart Rate Variability parameters (HRV) and audio video recordings and also subjective data like the user’s subjective stress feeling and observation of the user’s behavior.

The scholarly impact of TRECVid (2003–2009)

AbstractThis paper reports on an investigation into the scholarly impact of the TRECVid (Text Retrieval and Evaluation Conference, Video Retrieval Evaluation) benchmarking conferences between 2003 and 2009. The contribution of TRECVid to research in video retrieval is assessed by analyzing publication content to show the development of techniques and approaches over time and by analyzing publication impact through publication numbers and citation analysis. Popular conference and journal venues for TRECVid publications are identified in terms of number of citations received. For a selection of participants at different career stages, the relative importance of TRECVid publications in terms of citations vis à vis their other publications is investigated. TRECVid, as an evaluation conference, provides data on which research teams ‘scored’ highly against the evaluation criteria and the relationship between ‘top scoring’ teams at TRECVid and the ‘top scoring’ papers in terms of citations is analyzed. A strong relationship was found between ‘success’ at TRECVid and ‘success’ at citations both for high scoring and low scoring teams. The implications of the study in terms of the value of TRECVid as a research activity, and the value of bibliometric analysis as a research evaluation tool, are discussed.

Bibliometrics as a Performance Measurement Tool for Research Evaluation: The Case of Research Funded by the National Cancer Institute of Canada

As bibliometric indicators are objective, reliable, and cost-effective measures of peer-reviewed research outputs, they are expected to play an increasingly important role in research assessment/management. Recently, a bibliometric approach was developed and integrated within the evaluation framework of research funded by the National Cancer Institute of Canada (NCIC). This approach helped address the following questions that were difficult to answer objectively using alternative methods such as program documentation review and key informant interviews: (a) Has the NCIC peer-review process selected outstanding Canadian scientists in cancer research? (b) Have the NCIC grants contributed to increasing the scientific performance of supported researchers? (c) How do the NCIC-supported researchers compare to their neighbors supported by the U.S. National Cancer Institute? Using the NCIC evaluation as a case study, this article demonstrates the usefulness of bibliometrics to address key evaluation questions and discusses its integration, along complementary indicators (e.g., peer ratings), in a practice-driven research evaluation continuum.

OMB should rethink research evaluation tool

ADVERTISEMENT RETURN TO ISSUEPREVNEWSNEXTOMB should rethink research evaluation toolRebecca RennerCite this: Environ. Sci. Technol. 2008, 42, 8, 2715–2716Publication Date (Web):April 15, 2008Publication History Published online15 April 2008Published inissue 1 April 2008https://doi.org/10.1021/es0870496RIGHTS & PERMISSIONSArticle Views145Altmetric-Citations-LEARN ABOUT THESE METRICSArticle Views are the COUNTER-compliant sum of full text article downloads since November 2008 (both PDF and HTML) across all institutions and individuals. These metrics are regularly updated to reflect usage leading up to the last few days.Citations are the number of other articles citing this article, calculated by Crossref and updated daily. Find more information about Crossref citation counts.The Altmetric Attention Score is a quantitative measure of the attention that a research article has received online. Clicking on the donut icon will load a page at altmetric.com with additional details about the score and the social media presence for the given article. Find more information on the Altmetric Attention Score and how the score is calculated. Share Add toView InAdd Full Text with ReferenceAdd Description ExportRISCitationCitation and abstractCitation and referencesMore Options Share onFacebookTwitterWechatLinked InReddit PDF (2 MB) Get e-Alerts Get e-Alerts

Bibliometrics and its applications as a research evaluation tool(<Special feature>Evaluation methods in library and information activities)

Bibliometrics and its applications as a research evaluation tool(<Special feature>Evaluation methods in library and information activities)

Marketing research performance and strategy

PurposeTo investigate whether strategic orientation affects the evaluation of specific market research projects in for‐profit firms.Design/methodology/approachA small‐scale follow‐up survey was conducted, building on qualitative and quantitative research among a sample of the top‐1,000 marketing managers in Australia. The study used an existing market research evaluation tool, the USER scale and items generated from the qualitative research, to investigate the firm's most recent market research project.FindingsFour market research performance factors were identified – market research as a knowledge enhancing (KE) function, the internal political use of market research, the misuse of market research and the generation of market understanding. The Miles and Snow strategy types were related to these factors, with Prospector types more likely to use market research rationally and less likely to use it for internal political purposes. Tactical projects were more likely to be misused than were those with a strategic orientation. Prospectors were far less likely and Analyzers far more likely to misuse tactical research projects. Prospectors were more often satisfied with the performance of their most recent market research. The Porter typology was less successful in predicting market research performance.Research limitations/implicationsThe study was based on a small sample of market research projects in Australian for‐profit firms. Future studies need to study these phenomena more intensively using ethnographic methods and more extensively using larger multi‐country samples.Practical implicationsMarket research suppliers should learn the nature of their client's strategic intent to improve their effectiveness. Defender firms should carefully monitor the use of market research, especially that of a tactical nature, which may be wasted or misused.Originality/valueThe paper contributes to an understanding of how strategic orientation relates to the ways market research information is used within the firm.

The construction of the Taiwan Humanities Citation Index

The Taiwan Humanities Citation Index (THCI) is Taiwan's effort to construct a search, research, and evaluation tool for research in the arts and humanities. This article describes the design, framework, features, and policies and rules of the THCI. Citation analysis has been regarded as a systematic way to investigate research developments and trends. Since the Arts & Humanities Citation Index (A&HCI) indexes mostly English journals, the THCI could become an auxiliary citation index of the A&HCI for Taiwanese researchers.