Citation-based Impact Research Articles

A Nine Year Story of the India Software Engineering Conference from 2008 to 2016

The India Software Engineering Conference (ISEC) is an annual conference in the field of Software Engineering (SE) in India. ISEC started in the year 2008 and completed 9 years in 2016. The ISEC conference has evolved into a high-quality academic event for SE researchers from universities and industry in India with considerable international participation. Assessment and evaluation of ISEC conference quality, status and evolution is important for the national SE scientific community, ISEC steering committee, sponsors and science and technology-related government bodies. In this paper, we conduct scientific paper publication mining and scientometric and bibliometric analysis of 9 years of ISEC publications and programs. We conduct an in-depth multi-dimensional analysis of the conference across various aspects such as a summary of 9 years of ISEC programs (paper submission data, tutorials, workshops, keynotes, invited talks, geographical location, program and general chairs), author-affiliation-based geographical contribution (analysis at the international and national levels), topic analysis, university and industry collaborations, contributions across university types in India, prolific and new authors, gender equality and imbalance, program committee characteristics, open-source or closed-source datasets and citation-based impact. We also present our recommendations for future editions of the ISEC based on our comprehensive analysis study presented in this paper.

Web indicators for research evaluation. Part 3: books and non standard outputs

This literature review describes web indicators for the impact of books, software, datasets, videos and other non-standard academic outputs. Although journal articles dominate academic research in the health and natural sciences, other types of outputs can make equally valuable contributions to scholarship and are more common in other fields. It is not always possible to get useful citation-based impact indicators for these due to their absence from, or incomplete coverage in, traditional citation indexes. In this context, the web is particularly valuable as a potential source of impact indicators for non-standard academic outputs. The main focus in this review is on books because of the much greater amount of relevant research for them and because they are regarded as particularly valuable in the arts and humanities and in some areas of the social sciences.

Computer science research: the top 100 institutions in India and in the world

This paper aims to perform a detailed scientometric and text-based analysis of Computer Science (CS) research output of the 100 most productive institutions in India and in the world. The analytical characterization is based on research output data indexed in Scopus during the last 25 years period (1989---2013). Our computational analysis involves a two-dimensional approach involving the standard scientometric methodology and text-based analysis. The scientometric characterization aims to assess CS domain research output in leading Indian institutions vis-a-vis the leading world institutions and to bring out the similarities and differences among them. It involves analysis along traditional scientometric indicators such as total output, citation-based impact assessment, co-authorship patterns, international collaboration levels etc. The text-based characterization aims to identify the key research themes and their temporal trends for the two sets. The key contribution of the experimental work is that it's an analytical characterization of its kind, which identifies characteristic similarities and differences in CS research landscape of Indian institutions vis-a-vis world institutions.

Comparison of a citation-based indicator and peer review for absolute and specific measures of research-group excellence

Many different measures are used to assess academic research excellence and these are subject to ongoing discussion and debate within the scientometric, university-management and policy-making communities internationally. One topic of continued importance is the extent to which citation-based indicators compare with peer-review-based evaluation. Here we analyse the correlations between values of a particular citation-based impact indicator and peer-review scores in several academic disciplines, from natural to social sciences and humanities. We perform the comparison for research groups rather than for individuals. We make comparisons on two levels. At an absolute level, we compare total impact and overall strength of the group as a whole. At a specific level, we compare academic impact and quality, normalised by the size of the group. We find very high correlations at the former level for some disciplines and poor correlations at the latter level for all disciplines. This means that, although the citation-based scores could help to describe research-group strength, in particular for the so-called hard sciences, they should not be used as a proxy for ranking or comparison of research groups. Moreover, the correlation between peer-evaluated and citation-based scores is weaker for soft sciences.

Scholarometer: A Social Framework for Analyzing Impact across Disciplines

The use of quantitative metrics to gauge the impact of scholarly publications, authors, and disciplines is predicated on the availability of reliable usage and annotation data. Citation and download counts are widely available from digital libraries. However, current annotation systems rely on proprietary labels, refer to journals but not articles or authors, and are manually curated. To address these limitations, we propose a social framework based on crowdsourced annotations of scholars, designed to keep up with the rapidly evolving disciplinary and interdisciplinary landscape. We describe a system called Scholarometer, which provides a service to scholars by computing citation-based impact measures. This creates an incentive for users to provide disciplinary annotations of authors, which in turn can be used to compute disciplinary metrics. We first present the system architecture and several heuristics to deal with noisy bibliographic and annotation data. We report on data sharing and interactive visualization services enabled by Scholarometer. Usage statistics, illustrating the data collected and shared through the framework, suggest that the proposed crowdsourcing approach can be successful. Secondly, we illustrate how the disciplinary bibliometric indicators elicited by Scholarometer allow us to implement for the first time a universal impact measure proposed in the literature. Our evaluation suggests that this metric provides an effective means for comparing scholarly impact across disciplinary boundaries.

Ranking Finance Journals Using Author Affiliation Index

In this paper we use a new method to rank finance journals. Traditionally, journal qualities are measured either by the citation-based impact factor approach, or by the survey method. Although these two approaches have merits, their efficacy is also limited in many ways. Author Affiliation Index is a cost-effective and intuitively easy to understand approach to journal rankings. We define Author Affiliation Index as the ratio of articles authored by faculties at the world's top 80 finance programs divided by the total number of articles by all authors. Forty-one finance journals are ranked according to this index. The effectiveness of the index is comparable to prior citation-based and survey-based studies. We find Author Affiliation Index, if properly constructed, provides an easy and credible way to supplement the existing journal ranking methods. Due to its easiness to construct, the index is particularly useful for junior faculty seeking the right journals to publish but is unable to justify the journal quality because these journals are not ranked in existing studies. Based upon various definitions of the Author Affiliation Index, our findings indicate that Journal of Finance, Review of Financial Studies, Journal of Financial Economics, Journal of Financial and Quantitative Analysis, and Journal of Business unambiguously claim the top five spots in finance journal rankings. Some newer journals, such as Journal of Corporate Finance and Journal of Financial Markets, while ranked lower in the other studies, score impressively in our analysis. Compared with prior studies, our ranking is highly correlated with most of those journal rankings.

Assessing the quality of scholarly journals in Linguistics:An alternative to citation-based journal impact factors

Methods were developed to allow quality assessment of academic research in linguistics in allsub-disciplines. Data were obtained from samples of respondents from Flanders, the Netherlands,as well as a world-wide sample, evaluated journals, publishers, and scholars. Journals andpublishers were ranked by several methods. First, we weighted the number of times journals orpublishers were ranked as 'outstanding', 'good', or 'occasionally/not at all good'. To reduce theinfluence of unduly positive or negative biases of respondents, the most extreme ratings weretrimmed. A second weight reflects the (international) visibility of journals and publishers. Here,journals or publishers nominated by respondents from various countries or samples received agreater weight than journals or publishers nominated by respondents from one country or onesample only. Thirdly, a combined index reflects both quality and international visibility. Its use isillustrated on the output of scholars in linguistics. Limitations and potentials for application ofbibliometric methods in output assessments are discussed.

Risk and Insurance Research Productivity: 1987-1996

INTRODUCTION Several studies have been conducted evaluating research productivity in the areas of finance and economics. These studies tend to focus on one or more of the following major categories: research productivity of individual authors and the institutions that employ them, research productivity of the universities that awarded the authors their highest academic degrees, and/or the quality of the journals in which the research is published. Studies of these types also have been applied specifically to the risk and insurance discipline, although most are now several years old and must be updated to yield useful, current information. Research documenting productivity within the risk and insurance discipline is important for several reasons. Decision-making by administrators, prospective faculty, potential graduate students, and accreditation bodies is enhanced when reliable information is available about individuals and institutions active in risk and insurance research. The value of such information may he even more important when changes in research performance can he documented over various historical periods. The purpose of this study is to evaluate risk and insurance research productivity for the years 1987 through 1996, focusing on individual authors, their employing institutions, and the institutions from which authors obtained their highest academic degrees. The study is designed to allow historical comparisons to other studies for years prior to 1987. Further, the study is designed in recognition of the fact that research in risk and insurance is published in specialized journals as well as general finance and economics outlets. Previous relevant research is briefly discussed in the next section, followed by sections on the research methods and results. The final section consists of a brief summary and statement of conclusions. PRIOR RESEARCH Cox and Gustavson (1990) [hereafter CG] published the first comprehensive study of risk and insurance research. They analyzed the research productivity of authors, employing institutions, and degree-granting institutions during the eleven-year period 1976 through 1986. CG included twenty-two different journals in their study, drawing from the finance, economics, and actuarial literature in addition to journals focused exclusively on risk and insurance topics. The CG study followed the methodology of similar, more general finance studies such as Moore and Taylor (1980), Niemi (1987), and Heck, et. al. (1986). Chung and Puelz (1992) identified an empirical regularity in the frequency distribution of articles published among six different risk and insurance journals, thereby providing an alternative way of assessing individual authors' productivity. The issue of journal quality is important for research productivity studies that incorporate numerous different journals. CG dealt with this problem by adjusting their reported page and article counts for journal impact scores, based on a survey of insurance academics conducted by Outreville and Malouin (1985). The more commonly accepted practice within the finance literature is to judge research significance on the basis of citations reported by the Social Sciences Citation Index (SSCI). This practice was illustrated for publications in two finance journals by Ederington (1979) and has since been used (e.g., see Borokhovich, et. al. 1995) to differentiate among finance journals on the basis of impact. Studies of risk and insurance productivity are handicapped in their use of citations as a measure of impact, because relatively few journals in the field are included in the SSCI. Thus, alternatives have been developed, including survey rankings compiled by McNamara and Kolbe (1996) and Baur, et. al. (1996), and citation-based insurance impact factors reported by Colquitt (1997). Viewed in relationship to prior related research, the current study is both an update and an extension of the CG study. …

Citation-Based Impact Measures and the Bradfordian Selection Criteria

A comparison of a Bradford-ranked list of journals in the field of mathematics with a list of the same journals ranked by citation counts finds little correlation between the two. A previous study by Lamb shows a high correlation between a ranking based on citation frequency for these journals and a ranking based on the prestige of individual contributors. Lamb's study suggests that a selection policy based on choosing journals publishing the most papers on a topic will result in the acquisition of high quality papers. The results of this study cast doubt on Lamb's conclusions.