Excellence In Research For Australia Research Articles

Evaluating journals in business and related fields: A guide for faculty

This guide for faculty presents the data sources, online tools, and citation metrics that are most widely used to evaluate the quality, scholarly impact, or reputation of business and economics journals. The guide, which is distinctive in its subject emphasis and its intended audience, may be useful to scholars, departments, and committees that seek to identify appropriate publication outlets, to evaluate or demonstrate the impact of particular research contributions, or to distinguish between legitimate and predatory journals. The topics and resources covered include the multiple dimensions of journal quality, journal rankings based on citation impact (e.g., Scopus, Web of Science, Google Scholar Metrics, Cabell’s Journalytics, RePEc), journal ratings based on expert opinion (e.g., Australian Business Deans Council, Chartered Association of Business Schools, Harzing’s Journal Quality List, Excellence in Research for Australia), other means of assessing journal quality (e.g., Directory of Open Access Journals, Beall’s List, Cabell’s Predatory Reports), and resources for gauging the citation impact of particular articles.

Valuing and evaluating musical practice as research in ethnomusicology and its implications for research assessment

ABSTRACT In this article, we argue that ethnomusicology holds valuable epistemic insights for considering how to measure and evaluate research for academics, as well as for research policy and management professionals. We focus on two notable instances of standardised national research assessment frameworks: the UK’s Research Excellence Framework (REF), and Excellence in Research for Australia (ERA) and identify the advantages of practice as research models for music research by considering the benefits of ethnomusicology’s reflexive and relativist methodologies to formal research assessment processes. To support our argument, we refer to published case studies of ethnomusicological research that reach beyond Western practice and thought to highlight the advantages recognising practice as research as a more inclusive modality of original knowledge production. We call upon ethnomusicologists to pro-actively engage with the formal processes of research assessment to make them more equitable and representative of our discipline’s broad commitment to decolonising academic practice.

Excellence in research in Australia: the souffle keeps on rising

ObjectiveThe Excellence in Research for Australia (ERA) data collections completed in 2010, 2012, 2015, and 2018 were developed by the Australian Research Council (ARC) to measure the quality of the research produced by the broad range of Australian tertiary education institutions.MethodThis analysis compared the ratings of the 41 Australian universities across the years 2010, 2012, 2015, and 2018 for the psychology codes (i.e., 17, 1701, 1702, and 1799), as well as examining the combined means for all four data collections and compared these across institutions. The universities were also compared according to institution type (i.e., Group of 8 (Go8), the Australian Innovative Research Universities (IRU), the Australian University Technology Network (ATN), the Regional Universities Network (RUN), and the un‐aligned Universities across years. The final analysis examined the performance of the institutions on the engagement and impact assessment (EI) and compared this with the data for the research publications. The universities were also compared with international benchmarks as gathered from the international ratings agencies.ResultsResults of the analyses revealed an overall improvement in research ratings by the universities, with most improving across all four data collections. The performance improved significantly in the 2018 data collection in comparison to each of the three previous data collections.ConclusionIt is now the case that 78% of AOUs are performing at or above world standard in psychology according to the ARC's definition of world standard, an outstanding result and double the level seen in 2010. Some of the issues associated with the ERA data collections are discussed, and suggestions are made for improving this process.

The Production and Dissemination of Australian Social Work Scholarship: A Citation Analysis

ABSTRACT Understanding the scholarly impact of social work research can inform strategies to strengthen the profession’s research foundations. This study examined research productivity and academic impact using h-indexes for a sample of 112 Australian social work researchers according to the stage of their career. Annual research output was calculated using Excellence in Research for Australia (ERA) data. The average h-index for all social work researchers in the sample was 6.39 (Scopus) and 12.5 (Google Scholar). Web of Science was not used due to the number of inaccurate researcher profiles. One hundred and thirty-nine publications received 100 or more citations, but there were many never-cited publications. A social work benchmark for the number of publications is around 1.4 publications per year, depending upon a person’s stage of career and workload profile. Citation analysis can supplement peer review in evaluating research quality, but there are disciplinary differences in publication and citation practices, so it is useful to know where social work is positioned. IMPLICATIONS Citation analysis is used to assess research productivity and quality alongside qualitative assessment, such as peer review. It is imperative for researchers to check, correct, and claim Google Scholar, Scopus and Web of Science records, and to link these to their ORCID profile. A citation benchmark for social work would be in the range of h-index 3.2 for early-career researchers to 15.8 for research-focused positions (Scopus) or h-index 6.1–25.0 (Google Scholar).

A Study of Core Competencies for Supporting Roles in Engagement and Impact Assessment in Australia

ABSTRACTIn 2018, Australia will include, for the first time, an engagement and impact (EI) assessment exercise as a companion to the Excellence in Research for Australia (ERA) national framework. The research underpinning this paper sought to identify the core competencies that are crucial to supporting roles within emerging impact and engagement expectations. A content analysis of research support position descriptions and job advertisements from the UK and Australia was conducted and these were mapped to current industry skill audits for library and information science (LIS) professionals. Also discussed are the current knowledge and skills of LIS professionals and what additional competencies they may need to acquire or develop if they are to contribute to institutions and researchers meeting the requirements of an EI assessment exercise. The hard skills required by both UK and Australian EI support roles are aligned to those endorsed and promoted by the LIS industry and are reflected in the Australian Library and Information skills templates. The study points to the viability of using current academic library networks and LIS professionals to meet the demands of future EI assessment in Australia.

Understanding research strategies to improve ERA performance in Australian universities: circumventing secrecy to achieve success

ABSTRACTMany Australian universities have prioritised improving discipline performance on the national research assessment – Excellence for Research in Australia. However, a culture of secrecy pervades Excellence in Research for Australia (ERA). There are no specified criteria for the assignment of ratings on a 5-point scale ranging from ‘well above world standard’ (5) to ‘well below world standard’ (1). No rationale is provided to institutions for their discipline ratings and university staff on the ERA panels sign confidentiality agreements. However, what is available to universities are the research strategies that each university documents to improve its ERA performance in its Mission-based Compact, a government funding agreement. Thus, the purpose of this paper is to investigate the similarities and differences in the research strategies that universities with different performance profiles employ. Following an analysis of the strategies, substantial commonality was identified in strategy use. However, what was different was how universities employed these strategies and the associated contexts.

Excellence in Research in Australia 2010, 2012, and 2015: The Rising of the Curate's Soufflé?

ObjectiveThe Excellence in Research for Australia (ERA) data collections completed in 2010, 2012, and 2015 were developed by the Australian Research Council to identify the quality of research produced by the broad range of Australian tertiary education institutions. In each evaluation, the quality of research produced by the institutions was rated on a 5‐point scale ranging from “well above world standard” to “well below world standard” within each field of research (FoR). The FoRs relevant to psychology include codes 1701 (Psychology), 1702 (Cognitive Sciences), and 17 (Psychology and Other Cognitive Sciences), a combination of the aforementioned codes.MethodThis analysis examined and compared the ratings of the 41 Australian universities across the years 2010, 2012, and 2015 for the psychology codes (i.e., 17, 1701, and 1702), as well as examining the combined means for all three data collections. The universities were also compared according to institution type (i.e., Group of 8, the Australian Innovative Research Universities, the Australian University Technology Network, and the Regional Universities Network across years. The final analysis compared other codes relevant to psychology across the years 2010, 2012, and 2015.ResultsResults of the analyses revealed an overall improvement in research ratings by the universities, with most improving or at least holding their ground.ConclusionIt is still concerning that almost 40% of the institutions did not meet the benchmark of at or above world standard. Some of the issues associated with the ERA data collections are discussed, and suggestions are made for improving this process.

Exploiting Data Reliability and Fuzzy Clustering for Journal Ranking

Journal impact indicators are widely accepted as possible measurements of academic journal quality. However, much debate has recently surrounded their use, and alternative journal impact evaluation techniques are desirable. Aggregation of multiple indicators offers a promising method to produce a more robust ranking result, avoiding the possible bias caused by the use of a single impact indicator. In this paper, fuzzy aggregation and fuzzy clustering, especially the ordered weighted averaging (OWA) operators are exploited to aggregate the quality scores of academic journals that are obtained from different impact indicators. Also, a novel method for linguistic term-based fuzzy cluster grouping is proposed to rank academic journals. The paper allows for the construction of distinctive fuzzy clusters of academic journals on the basis of their performance with respect to different journal impact indicators, which may be subsequently combined via the use of the OWA operators. Journals are ranked in relation to their memberships in the resulting combined fuzzy clusters. In particular, the nearest-neighbor guided aggregation operators are adopted to characterize the reliability of the indicators, and the fuzzy clustering mechanism is utilized to enhance the interpretability of the underlying ranking procedure. The ranking results of academic journals from six subjects are systematically compared with the outlet ranking used by the Excellence in Research for Australia, demonstrating the significant potential of the proposed approach.

Assessing the performance of educational research in Australian universities: an alternative perspective

ABSTRACTThis study uses bibliometric data to assess the performance of educational research in Australian universities. It provides an alternative perspective to the Australian government’s Excellence in Research for Australia (ERA) assessment. ERA results suggest that the performance of educational research is substantially less compared to other disciplines, with only a handful of universities performing ‘above world standard’. I use a range of indicators that capture citation impact and journal quality from Elsevier’s SciVal research benchmarking tool to provide an alternative perspective that is authentic, transparent, comprehensive and multi-dimensional. The findings show that most Australian universities are performing above the world average in educational research. Australian universities perform especially well on the citation indicators, with more than 75% of universities performing above the world average on all three indicators. Performance on ERA is only moderately correlated with research performance in my analysis but is strongly related to one indicator, publication volume. This finding suggests that ERA assessments favour large entities and disadvantage smaller ones. The findings also suggest that the current ERA peer-review process may not accurately or fully reflect the performance of educational research in Australia. I anticipate that this alternative, bibliometric-based assessment will be a useful benchmarking tool for a range of purposes. I conclude with recommendations for future research.

A Time For Renewal

The Telecommunications Association has commenced the second phase of the renewal process that started in 2013. As part of the first phase of this renewal process a key decision was to relaunch the Telecommunications Journal of Australia as the Australian Journal of Telecommunications and the Digital Economy. In addition to the Journal, the Telecommunications Association holds the Henry Sutton and Charles Todd Orations each year in Melb0urne and Sydney respectively. As the second phase progresses, the Telecommunications Association will launch a new brand, update the Association’s website and host the first of what should become an annual two-day telecommunications forum. The first event is to be held in Melbourne in November. For the Journal, key milestones have now been achieved, including being added to the SCOPUS list of indexed Journals and the Australian Research Council’s Excellence in Research for Australia, which is the national research evaluation framework.

A Time For Renewal

The Telecommunications Association has commenced the second phase of the renewal process that started in 2013. As part of the first phase of this renewal process a key decision was to relaunch the Telecommunications Journal of Australia as the Australian Journal of Telecommunications and the Digital Economy. In addition to the Journal, the Telecommunications Association holds the Henry Sutton and Charles Todd Orations each year in Melb0urne and Sydney respectively. As the second phase progresses, the Telecommunications Association will launch a new brand, update the Association’s website and host the first of what should become an annual two-day telecommunications forum. The first event is to be held in Melbourne in November. For the Journal, key milestones have now been achieved, including being added to the SCOPUS list of indexed Journals and the Australian Research Council’s Excellence in Research for Australia, which is the national research evaluation framework.

Research Support in Australian Academic Libraries: Services, Resources, and Relationships

ABSTRACTIn the last decade Australian academic libraries have increasingly aligned their research support services with assessment criteria used in the national research evaluation exercise (Excellence for Research in Australia). The same period has seen growing interest in research impact outside of traditional measures, such as bibliometrics. Social media has provided opportunities for research dissemination and new tools, altmetrics, to measure these activities have emerged. This article reports on research into the extent and nature of research support services at Australian academic libraries, how the services are managed, and the factors that influence their development and delivery. Quantitative and qualitative research methods were used to compare the findings with an earlier study and to provide a deeper understanding of research support in Australia. Three key themes, services, staff and resourcing, and relationships, are discussed in relation to the management and challenges faced in providing research support.

Supporting Researchers and Institutions in Engagement and Impact Assessment: A Gap Analysis Concerning the Core Skills of LIS Professionals in Australia

Academic institutions have been evaluating research for many years. There has been a focus towards measuring and demonstrating research excellence in Australia with the Excellence in Research for Australia (ERA) replacing the Research Quality Framework in 2007. Since then Australia have moved to evaluate not just the quality of research but its impact beyond academia. The aim of this study is to investigate the skills needed by information professionals to guide researchers and universities in fulfilling their requirements of the EI Assessment that will be implemented alongside ERA 2018. This study will involve an inductive framework by using multiple analytical techniques. It will gather job adverts and role descriptions which are associated with research support in engagement and impact assessment. Through thematic coding, a content analysis will be conducted, identifying the core competencies required in these roles. This will lead to a gap analysis / mapping exercise to understand the relationship between expected and highly regarded core competencies required by employers to fulfil supporting roles in engagement and impact assessment, and the current level of skill expected by industry body the Australian Library and Information Association, via their research focused skills audit template.

The Research Excellence Framework and the Stern Review – Implications for Australia’s Engagement and Impact Assessment Framework

This article explores the advancement of a national research impact assessment framework for Australia. It focuses on the historical development of an Australian research impact agenda inspired by neoliberal ideologies, which began with the conservative Howard Government in the late 1990s and continues today. The article will also discuss the findings of the UK’s Independent Review of the Research Excellence Framework (REF) by Lord Nicholas Stern and how the Australian Government, in the development of a research engagement and impact assessment framework, could draw on and learn from the findings conveyed in this report. Finally, the article suggests ways forward for Australia as it approaches its 2017 research impact pilot and 2018 full research impact assessment alongside the 2018 Excellence in Research for Australia (ERA).

Accounting for Research: Academic Responses to Research Performance Demands in an Australian University

This study examines the perceptions of individual academics about performance management systems (PMS) developed by an Australian university to meet government research assessment requirements. Using a case study method, the research examines the period 2006–2010 within a university, relying on academics’ responses to a survey on the effect of research PMS, contextualised with publicly available organisational documents. The case study university performed well in the Australian Government's first research assessment exercise, Excellence in Research for Australia (ERA) 2010. However, academics reported increasing levels of stress and decreasing job satisfaction, consistent with research that identifies the commodification of academic research. This process of commodification has occurred as a result of the implementation of PMS designed to assess academics’ research‐focused performance and thereby strengthen the university's performance under ERA. In investigating in detail the responses of individual academics to Australia's research assessment initiative, the paper reveals a disconnect between the macro‐institutional demands placed on the higher education sector, university changes made to accommodate these demands, and the ability of academics to meet these demands in a sustainable way.

Policy documents as sources for measuring societal impact: how often is climate change research mentioned in policy-related documents?

In the current UK Research Excellence Framework (REF) and the Excellence in Research for Australia (ERA), societal impact measurements are inherent parts of the national evaluation systems. In this study, we deal with a relatively new form of societal impact measurements. Recently, Altmetric—a start-up providing publication level metrics—started to make data for publications available which have been mentioned in policy documents. We regard this data source as an interesting possibility to specifically measure the (societal) impact of research. Using a comprehensive dataset with publications on climate change as an example, we study the usefulness of the new data source for impact measurement. Only 1.2 % (n = 2341) out of 191,276 publications on climate change in the dataset have at least one policy mention. We further reveal that papers published in Nature and Science as well as from the areas “Earth and related environmental sciences” and “Social and economic geography” are especially relevant in the policy context. Given the low coverage of the climate change literature in policy documents, this study can be only a first attempt to study this new source of altmetrics data. Further empirical studies are necessary, because mentions in policy documents are of special interest in the use of altmetrics data for measuring target-oriented the broader impact of research.

Evaluating Research Impact: The Development of a Research for Impact Tool.

This paper examines the process of developing a Research for Impact Tool in the contexts of general fiscal constraint, increased competition for funding, perennial concerns about the over-researching of Aboriginal and Torres Strait Islander issues without demonstrable benefits as well as conceptual and methodological difficulties of evaluating research impact. The aim is to highlight the challenges and opportunities involved in evaluating research impact to serve as resource for potential users of the research for impact tool and others interested in assessing the impact of research. A combination of literature reviews, workshops with researchers, and reflections by project team members and partners using participatory snowball techniques. Assessing research impact is perceived to be difficult, akin to the so-called "wicked problem," but not impossible. Heuristic and collaborative approach to research that takes the expectations of research users, research participants and the funders of research offers a pragmatic solution to evaluating research impact. The logic of the proposed Research for Impact Tool is based on the understanding that the value of research is to create evidence and/or products to support smarter decisions so as to improve the human condition. Research is, therefore, of limited value unless the evidence created is used to make smarter decisions for the betterment of society. A practical way of approaching research impact is, therefore, to start with the decisions confronting decision makers whether they are government policymakers, industry, professional practitioners, or households and the extent to which the research supports them to make smarter policy and practice decisions and the knock-on consequences of doing so. Embedded at each step in the impact planning and tracking process is the need for appropriate mix of expertise, capacity enhancement, and collaborative participatory learning-by-doing approaches. The tool was developed in the context of Aboriginal and Torres Strait Islander research but the basic idea that the way to assess research impact is to start upfront with the information needs of decisions makers is equally applicable to research in other settings, both applied (horizontal) and basic (vertical) research. The tool will be further tested and evaluated with researchers over the next 2 years (2016/17). The decision by the Australian Government to include 'industry engagement' and 'impact' as additions to the Excellence in Research for Australia (ERA) quality measures from 2018 makes the Research for Impact Tool a timely development. The wider challenge is to engage with major Australian research funding agencies to ensure consistent alignment and approaches across research users, communities, and funders in evaluating impact.

The work of the teacher-educator in Australia: Reconstructing the “superhero” performer/academic in an audit culture

ABSTRACTWhat is it that teacher-educators “do”? This paper draws on interview data with Deans/Heads of Schools of Education in the Australian context to explore this question by asking: How is the teacher-educator produced as a category of academic worker? Using critical approaches to discourse analysis, the paper presents two interlocked storylines woven with varying emphasis through the interviews. First, the teacher-educator is produced as a superhero researcher and teacher, elevated by the expectations of the Excellence in Research for Australia audit/surveillance tools. Second, there is a concomitant struggle to reconcile pressure to research with commitment to meeting the needs of schooling systems, and to addressing the work of the teacher-educator in ethical terms. It is suggested that teacher-educators’ work reflects a desire to legitimate various performances of teacher education within current dynamics of audit culture pressures, set against the less visible ethical project of education more broadly conceived.

Embodying pre-tense conditions for research among teacher educators in the Australian university sector: a Bourdieusian analysis of ethico-emotive suffering

ABSTRACTInterviews conducted as part of the Work of Teacher Education (WoTE) project in Australia highlight emotional and ethical suffering embodied by teacher educators who find their research aspirations thwarted in the context of high-stakes research assessment exercises. We argue that government-run assessments, such as Excellence in Research for Australia, and localised institutional strategies developed in response, provoke “pre-tense” conditions that unsettle institutions of the Australian university sector regarding future claims for research status. Drawing on interviews with an early- and a mid-career teacher educator, both of whom evidence significant research aspirations, we portray and analyse their ethico-emotive sufferings, linked to contemporary pre-tense conditions in which they work, which thwart their dispositions to do research. We conclude by reflecting on the need for systemic response within the field of teacher education to ensure its research future, including an ethico-emotive politics that mobilises across generations of academics, with particular responsibilities for senior researchers.

Incoming Editor-in-Chief Note

I am honored to be the new editor-in-chief of ASCE’s Journal of Management in Engineering (JME). I would like to thank the search committee for entrusting me with this opportunity. A special thanks goes to the outgoing editor-in-chief, Professor Makarand Hastak at Purdue University, who has demonstrated great leadership for the last seven years to achieve significant milestones for JME. I have been a faculty member at George Washington University School of Business in Washington, DC, since 1999. I joined the JME editorial board in 2008 upon invitation from Professor Irtishad Ahmad at Florida International University, who was the editor-in-chief at the time. During the last couple of years, I have served as an associate editor for JME. In addition to JME, I serve as a specialty editor for the case studies section of ASCE’s Journal of Construction Engineering and Management. I also serve on the editorial boards of IEEE’s Transactions on Engineering Management and International Journal of Project Management, the leading journals in engineering management and project management. Therefore, I am very familiar with the recent research trends in our field. JME has its competitive advantages as one of the few management in engineering journals that are recognized by the community of scholars. It receives a respectable impact factor from the Thomson Reuters journal-citation reports, is listed as a wellregarded journal in the United Kingdom’s Association of Business Schools (ABS) academic journal guide for business and management academics, and was ranked an A* journal by the Australian Research Council’s Excellence in Research for Australia (ERA), just to name a few. I believe that there is room for improvement to raise the visibility and reputation of JME by attracting new readers and contributors who conduct management in engineering research that incorporates economics, technology, innovation, decision sciences, psychology, social and behavioral sciences, leadership, organizational theory, etc., into civil engineering, construction, infrastructure, and the built environment. I am committed to bringing JME to the next level, as the management in engineering field is maturing into an interdisciplinary, vibrant, and dynamic discipline. Exposing JME to various disciplines will bring new insight and great appreciation to the growing field, and I would like to be a facilitator to introduce the latest trends, challenges, and opportunities. My ultimate goal is to see JME become a preferred journal by researchers and practitioners to acquire and disseminate essential ideas, theories, knowledge, and practices in management in engineering. I will work closely with associate editors, editorial board members, and the ASCE journal publications staff to make this possible. Most importantly, I will seek your continued interest, support, and commitment. I look forward to working with all of you to receive high-quality papers as an author and constructive feedback as a reviewer. If you have any suggestions or comments regarding JME, please do not hesitate to contact me. Thank you very much.