Higher Education Evaluation Research Articles

The impact of integrating technology into students’ presentations on peer evaluation in higher education

This study investigated the impact of technology in presentations on students’ perception of quality. Students peer reviewed presentations and two external raters evaluated the presentations based on a rubric adapted from Savory (2009). Students reviewed activity using two assessment instruments: a seven-point attitudinal scale and a 1–5 ranking scale. The study utilized a mixed-methods, embedded QUAN:qual design, where statistical analysis of Pearson Correlation coefficient was paired with qualitative description to discuss the data gathered. The findings showed that students’ scores on the attitudinal scale and their holistic rankings correlated positively with the degree of technology employed in the presentations. The greater the integration of technology in a presentation, the higher the peer rating. However, the external raters’ evaluations did not generally accord with the student-raters’.

Barriers in teacher perception about the use of technology for evaluation in Higher Education

This article addresses perceptions that higher education teachers have on the integration of technology in evaluation processes, focusing on their beliefs about learning, evaluation and technology that enable or hinder this pedagogical innovation.This study considered teachers as part of a project using rubrics in Moodle to evaluate learning in a Chilean higher education institution in 2014, 2015 and 2016.Forming and evaluating in a competency-based approach involves challenges for the teaching staff and changes in their role that are accepted or resisted. The same happens in the face of innovation processes with information and communication technologies (ICT) integration in teaching practices.The primary results show a relationship between pedagogical beliefs and evaluation which is consistent with adoption and assessment of the digital tool used. In addition, it reveals the role of beliefs as secondary barriers to change in the face of access or primary barriers (Ertmert, 1999). The value of autonomy in student work and feedback are identified as key beliefs in technology adoption.

Shortcomings of Higher Education Evaluation Systems

Higher Education Evaluation Systems supply information for diverse stakeholders. A “one size fits all” approach in university rankings is not enough. Looking to the future, evaluation may need to take into account criteria such as links with employers, lifelong education, implications of digitization, and interdisciplinary and interinstitutional collaboration across borders. The extensive possibilities of today’s research data based analyses are analysed, against the background of a whole industry devoted to this. Shortcomings, challenges and unintended consequences of the current approach are discussed. Impact analyses are seen as one of the ways forward, taking into account contributions to societies and their transformations. Diversity, “glocal” mindset and international collaboration are suggested as additional criteria for the competitive rankings of the future.

Beyond Performance Indicators: Reforming Higher Education Evaluation Systems to Promote Improvement

As higher education attainment has become increasingly essential for both individual socioeconomic outcomes and the economic competitiveness of nation-states, and as the cost of financing the higher education enterprise continues to rise, university quality has become an urgent concern for students, families, and policy makers around the globe. The widespread interest in assessing university quality manifests itself in the rise of global rankings (Hazelkorn, 2015) and the increasing use of so-called performance indicators by government agencies. This paper focuses on the latter phenomenon. The first part of the paper examines the benefits and limitations of higher education performance indicators as conventionally implemented, and the second part advances a set of suggestions to address these shortcomings by adapting performance systems to represent and incentivize evidence-informed improvement efforts.

Escala de Motivação Académica: Validação no Ensino Superior Público Português

Resumo Esta investigação teve como objetivo o estudo das propriedades psicométricas e posterior validação da Escala de Motivação Académica proposta por Guimarães e Bzuneck (2008), em Portugal, através da utilização de duas amostras, a Amostra A constituída por 568 estudantes do ensino superior público e a Amostra B constituída por 589 estudantes igualmente do ensino superior publico, com uma abrangência nacional. Para tratamento dos dados foram utilizados os softwares SPSS 25 e AMOS 25. Foram conduzidas duas análises fatoriais confirmatórias em cada amostra e uma análise de grupos múltiplos, sendo que os resultados apontaram para a validade da escala reespecificada na avaliação da motivação no ensino superior e sua robustez, com destaque para correlações elevadas entre as dimensões Motivação Extrínseca por Regulação Integrada e Motivação Intrínseca, Motivação Extrínseca, por Regulação Integrada e Motivação Extrínseca por Regulação Introjetada e por Motivação Extrínseca por Regulação Externa e Desmotivação. Por outro lado, verificou-se que o modelo de escala proposto é invariante.

Exame Nacional de Desempenho de Estudantes

O Exame Nacional de Desempenho de Estudantes (ENADE) integra o Sistema Nacional de Avaliação da Educação Superior (SINAES) aferindo o rendimento dos alunos quanto às Diretrizes Curriculares Nacionais. O objetivo deste trabalho foi analisar a opinião de professores dos Cursos de Farmácia de Instituições públicas e privadas do Estado do Rio Grande do Sul, em relação ao ENADE. Através do sistema Qualtrics, 168 docentes participaram dos questionários, com retorno submetido à Análise Textual Discursiva. As respostas revelaram que 28% dos professores (47/168) apresentam opinião favorável ao método avaliativo; 45,2% (76/168) opinião moderada; 14,3% (24/168) indiferente e 12,5% (21/168) desfavorável, cuja abordagem, principal, ao aluno, mencionada por 22% (37/168) dos professores. Considerando que o sistema está em construção e assim deve permanecer para manter-se atual através dos tempos, a opinião dos envolvidos e constante monitoramento é crucial para que o instrumento de aferição efetivamente promova a qualificação constante da educação superior.

O status de bem público e democrático nas avaliações nacionais

Este artigo é resultado de uma elaboração teórico-crítica e tem o objetivo de analisar os dois principais instrumentos de avaliação da educação brasileira: o Sistema de Avaliação da Educação Básica e o Sistema Nacional de Avaliação da Educação Superior no tocante a princípios que privilegiem a participação democrática e com status de bem público. Como resultado depreende-se que, embora se busque a participação de diferentes atores da instituição no intuito de melhorias para o estabelecimento educacional, ainda é forte a presença de conceitos mercadológicos nas avaliações institucionais que silenciam e não retratam a qualidade da educação ofertada pela instituição.

XP Model for Information Systems Recording Academic Data

Fulfillment of academic information to support a decision in a college is very necessary, especially to support an academic planning and evaluation in higher education. Often management asks the student academic bureau to find out the number of students based on their class, period, number of graduates, class schedules to student grades. This encourages the student academic bureau to create a desktop-based application that can be used to find out more quickly the needs requested by management. By using the extreme programming (XP) model, it can accommodate the academic information needs of student affairs in universities in the form of a recap of academic data that can be presented in real time and quickly for the upper level management.

Assessment & Evaluation in Higher Education Using the margin of error statistic to examine the effects of aggregating student evaluations of teaching

We proposed an extended form of the Govindarajulu and Barnett margin of error (MOE) equation and used it with an analysis of variance experimental design to examine the effects of aggregating student evaluations of teaching (SET) ratings on the MOE statistic. The interpretative validity of SET ratings can be questioned when the number of students enrolled in a course is low or when the response rate is low. A possible method of improving interpretative validity is to aggregate SET ratings data from two or more courses taught by the same instructor. Based on non-parametric comparisons of the generated MOE, we found that aggregating course evaluation data from two courses reduced the MOE in most cases. However, significant improvement was only achieved when combining course evaluation data for the same instructor for the same course. Significance did not hold when combining data from different courses. We discuss the implications of our findings and provide recommendations for practice.

Developing a Service Quality Framework for a Special Type of Course

This paper addresses the issue of service quality measurement and evaluation in higher education and stresses the need to develop sound measures for special types of courses. During these courses students carry out project works under special circumstances and with special characteristics compared to "ordinary" courses where traditional course evaluation methods have been applied for a long time. The primary aim of the paper is to support the need to develop valid, reliable and replicable measures of service quality in case of these courses. Therefore, a questionnaire was designed for these courses to collect students' perceptions. The results are reported by using an importance-performance analysis supplemented by the draw of statistical conclusions. The presented methodology allows the identification of importance-performance gaps and supports the assessment of quality improvement programs.

Analysis on Performance Management of Financial Funds in China's Higher Education

As an effective educational management tool, budget performance management in colleges and universities has become an important basis and management means for allocating educational resources and formulating corresponding educational policies. It plays an important role in promoting colleges and universities to improve their management level, cultivate outstanding talents and serve social development. In recent years, with the rapid development of higher education in our country, the financial expenditure in the field of education has gradually increased, the government departments have continuously increased their investment in financial education funds, and the state's investment in higher education has also increased year by year. In addition, with the reform of the "performance-oriented" financial expenditure management system, the relevant departments in China have also strengthened the performance management of colleges and universities. This paper sorts out the practice of financial fund performance evaluation in higher education at central and local levels in China since 1988, summarizes the experience and practice of financial fund performance evaluation in higher education in China, and thinks that further improvement is needed in legal system, evaluation consciousness, evaluation scope, evaluation index, evaluation subject, evaluation content and application of evaluation results, hoping to have certain reference value for improving the financial fund performance management in higher education in China.

The Plight of Course Evaluations in Higher Education

The Department ChairVolume 29, Issue 3 p. 24-25 Articles The Plight of Course Evaluations in Higher Education Pamela D. Wash, Pamela D. WashSearch for more papers by this authorAlan Davis, Alan DavisSearch for more papers by this author Pamela D. Wash, Pamela D. WashSearch for more papers by this authorAlan Davis, Alan DavisSearch for more papers by this author First published: 09 January 2019 https://doi.org/10.1002/dch.30239Citations: 1Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat No abstract is available for this article.Citing Literature Volume29, Issue3Winter 2019Pages 24-25 RelatedInformation

Research on the Avoidance of Moral Hazard in Higher Education Evaluation of China Under Multi-agent Mode

The education evaluation agencies will undertake education evaluation task and will form principal-agent relationships with the governments with the separation reform of supervision, running and evaluation in China. Because of bounded rationality and information asymmetry, it was already a big problem to avoid the moral hazard of an education evaluation agency in a principal-agent relationship involving only one principal and one agent. In reality, the principal-agent relationship is often more complicated. Of particular concern is that the situation of “one principal–multiple agents” in the evaluation of higher education has emerged. The coexistence of “multiple” education evaluation agencies, such as the official agencies, the semi-official agencies, the agencies affiliated with the university, and the third-party agencies, has taken shape. By using critical discourse analysis, this study attempts to argue that avoiding the moral hazard of higher education evaluation under the “multi-agent” mode becomes a research issue that must be squarely addressed. The paper concluded that in “multi-agent” mode, although the possibility of implied moral hazard in higher education evaluation increases, if the “relative performance” evaluation mechanism can be understood and effective measures can be taken, not only will it not lead to the consequences of moral hazard being more difficult to avoid, but it will greatly enhance the effectiveness of the avoidance of moral hazard. The conclusion has important reference significance on how to effectively use the “multi-agent” mode to avoid the moral hazard of higher education evaluation.

Organization of monitoring and evaluation of higher education in the United States

Organization of monitoring and evaluation of higher education in the United States

Arabic Language Learning Evaluation in Higher Education with Context Input Process Product (CIPP) Model

Evaluation of Arabic language learning in higher education is needed for quality assurance of a learning system. The evaluation should not assess the mere students’ learning output, but involve all aspects of learning program including objectives, planning, implementation and output. This study aims to uncover the implementation of evaluation of Arabic learning program in higher education with the CIPP model, from the emergence of the idea to the result achieved after the program is implemented. It is a theoretical study, which is not based on empirical data in the field. The application of the CIPP model in evaluating Arabic language learning in higher education is conducted by collecting data and information about context, input, process and products of a learning program comprehensively and continuously. More specifically, context evaluation is directed to assess data concerning the purpose and background of a learning program. Input evaluation is conducted to examine the accuracy of the curriculum, students, lecturers and infrastructures. Meanwhile, process evaluation is carried out to reveal the suitability of the curriculum, teaching materials and media in a learning activity. Product evaluation is intended to provide data and information about the outputs and outcomes of Arabic language learning.

Towards a Definition of Environmental Sustainability Evaluation in Higher Education

Sustainability is increasing their presence at universities, so it is convenient to reflect on the impact and effectiveness that university sustainability actions are having. Several authors have recognized mature experiences about environmental sustainability in the different dimensions of higher education: teaching, research, operations and outreach. Assessment of university sustainability is an emerging field of research of Education for Sustainable Development in Higher Education, because of the use by universities of assessment tools to improve the performance of its policies, but also to disseminate their results. This paper will try to define what is meant by ‘assessment of university environmental sustainability’ based on different evaluation approaches found in an integrative meta-analysis of specialized literature on the subject and review of assessment tools. While the most common evaluative approach is the self-assessment, to improve the implementation of policies, other approaches aimed at promoting university activity in sustainability through its participation in rankings or accreditation system increasingly are becoming greater presence. This leads to identifying a particular concern among universities to compete and appear in the university context as ‘sustainable’ without ensuring that their actions are being designed really to improve sustainability, at a university and global context.

Dynamic Quality Evaluation in Higher Education

The paper aims to propose, investigate and test suitable means for automation of the processes for dynamic quality evaluation of objects in a given subject area. On the basis of the theoretical study a number of conceptual and computational models are proposed. In consequence, the architecture of a software system for dynamic quality evaluation is defined and a corresponding software prototype is built over an existing university information infrast-ructure. General models are applied for dynamic quality evaluation of different objects in the field of higher education.

Evaluating learning outcomes of an asynchronous online discussion assignment: a post-priori content analysis

ABSTRACTUse of learning management systems is prevalent across the continuum of education formats (online, blended, face-to-face). Specific asynchronous tools such as the discussion board are effective for student–instructor and student–student communication [Calderon, Ginsberg, and Ciabocchi (2012); Jorczak, R. L., & Dupuis, D. N. (2014). Differences in classroom versus online exam performance due to asynchronous discussion. Journal of Asynchronous Learning Networks, 18(2), 67–75; Lane, L. M. (2014). Constructing the past online: Discussion board as history Lab. History Teacher, 47(2), 197–207. Retrieved from https://eric.ed.gov/?redir=http://www.societyforhistoryeducation.org%2fpdfs%2fF14_Lane.pdf]. However, designing learning assignments that optimize the educational value of web-based tools requires a careful consideration of strategies to evaluate students’ learning outcomes. To date, there is a lack of systematic analysis that focuses on identifying learning evaluation dimensions beyond the traditional strategies to evaluate written assignments or face-to-face communication. Previous work includes a-priori parameters or instrumental dimensions such as critical thinking [Lai, K. (2012). Assessing participation skills: Online discussions with peers. Assessment & Evaluation in Higher Education, 37(8), 933–947. doi:10.1080/02602938.2011.590878] or length of communication [Brooks, C. F., & Bippus, A. M. (2012). Underscoring the social nature of classrooms by examining the amount of virtual talk across online and blended college courses. European Journal of Open, Distance and E-Learning. Retrieved from http://www.eurodl.org/materials/contrib/2012/Brooks_Bippus.pdf]. This study utilized a post- priori analysis of students’ responses to an online discussion board assignment in a graduate course to identify organically emerging indicators of learning that occur when completing such assignments, thus capturing learning outcomes that are unique to an online group discussion assignment. Three dimensions of learning emerged: contextual (accuracy and sophistication of response content); inter-personal communication (quality of peer-to-peer interaction); meta-learning (reflections on the assignment and the learning process). These can serve as parameters for evaluating educational outcomes unique to the online environment.

Fases de submissão do processo de reconhecimento de cursos superiores: o caso do Instituto Federal Farroupilha

Este artigo tem por objetivo apresentar as fases do processo para a solicitação de reconhecimento dos Cursos Superiores de Tecnologia em Gestão Pública do Instituto Federal Farroupilha, considerando as normativas da plataforma do e-MEC. Foi realizada uma investigação do tipo estudo de caso com entrevista semiestruturada, subsidiada pela fundamentação teórica dos autores: Barreyro e Rothen (2008), Ristoff (1997), legislação específica, entre outros. Constatou-se que os programas implantados pelo MEC, em cada momento propício, tiveram contribuições importantes e positivas de acordo com o contexto da época de sua vigência. Verificou-se, a respeito das questões sobre o processo de avaliação de cursos, que o sistema tem como prioridade a transparência para a melhoria contínua do processo. No entanto, existem dificuldades de inserção de dados na plataforma e-MEC, bem como se destaca a rigidez do processo, referentes aos prazos de cada fase.

Assessment co-creation: an exploratory analysis of opportunities and challenges based on student and instructor perspectives

ABSTRACTIn recent years, research and practice focused on academics and students working in partnership to co-design learning and teaching in higher education has increased [Deeley and Bovill 2017. ‘Staff student partnership in assessment: enhancing assessment literacy through democratic practices.’ Assessment & Evaluation in Higher Education 42 (3): 463–477. doi:10.1080/02602938.2015.1126551]. Student generation of content has been advocated as a means of fostering deep learning and high levels of students engagement, leading to enhanced conceptual understanding [Draper 2009. ‘Catalytic Assessment: Understanding how MCQs and EVS can Foster Deep Learning.’ British Journal of Educational Technology 40 (2): 285–293.]. However, within the discourse exploring staff-student co-creation, some concerns have been raised about whether students ought to, or indeed can, meaningfully act as partners in assessment [Cook-Sather, Bovill, and Felten 2014. Engaging Students as Partners in Learning and Teaching. San Francisco, CA: Jossey Bass.]. This paper describes the introduction of a co-creation assignment into an undergraduate tax module. Students’ and instructors’ perceptions of several aspects of the assignment were examined, facilitating the cataloguing and elucidation of the tensions involved in assessment co-creation. The paper suggests techniques to manage these challenges effectively, thereby providing a case study for others with an interest in implementing assessment co-creation within their own disciplinary context.