Evaluation Of Educational Software Research Articles

Evaluation of modern educational software and basic digital competences among teachers in Italy

ABSTRACT The aim of this paper is to evaluate the usefulness of educational software among teachers in Italy (N = 559). The evaluation of usefulness was conducted on the latest and most popular web-based software used by European teachers, namely: Quizizz, Mentimeter, Wakelet, Padlet, Canva, Emaze, Answergarden, Jamboard, Coggle, Createley, Wordwall, LearningApps, Prezi, Kahoot, Plickers, Trimino, Dobble, Genial.ly, ClassDojo, Explain Everything, KhanAcademy, and Easelly. Based on data collected in the first half of 2022 using an online survey, it was noted that: 1) The majority of teachers are not able to evaluate modern software because they do not use such teaching resources as those listed; 2) About 9.3% of teachers rate the listed software highly; 3) Most commonly teachers use and also rate highly: Canva, Wordwall, Padlet, LearningApps, Kahoot, Coggle, Jamboard, and Prezi; 4) Seniority and workplace does not always differentiate the evaluation of educational software.

El modelo MOSCA como estrategia para la evaluación de software educativo: estudio de caso SOLVE ELECv2.5

This research work makes an approach to the evaluation of educational software through the case study of the application of the SOLVE ELECv2.5 software tool in higher education; it bases its development on the implementation of its own model based on usability and criteria established in the software evaluation model under a systemic approach to quality (MOSCA), as well as the ISO/IEC standard. It performs the usability evaluation based on the application of instruments designed for technical personnel specialized in software development and actors of the teaching learning process: students and teachers. As a result, the use of the SOLVE ELECv2.5 tool in the subjects related to the area of electrical and electronic circuits is effective for the consolidation of knowledge.

Incorporation of a software engineering evaluation method in an educational software course in Turkey

ABSTRACT The research literature identifies the limitations of pre-service teacher training in the evaluation and assessment of educational software for classroom use. This study investigated the enhancement of IT pre-service teacher training in educational software evaluation through the incorporation of software evaluation techniques from the software engineering discipline. Participants in this study were 25 final year pre-service IT teachers at Faculty of Education at a public university in Turkey. The pre-service teachers were introduced to the software engineering evaluation methods and an instructional educational software evaluation method. The results illustrated that the software engineering evaluation methods enhanced the technical evaluation skills of pre-service teachers and increased their confidence. Pre-service teachers reported that they considered themselves more professional after applying these methods. We believe this work highlighted a method to enhance the educational software evaluation skills of IT pre-service teachers through combining technical evaluation skills with traditional educational methods.

Heuristic Evaluation on Usability of Educational Games: A Systematic Review

This work presents a systematic review whose objective was to identify heuristics applicable to the evaluation of the usability of educational games. Heuristics are usability engineering methods that aim to detect problems in the use of a system during its development and / or when its interface is in interaction with the user. Therefore, applying heuristics is an essential part of developing digital educational games. Search sources were articles available in all the databases present in the Capes / MEC / Brazil periodicals portal, in the available languages. The descriptors adopted were "educational games", "heuristic" and "usability" in Boolean search in titles, abstracts and keywords, with AND operator, for publications starting in 2014. The inclusion criteria were: (a) articles with a clear description of the methodology used in the usability analysis; (b) studies presenting primary data and (c) articles whose focus corresponds to the investigated question. Two examiners conducted the searches in the databases and a third the evaluation and general review of the data. Initially, 93 articles were identified, of which 19 were repeated, 5 were literature reviews. Of the 69 that remained, 57 were elected as not eligible with only 12 selected for full studies, of which 6 entered the final review. With this review we can deduce that the field of heuristics and usability for educational games is still little explored, with few specific evaluations validated or in the process of validation, requiring greater investment in the area. Through this review, we found at least one heuristic that meets the usability evaluation of educational software: Game User Experience Satisfaction Scale (GUESS).

Empirical evaluation of educational interactive systems

Educational software systems are usually highly interactive systems that needs to be evaluated from two different perspectives, first as interactive software, mainly using usability and accessibility approaches, and second as an educational product that needs to be evaluated from its learning impact. The empirical evaluation methods may be quantitative, with a more deductive reasoning, or qualitative, with a more inductive reasoning, oriented, however mixed approaches have been used extensively because they combine the both main methods with a more pragmatic knowledge focusing on knowing the consequences or solving problems. The empirical evaluation of interactive educational software due to its own characteristics requires a mixed approach typically. This special issue is devoted to identify, share and valorise best practices and experiences, with a very solid modelling background, that are focused on the empirical evaluation of educational interactive systems with a special emphasis on mixed quantitative and qualitative approaches.

Utilização e avaliação de software educacional para ensino de equilíbrio químico

A interação entre aluno e conteúdo através da utilização de softwares educacionais pode favorecer uma postura mais ativa na construção do conhecimento, e também, contribuir para a significação do conteúdo científico. Este artigo apresenta os resultados parciais de uma investigação cujo objetivo é a utilização e avaliação do software educacional “The Law of Mass Action” no ensino do Equilíbrio Químico. Através de uma proposta didática constituída por quatro encontros, aplicada a alunos do segundo ano do ensino médio, foi possível identificar as potencialidades e limitações do software enquanto ferramenta educacional. A constituição dos dados conta com observação, registros de áudio e questionários. A análise e interpretação, dá-se a partir dos pressupostos da Análise Textual Discursiva (MORAES, GALIAZZI, 2011) e da Teoria da ação Mediada (WERTSCH, 1999), permitindo identificar graus de domínio e apropriação dos alunos em relação ao equilíbrio químico após interação com o software.

Field Test Evaluation of Educational Software: A Description of One Approach

Educational evaluators in general have traditionally recognized the needto incorporate data from potential users in designing evaluation studies. In the field of courseware evaluation, however, there has been a great deal of emphasis placed on expert judgment as a source of data for evaluating computer-based educational materials. Although courseware reviews are extremely useful, they are not substitutes for field tests;each provides a different type of information that evaluators may use in order to determinethe quality of an instructional product.This paper reports on the evaluation of a courseware designed to assist the writing of the lower-case alphabet. The main objective of the article is to demonstrate an evaluation design which provided adequate answers to our evaluation questions, allowed us to perform multiple comparisons to support our conclusions, and was also practical enough to be used in a normal classroom situation without disturbing everyday activities. Three criteria for selecting a design are presented followed by a description of the courseware evaluation.

Data mining in educational technology classroom research: Can it make a contribution?

The paper addresses and explains some of the key questions about the use of data mining in educational technology classroom research. Two examples of use of data mining techniques, namely, association rules mining and fuzzy representations are presented, from a study conducted in Europe and another in Australia. Both of these studies examine student learning, behaviors, and experiences within computer-supported classroom activities. In the first study, the technique of association rules mining was used to understand better how learners with different cognitive types interacted with a simulation to solve a problem. Association rules mining was found to be a useful method for obtaining reliable data about learners' use of the simulation and their performance with it. The study illustrates how data mining can be used to advance educational software evaluation practices in the field of educational technology. In the second study, the technique of fuzzy representations was employed to inductively explore questionnaire data. The study provides a good example of how educational technologists can use data mining for guiding and monitoring school-based technology integration efforts. Based on the outcomes, the implications of the study are discussed in terms of the need to develop educational data mining tools that can display results, information, explanations, comments, and recommendations in meaningful ways to non-expert users in data mining. Lastly, issues related to data privacy are addressed.

The construction and evaluation of new educational software for nursing diagnoses: a randomized controlled trial

Student nurses often have difficulties with diagnostic inferences. To achieve high accuracy nursing diagnoses, effective learning strategies are required. To describe the construction and evaluation of new educational software called Wise Nurse, which was developed to increase the capacity of student nurses to identify nursing diagnoses (NDx) and to establish relationships between NDx, defining characteristics (DC), and related factors (RF). Randomized controlled trial. Participants were 2nd to 4th year student nurses from an undergraduate program at a university in Brazil. Of the 47 recruited students, 37 completed the survey. Students were randomly assigned to test the software (experimental group) and to solve printed clinical cases (comparison group). A pretest and post-test were applied before and after the experiment. Statistical analyses of the students' performance in the tests were conducted. The primary outcome was the students' progress in solving questions and clinical cases regarding NDx. The System Usability Scale was used to measure the software's ease of use. No significant difference was found between the experimental and comparison groups before and after the experiment. The average students' performance in identifying RF and NDx was higher than in identifying DC. The post-test score was higher than the pretest score in both groups (P=0.022). The usability score was good (average score 83.75, N=20). The use of Wise Nurse supported an improvement in student diagnostic reasoning equivalent to that of the traditional NDx training, but the software stands out as an innovative teaching tool.

The Comprehensive Evaluation of Electronic Learning Tools and Educational Software (CEELTES)

Despite the fact that digital technologies are more and more used in the learning and education process, there is still lack of professional evaluation tools capable of assessing the quality of used digital teaching aids in a comprehensive and objective manner. Construction of the Comprehensive Evaluation of Electronic Learning Tools and Educational Software (CEELTES) tool was preceded by several surveys and knowledge obtained in the course of creation of digital learning and teaching aids and implementation thereof in the teaching process. The evaluation tool as such consists of sets (catalogues) of criteria divided into four separately assessed areas - the area of technical, technological and user attributes; the area of criteria evaluating the content, operation, information structuring and processing; the area of criteria evaluating the information processing in terms of learning, recognition, and education needs; and, finally, the area of criteria evaluating the psychological and pedagogical aspects of a digital product. The specified areas are assessed independently, separately, by a specialist in the given science discipline. The final evaluation of the assessed digital product objectifies (quantifies) the overall rate of appropriateness of inclusion of a particular digital teaching aid in the teaching process.

El discurso del profesorado de Educación Musical en la innovación educativa con TIC: Posicionamientos en la evaluación del software Tactus

This paper presents teachers positioning analysis around the assessment of ad hoc software designed for learning musical rhythm called “Tactus”. The main objective is to study the teachers speeches and opinions in software assessment, which is built through the speeches analysis (n = 115) in 8 focus groups, with a descriptive model of 5 clusters. The study gives us a diverse, varied and a full picture in educational software evaluation trough teachers’ discourses analysis.

Educational Software Evaluation : A Study from an Educational Data Mining Perspective

In this paper the complicated task of educational software evaluation is revisited and examined from a different point of view. By the means of Educational Data Mining (EDM) techniques, in the present study 177 of the most common evaluation standards that have been proposed by various researchers are examined and evaluated with regards to the degree they affect the effectiveness of educational software. More specifically, via the employment of prediction, feature selection and relationship mining techniques we investigate for the underlying rationale hidden within the data collected from experiments conducted at the Department of Education of the University of Patras with regards to the software evaluation task and the results of this study are presented and discussed in a quantitative and qualitative way.

An Evaluation of the Educational Software about Precipitation Titrations

The aim of study is to evaluate the views of student chemistry teachers about an education software concerning precipitation titrations. The participants are 13 student chemistry teachers. The study employs the method of survey and the data were collected through the use of “the scale of educational software evaluation” developed by Kara (2007). The reliability and validity study of the scale was also carried out. The scale has a total of five dimensions, namely content, easiness of the use, technical specifications, educational specifications and personal views. After the analysis of the data, it is observed that, computer-assisted teaching material has an important effect on understanding the subjects.

Standardization of Quality Evaluation of Educational Software and Electronic Learning Tools—Analysis of Opinions of Selected Experts

Our research was focused on the identification of features, which was essential for educational digital products and the determination of their quality. The introductory analytical part of our research is focused on the analysis of existing sources of information related to the problems of research, production, appropriate use and evaluation of educational software environments. Consequently, we have divided the existing software products into three basic groups according to our main distinguishing feature. Second part of our paper is focused on various aspects, which are to be considered when assessing the quality of software solutions. The final part contains the presentation of results of our findings related to the most important features expected and required from digital learning tools by professional experts and specialists in given field.

Understanding expertise-based training effects on the software evaluation process of Mathematics Education teachers

This case study examined the educational software evaluation processes of pre-service teachers who attended either expertise-based training (XBT) or traditional training in conjunction with a Software-Evaluation checklist. Forty-three mathematics teacher candidates and three experts participated in the study. All participants evaluated educational software selected by the novice groups in accordance with Software Evaluation checklist. Two analysis procedures were applied: (a) similarities in rankings of the same educational software between the XBT and traditional groups using the checklist and (b) organizing common themes in the data collected through journals and in-class observations. The results showed that the expert-XBT group agreement rate was 63%, while the expert-traditional group agreement rate was 42%. Moreover, the XBT group showed a similar approach to the experts, including assessing each criterion and evaluating the software as a whole.

Differences in the educational software evaluation process for experts and novice students

<span>This comparative case study investigated the educational software evaluation processes of both experts and novices in conjunction with a software evaluation checklist. Twenty novice elementary education students, divided into groups of five, and three experts participated. Each novice group and the three experts evaluated educational software selected by the novice groups in accordance with the checklist. Data were collected through focus group interviews, classroom observations and document analysis. Evaluation processes were analysed through thematic comparisons. The results showed that the expert-novice agreement rate was as low as 48%, with novice students tending to grade the software higher. While the experts used a systematic approach, including understanding and assessing each criterion, supporting the process with literature and evaluating the software as a whole, novice students lacked such methods, indicating a need for additional training and development.</span><br />

Development of Usability Criteria for E-Learning Content Development Software.

Revolutionary advancements have been observed in e-learning technologies though an amalgamated evaluation methodology for new generation e-learning content development tools is not available. The evaluation of educational software for online use must consider its usability and as well as its pedagogic effectiveness. This study is a first step towards the definition of criteria for evaluating e-learning tools. A preliminary user study involving a group of pre-service instructional designers, observed during their interaction with e-learning tools, is reported. Throughout the study, specific usability attributes of these e-learning tools were identified. Participants were assigned to rate the importance of functional and pedagogical competences proposed during the criteria development phase. The findings of the study revealed 31 evaluation criteria under the headings of technical, media, and assessment competences. Among the groups of benchmarks proposed and rated by the users, assessment was considered as the most important one while technical and media features were even.The following step was actual implemention of the usability criteria into evaluation of fifteen leading software used in e-learning across the world. Mostly, tools were observed as having limitations in terms of capabilities. Comparing to the other software, Captivate, Softchalk, and Lectora were regarded as outstanding tools by the participants. Following the discussion on the limitations of the study, some implications for further research were proposed.

Designing software to maximize learning

I take it as axiomatic that those involved in developing educational software intend that it should maximize learning. The evaluation of educational software, therefore, needs to focus mainly on this central issue. There are still, however, two ways in which 'maximization' can be measured: either in terms of the increase in the amount of learning, or in terms of the increase in the quality of learning. In either case, an important contributory factor in the evaluation of educational software is the amount of take-up in terms of purchase and use. The crude indicator of BSH (number of Bums on Seats for number of Hours) is of some value, because if software is not used it cannot have any impact at all on learning. But it is an indicator which should only be used with caution because badly designed software used frequently presumably does less to maximize learning than well designed software used infrequently. I am concerned here with the design of software which maximizes the quality of learning. I will also deal briefly with the issue of sensitivity to the context of use, but the wider problem of the take-up and use of software, although important and interesting, is beyond the scope of this paper. The paper is based on a review of key texts in the literature on learning, including some which relate directly to software development. I hope it will make some contribution to both the design and the evaluation of educational software.DOI:10.1080/0968776960040302

Internet‐based information‐seeking behaviour amongst doctors and nurses: a short review of the literature

Reviews of how doctors and nurses search for online information are relatively rare, particularly where research examines how they decide whether to use Internet-based resources. Original research into their online searching behaviour is also rare, particularly in real world clinical settings. as is original research into their online searching behaviour. This review collates some of the existing evidence, from 1995 to 2009. To establish whether there are any significant differences in the ways and reasons why doctors and nurses seek out online information; to establish how nurses and doctors locate information online; to establish whether any conclusions can be drawn from the existing evidence that might assist health and medical libraries in supporting users. An initial scoping literature search was carried out on PubMed and CINAHL to identify existing reviews of the subject area and relevant original research between 1995 and 2009. Following refinement, further searches were carried out on Embase (Ovid), LISA and LISTA. Following the initial scoping search, two journals were identified as particularly relevant for further table of contents searching. Articles were exclused where the main focus was on patients searching for information or where the focus was the evaluation of online-based educational software or tutorials. Articles were included if they were review or meta-analysis articles, where they reported original research, and where the primary focus of the online search was for participants' ongoing Continuing Professional Development (CPD). The relevant articles are outlined, with details of numbers of participants, response rates, and the user groups. There appear to be no significant differences between the reasons why doctors and nurses seek online Internet-based evidence, or the ways in which they locate that evidence. Reasons for searching for information online are broadly the same: primarily patient care and CPD (Continuing Professional Development). The perceived barriers to accessing online information are the same in both groups. There is a lack of awareness of the library as a potential online information enabler. Libraries need to examine their policy and practice to ensure that they facilitate access to online evidence-based information, particularly where users are geographically remote or based in the community rather than in a hospital setting. Librarians also need to take into account the fact that medical professionals on duty may not be able to take advantage of the academic model of online information research. Further research is recommended into the difference between the idealised academic model of searching and real world practicalities; and how other user groups search, for example patients.

Teachers’ evaluation of preschool educational software: the case of probabilistic thinking

Nowadays, children interact in formal and informal settings with technology from their very early age. As a result, a wide range of software has emerged with diverse purposes, usages, and applications. One such category is the software that is designed to assess children's prior knowledge and skills concerning a particular subject area. In this study, software testing children's abilities to make estimations and judgments based on probabilities was evaluated by preschool teachers. This pilot study took place in Greece, during 2009-2010, in a self selected group of in-service teachers (N = 45). After personal interaction with the software, educators filled in a questionnaire with their opinions. Further analysis was made in order to discriminate the criteria of great importance that comprise software as developmentally appropriate. The results revealed that not only the design but also the content, the process and the purpose of preschool software should be taken into consideration when implementing pedagogical software in Preschool Education. © 2010 Published by Elsevier Ltd.