Teaching Software Testing Research Articles

Learning environments in software testing education: An overview

AbstractNowadays, we heavily depend on software, as we utilize it daily. Bugs in the software can cause significant damage, loss of private data and money, and even loss of human lives. Software testing and quality assurance discipline, which belongs to the software engineering domain, aims to ensure that the software is good enough before its release to the market. Therefore, it is vital to ensure that software engineers are highly trained in this domain. Unfortunately, this discipline is frequently neglected throughout software engineering courses. Students are mostly taught just the theoretical principles of software testing and gain little practical experience. Additionally, the software testing lectures are considered to be dull, especially if they are not followed by practical tasks where the students could apply learned techniques and gain hands‐on experience. Several tools, games, and collaborative learning environments have been proposed to make the lectures more interesting and increase student engagement. The main goal of this article is to provide a systematic survey of such environments, where each environment was evaluated in terms of major software testing topics covered and the approach used to engage the students.

Can gamification help in software testing education? Findings from an empirical study

Software testing is an essential knowledge area required by industry for software engineers. However, software engineering students often consider testing less appealing than designing or coding. Consequently, it is difficult to engage students to create effective tests. To encourage students, we explored the use of gamification and investigated whether this technique can help to improve the engagement and performance of software testing students. We conducted a controlled experiment to compare the engagement and performance of two groups of students that took an undergraduate software testing course in different academic years. The experimental group is formed by 135 students from the gamified course whereas the control group is formed by 100 students from the non-gamified course. The data collected were statistically analyzed to answer the research questions of this study. The results show that the students that participated in the gamification experience were more engaged and achieved a better performance. As an additional finding, the analysis of the results reveals that a key aspect to succeed is the gamification experience design. It is important to distribute the motivating stimulus provided by the gamification throughout the whole experience to engage students until the end. Given these results, we plan to readjust the gamification experience design to increase student engagement in the last stage of the experience, as well as to conduct a longitudinal study to evaluate the effects of gamification.

Design and Implementation of Gamified Learning System for Mutation Testing

Mutation Testing is used to test case improvement mechanism for evaluating the effectiveness of test cases by generating a large number of mutants. In the past, some approaches are proposed to improve the performance of generating the mutants. In recent years, many studies have begun to explore the software engineering education of mutation testing, trying to make students understand its concept through gamification. In this paper, we apply gamification theory and build a gamified learning system for mutation testing, named code immunity boost (CIB), taking the story of vaccine development as a sense of mission. We invited students to learn mutation testing through the relationship between leukocyte (test case), vaccine (mutant) and human body (program). Students can play the role of a vaccine and stimulate the testing of test cases by writing mutants to improve the quality of the program. We adopted the benchmark programs commonly used in mutation testing research, and developed a code vaccine incubator (CVI) tool to generate a large number of mutants as the experimental cornerstone of this study. Final experiments show that our tool can help mutation testing education, as the performance and quality of the experimental group is better than that of the control group. The response from the questionnaire also shows students like learning by our gamified tool. We therefore recommend to promote such software testing education approach by integrating our tool with popular online programming tools such as Online Judge system.

Integrating SPOCs in Software Testing Education: Evidence in Emergency Remote Courses

Nowadays, SPOCs (Small Private Online Courses) have been used as complementary methods to support classroom teaching. SPOCs are courses that apply the usage of MOOCs (Massive Open Online Courses), combining classroom with online education, making them an exciting alternative for contexts such as emergency remote teaching. Although SPOCs have been continuously proposed in the software engineering teaching area, it is crucial to assess their practical applicability via measuring the effectiveness of this resource in the teaching-learning process. In this context, this paper aims to present an experimental evaluation to investigate the applicability of a SPOC in a Verification, Validation, and Software Testing course taught during the period of emergency remote education during the COVID-19 pandemic in Brazil. Therefore, we conducted a controlled experiment comparing alternative teaching through the application of a SPOC with teaching carried out via lectures. The comparison between the teaching methods is made by analyzing the students’ performance during the solving of practical activities and essay questions on the content covered. In addition, we used questionnaires to analyze students’ motivation during the course. Study results indicate an improvement in both motivation and performance of students participating in SPOC, which corroborates its applicability to the software testing teaching area.

A Quasi-Experimental Evaluation of Teaching Software Testing in Software Quality Assurance Subject during a Post-Graduate Computer Science Course

Software testing is regarded as a key activity in the software development cycle, as it helps information technology professionals to design good quality software. Thus, this is an essential activity for the software industry, although with all its nuances high priority is still not being given to learning about it at an academic level. The purpose of this work is to investigate a teaching strategy for software testing which involves acquiring academic skills within a curriculum based on active teaching methodologies. A teaching model was designed for this to coordinate the different areas of a subject, and then a controlled quasi-experiment was carried out in a post-graduate course to evaluate the application of this model. The results obtained demonstrate that there was a considerable learning gain in the experimental group that adopted the teaching approach when compared with the control group that relied on a traditional approach. The student t-test was employed to determine the learning efficiency.

Software Testing Education: A Systematic Literature Review

Software Testing is the core part of computer science & engineering curriculum. It has been observed that software testing has been taught much in Computer science & engineering disciplines at undergraduate level. Software Testing Education (STE) involves time, cost, risk, quality, integration, communication, human resource, and procurement management skills. The STE at undergraduate and graduate level is a challenging task because it also requires knowledge and experience. This article aims to investigate and synthesize the state-of-the-art research in STE for the improvement of STE curriculum, pedagogical tools and techniques, cognitive, empirical and assessments methods. STE research approaches has been categorized into five categories including empirical type, research approaches, software testing education processes, key areas, and curricula. The ninety-seven articles for the area of STE have been chosen after rigorous systematic screening process published in during 2004 to 2021. Furthermore, tools and techniques, testing processes, pedagogy and student performance are the frequently addressed in STE; whereas assessment methods, gamification, curriculum and exemplary program development appeared as mostly ignored areas. Lastly, research gaps and challenges relate to STE has been presented as future directions for the faculty, software industry, and researchers.

Teaching Software Testing Using Automated Grading

Teaching Software Testing Using Automated Grading

Design and Evaluation of a Mobile Application for an Educational Card Game

Educational games can provide players with rich learning and socializing experiences through different interac-tion paradigms, such as board games, card games, and, more recently, hybrid (physical-digital) games. However, the process of making, maintaining, and evolving an educational analog game is not a trivial task. Balancing the game mechanics and dynamics to provide a pleasant and educational gaming experience can be very difficult to achieve. Furthermore, adding a digital component in the gameplay can disturb the experience of the game and learning ob-jectives, and this kind of insertion should be evaluated. This work then aims to report the process of developing and evaluating a mobile application for helping the gameplay of a card game that focuses on on teaching software testing concepts. Our primary concern during the development of the application was whether its insertion would compromise the learning process or the social experience of the card game. The developed application has the fol-lowing functionalities: point counter, dice scrolling, timer, and a summary of the rules. We designed and developed a first version of the application. Then, we evaluated the impact of its insertion on gameplay by applying the game assisted with the application with students from Computer Science and Computer Engineering courses and, after the end of the game, the participants answered questionnaires about the players’ experience and their impressions about the application. Based on the results, we perceived that the use of the application provided benefits to the players’ experience, although the evaluation highlighted some opportunities for improvement. Thus, we evolve the mobile application based on the comments gathered in this evaluation. This new version has improvements on user’s interface, aiming to provide a better user experience, and new functionalities. Furthermore, we assessed the second version and compared both versions of the mobile app in order to collect evidence regarding improvements in the game experience.

Is It Worth Using Gamification on Software Testing Education? An Extended Experience Report in the Context of Undergraduate Students

Context: Testing is fundamental in the software development process. Nevertheless, testing education faces the key challenge of ensuring that undergraduate students acquire knowledge and skills they need for their future careers by matching what is taught in the classroom to industry standards. In this context, gamification can be used as an alternative educational approach. It uses game elements in real-world contexts in order to increase people's motivation and engagement in tasks that require external stimuli, especially in educational contexts.Objective: Reporting on results of an experimental study designed to assess the impact of gamification on software testing education, as well as reporting on the experience of building a supporting gamified platform. Method: We performed a systematic literature mapping aiming at characterizing how gamification has been explored in the software testing context. In addition, some of the problems faced by testing education were identified through an ad-hoc search. Then, we developed a gamified approach and a platform that have been used to run five 4-hour functional testing classes with undergraduate students from four Brazilian institutions of higher education.Results: Overall, students who learned with the traditional approach felt more motivated than those who learned with the gamified approach, whereas the performance of both groups were similar. On the other hand, feedback questionnaires indicated that the gamified class was more attractive (in terms of attention) and funny. Moreover, we observed that building a gamified platform is complex and challenging, particularly for the definition of the game mechanics and dynamics.Conclusion: Even though the results in terms of students' performance was neutral, and the motivation of students did not increase due to the introduction of gamification, the experience of having used this alternative approach is considered positive, as it provided a more enjoyable and funny environment, both from the researcher's and students' point of view. Furthermore, with this experience we foresee we can do better in terms of gamified teaching in future work.

Software-testing education: A systematic literature mapping

ContextWith the rising complexity and scale of software systems, there is an ever-increasing demand for sophisticated and cost-effective software testing. To meet such a demand, there is a need for a highly-skilled software testing work-force (test engineers) in the industry. To address that need, many university educators worldwide have included software-testing education in their software engineering (SE) or computer science (CS) programs. Many papers have been published in the last three decades (as early as 1992) to share experience from such undertakings. ObjectiveOur objective in this paper is to summarize the body of experience and knowledge in the area of software-testing education to benefit the readers (both educators and researchers) in designing and delivering software testing courses in university settings, and to also conduct further education research in this area. MethodTo address the above need, we conducted a systematic literature mapping (SLM) to synthesize what the community of educators have published on this topic. After compiling a candidate pool of 307 papers, and applying a set of inclusion/exclusion criteria, our final pool included 204 papers published between 1992 and 2019. ResultsThe topic of software-testing education is becoming more active, as we can see by the increasing number of papers. Many pedagogical approaches (how to best teach testing), course-ware, and specific tools for testing education have been proposed. Many challenges in testing education and insights on how to overcome those challenges have been proposed. ConclusionThis paper provides educators and researchers with a classification of existing studies within software-testing education. We further synthesize challenges and insights reported when teaching software testing. The paper also provides a reference (“index”) to the vast body of knowledge and experience on teaching software testing. Our mapping study aims to help educators and researchers to identify the best practices in this area to effectively plan and deliver their software testing courses, or to conduct further education-research in this important area.

UTCPredictor: An uncertainty‐aware novel teaching cases predictor

AbstractTeaching cases are crucial for computer science (e.g., software testing) teaching. With the fast development of computer science, old and outdated teaching cases cannot meet the requirements of teaching. Therefore, teachers need to update teaching case repository continually and timely. However, teaching case development is an extremely time‐consuming work. Given today's complex and fast‐moving environment of computer science, teachers often feel blind about about what types of cases should be added for teaching. This paper presents UTCPredictor—an automated approach of predicting novel teaching cases from real production by identifying the most uncertain data, which are always with new features that reflect the latest developments and trends in the field. The implementation of UTCPredictor is based on the idea of interactive machine learning as well as several text mining techniques. To evaluate the effectiveness of UTCPredictor, we take bug report case building in software testing teaching as an example, using UTCPredictor to perform 10‐fold cross‐validation on an existing teaching case set. The performance in terms of indicators; Recall, Precision, and F1‐score, achieved three very competitive values—0.91, 0.94, and 0.85, respectively. We further evaluate the effectiveness of UTCPredictor through a user study and a questionnaire. The results are very positive; the user study indicates that educators can build a teaching case set from latest bug report repository by spending only 8.16%–18.11% time costs compared with traditional manual approach; the responses from 2,000 students for the questionnaire show that the teaching cases built with UTCPredictor are very popular among students.

Diagnóstico tecnológico de la pertinencia al implementar un laboratorio de testing de software. Caso: Universidad Cooperativa de Colombia, campus Popayán

Contexto: Actualmente la Universidad Cooperativa de Colombia, campus Popayán, no cuenta con un espacio o laboratorio de testing de software, sin embargo, el testing es un elemento fundamental del control de calidad enfocado tanto en el producto como en el proceso. Se plantea un diagnostico tecnológico con el objeto de validar la pertinencia de su implementación.
 Método: En el artículo se realiza una revisión de las definiciones teóricas, las técnicas, los métodos, las herramientas y las características que se sugiere se deben tener en cuenta para la implementación de un laboratorio de testing a partir del marco de referencia CMMI y TMMI (referentes de la bibliografía científica). Con el método se enfatiza también la importancia del rol de las instituciones de educación superior y la enseñanza del testing de software.
 Resultados: El diagnostico tecnológico es una herramienta que, de forma ágil y eficiente, permite obtener claridad en las necesidades y oportunidades para proponer soluciones reales, como elemento fundamental en la pertinencia y viabilidad de implementar un espacio para las prácticas y experimentación referente al testing y la calidad de software, en la Universidad Cooperativa de Colombia, campus Popayán.
 Conclusiones: El diagnostico tecnológico abre las puertas a profundizar en el contexto del control de calidad y su pertinencia desde los entornos académicos y contribuye a la cohesión entre la industria software y el modelo enseñanza/aprendizaje del software.

An Alliance-Based Term Project in Software Quality Courses: a Lesson Learned

Software testing education has become important in the field of software engineering education. In the previous software quality assurance course, students were asked to form teams to complete a term project. By working on term projects, students can learn programming skills and test skills in a practical way. However, from the experience of the last 3 years, we found that students only did unit testing and system performance testing well but did poorly in integration testing. In addition, students do not yet have the concept of system decomposition and integration, even though it is important during software development. In this paper we report our improvements to software testing course design - an alliance-based approach. In the term project, students are organized into teams and many teams are grouped into alliances. Each alliance has a team of masters building game platforms for others. The master team must define the application interface to interact with other gaming teams, and they must perform integration tests based on the defined interface. In this paper we report our experiences and student feedback on the educational approach.

Research on the Application of Yin Yang Theory in Software Testing Teaching

Research on the Application of Yin Yang Theory in Software Testing Teaching

The impact of Software Testing education on code reliability: An empirical assessment

Abstract Software Testing (ST) is an indispensable part of software development. Proper testing education is thus of paramount importance. Indeed, the mere exposition to ST knowledge might have an impact on programming skills. In particular, it can encourage the production of more correct - and thus reliable - code. Although this is intuitive, to the best of our knowledge, there are no studies about such effects. Concerned with this, we have conducted two investigations related to ST education: (1) a large experiment with students to evaluate the possible impact of ST knowledge on the production of reliable code; and (2) a survey with professors that teach introductory programming courses to evaluate their level of ST knowledge. Our study involved 60 senior-level computer science students, 8 auxiliary functions with 92 test cases, a total of 248 implementations, and 53 professors of diverse subfields that completed our survey. The investigation with students shows that ST knowledge can improve code reliability in terms of correctness in as much as 20%, on average. On the other hand, the survey with professors reveals that, in general, university instructors tend to lack the same knowledge that would help students increase their programming skills toward more reliable code.

Scrumify: A Software Game to Introduce Agile Software Development Methods

Software Engineering courses are perceived as being dry and boring. The education community has been looking at active learning and hands-on training techniques to engage students better and thereby provide a better understanding of concepts. This paper presents a web-based game called Scrumify, developed by undergraduate students, that is designed to introduce students to the Scrum software development process through story elements, and quiz mechanics. The game is designed to reinforce software engineering concepts more specifically agile software processes. Quizzes within the game are designed to grant points that inject a sense of competition for the users and motivation to learn the agile methodology concepts. Scrumify serves two objectives: software engineering education for audience of the game and web-based game development and software testing education for the developers of the game. Use of this game in classroom provides an active learning technique for agile software processes. Students are more engaged with games and thereby absorb the concepts better. The game was evaluated in a sophomore software engineering class and the results are presented.

Experimental Model for Teaching Software Testing

The topic of software testing in education has become appealing to some although most researchers focused on researching software testing tools and methods. This is because of the increase awareness of the importance of adding software testing to the curricula in universities. In this paper, we propose the concept of teaching software testing with an experimental model for pairwise testing; developed with Migrating Birds Optimization (MBO) algorithm called Experimental Model for Teaching Software Testing (EMTST). The EMTST framework is explained in details and along with the comparison of MBO algorithm with other pairwise strategies. In conclusion, this strategy with MBO can help students understand the concept and practical aspect of the development of a pairwise strategy and how to evaluate the effectiveness of the algorithm in AI-based pairwise strategy.

The perception of software quality and testing in Czech software companies

The aim of this article is to introduce the survey results of the current state of testing and quality management in software companies in the Czech Republic. The article includes answers to questions about the view of quality management in these companies and their employees' level of knowledge of software testing as well as an analysis of these results and a proposal for solving the identified issues. Moreover it describes the current software testing education system and its characteristics as it has a direct impact on this perception.

Application of Project-Driven Teaching Model on Software Functional Testing

in order to efficient and rapid training student mastery the all aspects application skills of software test, it was adopted project-driven teaching method to teach based on the characteristic of software test theory knowledge and operation requirements of job. This essay is main introduced that study and application of function testing teaching of software testing by project-driven teaching method. It is proved the method obtained good teaching results in fact teaching.

Designing an adaptive learning module to teach software testing

Adaptive learning systems aim to precisely tailor education and training to the individual needs of learners. Such systems use an internal model of a user's current knowledge to adjust the navigational affordances and presentation order of material. The user model is incrementally built and updated as the user demonstrates mastery by completing exercises and tests. Designing courses that are delivered adaptively involves addressing many complexities. This paper describes experiences designing the first adaptive module in a series intended to teach software testing skills. Experiences in using the first module and a preliminary evaluation of its effectiveness are presented.