Guidelines for Conducting Action Research Studies in Software Engineering

Background: Action research is a well-established research methodology. It is following a post-positivist research philosophy grounded in critical thinking. The methodology is driven by practical problems, emphasis participatory research, and develops practically useful solutions in an iterative manner.Objective: Two objectives are to be achieved: (1) Understanding the state of the art with respect to action research usage in the software engineering literature, and (2) reflecting and providing recommendations of how to foster industry-academia collaboration through action research.Method:} Based on our experience with two action research studies in close collaboration with Ericsson lessons learned and guidelines are presented.Results: In both cases presented action research led to multiple refinements in the interventions implemented. Furthermore, the close collaboration and co-production with the industry was essential to identify and describe the required refinements to provide an in-depth understanding. In comparison with previous studies, we required multiple iterations while previous software engineering studies reported mostly one iteration, or were not explicit regarding the number of iterations studied.Conclusion: We conclude that action research is a powerful tool for industry-academia collaboration. The success of the method highly depends on the researchers and practitioners working in a team. Future studies need to improve the reporting with respect to describing the type of action research used, the iterations, the model of collaboration, and the rationales for changes in each iteration.

Chapter 5 - Action Research Can Swing the Balance in Experimental Software Engineering

Context : In any discipline, replications of empirical studies are necessary to consolidate the acquired knowledge. In Software Engineering, replications have been reported since the 1990s, although their number is still small. The difficulty in publishing, the lack of guidelines, and the unavailability of replication packages are pointed out by the community as some of the main causes. Objective : Understanding the current state of replications in Software Engineering studies by evaluating current trends and evolution during the last 6 years. Method : A Systematic Mapping Study including articles published in the 2013–2018 period that report at least one replication of an empirical study in Software Engineering. Results : 137 studies were selected and analysed, identifying: ${i}$ ) forums; ii) authors, co-authorships and institutions; iii) most cited studies; iv) research topics addressed; ${v}$ ) empirical methods used; vi) temporal distribution of publications; and vii) distribution of studies according to research topics and empirical methods. Conclusions : According to our results, the most relevant forums are the Empirical Software Engineering and Information and Software Technology journals, and the Empirical Software Engineering and Measurement conference. We observed that, as in previous reviews by other researchers, most of the studies were carried out by European institutions, especially Italian, Spanish, and German researchers and institutions. The studies attracting more citations were published mainly in journals and in the International Conference on Software Engineering . Testing, requirements, and software construction were the most frequent topics of replication studies, whereas the usual empirical method was the controlled experiment. On the other hand, we identified research gaps in areas such as software engineering process, software configuration management, and software engineering economics. When analysed together with previous reviews, there is a clear increasing trend in the number of published replications in the 2013–2018 period.


 
 
 CONTEXT: Despite the possible lack of validity when compared with other science areas, Simulation-Based Studies (SBS) in Software Engineering (SE) have supported the achievement of some results in the field. However, as it happens with any other sort of experimental study, it is important to identify and deal with threats to validity aiming at increasing their strength and reinforcing results confidence. OBJECTIVE: To identify potential threats to SBS validity in SE and suggest ways to mitigate them. METHOD: To apply qualitative analysis in a dataset resulted from the aggregation of data from a quasi-systematic literature review combined with ad-hoc surveyed information regarding other science areas. RESULTS: The analysis of data extracted from 15 technical papers allowed the identification and classification of 28 different threats to validity concerned with SBS in SE according Cook and Campbell’s categories. Besides, 12 verification and validation procedures applicable to SBS were also analyzed and organized due to their ability to detect these threats to validity. These results were used to make available an improved set of guidelines regarding the planning and reporting of SBS in SE. CONCLUSIONS: Simulation based studies add different threats to validity when compared with traditional studies. They are not well observed and therefore, it is not easy to identify and mitigate all of them without explicit guidance, as the one depicted in this paper.
 
 

Doing Safer Masculinities: Addressing at-Risk Gendered Behaviours on Mine Sites

Since the development of the notion of the teacher-researcher, a range of published action research studies have focused on school-based pedagogy. Scholars agree that action research is an essential tool for teachers to improve their practice, but there is little known about the process underpinning teachers’ choice of particular action research methodologies in primary science education. In this systematic literature review, 33 articles were reviewed to examine the methodological considerations teacher-researchers made when conducting action research within the primary school context, as well as the quality of the action plans and the impacts of the research on children’s learning in science. The systematic review navigated existing primary science studies, focusing on methodological considerations in the choice of particular types of action research. With regards to the quality of action research studies, the reviewed articles had a good average score for all three types of validity. In terms of the effects of the employed action research methodologies, most studies demonstrated positive impacts on children’s learning in science. Based on this review, it is recommended that future researchers be encouraged to carry out action research in their classroom settings, as most authors in the review favour this approach. Researchers should also examine the outcome, process, and democratic validity of their action research studies to generate better quality action plans. Lastly, primary school science teachers can adopt the various action research methodologies employed in this review since most studies support their positive impacts on children’s learning in science.

This article considers some paradigms of educational research, and their relation to teachers’ action research in their classrooms or studios. The positivist/scientific paradigm and the interpretive/naturalist paradigm are examined, with reference to two cases of music teachers’ action research studies. These studies are found to be flawed because the paradigms underpinning them are inappropriate for classroom-based action research. The critical theory approach is also discussed but only briefly, because no instances of music teachers’ action research in this paradigm have been found. The participatory paradigm is explained, with reference to a third case of music teachers’ action research. The article argues that, for teachers’ classroom-based action research, this paradigm is more appropriate than others. It suggests that music teachers’ action research in the participatory paradigm: (a) includes self-study, (b) involves students, (c) considers the influence of context, (d) involves more than one turn of the action research cycle, and (e) engages with, and contributes to, the development of theory.

Action research studies in education often address learners’ needs and empower practitioners to effectively change instructional practices and school communities. A systematic review of action research (AR) studies undertaken in EFL/ESL setting was conducted in this paper to systematically analyze empirical studies on action research published within a ten-year period (between 2010 and 2019). The review also aimed at investigating the focal themes in teaching the language skills at school level and evaluating the overall quality of AR studies concerning purpose, participants, and methodology. Inclusion criteria were established and 40 studies that fit were finally selected for the systematic review. Garrard’s (2007) Matrix Method was used to structure and synthesize the literature. Results showed a significant diversity in teaching the language skills and implementation of the AR model. Moreover, findings revealed that (50%) of the studies used a mixed-method approach followed by a qualitative method (37.5%); whereas only (12.5%) employed quantitative methodology. Research gaps for future action research in developing language skills were highlighted and recommendations were offered.

Authors of interpretive information systems (IS) work are claiming more frequently to be drawing upon action research as a means of undertaking IS research. The literature, while containing many useful discussions about the fundamental concepts of action research and case studies reporting its use, has little to offer the IS researcher in terms of practical guidance about conducting an action research study. The intention in this paper is to draw upon personal experiences to offer some practical guidance and warnings for the would-be IS action researcher. To do this we use Checkland's description of research (what we refer to as the FMA model) as a way of helping us to consider the action research process of our own formal IS studies and to report our experiences and lessons. We attempt to set out these experiences and lessons in a way that is consistent with an action research study.

Is Action Research just about telling a good story?

Background: Traditional research approaches are increasingly challenged in healthcare contexts as they produce abstract thinking rather than practical application. In this regard, action research is a growing area of popularity and interest, essentially because of its dual focus on theory and action. However, there is a need for action researchers not only to justify their research approach but also to demonstrate the quality of their empirical studies. Therefore, the authors set out to examine the current status of the quality of extant action research studies in healthcare to encourage improved scholarship in this area. The aim of this scoping review is to identify, explore and map the literature regarding the application of action research in either individual, group or organisational domains in any healthcare context. Methods: The systematic scoping review will search the literature within the databases of CINAHL, PubMed and ABI/Inform within the recent five-year period to investigate the scientific evidence of the quality of action research studies in healthcare contexts. The review will be guided by Arksey and O'Malley’s five mandatory steps, which have been updated and published online by the Joanna Briggs Institute. The review will follow the PRISMA-ScR framework guidelines to ensure the standard of the methodological and reporting approaches are exemplary. Conclusion: This paper outlines the protocol for an exploratory scoping review to systematically and comprehensively map out the evidence as to whether action research studies demonstrate explicitly how the essential factors of a comprehensive framework of action research are upheld. The review will summarise the evidence on the quality of current action research studies in healthcare. It is anticipated that the findings will inform future action researchers in designing studies to ensure the quality of the studies is upheld.

Background: Traditional research approaches are increasingly challenged in healthcare contexts as they produce abstract thinking rather than practical application. In this regard, action research is a growing area of popularity and interest, essentially because of its dual focus on theory and action. However, there is a need for action researchers not only to justify their research approach but also to demonstrate the quality of their empirical studies. Therefore, the authors set out to examine the current status of the quality of extant action research studies in healthcare to encourage improved scholarship in this area. The aim of this scoping review is to identify, explore and map the literature regarding the application of action research in either individual, group or organisational domains in any healthcare context.Methods: The systematic scoping review will search the literature within the databases of CINAHL, PubMed and ABI/Inform within the recent five-year period to investigate the scientific evidence of the quality of action research studies in healthcare contexts. The review will be guided by Arksey and O'Malley’s five mandatory steps, which have been updated and published online by the Joanna Briggs Institute. The review will follow the PRISMA-ScR framework guidelines to ensure the standard of the methodological and reporting approaches are exemplary.Conclusion: This paper outlines the protocol for an exploratory scoping review to systematically and comprehensively map out the evidence as to whether action research studies demonstrate explicitly how the essential factors of a comprehensive framework of action research are upheld. The review will summarise the evidence on the quality of current action research studies in healthcare. It is anticipated that the findings will inform future action researchers in designing studies to ensure the quality of the studies is upheld.

Background: Traditional research approaches are increasingly challenged in healthcare contexts as they produce abstract thinking rather than practical application. In this regard, action research is a growing area of popularity and interest, essentially because of its dual focus on theory and action. However, there is a need for action researchers not only to justify their research approach but also to demonstrate the quality of their empirical studies. Therefore, the authors set out to examine the current status of the quality of extant action research studies in healthcare to encourage improved scholarship in this area. The aim of this scoping review is to identify, explore and map the literature regarding the application of action research in either individual, group or organisational domains in any healthcare context. Methods: The systematic scoping review will search the literature within the databases of CINAHL, PubMed and ABI/Inform within the recent five-year period to investigate the scientific evidence of the quality of action research studies in healthcare contexts. The review will be guided by Arksey and O'Malley's five mandatory steps, which have been updated and published online by the Joanna Briggs Institute. The review will follow the PRISMA-ScR framework guidelines to ensure the standard of the methodological and reporting approaches are exemplary. Conclusion: This paper outlines the protocol for an exploratory scoping review to systematically and comprehensively map out the evidence as to whether action research studies demonstrate explicitly how the essential factors of a comprehensive framework of action research are upheld. The review will summarise the evidence on the quality of current action research studies in healthcare. It is anticipated that the findings will inform future action researchers in designing studies to ensure the quality of the studies is upheld.

Background: The challenges being experienced by health and care systems across the globe are not being adequately addressed by current research paradigms and approaches. The scientific literature has recommended that in order to understand complex interventions, such as the implementation of integrated care, in complex systems, longitudinal and qualitative research approaches are required. Action research approaches have the potential to be a transformative force in healthcare through its dynamic dual intention of action and research that facilitates organisational change, teamwork and the empowerment of heath care professionals in hospitals and communities thus contributing to improvements in the quality of care. Despite its potential value, there is no clear consensus on what constitutes ‘good’ action research with many empirical investigations appearing to insufficiently engage with quality criteria that has been described in some detail in the literature including four factors from Shani and Pasmore’s complete theory of action research. Objectives: The aim of this scoping review was to explore the extant literature on action research studies in healthcare and assess and synthesise how researchers addressed criteria for quality of action research studies in healthcare. Eligibility criteria: This scoping review considered qualitative and quantitative primary research utilising action research approaches, published in the English language between the years 2016 and 2021. Sources of evidence: The following databases were searched: PubMed, CINAHL and ProQuest. In addition, a manual search of key action research journals was performed. Data extraction was conducted using Rayyan software and a bespoke data extraction table was developed to produce a descriptive summary of the results, addressing the objectives and research question. The review used the Arksey, and O’Malley (2005) revised six-step framework and adhered to the Preferred Reporting Items for Systematic Reviews and Meta-analysis Extension for Scoping Reviews (PRISMA-ScR). As recommended, the findings are presented in narrative and tabular form with figures to support the data where appropriate. Results: 3263 articles were originally identified. 641 articles were identified as duplicates entries and were removed before screening. A further 2565 articles were excluded as they did not meet the inclusion criteria. A total of 57 articles were included in the review. Most publications were in 2016. Several different action research approaches were used. Most studies had nursing participants and were set in hospitals with most publications in Europe. Only 13 of the 57 articles considered all 4 of the Shani and Pasmore quality factors and none considered the integration of these factors. Conclusion: This Scoping review builds upon knowledge generated by previous systematic reviews and contributes to knowledge and practice by providing coherence on how best to approach an action research project and attend to quality. If action research is to fulfil its potential as a transformative power for change within health care, action researchers need to demonstrate the rigor in their work. The authors offer some guidance in this area. Just as AR has dual intention, researchers have a dual obligation, to do good and to do good action research.

Context: Two of the most common external threats to validity in quantitative studies in software engineering (SE) are concerned with defining the population by convenience and nonrandom sampling assignment. Although these limitations can be reduced by increasing the number of replications and aggregating their results, the acquired evidence rarely can be generalized to the field.Objective: To investigate the state of practice of meta-analysis in SE and its limitations, intending to propose an alternative perspective to understand the relationships among experimentation, production, threats to validity and evidence. To propose and evaluate means to strengthen quantitative studies in software engineering and making them less risky due to population and sampling issues.Method: To use the underlying idea from the Theory of Food Chains to alternatively understand the impact of external threats to validity in the SE experimental cycle (experimental chains). Next, to accomplish an initial technical literature survey to observe basic features of secondary studies aggregating primary studies results. Third, to organize a set of experimental chain's concepts and make initial discussions regarding the observed secondary studies concerned with this metaphor.Results: By applying the experimental chains concepts it was initially observed that, although important and necessary, most of the current effort in the conduction of quantitative studies in SE does not produce (mainly due to population/sampling constraints) results strong enough to positively impact the engineering of software. It promotes an imbalance between research and practice. However, more investigation is necessary to support this claim.Conclusion: We argue that research energy has been lost in SE studies due to population/sampling constraints. Therefore, we believe more investigation must be undertaken to understand how better organizing, enlarging, setting up and sampling SE quantitative studies' population by using, for instance, alternative technologies such as social networks or other crowdsourcing technologies.