Large-scale Software Development Research Articles

The Importance of Robust Communication in Large-Scale Agile Development

The agile methodology stands out as a prevalent model for efficient software development, particularly favored for its adaptability and suitability in small-scale projects across various software industries. Nevertheless, its widespread adoption has brought to light certain communication challenges, particularly when applied to large-scale distributed teams. Agile, it appears, may not be the optimal choice for extensive teams engaged in global software development efforts. This study delves into the intricacies of issues faced by teams employing agile in the context of large-scale distributed development, particularly focusing on communication-related challenges and their repercussions. Our approach involved in-depth interviews with diverse developers and teams hailing from various sectors within the software industry. Moreover, we conducted an extensive quantitative analysis, surveying 50 developers representing different distributed teams. The outcomes of our investigation unearthed several communication-related deficiencies that significantly impact the development process. To arrive at these insights, we employed two robust statistical analysis methods: descriptive analysis and regression analysis. The implications of our findings have led us to propose innovative software solutions, bearing distinctive features engineered to mitigate the communication issues often encountered in large-scale software development. These solutions have the potential to enhance the efficiency and effectiveness of agile practices when applied in extensive and globally dispersed development endeavors.

The Potential of AI-Driven Assistants in Scaled Agile Software Development

Scaled agile development approaches are now used widely in modern software engineering, allowing businesses to improve teamwork, productivity, and product quality. The incorporation of artificial intelligence (AI) into scaled agile development methods (SADMs) has emerged as a potential strategy in response to the ongoing demand for simplified procedures and the increasing complexity of software projects. This paper explores the intersection of AI-driven assistants within the context of the scaled agile framework (SAFe) for large-scale software development, as it stands out as the most widely adopted framework. Our paper pursues three principal objectives: (1) an evaluation of the challenges and impediments encountered by organizations during the implementation of SADMs, (2) an assessment of the potential advantages stemming from the incorporation of AI in large-scale contexts, and (3) the compilation of aspects of SADMs that AI-driven assistants enhance. Through a comprehensive systematic literature review, we identified and described 18 distinct challenges that organizations confront. In the course of our research, we pinpointed seven benefits and five challenges associated with the implementation of AI in SADMs. These findings were systematically categorized based on their occurrence either within the development phase or the phases encompassing planning and control. Furthermore, we compiled a list of 15 different AI-driven assistants and tools, subjecting them to a more detailed examination, and employing them to address the challenges we uncovered during our research. One of the key takeaways from this paper is the exceptional versatility and effectiveness of AI-driven assistants, demonstrating their capability to tackle a broader spectrum of problems. In conclusion, this paper not only sheds light on the transformative potential of AI, but also provides invaluable insights for organizations aiming to enhance their agility and management capabilities.

Responding to change over time: A longitudinal case study on changes in coordination mechanisms in large-scale agile

ContextResponding to change and continuously improving processes, practices, and products are core to agile software development. It is no different in large-scale agile, where multiple software development teams need to respond both to changes in their external environments and to changes within the organization.ObjectiveWith this study, we aim to advance knowledge on coordination in large-scale agile by developing a model of the types of organizational changes that influence coordination mechanisms.MethodWe conducted a longitudinal case study in a growing large-scale agile organization, focusing on how external and internal changes impact coordination over time. We collected our data through 62 days of fieldwork across one and a half years. We conducted 37 interviews, observed 118 meetings at all organizational levels, collected supplementary material such as chat logs and presentations, and analyzed the data using thematic analysis.ResultsOur findings demonstrate how external events, such as onboarding new clients, and internal events, such as changes in the team organization, influence coordination mechanisms in the large-scale software development program. We find that external and internal change events lead to the introduction of new coordination mechanisms, or the adjustment of existing ones. Further, we find that continuous scaling requires continuous change and adjustment. Finally, we find that having the right mechanisms in place at the right time strengthens resilience and the ability to cope with change in coordination needs in complex large-scale environments.ConclusionsOur findings are summarized in an empirically based model that provides a practical approach to analyzing change, aimed at supporting both researchers and practitioners dealing with change in coordination mechanisms in large-scale agile development contexts.

Identifying and Categorizing Challenges in Large-Scale Agile Software Development Projects: Insights from Two Swedish Companies

We conducted a case study to examine the challenges encountered in large-scale agile development (LSAD) within two Swedish software companies. While agile methodologies have proven successful in small and medium-sized projects, their implementation in large-scale software development projects can be problematic. To identify these challenges, we employed thematic analysis, which revealed a total of 26 distinct challenges. These challenges were categorized into three main themes: Processes and practices, Teams, and Organizational-level challenges in LSAD. By recognizing and addressing these challenges, projects operating in similar contexts can synchronize their activities and harness the advantages of agile methodologies at a large scale. The article delves into comprehensive discussions on these challenges, offering valuable insights and directions for future research endeavors.

Identifying and Categorizing Challenges in Large-Scale Agile Software Development Projects: Insights from Two Swedish Companies

We conducted a case study to examine the challenges encountered in large-scale agile development (LSAD) within two Swedish software companies. While agile methodologies have proven successful in small and medium-sized projects, their implementation in large-scale software development projects can be problematic. To identify these challenges, we employed thematic analysis, which revealed a total of 26 distinct challenges. These challenges were categorized into three main themes: Processes and practices, Teams, and Organizational-level challenges in LSAD. By recognizing and addressing these challenges, projects operating in similar contexts can synchronize their activities and harness the advantages of agile methodologies at a large scale. The article delves into comprehensive discussions on these challenges, offering valuable insights and directions for future research endeavors.

Checklists to support decision-making in regression testing

Context:Practitioners working in large-scale software development face many challenges in regression testing activities. One of the reasons is the lack of a structured regression testing process. In this regard, checklists can help practitioners keep track of essential regression testing activities and add structure to the regression testing process to a certain extent. Objective:This study aims to introduce regression testing checklists so test managers/teams can use them: (1) to assess whether test teams/members are ready to begin regression testing, and (2) to keep track of essential regression testing activities while planning and executing regression tests. Method:We used interviews, workshops, and questionnaires to design, evolve, and evaluate regression testing checklists. In total, 25 practitioners from 12 companies participated in creating the checklist. Twenty-three of them participated in checklists evolution and evaluation. Results:We identified activities practitioners consider significant while planning, performing, and analyzing regression testing. We designed regression testing checklists based on these activities to help practitioners make informed decisions during regression testing. With the help of practitioners, we evolved these checklists into two iterations. Finally, the practitioners provided feedback on the proposed checklists. All respondents think the proposed checklists are useful and customizable for their environments, and 80% think checklists cover aspects essential for regression testing. Conclusion:The proposed regression testing checklists can be useful for test managers to assess their team/team members’ readiness and decide when to start and stop regression testing. The checklists can be used to record the steps required while planning and executing regression testing. Further, these checklists can provide a basis for structuring the regression testing process in varying contexts.

Software selection in large-scale software engineering: A model and criteria based on interactive rapid reviews

ContextSoftware selection in large-scale software development continues to be ad hoc and ill-structured. Previous proposals for software component selection tend to be technology-specific and/or do not consider business or ecosystem concerns.ObjectiveOur main aim is to develop an industrially relevant technology-agnostic method that can support practitioners in making informed decisions when selecting software components for use in tools or in products based on a holistic perspective of the overall environment.MethodWe used method engineering to iteratively develop a software selection method for Ericsson AB based on a combination of published research and practitioner insights. We used interactive rapid reviews to systematically identify and analyse scientific literature and to support close cooperation and co-design with practitioners from Ericsson. The model has been validated through a focus group and by practical use at the case company.ResultsThe model consists of a high-level selection process and a wide range of criteria for assessing and for evaluating software to include in business products and tools.ConclusionsWe have developed an industrially relevant model for component selection through active engagement from a company. Co-designing the model based on previous knowledge demonstrates a viable approach to industry-academia collaboration and provides a practical solution that can support practitioners in making informed decisions based on a holistic analysis of business, organisation and technical factors.

A Taxonomy of Inter-Team Coordination Mechanisms in Large-Scale Agile

In large-scale agile software development, many teams work together to achieve overarching project goals. The more teams, the greater the coordination requirements. Despite the growing popularity of large-scale agile, inter-team coordination is challenging to practice and research. We conducted a case study over 1.5 years in a large-scale software development firm to better understand which inter-team coordination mechanisms are used in large-scale agile and how they support inter-team coordination. Based on a thematic analysis of 31 interviews, 113 hours of observations, and supplemental material, we identified 27 inter-team coordination mechanisms. From this, we offer the following contributions. First, we propose a taxonomy of inter-team coordination with three categories: coordination meetings, such as communities of practice, inter-team stand-ups, and retrospectives; coordination roles, such as the program architects and the platform team; and coordination tools and artefacts, such as Slack and JIRA as well as inter-team task boards, product backlogs, and roadmaps. Second, the coordination mechanisms displayed combinations of four key characteristics, technical, organizational, physical, and social (TOPS), which form the basis of the TOPS framework to capture the multifaceted characteristics of coordination mechanisms. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Technical</i> relates to the software product and/or technical tools supporting software development. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Organizational</i> pertains to the structural aspects of the organization. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Physical</i> refers to tangible or spatial characteristics. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Social</i> captures interpersonal and community-based characteristics. Finally, the taxonomy and the TOPS framework provide a knowledge base and a structured approach for researchers to study as well as for software practitioners to understand and improve inter-team coordination in large-scale agile.

Leveraging Structure in Software Merge: An Empirical Study

Large-scale software development today relies heavily on version control systems facilitating distributed development of software projects. For the purpose of merging diverging versions of the code base, version control systems employ line-based merge algorithms, which are applicable to all text files. Structured merge algorithms have been proposed as an alternative to unstructured, line-based merging, with the goal of reducing the number of merge conflicts that have to be manually resolved by the developer. By leveraging the structure inherent in source code (i.e., by representing source code files in terms of abstract syntax trees instead of sequences of text lines), these algorithms are able to merge revisions in various situations (e.g., reordering of methods) that would cause conflicts when merged using an unstructured approach. However, merging abstract syntax trees is inherently more complex than merging sequences of text lines, which makes structured merge algorithms computationally more expensive than an unstructured merge. To reduce the runtime cost of structured merge algorithms, semistructured merge as well as combinations of different merge strategies were proposed. As such, we observe a range of increasingly structured merge algorithms, which feature different characteristics in terms of conflict resolution and runtime. The progressively increasing use of structure to avoid merge conflicts or to automate conflict resolution raises a number of questions: How is the correctness of the code resulting from a merge affected when employing structured merge algorithms? Which algorithm strikes the best balance between runtime, conflict resolution potential, and correctness of the merge result? For the first time, we evaluate a whole range of merge algorithms (from unstructured over semistructured to structured as well as combinations) by replaying merge commits in a controlled setting. We employ the test suite of the projects in question as an oracle for the correctness of the resulting code, triangulated by a thorough manual analysis. Using 7727 merge commits from 10 open-source projects, we find that combined strategies appear to be the best of both worlds: They resolve as many conflicts as structured merge at a significantly lower runtime per merge commit. Notably, structured merge strategies do cause more test failures, however, the increase is small.

Changes to team autonomy in large-scale software development: a multiple case study of Scaled Agile Framework (SAFe) implementations

Large-scale transformations of agile ways of working have received more attention in the industry in recent years. Some organizations have developed their own solutions for scaling, whereas many have chosen trademarked frameworks. In large-scale agile software development, many developers and development teams carry out work simultaneously. When autonomous teams need to coordinate toward a common goal, they must sacrifice some level of autonomy. Development, testing, and integrations need to be coordinated with other teams and aligned with an organization´s programs or portfolio. Through the conducting of 28 interviews and 17 on-site visits, this multiple case study explored how team autonomy changed in three agile software development organizations that implemented the Scaled Agile Framework (SAFe). The positive changes to team autonomy that they experienced as a result included getting a better overview, making better long-term decisions, giving and receiving help, and signaling limitations. We found two negative impacts on team autonomy: limited feature choice and enforced refinement. The study extends previous research on large-scale agile software development and improves our understanding of impacts on team autonomy.

To schedule or not to schedule? An investigation of meetings as an inter-team coordination mechanism in largescale agile software development

Coordination of teams is critical when managing large programmes that involve multiple teams. In large-scale software development, work is carried out simultaneously by many developers and development teams. Results are delivered frequently and iteratively, which requires coordination on different levels, e.g., the programme, project, and team levels. Prior studies of knowledge work indicate that such work relies heavily on coordination through "personal" modes such as mutual adjustment between individuals or through scheduled or unscheduled meetings. In agile software development processes, principles and work structures emerge during the project and are not predetermined. We studied how coordination through scheduled and unscheduled meetings changes over time in two large software development programmes relying on agile methods. Our findings include transitions from scheduled to unscheduled meetings and from unscheduled to scheduled meetings. The transitions have been initiated both bottom-up and top-down in the programme organizations. The main implication is that programme management needs to be sensitive to the vital importance of coordination and the coordination needs as they change over time. Further, when starting a program, we recommend to early identify the important scheduled meetings, as having enough scheduled meetings is important to develop a common understanding of domain knowledge.

ConE: A Concurrent Edit Detection Tool for Large-scale Software Development

Modern, complex software systems are being continuously extended and adjusted. The developers responsible for this may come from different teams or organizations, and may be distributed over the world. This may make it difficult to keep track of what other developers are doing, which may result in multiple developers concurrently editing the same code areas. This, in turn, may lead to hard-to-merge changes or even merge conflicts, logical bugs that are difficult to detect, duplication of work, and wasted developer productivity. To address this, we explore the extent of this problem in the pull-request-based software development model. We study half a year of changes made to six large repositories in Microsoft in which at least 1,000 pull requests are created each month. We find that files concurrently edited in different pull requests are more likely to introduce bugs. Motivated by these findings, we design, implement, and deploy a service named Concurrent Edit Detector (ConE) that proactively detects pull requests containing concurrent edits, to help mitigate the problems caused by them. ConE has been designed to scale, and to minimize false alarms while still flagging relevant concurrently edited files. Key concepts of ConE include the detection of the Extent of Overlap between pull requests, and the identification of Rarely Concurrently Edited Files . To evaluate ConE, we report on its operational deployment on 234 repositories inside Microsoft. ConE assessed 26,000 pull requests and made 775 recommendations about conflicting changes, which were rated as useful in over 70% (554) of the cases. From interviews with 48 users, we learned that they believed ConE would save time in conflict resolution and avoiding duplicate work, and that over 90% intend to keep using the service on a daily basis.

Finding the sweet spot for organizational control and team autonomy in large-scale agile software development

Agile methods and the related concepts of employee empowerment, self-management, and autonomy have reached large-scale software organizations and raise questions about commonly adopted principles for authority distribution. However, the optimum mechanism to balance the need for alignment, quality, and process control with the need or willingness of teams to be autonomous remains an unresolved issue. In this paper, we report our findings from a multiple-case study in two large-scale software development organizations in the telecom industry. We analysed the autonomy of the agile teams in the organizations using Hackman’s classification of unit authority and found that the teams were partly self-managing. Further, we found that alignment across teams can be achieved top-down by management and bottom-up through membership in communities or through dialogue between the team and management. However, the degree of team autonomy was limited by the need for organizational alignment. Top-down alignment and control were maintained through centralized decision-making for certain areas, the use of supervisory roles, mandatory processes, and checklists. One case employed a bottom-up approach to alignment through the formation of a community composed of all teams, experts, and supporting roles, but excluding managers. This community-based alignment involved teams in decision-making and engaged them in alignment initiatives. We conclude that implementation of such bottom-up structures seems to provide one possible mechanism for balancing organizational control and team autonomy in large-scale software development.

A Study on the Direction of Modeling Techniques for the Development of Large Scale Software

A Study on the Direction of Modeling Techniques for the Development of Large Scale Software

Моделирование операционных, программных и технических систем в проектах РФФИ

A wide range of issues of theory and practice in the development of large-scale software and hybrid software intensive systems, including operating systems, is considered. Issues of configuration management, modeling and verification of such systems, construction of ontological models of subject areas related to application and system software are touched upon. Such multilateral consideration is necessary to ensure reliability, security and elastic development during the multi-year period of operation of infrastructural and mission-critical systems. The paper is based on the materials of studies carried out within the two RFBR projects. In addition to recent results, the authors pay attention to the history of the development of research in the relevant areas in the Soviet Union.

SiameseQAT: A Semantic Context-Based Duplicate Bug Report Detection Using Replicated Cluster Information

In large-scale software development environments, defect reports are maintained through bug tracking systems (BTS) and analyzed by domain experts. Different users may create bug reports in a non-standard manner and may report a particular problem using a particular set of words due to stylistic choices and writing patterns. Therefore, the same defect can be reported with very different descriptions, generating non-trivial duplicates. To avoid redundant work for the development team, an expert needs to look at all new reports while trying to label possible duplicates. However, this approach is neither trivial nor scalable and directly impacts bug fix correction time. Recent efforts to find duplicate bug reports tend to focus on deep neural approaches that consider hybrid representations of bug reports, using both structured and unstructured information. Unfortunately, these approaches ignore that a single bug can have multiple previously identified duplicates and, therefore, multiple textual descriptions, titles, and categorical information. In this work, we propose SiameseQAT, a duplicate bug report detection method that considers information on individual bugs as well as information extracted from bug clusters. The SiameseQAT combines context and semantic learning on structured and unstructured features and corpus topic extraction-based features, with a novel loss function called Quintet Loss, which considers the centroid of duplicate clusters and their contextual information. We validated our approach on the well-known open-source software repositories Eclipse, NetBeans, and Open Office, comprised of more than 500 thousand bug reports. We evaluated both the retrieval and classification of duplicates, reporting a Recall@25 mean of 85% for retrieval and 84% AUROC for classification tasks, results that were significantly superior to previous works.

Behavioral Science of Software Engineering

Large-scale software development is a sociotechnical activity only bounded by human imagination, ingenuity, and creativity. It involves teams of developers progressing by coordinating their activities and communicating their bottlenecks, goals, and advancements toward the wider goal of creating large, high-quality software systems. The stakeholders they serve are diverse (for example, clients, infrastructure providers, open source communities, project managers, and regulatory authorities), and often they have many competing, implicit requirements. But, as the political and legal implications of algorithms and data (https://harvardmagazine.com/2000/01/code-is-law-html) increasingly affect society, it is imperative that the systems the developers build are high quality in terms of accurately embodying all of those requirements.

Implementing Agility in Large Software Development Projects

The present paper focuses on providing an in-depth understanding of the concept of agility and its uses in the software development projects. The paper also explains the key issues experienced by large companies while implementing agility practices and frameworks in the large-scale software development projects. The paper also develops key ideas about the procedure involved in implementing agility practices through extensive study of previous literature in this context. It also recommends certain methods in which the issues experienced during the implementation of agility in large scale software development projects can be minimized. The findings provide insights about the various challenges and reasons behinds such issues that are common in such large-scale software development projects drawing instances of international companies providing software development services.

When Agile Harms Learning and Innovation: (and What Can Be Done About It)

Originally developed for software development, Agile approaches are increasingly adopted by organizations that seek flexibility in the face of rapid change. However, little attention has been paid to the potential negative consequences of the implementation of Agile in large-scale settings. This article presents the results of a multi-site study of a multinational telecom company over five years during its implementation of Agile practices in the context of large-scale software development. The article points to six potential pitfalls of implementing such practices that may negatively influence individual learning, ideation, and exploitation capabilities. It offers advice on how to avoid these consequences in large, established firms.

A generalized multiple environmental factors software reliability model with stochastic fault detection process

Software systems have been widely applied in numerous safety–critical domains; however, large-scale software development is still considered as a complicated and expensive activity. As the latest trends in software industry accelerate the complexity and dependency of software development, such complicated and human-centered process needs to be addressed well. Meanwhile, recent survey investigations (Zhu et al. in J Syst Softw 109:150–160, 2015; Zhu and Pham in J Syst Softw 132:72–84, 2017) revealed that environmental factors, defined from software development, have significant impacts on software reliability. Considering such significant impacts, we first propose a generalized multiple-environmental-factors software reliability growth model with multiple environmental factors and the associated randomness under the martingale framework. The randomness is reflected on the process of detecting software fault. Indeed, this is a stochastic fault detection process. As an illustration, a specific multiple-environmental-factors software reliability growth model incorporating two specific environmental factors, percentage of reused modules and frequency of program specification change, is further developed. Lastly, we employ two real-world data sets to demonstrate the prediction performance of the proposed generalized multiple-environmental-factors software reliability growth model.