The Impact of LLM-Assistants on Software Developer Productivity: A Systematic Review and Mapping Study
This systematic review of 39 studies examines how large language model assistants impact software developer productivity, revealing mostly benefits such as faster development and task automation, but also concerns like reduced collaboration and unresolved effects on code quality, with most evaluations being exploratory and multi-dimensional.
Large language model assistants (LLM-assistants) present new opportunities to transform software development. Developers are increasingly adopting these tools across tasks, including coding, testing, debugging, documentation, and design. Yet, despite growing interest, there is no synthesis of how LLM-assistants affect software developer productivity. In this paper, we present a systematic review and mapping of 39 peer-reviewed studies published between January 2014 and December 2024 that examine this impact. Our analysis reveals that the majority of studies report considerable benefits from LLM-assistants, though a notable subset identifies critical risks. Commonly reported gains include accelerated development, minimized code search, and the automation of trivial and repetitive tasks. However, studies also highlight concerns around cognitive offloading and reduced team collaboration. Our study reveals that whether LLM-based assistants improve or degrade code quality remains unresolved, as existing studies report contradictory outcomes contingent on context and evaluation criteria. While the majority of studies (90%) adopt a multi-dimensional perspective by examining at least two SPACE dimensions, reflecting increased awareness of the complexity of developer productivity, only 15% extend beyond three dimensions, indicating substantial room for more integrated evaluations. Satisfaction, Performance, and Efficiency are the most frequently investigated dimensions, whereas Communication and Activity remain underexplored. Most studies are exploratory (59%) and methodologically diverse, but lack longitudinal and team-based evaluations. This review surfaces key research gaps and provides recommendations for future research and practice. All artifacts associated with this study are publicly available at https://zenodo.org/records/18489222 .
- Research Article
1
- 10.1016/0066-4138(90)90011-f
- Jan 1, 1988
- Annual Review in Automatic Programming
Methods for monitoring productivity in applicative software development
- Research Article
- 10.1016/s1474-6670(17)53682-4
- Sep 1, 1988
- IFAC Proceedings Volumes
Methods for Monitoring Productivity in Applicative Software Development
- Research Article
116
- 10.1016/j.infsof.2013.02.006
- Mar 1, 2013
- Information and Software Technology
A systematic mapping study of web application testing
- Conference Article
2
- 10.1109/clei.2018.00021
- Oct 1, 2018
Startup companies are seeking for a business model by means of innovative products and need external investments until they can stabilize this product in the market and then start to grow to become a mature company. Nevertheless, many of these companies fail before the growth phase, emerging the need to seek for methods that help these companies to achieve their goals. This paper intends to identify the techniques, practices and tools used in software development on startups and from the results present a knowledge base that can be transferable to the industry. A systematic mapping study was conducted using a classification schema to attest the primary studies quality as well as a exclusion and inclusion criteria to select them. A total of 112 studies are found at the end of the searches and 19 of them were selected to form the mapping study. From these studies a total of 24 techniques, 31 practices and 37 tools were found. As a result the mapping study presents a technical knowledge base that seeks to fill the research gap and that can also be used as a starting point for both researchers and startups' practitioners.
- Conference Article
34
- 10.1049/ic.2012.0016
- Jan 1, 2012
Context: We have been undertaking a series of case studies to investigate the value of mapping (scoping) studies in software engineering. Our previous studies have assessed these using the subjective opinions of researchers. Objective: In order to provide a more objective assessment of value, for this study, we used the results of a systematic mapping study to investigate how well mapping studies identify clusters of related studies and to what extent such clusters are complete. Method: In this participant-observer case study, we undertook a mapping study of unit testing and regression testing empirical studies, which we compared with a previous expert literature review and with six other mapping studies and systematic literature reviews (SLRs) that addressed overlapping topics. Results: Our mapping study found more clusters than the expert literature review although it benefited from the set of studies identified by the expert review when refining our search process. The set of studies found by our searches were less complete than those found by SLRs addressing more specific topics, although we found some studies missed by those SLRs. Conclusions: Researchers undertaking systematic reviews and mapping studies should make use of related systematic reviews and mapping studies to identify known studies in order to refine search strings and validate search results. For completeness and traceability, mapping studies should keep a record of all multiple reports of a single study. Meta-analyses and other systematic literature reviews undertaking detailed aggregation should report on candidate primary studies that were rejected in the final screening process, as well as candidate studies that were included. This helps ensure the repeatability of aggregation results.
- Conference Article
3168
- 10.14236/ewic/ease2008.8
- Jun 1, 2008
- Electronic workshops in computing
BACKGROUND: A software engineering systematic map is a defined method to build a classification scheme and structure a software engineering field of interest. The analysis of results focuses on frequencies of publications for categories within the scheme. Thereby, the coverage of the research field can be determined. Different facets of the scheme can also be combined to answer more specific research questions. OBJECTIVE: We describe how to conduct a systematic mapping study in software engineering and provide guidelines. We also compare systematic maps and systematic reviews to clarify how to chose between them. This comparison leads to a set of guidelines for systematic maps. METHOD: We have defined a systematic mapping process and applied it to complete a systematic mapping study. Furthermore, we compare systematic maps with systematic reviews by systematically analyzing existing systematic reviews. RESULTS: We describe a process for software engineering systematic mapping studies and compare it to systematic reviews. Based on this, guidelines for conducting systematic maps are defined. CONCLUSIONS: Systematic maps and reviews are different in terms of goals, breadth, validity issues and implications. Thus, they should be used complementarily and require different methods (e.g., for analysis).
- Research Article
4
- 10.1002/spe.2504
- May 12, 2017
- Software: Practice and Experience
Special issue on software reuse
- Conference Article
6
- 10.1109/icsa-c50368.2020.00050
- Mar 1, 2020
Context: System of Systems (SoS) is a set of independent systems that cooperate to achieve an emergent behavior. SoSs have been used in different domains such as defense, transportation, energy, and health care, which directly impact on the society. The critical nature of SoS, in which a failure in one of its Constituent Systems (CSs) may lead to catastrophic damages to the property, environment, injuries or loss of human's life, demands risk management activities. Existing risk management practices applied to SoS are extensions of risk management techniques at the CS level. Objective: in this paper, we present an overview of risk management approaches and tools for SoS. Method: we performed a Systematic Mapping (SM) study by searching into five databases to identify primary studies. We identified 22 primary studies related to risk management practices for SoS. Results: from the analysis of these primary studies, we identified a set of risks and risk management practices for SoS and their differences to risk management techniques at the CS level. Conclusion: the identified approaches and support tools for risk management in the SoS level are not well established yet.
- Research Article
- 10.1002/widm.70078
- Mar 1, 2026
- WIREs Data Mining and Knowledge Discovery
This review provides a comprehensive Systematic Literature Review (SLR) and Systematic Mapping Study (SMS) of regression research published in Wiley‐indexed venues between 1975 and 2025 . Analyzing a validated corpus of 1466 publications—of which 1174 dated studies form the empirical base—this study traces the theoretical, methodological, and epistemic evolution of regression modeling. The findings reveal a progressive convergence between classical penalized methods (e.g., LASSO, Elastic Net), Bayesian shrinkage approaches, and machine learning‐based regressors such as ensembles and neural networks. From 2022 onward, a distinct interpretability‐driven era emerges, marked by the selective uptake of post hoc explanation techniques like SHAP and LIME. Despite their conceptual appeal, these tools remain underutilized in the corpus—due in part to computational costs, concerns over stability, and the prioritization of embedded interpretability strategies. Through thematic synthesis, the review identifies persistent trade‐offs in regression design—bias versus variance, sparsity versus signal retention, and accuracy versus interpretability—and highlights exemplar hybrid approaches that balance these tensions. The study concludes with a research agenda targeting five frontiers: (1) integrated interpretability systems, (2) unified theoretical modeling frameworks, (3) scalable regression for high‐dimensional and streaming data, (4) robust and uncertainty‐aware modeling, and (5) fairness and transparency in applied settings. Together, these contributions reposition regression as a unified, interdisciplinary field grounded in statistical insight, computational design, and ethical accountability.
- Conference Article
11
- 10.1109/cbi49978.2020.10055
- Jun 1, 2020
Nowadays, education is called upon to fulfill its mission in complex and fast-changing environments. According to the OECD, globalization, democracy issues, security risks, ageing societies and modern cultures are the global mega-trends affecting the way education deploys its functions. Enterprise Architecture (EA), as a key enabler of strategy formulation and business-IT alignment, could play a central role in helping ministries and educational organizations develop their full IT strategy and gain a competitive advantage. In this perspective, a systematic EA mapping study was conducted on major research databases, academic journals and conference proceedings, in order to identify the major approaches and challenges of EA in the education domain. A total amount of 60 articles from the past 10 years were found and analyzed (e.g. the EA lifecycle phase of each initiative). Our analysis reveals that most research focuses on the development of EA for Higher Education Institutions, while the primary, secondary education and lifelong learning are almost ignored. A notable research gap is also uncovered, namely, the small number of articles focusing on EA implementation and assessment even in the educational systems of countries considered pioneers in the field.
- Book Chapter
21
- 10.1201/9781315154855-2
- Nov 28, 2017
Security engineering in the software lifecycle aims at protecting information and systems to guarantee confidentiality, integrity, and availability. As security engineering matures and the number of research papers grows, there is an increasing need for papers that summarize results and provide an overview of the area. A systematic mapping study maps a research area by classifying papers to identify which topics are well-studied and which need additional study. Therefore, systematic mapping studies are becoming increasingly important in security engineering. This chapter provides methodological support for systematic mapping studies in security engineering based on examples from published security engineering papers. Because security engineering is similar to software engineering in that it bridges research and practice, researchers can use the same basic systematic mapping process, as follows: (1) study planning, (2) searching for studies, (3) study selection, (4) study quality assessment, (5) data extraction, (6) data classification, (7) data analysis, and (8) reporting of results. We use published mapping studies to describe the tailoring of this process for security engineering. In addition to guidance on how to perform systematic mapping studies in security engineering, this chapter should increase awareness in the security engineering community of the need for additional mapping studies.
- Conference Article
12
- 10.1049/ic.2012.0031
- Jan 1, 2012
Background- Risks associated with a software project have the potential to affect all stakeholders. Today much software makes use of off-the-shelf (OTS) components. A better understanding of OTS-derived software risks will help to define responsibilities for these risks, and also to avoid them. Aim- Our objective is to identify, classify and compare risks of OTS-based software projects from both a software development and a software acquisition perspective. Method- To identify and classify the risks, we performed a systematic mapping study. In order to compare risks of OTS-based software development and acquisition in the real world setting, we used the mapping study results to survey occurrences of 11 shared risks in OTS-based software, in 35 OTS-based software developments and 34 OT-Sbased software acquisitions of Indonesian background. The survey is a partial replication of a previous study. Results- We identified 133 risks associated with OTS-based software development and 36 risks associated with OTS-based software acquisition. These risks are grouped into 17 risk categories. Risks occurred more frequently in software acquisition than in software development. In addition, two risks, insufficient OTS component documents and lack of provider technical support and training, frequently occurred only in the software development. Conclusions- In OTS-based projects, most risks for acquisition and development are similar. Technical-related risks are found less often in acquisition and project management related risks are found less often in development. Shared risks are perceived differently by developers and acquirers. Better understanding of actual and perceived risk in OTS-based software projects will improve risk management. Further work to validate these results is ongoing.
- Research Article
76
- 10.1109/access.2021.3052311
- Jan 1, 2021
- IEEE Access
In the modern digital era, software systems are extensively adapted and have become an integral component of human society. Such wide use of software systems consists of large and more critical data that inevitably needs to be secured. It is imperative to make sure that these software systems not only satisfy the users’ needs or functional requirements, but it is equally important to make sure the security of these software systems. However, recent research shows that many software development methods do not explicitly include software security measures during software development as they move from demand engineering to their final losses. Integrating software security at each stage of the software development life cycle (SDLC) has become an urgent need. Tackling software security, various methods, techniques, and models have been suggested and developed, however, only a few of them provide strong evidence for building secure software applications. The main purpose of this research is to study security measures in the context of the development of secure software (SSD) during the study of systematic mapping (SMS). Based on the inclusion and exclusion criteria, 116 studies were selected. After the data extraction from the selected 116 papers, these were classified based on the quality assessment, software security method, SDLC phases, publication venue, and SWOT analysis. The results indicate that this domain is still immature and sufficient research work needs to be carried out particularly on empirically evaluated solutions.
- Research Article
98
- 10.1016/j.jss.2016.08.039
- Aug 21, 2016
- Journal of Systems and Software
Software architectures for robotic systems: A systematic mapping study
- Research Article
7
- 10.4236/jsea.2011.411072
- Jan 1, 2011
- Journal of Software Engineering and Applications
In this paper, we identify a set of factors that may be used to forecast software productivity and software development time. Software productivity was measured in function points per person hours, and software development time was measured in number of elapsed days. Using field data on over 130 field software projects from various industries, we empirically test the impact of team size, integrated computer aided software engineering (ICASE) tools, software development type, software development platform, and programming language type on the software development productivity and development time. Our results indicate that team size, software development type, software development platform, and programming language type significantly impact software development productivity. However, only team size significantly impacts software development time. Our results indicate that effective management of software development teams, and using different management strategies for different software development type environments may improve software development productivity.