Code Review Research Articles

The upper bound of information diffusion in code review

BackgroundCode review, the discussion around a code change among humans, forms a communication network that enables its participants to exchange and spread information. Although reported by qualitative studies, our understanding of the capability of code review as a communication network is still limited.ObjectiveIn this article, we report on a first step towards understanding and evaluating the capability of code review as a communication network by quantifying how fast and how far information can spread through code review: the upper bound of information diffusion in code review.MethodIn an in-silico experiment, we simulate an artificial information diffusion within large (Microsoft), mid-sized (Spotify), and small code review systems (Trivago) modelled as communication networks. We then measure the minimal topological and temporal distances between the participants to quantify how far and how fast information can spread in code review.ResultsAn average code review participants in the small and mid-sized code review systems can spread information to between 72 % and 85 % of all code review participants within four weeks independently of network size and tooling; for the large code review systems, we found an absolute boundary of about 11 000 reachable participants. On average (median), information can spread between two participants in code review in less than five hops and less than five days.ConclusionWe found evidence that the communication network emerging from code review scales well and spreads information fast and broadly, corroborating the findings of prior qualitative work. The study lays the foundation for understanding and improving code review as a communication network.

Disturbances of Interoception in Schizophrenia – Focus on Neuroimaging

Abstract: The current paper presents the most recent theoretical understanding of interoception (INTC) in light of the predictive processing framework and the Embodied Predictive Interoceptive Coding model and reviews recent experimental data on different interoceptive disturbances in schizophrenia (SCZ) such as lower interoceptive accuracy, nociceptive dysfunction reflected by reduced pain sensitivity, disturbed thermoregulation, impaired gastrointestinal regulation, cardiovascular and respiratory dysfunction, and immune disturbances. The immense clinical implications of these disturbances are discussed in the context of the high rates of morbidity and mortality in SCZ related to suicide and cardiac death. Next, the potential of INTC-related treatments is reviewed, and conclusions are made on the evidence for the use of vagus nerve stimulation, mindfulness-based interventions, and exercise therapy. The neuroimaging perspective is given in a brief account of the INTC-related findings in healthy individuals suggesting the right anterior insula and anterior cingulate cortex as key brain areas for interoceptive awareness, accuracy, and sensibility, by supporting higher-order representation and integration of visceral, and somatosensory interoception with exteroceptive information. In SCZ, on the other hand, neuroimaging research on specific interoceptive abilities is lacking but indirect evidence points to disrupted activation of major INTC network hubs. There is converging evidence of changes in these hubs in terms of both structure and function (volume reductions, aberrant activation, and connectivity). This narrative review identifies several gaps in contemporary INTC-related research in SCZ that need to be addressed in future experimental and clinical studies to elucidate the exact nature of the disturbances of interoceptive abilities in modalities other than cardioception; the interactions between different interoceptive modalities; the association between interoceptive abilities and other mental functions (such as emotion recognition and regulation, decision making, learning, memory, etc.); the efficacy of nonpharmacological and pharmacological INTC-based interventions; and the relation of disturbed interoceptive abilities with brain structure and function.

Exploring Debugging Challenges and Strategies Using Structural Topic Model: A Comparative Analysis of High and Low-Performing Students

Debugging is essential for identifying and rectifying errors in programming, yet time constraints and students’ trivialization of errors often hinder progress. This study examines differences in debugging challenges and strategies among students with varying computational thinking (CT) competencies using weekly coding journals from an online undergraduate CT course. Participants used Scratch, a block-based programming language, and their journals from five assignments were analyzed using Term Frequency-Inverse Document Frequency and Structural Topic Modeling. High-performing students engaged with diverse topics and specific blocks tied to their weekly projects while low-performing students faced repetitive and broad challenges, such as understanding motion blocks and broadcast concepts. These patterns reveal that low-performing students struggle particularly during the ‘diagnose the fault’ phase of debugging, often hindering their progress in the final stage. Such challenges highlight the necessity for targeted educational interventions to improve the debugging proficiency and overall CT skills. The study underscores the importance of further research into students’ logical thinking processes during code review and debugging, suggesting the use of think-aloud protocols and detailed tracking of debugging practices for deeper insights. This research contributes to the field by showing that differentiated instruction and strategic support can enhance debugging skills across different student performance levels.

Understanding and effectively mitigating code review anxiety

Anxiety about giving and receiving code reviews has been documented as a common occurrence that leads to developers avoiding code reviews by procrastinating and limiting their cognitive engagement with them. This avoidance not only increases anxiety in the long term, but also prevents developers, their teams, and their organizations from accessing the technical and sociocognitive benefits of effective and efficient code reviews. However, software research has not yet empirically examined code review anxiety, and from this, tractable intervention targets and strategies for mitigating code review anxiety. In this study, we present an empirical framework for understanding the factors maintaining and exacerbating code review anxiety. Utilizing a randomized waitlist control trial, we also tested the effectiveness of a novel single-session cognitive-behavioral workshop intervention. Our results show evidence that positive impact can be obtained from a brief intervention and suggest code review anxiety can be successfully mitigated by targeting developers’ cost bias, anxiety self-efficacy, and self-compassion.

Deciphering refactoring branch dynamics: An empirical study on Qt

Context:Modern code review is a widely employed technique in both industrial and open-source projects, serving to enhance software quality, share knowledge, and ensure compliance with coding standards and guidelines. While code review is extensively studied for its general challenges, best practices, outcomes, and socio-technical aspects, little attention has been paid to how refactoring is reviewed and what developers prioritize when reviewing refactored code in the ‘Refactor’ branch. Objective:The goal is to understand the review process for refactoring changes in the ‘Refactor’ branch and to identify what developers care about when reviewing code in this branch. Method:In this study, we present a quantitative and qualitative examination to understand the main criteria developers use to decide whether to accept or reject refactored code submissions and identify the challenges inherent in this process. Results:Analyzing 2,154 refactoring and non-refactoring reviews across Qt open-source projects, we find that reviews involving refactoring from the ‘Refactor’ branch take significantly less time to resolve in terms of code review efforts. Additionally, documentation of developer intent is notably sparse within the ‘Refactor’ branch compared to other branches. Furthermore, through thematic analysis of a substantial sample of refactoring code review discussions, we construct a comprehensive taxonomy consisting of 12 refactoring review criteria. Conclusion:Our findings underscore the importance of developing precise and efficient tools and techniques to aid developers in the review process amidst refactorings.

Code review in digital humanities

Abstract Software and computational methods offer tremendous possibilities for digital humanities research, both accelerating existing work and opening up entirely new questions. However, software also has the potential to introduce new kinds of errors into the research workflow. How do we know that the software developed for a digital humanities project is error free and does what we think it does? Code review is a widespread technique to improve software quality and reduce the number of flaws, where a programmer other than the author reviews and comments on the source code. However, given that many digital humanities developers work in developer teams of one, code review is often not possible. In this article, we share progress and insights from an effort to establish a community code review process for digital humanities, and provide background to help understand the need and potential impacts of this work.

Understanding Barriers to STEM: Teachers' Insights on African American Underrepresentation

This study focuses on the development of a measurement instrument to identify Middle and High School mathematics and science teachers’ beliefs about the main factors of underrepresentation of African American students in STEM (Science, Technology, Engineering, and Mathematics) fields. The research method using the stages of instrument development were (1) test design which involved construction of initial item pool by conducting extensive review of literature and coding of participating teachers personal statements (2) determination of validity which involved expert review process to confirm construct and face validity of items, (3) pilot testing which involved collecting data from a second set of participants, (4) determination of reliability which involves conducting a reliability analysis based on data collected from pilot testing and (5) determination of factorial structure which involved Exploratory Factor Analysis (EFA) to identify underlying factors in participating teachers’ belief structures. First, to obtain trustworthy information, voluntary middle and high school mathematics and science teachers were asked to write personal statements where they explain their main reasons for choosing teaching as a career. Secondly, qualitative data obtained from participating teachers’ personal statements were coded. Coding was applied to identify and to group the phrases or sentences that convey the same or similar reasons. Then results from an extensive literature review on this topic were blended in teachers’ common expressions which were determined because of coding procedure. Then 8 content experts assessed content validity and face validity. Finally, the survey was piloted to African American middle and high school mathematics and science teachers. Fifty-three (53) surveys were completed and received back from teachers. Then, exploratory factor analysis was conducted to identify any underlying factors in the scale. Reliability analyses were also conducted for both the entire survey and for each of the factors. Results of the study suggested four underlying components: (1) Access and Resources, (2) Role Models and Representation, (3) Bias and Discrimination, (4) Curriculum and Pedagogy.

A preliminary investigation on using multi-task learning to predict change performance in code reviews

A preliminary investigation on using multi-task learning to predict change performance in code reviews

Industry perceptions of mass engineered timber (MET) construction: a comparison of South-East Asia with other regions

ABSTRACT Mass Engineered Timber (MET) has emerged as an innovative construction material used in the creation of low-carbon buildings. However, more built examples are found in Europe and North America, with far fewer in Asia, despite its generally higher levels of construction. This research explores these trends by measuring building industry practitioners’ perceptions – their awareness, attitudes and perceived obstacles – across three regions experiencing different levels of MET adoption: Europe and North America (early adopters), Australia and New Zealand (early developing) and South-East Asia (under developed). The research develops a novel framework to understand industry perceptions of MET composed of ‘Enabling Parameters’ and ‘Decision-making Parameters’. This is used to develop a survey which was completed by 186 participants from the three regions. Results show that participants’ perceptions, knowledge and experience of MET are consistently lower in South-East Asia, than the other regions – a finding previously not identified in existing research. Protection against fire and water damage, ease of repair and maintenance, and perceived unsuitability of MET for high-rise building types are the primary concerns of practitioners in South-East Asia. This paper calls for technological advancements for taller MET and hybrid buildings, a review of building codes, provision of incentives, and education among practitioners to improve the perception and adoption of MET in South-East Asia.

Fine-tuning Large Language Models to Improve Accuracy and Comprehensibility of Automated Code Review

Fine-tuning Large Language Models to Improve Accuracy and Comprehensibility of Automated Code Review

Delta GUI change detection using inferred models

Recent software development methodologies emphasize iterative and incremental evolution to align with stakeholders’ needs. This perpetual and rapid software evolution demands ongoing research into verification practices and technologies that ensure swift responsiveness and effective management of software delta increments. Strategies such as code review have been widely adopted for development and verification, ensuring readability and consistency in the delta increments of software projects. However, the integration of techniques to detect and visually report delta changes within the Graphical User Interface (GUI) software applications remains an underutilized process. In this paper, we set out to achieve two objectives. First, we aim to conduct a comprehensive review of existing studies concerning GUI change detection in desktop, web, and mobile applications to recognize common practices. Second, we introduce a novel change detection tool capable of highlighting delta GUI changes for this diverse range of applications. To accomplish our first objective, we performed a systematic mapping of the literature using the Scopus database. To address the second objective, we designed and developed a GUI change detection tool. This tool simultaneously transits and compares state models inferred by a scriptless testing tool, enabling the detection and highlighting of GUI changes to detect the widgets or functionalities that have been added, removed, or modified. Our study reveals the existence of a multitude of techniques for change detection in specific GUI systems with different objectives. However, there is no widely adopted technique suitable for the diverse range of existing desktop, web, and mobile applications. Our tool and findings demonstrate the effectiveness of using inferred state models to highlight between 8 and 20 GUI changes in software delta increments containing a large number of changes over months and between 4 and 6 GUI changes in delta increments of small iterations performed over multiple weeks. Moreover, some of these changes were recognized by the software developers as GUI failures that required a fix. Finally, we expose the motivation for using this technique to help developers and testers analyze GUI changes to validate delta increments and detect potential GUI failures, thereby fostering knowledge dissemination and paving the way to standard practices.

Identification of crack width behavior of one-way reinforced concrete slab structure at different steel reinforcement area

An evaluation study of crack limit states based on design codes and prior research is presented in this publication. Its main goal is to connect research findings to common design codes. Researchers continue to face a difficult dilemma when it comes to reinforced concrete structure fractures, particularly in one-way slab constructions where there is still significant damage and corrosion in the reinforcement because of cracks. Practitioners will find it easier to construct these structures and solve the slab durability issue if the proper formula is discovered. One can overcome reinforced concrete. A method for estimating the maximum fracture width formula in one-way reinforced concrete slabs with varying steel areas is suggested based on this research. Slabs use a variety of steel areas, including 1000 mm2, 1200 mm2, and 1400 mm2.The test specimens are the same length of 2 meters and have a slab width of 0.6 meters with steel reinforcement. Findings from a literature review of research codes and prediction formulas from earlier studies, namely wmax(prop)=1.5*10-2fsAs-0.4, indicate that the maximum crack width is not significantly influenced by steel area (As). Overall, the findings from the two methods used in this analysis match the suggested formula and the observed experimental testing. This data indicates that the maximum fracture width has been greatly lowered by increasing the steel Area (As) of the reinforced concrete slab, leading to the determination of the experimental formula, As a result, a unique approximation formula has been developed to assess the impact of steel area parameters for pure slabs on the maximum crack width formula for one-way reinforced concrete slabs. This crack width formula is only applicable to one-way slabs in practice.

Barriers to Childhood Immunization in Rural and Remote Areas: A Qualitative Exploration From the Perspectives of Community Leaders in Sindh, Pakistan.

Despite overall improvements observed at the national level, there is a significant disparity in childhood vaccination coverage rates between urban and rural regions, particularly within tribal and remote areas in Pakistan. Our study aimed to explore the views of community/tribal leaders concerning barriers and their local solutions for improving vaccine uptake, with a specific focus on remote settings. An exploratory qualitative research study was conducted from July to September 2019 in Shikarpur, Sindh, Pakistan. We interviewed 11 community leaders, after developing a semi-structured interview guide based on the health belief model. Following validation, the collected data was transcribed and subsequently translated into the English language. A stepwise process of manual familiarization, coding, theme generation, and theme review using an inductive approach was followed. While most of the rural community leaders expressed a willingness to support vaccinations, numerous unexplored barriers were identified. These barriers included an unsustainable communication system heavily dependent on polio mobile teams, discourteous behavior of healthcare personnel, cultural restrictions that limited women's involvement, economic hardships, limited transportation options, insufficient collaboration with the education sector, poor knowledge among community leaders, and security concerns. Additionally, they proposed some innovative solutions of information dissemination methods through the Ottaque system, ethics-based training, subsidized transportation services or voucher system, and addressing security concerns in collaboration with community leaders. Our findings suggest policymakers take both community leaders and members for an inclusive policy-making process to redraft the special policy for these remote and rural areas.

Review of historical plumbing codes for lead service line inventories in the United States

ABSTRACT Inventorying and replacing lead service lines (LSLs) is a national priority in the United States (US), with records review conducted as a first step. Among other records, historical plumbing codes can provide cutoff dates for LSL allowance in the US. This work identified and reviewed historical national plumbing codes for mention of lead for drinking water pipe and found that lead pipe was permissible for service lines by the Basic Plumbing Code until 1978, the Uniform Plumbing Code until 1988, and the Southern Standard Plumbing Code until 1977/78. Overall, identifying pertinent code titles over time and obtaining them was challenging because those records were not readily available. Once obtained, interpreting the code language also proved challenging. Terminology initially reflected sanitary systems before drinking water systems were incorporated, and code editions through the years reorganized the content and specific mentions of lead in service lines within text. This review focused on model plumbing codes, which are then adopted at the state or local level, typically with modifications that also need to be understood. Two practical examples of service line inventory development were also summarized. These examples highlighted placing different emphasis on plumbing code review.

Comparison of Claudin-4, BerEP4, Carcinoembryonic Antigen and MOC31 in Serous Fluids Metastases Demonstrate High Sensitivity of Claudin-4 at Low Cellularity.

Claudin-4 has been described as a highly sensitive immunocytochemical marker for detection of metastatic carcinoma cells in effusion cytology specimens. This study aims to challenge the performance of claudin-4 in different types of malignancies and low cellularity specimens, by comparison with other markers in a large cohort of carcinomatous effusion specimens. Cell block preparations from peritoneal and pleural fluid specimens were retrieved, with malignant (carcinoma) diagnoses confirmed by review of hospital diagnosis code and pathology reports. Claudin-4, BerEP4, CEA, and MOC31 immunocytochemistry were performed and scored by expression proportion and intensity. Tumor cellularity was assessed for subgroup analysis of low cellularity specimens. Totally 147 specimens (70 pleural, 77 peritoneal) of 68 lung, 62 breast, 9 gynecological, and 7 gastrointestinal carcinomas were retrieved. The average proportion expression of claudin-4 was highest (89.6%, vs. CEA 40.5%, BerEp4 18.6%, MOC31 16.8%) and the percentage of strong expression was highest for claudin-4 (72.1%). Expression levels of claudin-4 were consistently higher than other markers in subgroups of all primary sites. The difference was more significant for low cellularity specimens. High (≥50%) proportion expression was seen for 96.61% of cases for claudin-4 (vs. BerEp4 8.77%, CEA 46.55%, MOC31 8.77%, p < 0.001). These factors contributed to a low concordance between claudin-4 and BerEp4, CEA and MOC31 (K = 0.010-0.043). Claudin-4 is more sensitive than CEA, BerEp4 and MOC31, suitable for low cellularity specimens of most types of metastatic carcinoma and is a robust immunocytochemical marker for carcinoma that can be used solitarily.

Don’t forget to change these functions! recommending co-changed functions in modern code review

Context:Code review is effective and widely used, yet still time-consuming. Especially, in large-scale software systems, developers may forget to change other related functions that must be changed together (aka. co-changes). This may increase the number of review iterations and reviewing time, thus delaying the code review process. Based on our analysis of 66 projects from five open-source systems, we find that there are 16%–33% of code reviews where at least one function must be co-changed, but was not initially changed. Objectives:This study aims to propose an approach to recommend co-changed functions in the context of modern code review, which could reduce reviewing time and iterations and help developers identify functions that need to be changed together. Methods:We propose CoChangeFinder, a novel method that employs a Graph Neural Network (GNN) to recommend co-changed functions for newly submitted code changes. Then, we conduct a quantitative and qualitative evaluation of CoChangeFinder with 66 studied large-scale open-source software projects. Results:Our evaluation results show that our CoChangeFinder outperforms the state-of-the-art approach, achieving 3.44% to 40.45% for top-k accuracy, 2.00% to 26.07% for Recall@k, and 0.04 to 0.21 for mean average precision better than the baseline approach. In addition, our CoChangeFinder demonstrates the capacity to pinpoint the functions related to logic changes. Conclusion:Our CoChangeFinder outperforms the baseline approach (i.e., TARMAQ) in recommending co-changed functions during the code review process. Based on our findings, CoChangeFinder could help developers save their time and effort, reduce review iterations, and enhance the efficiency of the code review process.

On the acceptance by code reviewers of candidate security patches suggested by Automated Program Repair tools

ObjectiveWe investigated whether (possibly wrong) security patches suggested by Automated Program Repairs (APR) for real world projects are recognized by human reviewers. We also investigated whether knowing that a patch was produced by an allegedly specialized tool does change the decision of human reviewers.MethodWe perform an experiment with n=72\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$n= 72$$\\end{document} Master students in Computer Science. In the first phase, using a balanced design, we propose to human reviewers a combination of patches proposed by APR tools for different vulnerabilities and ask reviewers to adopt or reject the proposed patches. In the second phase, we tell participants that some of the proposed patches were generated by security-specialized tools (even if the tool was actually a ‘normal’ APR tool) and measure whether the human reviewers would change their decision to adopt or reject a patch.ResultsIt is easier to identify wrong patches than correct patches, and correct patches are not confused with partially correct patches. Also patches from APR Security tools are adopted more often than patches suggested by generic APR tools but there is not enough evidence to verify if ‘bogus’ security claims are distinguishable from ‘true security’ claims. Finally, the number of switches to the patches suggested by security tool is significantly higher after the security information is revealed irrespective of correctness.LimitationsThe experiment was conducted in an academic setting, and focused on a limited sample of popular APR tools and popular vulnerability types.

MULTICR: Predicting Merged and Abandoned Code Changes in Modern Code Review Using Multi-Objective Search

Modern Code Review (MCR) is an essential process in software development to ensure high-quality code. However, developers often spend considerable time reviewing code changes before being merged into the main code base. Previous studies attempted to predict whether a code change was going to be merged or abandoned soon after it was submitted to improve the code review process. However, these approaches require complex cost-sensitive learning, which makes their adoption challenging since it is difficult for developers to understand the main factors behind the models’ predictions. To address this issue, we introduce in this paper, MULTICR, a multi-objective search-based approach that uses Multi-Objective Genetic Programming (MOGP) to learn early code review prediction models as IF-THEN rules. MULTICR evolves predictive models while maximizing the accuracy of both merged and abandoned classes, eliminating the need for misclassification cost estimation. To evaluate MULTICR, we conducted an empirical study on 146,612 code reviews from Eclipse, LibreOffice, and Gerrithub. The obtained results show that MULTICR outperforms existing baselines in terms of Mathew correlation coefficient (MCC) and F1 scores while learning less complex models compared to decision trees. Our experiments showed also how MULTICR allows identifying the main factors related to abandoned code reviews as well as their associated thresholds, making it a promising approach for early code review prediction with notable performance and interoperability. Additionally, we qualitatively evaluate MULTICR by conducting a user study through semi-structured interviews involving 10 practitioners from different organizations. The obtained results indicate that 90% of the participants find that MULTICR is useful and can help them to improve the code review process. Additionally, the learned IF-THEN rules of MULTICR are transparent.

EvaluateXAI: A framework to evaluate the reliability and consistency of rule-based XAI techniques for software analytics tasks

The advancement of machine learning (ML) models has led to the development of ML-based approaches to improve numerous software engineering tasks in software maintenance and evolution. Nevertheless, research indicates that despite their potential successes, ML models may not be employed in real-world scenarios because they often remain a black box to practitioners, lacking explainability in their reasoning. Recently, various rule-based model-agnostic Explainable AI (XAI) techniques, such as PyExplainer and LIME, have been employed to explain the predictions of ML models in software analytics tasks. In this paper, we assess the ability of these techniques, particularly the state-of-the-art PyExplainer and LIME, to generate reliable and consistent explanations for ML models across various software analytics tasks, including Just-in-Time (JIT) defect prediction, clone detection, and the classification of useful code review comments. Our manual investigations find inconsistencies and anomalies in the explanations generated by these techniques. Therefore, we design a novel framework: Evaluation of Explainable AI (EvaluateXAI), along with granular-level evaluation metrics, to automatically assess the effectiveness of rule-based XAI techniques in generating reliable and consistent explanations for ML models in software analytics tasks. After conducting in-depth experiments involving seven state-of-the-art ML models trained on five datasets and six evaluation metrics, we find that none of the evaluation metrics reached 100%, indicating the unreliability of the explanations generated by XAI techniques. Additionally, PyExplainer and LIME failed to provide consistent explanations for 86.11% and 77.78% of the experimental combinations, respectively. Therefore, our experimental findings emphasize the necessity for further research in XAI to produce reliable and consistent explanations for ML models in software analytics tasks.

Debunking the Myths of Code Velocity: The Elusive Search for Speed

We demonstrate that no empirical evidence supports long-held beliefs related to code velocity. The best-known pragmatic way to increase code velocity is to periodically nudge engineers to unblock the code review process. The industry needs to revisit the Modern Code Review process in light of the availability of various tools and techniques that analyze and prevalidate the proposed code changes. The code review bots are coming.