Software Maintenance Activities Research Articles

Commit-Level Software Change Intent Classification Using a Pre-Trained Transformer-Based Code Model

Software evolution is driven by changes made during software development and maintenance. While source control systems effectively manage these changes at the commit level, the intent behind them are often inadequately documented, making understanding their rationale challenging. Existing commit intent classification approaches, largely reliant on commit messages, only partially capture the underlying intent, predominantly due to the messages’ inadequate content and neglect of the semantic nuances in code changes. This paper presents a novel method for extracting semantic features from commits based on modifications in the source code, where each commit is represented by one or more fine-grained conjoint code changes, e.g., file-level or hunk-level changes. To address the unstructured nature of code, the method leverages a pre-trained transformer-based code model, further trained through task-adaptive pre-training and fine-tuning on the downstream task of intent classification. This fine-tuned task-adapted pre-trained code model is then utilized to embed fine-grained conjoint changes in a commit, which are aggregated into a unified commit-level vector representation. The proposed method was evaluated using two BERT-based code models, i.e., CodeBERT and GraphCodeBERT, and various aggregation techniques on data from open-source Java software projects. The results show that the proposed method can be used to effectively extract commit embeddings as features for commit intent classification and outperform current state-of-the-art methods of code commit representation for intent categorization in terms of software maintenance activities undertaken by commits.

Enhancing Software Project Monitoring with Multidimensional Data Repository Mining

Software project development and maintenance activities have been reported in various repositories. The data contained in these repositories have been widely used in various studies on specific problems, e.g., predicting bug appearance, allocating issues to developers, and identifying duplicated issues. Developed analysis schemes are usually based on simplified data models while issue report details are neglected. Confronting this problem requires a deep and wide-ranging exploration of software repository contents adapted to their specificities, which differs significantly from classical data mining. This paper is targeted at three aspects: the structural and semantic exploration of repositories, deriving characteristic features in value and time perspectives, and defining the space of project monitoring goals. The considerations presented demonstrate a holistic image of the project development process, which is useful in the assessment of its efficiency and identification of imperfections. The original analysis introduced in this work was verified using open source and some commercial software project repositories.

Forecasting the Principal of Code Technical Debt in JavaScript Applications

JavaScript (JS) is one of the most popular programming languages for developing client-side applications mainly due to allowing the adoption of different programming styles, not having strict syntax rules, and supporting a plethora of frameworks. The flexibility that the language provides may accelerate the development of application, but also pose threats to the quality of the final software product, e.g., introducing Technical Debt (TD). TD reflects the additional cost of software maintenance activities to implement new features, occurring due to poorly developed solutions. Being able to forecast the levels of TD in the future can be extremely valuable in managing TD, since it can contribute to informed decision making when designating future repayments and refactoring budget among a company's projects. Despite the popularity of JS and the undoubtful benefits of accurate TD forecasting, in the literature, there is available only a limited number of tools and methodologies that are able to: (a) forecast TD during software evolution, (b) provide a ground-truth TD quantifications to train forecasting, since TD tools that are available are based on different rulesets and none is recognized as a state-of-the-art solution, (c) take into consideration the language-specific characteristics of JS. As a main contribution for this study, we propose a methodology (along with a supporting tool) that supports the aforementioned goals based on the Backward Stepwise Regression and Auto-Regressive Integrated Moving Average (ARIMA). We evaluate the proposed approach through a case study on 19,636 releases of 105 open-source applications. The results point out that: (a) the proposed model can lead to an accurate prediction of TD, and (b) the Number of appearances of the “new” and “eval” keyword along with the number of “anonymous” and “arrow” functions are among the features of JavaScript language that are related to high levels of TD.

A Notional Understanding of the Relationship between Code Readability and Software Complexity

Code readability and software complexity are considered essential components of software quality. They significantly impact software metrics, such as reusability and maintenance. The maintainability process consumes a high percentage of the software lifecycle cost, which is considered a very costly phase and should be given more focus and attention. For this reason, the importance of code readability and software complexity is addressed by considering the most time-consuming component in all software maintenance activities. This paper empirically studies the relationship between code readability and software complexity using various readability and complexity metrics and machine learning algorithms. The results are derived from an analysis dataset containing roughly 12,180 Java files, 25 readability features, and several complexity metric variables. Our study empirically shows how these two attributes affect each other. The code readability affects software complexity with 90.15% effectiveness using a decision tree classifier. In addition, the impact of software complexity on the readability of code using the decision tree classifier has a 90.01% prediction accuracy.

On the diffusion of test smells and their relationship with test code quality of Java projects

AbstractTest smells are considered bad practices that can reduce the test code quality, thus harming software testing goals and maintenance activities. Prior studies have investigated the diffusion of test smells and their impact on test code maintainability. However, we cannot directly compare the outcomes of the studies as most of them use customized datasets. In response, we introduced the TSSM (Test Smells and Structural Metrics) dataset, containing test smells detected using the JNose Test tool and structural metrics (test code and production code) calculated with the CK metrics tool of 13,703 open‐source Java systems from GitHub. In addition, we perform an empirical study to investigate the relationship between test smells and structural metrics of test code and the relationship between test smells on a large‐scale dataset. We split the projects into three clusters to analyze the distribution of test smells, the co‐occurrences among test smells, and the correlation of test smells and structural metrics of test code. The ratio of smelly test classes with a specific test smell is similar among the clusters, but we could observe a significant difference in the number of test smells among them. The test smells Sleepy Test, Mystery Guest, and Resource Optimism rarely occur in the three clusters, and the last two are strongly correlated, indicating that those test smells are more severe than others. Our results point out that most test smells have a moderate correlation with high complexity, large size, and coupling of the test code, indicating that they can also negatively affect its quality. To support further studies, we made our dataset publicly available.

Automatic software vulnerability classification by extracting vulnerability triggers

AbstractVulnerability classification is a significant activity in software development and software maintenance. Natural Language Processing (NLP) techniques, which utilize the descriptions in public repositories, are widely used in automatic software vulnerability classification. However, vulnerability descriptions are ordinarily short and contain many technical terms, making them difficult for machines to automatically comprehend. In this paper, we present an approach based on vulnerability triggers to automatically classify vulnerabilities. First, we extract vulnerability triggers with Bert Question and Answer (Bert Q&A). Then, we use Recurrent Convolutional Neural Networks for Text classification (TextRCNN) to classify vulnerabilities based on Common Weakness Enumeration (CWE). We statistically perform an analysis of vulnerability triggers and comprehensively evaluate the classification performance of our approach on a set of 4769 prelabeled vulnerability entries, as well as compare it with state‐of‐the‐art vulnerability classification approaches. Experiment results show that our approach can achieve a F1‐measure of 95% on extraction and 80.8% on classification.

Using deep temporal convolutional networks to just-in-time forecast technical debt principal

Technical debt is a widely used metaphor to summarize all the consequences of poorly written code. Managing technical debt is important for software developers to allow adequate planning for software maintenance and improvement activities, such as refactoring and preventing system degradation. Several studies in the literature investigate the identification of technical debt and its consequences. This work aims to explore a deep learning approach to just-in-time predict the impact on technical debt when changes are performed on the source code. In this way the developer can work better, trying to improve the quality of the code that is being modified. Knowing what the TD trend will be in just-in-time source code with the change made is the key to avoiding a project taking a long time to remediate or improve. The model exploits the knowledge of quality and ad-hoc process metrics evolution over time. To validate the approach, a large dataset, including metrics evaluated from commits of ten Java software projects, was built. The results obtained show the effectiveness of the proposed approach in predicting the Technical Debt accumulation within the source code.

Patchworking: Exploring the code changes induced by vulnerability fixing activities

Identifying and repairing vulnerable code is a critical software maintenance task. Change impact analysis plays an important role during software maintenance, as it helps software maintainers to figure out the potential effects of a change before it is applied. However, while the software engineering community has extensively studied techniques and tools for performing impact analysis of change requests, there are no approaches for estimating the impact when the change involves the resolution of a vulnerability bug. We hypothesize that similar vulnerabilities may present similar strategies for patching. More specifically, our work aims at understanding whether the class of the vulnerability to fix may determine the type of impact on the system to repair. To verify our conjecture, in this paper, we examine 524 security patches applied to vulnerabilities belonging to ten different weakness categories and extracted from 98 different open-source projects written in Java. We obtain empirical evidence that vulnerabilities of the same types are often resolved by applying similar code transformations, and, thus, produce almost the same impact on the codebase. On the one hand, our findings open the way to better management of software maintenance activities when dealing with software vulnerabilities. Indeed, vulnerability class information could be exploited to better predict how much code will be affected by the fixing, how the structural properties of the code (i.e., complexity, coupling, cohesion, size) will change, and the effort required for the fix. On the other hand, our results can be leveraged for improving automated strategies supporting developers when they have to deal with security flaws. • Vulnerabilities of the same types are often resolved by applying similar code transformations. • Security patches to CWE-264 vulnerabilities involve more files than the other vulnerabilities. • Vulnerabilities caused by improper data handling entail the addition of new methods. • Vulnerabilities caused by improper input/output sanitization require the addition of new conditional branches.

A Review on Software Bug Localization Techniques using a Motivational Example

Software bug localization is an essential step within the software maintenance activity, consuming about 70% of the time and cost of the software development life cycle. Therefore, the need to enhance the automation process of software bug localization is important. This paper surveys various software bug localization techniques. Furthermore, a running motivational example is utilized throughout the paper. Such motivational example illustrates the surveyed bug localization techniques, while highlighting their pros and cons. The motivational example utilizes different software artifacts that get created throughout the software development lifecycle, and sheds light on those software artifacts that remain poorly utilized within existing bug localization techniques, regardless of the rich wealth of knowledge embedded within them. This research thus presents guidance on what artifacts should future bug localization techniques focus, to enhance the accuracy of bug localization, and speedup the software maintenance process.

Naming the Identified Feature Implementation Blocks from Software Source Code

Identifying software identifiers that implement a particular feature of a software product is known as feature identification. Feature identification is one of the most critical and popular processes performed by software engineers during software maintenance activity. However, a meaningful name must be assigned to the Identified Feature Implementation Block (IFIB) to complete the feature identification process. The feature naming process remains a challenging task, where the majority of existing approaches manually assign the name of the IFIB. In this paper, the approach called FeatureClouds was proposed, which can be exploited by software developers to name the IFIBs from software code. FeatureClouds approach incorporates word clouds visualization technique to name Feature Blocks (FBs) by using the most frequent words across these blocks. FeatureClouds had evaluated by assessing its added benefit to the current approaches in the literature, where limited tool support was supplied to software developers to distinguish feature names of the IFIBs. For validity, FeatureClouds had applied to draw shapes and ArgoUML software. The findings showed that the proposed approach achieved promising results according to well-known metrics in terms of Precision and Recall.

Unveiling process insights from refactoring practices

Context: Software comprehension and maintenance activities, such as refactoring, are said to be negatively impacted by software complexity. The methods used to measure software product and processes complexity have been thoroughly debated in the literature. However, the discernment about the possible links between these two dimensions, particularly on the benefits of using the process perspective, has a long journey ahead.Objective: To improve the understanding of the liaison of developers’ activities and software complexity within a refactoring task, namely by evaluating if process metrics gathered from the IDE, using process mining methods and tools, are suitable to accurately classify different refactoring practices and the resulting software complexity.Method: We mined source code metrics from a software product after a quality improvement task was given in parallel to (117) software developers, organized in (71) teams. Simultaneously, we collected events from their IDE work sessions (320) and used process mining to model their processes and extract the correspondent metrics.Results: Most teams using a plugin for refactoring (JDeodorant) reduced software complexity more effectively and with simpler processes than the ones that performed refactoring using only Eclipse native features. We were able to find moderate correlations (≈43%) between software cyclomatic complexity and process cyclomatic complexity. Using only process driven metrics, we computed ≈30,000 models aiming to predict the type of refactoring method (automatic or manual) teams had used and the expected level of software cyclomatic complexity reduction after their work sessions. The best models found for the refactoring method and cyclomatic complexity level predictions, had an accuracy of 92.95% and 94.36%, respectively.Conclusions: We have demonstrated the feasibility of an approach that allows building cross-cutting analytical models in software projects, such as the one we used for detecting manual or automatic refactoring practices. Events from the development tools and support activities can be collected, transformed, aggregated, and analyzed with fewer privacy concerns or technical constraints than source code-driven metrics. This makes our approach agnostic to programming languages, geographic location, or development practices, making it suitable for challenging contexts, such as, in modern global software development where many projects adopt agile methodologies, and low/no code platforms. Initial findings are encouraging, and lead us to suggest practitioners may use our method in other development tasks, such as, defect analysis and unit or integration tests.

Model Architecture for Subversion Software Repositories System

In the development cycle of any software, software evolution plays an important role. Software Evolution is essential and challenges characteristics in the field of software engineering. Many research studies have concluded that 60-80% of efforts are spent on the evolution and maintenance of the software. Software repositories, for example, version control and bug tracking systems, are vital fragments of numerous software maintenance activities. The version control system is an incremental version tracking system, introduced to avoid unnecessary overwriting of files such as programming code, web pages, and records. It also helps to decrease the confusion affected by duplicate or outdated data. In this proposed research, the SVN repository was maintained and analyzed for msitone.wikispaces.com to minimize the efforts as well as resources for future users. Two semesters' data was taken as the observations for the analysis. The obtained archives were implemented and deployed for postulating the local archives to the online server via the Visual SVN server, allowing remote users to access a repository.The tortoise SVN tool is used as an SVN client to access the repository of Wiki spaces. Analyses were observed with and without implementing the SVN repository. The result shows that implementing the SVN repositories is helpful for the maintenance of the Wiki spaces. It also reduces the cost, time and efforts for their evolution. Whereas without implementing the SVN repositories, Wikis paces were just supposed to be building the house byputting each brick from the start.

Augmenting commit classification by using fine-grained source code changes and a pre-trained deep neural language model

Analyzing software maintenance activities is very helpful in ensuring cost-effective evolution and development activities. The categorization of commits into maintenance tasks supports practitioners in making decisions about resource allocation and managing technical debt. In this paper, we propose to use a pre-trained language neural model, namely BERT (Bidirectional Encoder Representations from Transformers) for the classification of commits into three categories of maintenance tasks — corrective, perfective and adaptive. The proposed commit classification approach will help the classifier better understand the context of each word in the commit message. We built a balanced dataset of 1793 labeled commits that we collected from publicly available datasets. We used several popular code change distillers to extract fine-grained code changes that we have incorporated into our dataset as additional features to BERT’s word representation features. In our study, a deep neural network (DNN) classifier has been used as an additional layer to fine-tune the BERT model on the task of commit classification. Several models have been evaluated to come up with a deep analysis of the impact of code changes on the classification performance of each commit category. Experimental results have shown that the DNN model trained on BERT’s word representations and Fixminer code changes ([email protected]+Fix_cc) provided the best performance and achieved 79.66% accuracy and a macro-average f1 score of 0.8. Comparison with the state-of-the-art model that combines keywords and code changes ([email protected]+CD_cc) has shown that our model achieved approximately 8% improvement in accuracy. Results have also shown that a DNN model using only BERT’s word representation features achieved an improvement of 5% in accuracy compared to the [email protected]+CD_cc model.

A method of activity-based software maintenance cost estimation for package software

This paper defines software maintenance activities and develops a model for maintenance cost estimation of package software. First, we classified software maintenance activities which were collected from the literature reviews. Second, we developed a cost structure for package software maintenance based on the identified activities. Third, the activity-based software maintenance cost estimation model was developed based on the software maintenance activities and cost structure. Fourth, we defined the adjustment factors for the software maintenance cost estimation model to improve the accuracy of the developed model. Finally, the developed model was verified using actual data from software maintenance projects. The proposed model showed reliable performance in terms of the mean magnitude of relative error and prediction quality. Therefore, the proposed model is suitable for setting budgets and predicting costs associated with package software maintenance projects.

The Effect of Feature Characteristics on the Performance of Feature Location Techniques

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Feature Location (FL)</i> is a core software maintenance activity that aims to locate observable functionalities in the source code. Given its key role in software change, a vast array of Feature Location Techniques (FLTs) have been proposed but, as more and more FLTs are introduced, the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">selection of an appropriate FLT</i> is an increasingly difficult problem. One consideration is the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">characteristics of the features</i> being sought. For example, in the code associated with the feature, programmers may have named identifiers consistently, and with meaningful naming conventions, or not, and this may impact on the suitability of different FLTs. The suggestion that such characteristics matter has implicit support in the literature: An analysis of existing FLT empirical studies reveals that the system under study can often have a stronger impact on FLT performance than differing FLTs themselves. To understand this interaction between feature characteristics and FLTs better, this paper proposes <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a suite of feature-characteristic metrics</i> that are postulated to control FLTs’ performance, holistically across FLTs and impacting on individual FLTs to different degrees. To evaluate the suite, a controlled experiment is performed, using 878 features, to probe the relationship between the metrics and the performance of four FTL techniques: three commonly-used techniques and one state-of-the-art technique. The evaluation is performed using four commonly used evaluation measures and extended by employing 41 other established source-code metrics as extraneous variables. Results of the empirical evaluation suggest that the feature-metric suite presented impacts FLT performance holistically, and impacts different FLTs to different degrees. Thus, this paper moves towards the more standard selection of appropriate FLTs, with respect to the prominent feature characteristics in the software systems under study, and more rigorous consideration of the features selected to compare FLTs.

A Deep-Learning-Based Bug Priority Prediction Using RNN-LSTM Neural

Context: Predicting the priority of bug reports is an important activity in software maintenance. Bug priority refers to the order in which a bug or defect should be resolved. A huge number of bug reports are submitted every day. Manual filtering of bug reports and assigning priority to each report is a heavy process, which requires time, resources, and expertise. In many cases mistakes happen when priority is assigned manually, which prevents the developers from finishing their tasks, fixing bugs, and improve the quality. Objective: Bugs are widespread and there is a noticeable increase in the number of bug reports that are submitted by the users and teams’ members with the presence of limited resources, which raises the fact that there is a need for a model that focuses on detecting the priority of bug reports, and allows developers to find the highest priority bug reports. This paper presents a model that focuses on predicting and assigning a priority level (high or low) for each bug report. Method: This model considers a set of factors (indicators) such as component name, summary, assignee, and reporter that possibly affect the priority level of a bug report. The factors are extracted as features from a dataset built using bug reports that are taken from closed-source projects stored in the JIRA bug tracking system, which are used then to train and test the framework. Also, this work presents a tool that helps developers to assign a priority level for the bug report automatically and based on the LSTM’s model prediction. Results: Our experiments consisted of applying a 5-layer deep learning RNN-LSTM neural network and comparing the results with Support Vector Machine (SVM) and K-nearest neighbors (KNN) to predict the priority of bug reports. The performance of the proposed RNN-LSTM model has been analyzed over the JIRA dataset with more than 2000 bug reports. The proposed model has been found 90% accurate in comparison with KNN (74%) and SVM (87%). On average, RNN-LSTM improves the F-measure by 3% compared to SVM and 15.2% compared to KNN. Conclusion: It concluded that LSTM predicts and assigns the priority of the bug more accurately and effectively than the other ML algorithms (KNN and SVM). LSTM significantly improves the average F-measure in comparison to the other classifiers. The study showed that LSTM reported the best performance results based on all performance measures (Accuracy = 0.908, AUC = 0.95, F-measure = 0.892).

On the prediction of long-lived bugs: An analysis and comparative study using FLOSS projects

Software evolution and maintenance activities in today’s Free/Libre Open Source Software (FLOSS) rely primarily on information extracted from bug reports registered in bug tracking systems. Many studies point out that most bugs that adversely affect the user’s experience across versions of FLOSS projects are long-lived bugs. However, proposed approaches that support bug fixing procedures do not consider the real-world lifecycle of a bug, in which bugs are often fixed very fast. This may lead to useless efforts to automate the bug management process. This study aims to confirm whether the number of long-lived bugs is significantly high in popular open-source projects and to characterize the population of long-lived bugs by considering the attributes of bug reports. We also aim to conduct a comparative study evaluating the prediction accuracy of five well-known machine learning algorithms and text mining techniques in the task of predicting long-lived bugs. We collected bug reports from six popular open-source projects repositories (Eclipse, Freedesktop, Gnome, GCC, Mozilla, and WineHQ) and used the following machine learning algorithms to predict long-lived bugs: K-Nearest Neighbor, Naïve Bayes, Neural Networks, Random Forest, and Support Vector Machines. Our results show that long-lived bugs are relatively frequent (varying from 7.2% to 40.7%) and have unique characteristics, confirming the need to study solutions to support bug fixing management. We found that the Neural Network classifier yielded the best results in comparison to the other algorithms evaluated. Research efforts regarding long-lived bugs are needed and our results demonstrate that it is possible to predict long-lived bugs with a high accuracy (around 70.7%) despite the use of simple prediction algorithms and text mining methods.

How does code readability change during software evolution?

Code reading is one of the most frequent activities in software maintenance. Such an activity aims at acquiring information from the code and, thus, it is a prerequisite for program comprehension: developers need to read the source code they are going to modify before implementing changes. As the code changes, so does its readability; however, it is not clear yet how code readability changes during software evolution. To understand how code readability changes when software evolves, we studied the history of 25 open source systems. We modeled code readability evolution by defining four states in which a file can be at a certain point of time (non-existing, other-name, readable, and unreadable). We used the data gathered to infer the probability of transitioning from one state to another one. In addition, we also manually checked a significant sample of transitions to compute the performance of the state-of-the-art readability prediction model we used to calculate the transition probabilities. With this manual analysis, we found that the tool correctly classifies all the transitions in the majority of the cases, even if there is a loss of accuracy compared to the single-version readability estimation. Our results show that most of the source code files are created readable. Moreover, we observed that only a minority of the commits change the readability state. Finally, we manually carried out qualitative analysis to understand what makes code unreadable and what developers do to prevent this. Using our results we propose some guidelines (i) to reduce the risk of code readability erosion and (ii) to promote best practices that make code readable.

Automatic and Accurate Expansion of Abbreviations in Parameters

Abbreviations are widely used in identifiers. However, they have severe negative impact on program comprehension and IR-based software maintenance activities, e.g., concept location, software clustering, and recovery of traceability links. Consequently, a number of efficient approaches have been proposed successfully to expand abbreviations in identifiers. Most of such approaches rely heavily on dictionaries, and rarely exploit the specific and fine-grained context of identifiers. As a result, such approaches are less accurate in expanding abbreviations (especially short ones) that may match multiple dictionary words. To this end, in this paper we propose an automatic approach to improve the accuracy of abbreviation expansion by exploiting the specific and fine-grained context. It focuses on a special but common category of abbreviations (abbreviations in parameter names), and thus it can exploit the specific and fine-grained context, i.e., the type of the enclosing parameter as well the corresponding formal (or actual) parameter name. The recent empirical study on parameters suggest that actual parameters are often lexically similar to their corresponding formal parameters. Consequently, it is likely that an abbreviation in a formal parameter can find its full terms in the corresponding actual parameter, and vice versa. Based on this assumption, a series of heuristics are proposed to look for full terms from the corresponding actual (or formal) parameter names. To the best of our knowledge, we are the first to expand abbreviations by exploiting the lexical similarity between actual and formal parameters. We also search for full terms in the data type of the enclosing parameter. Only if all such heuristics fail, the approach turns to the traditional abbreviation dictionaries. We evaluate the proposed approach on seven well known open-source projects. Evaluation results suggest that when only parameter abbreviations are involved, the proposed approach can improve the precision from 26 to 95 percent and recall from 26 to 65 percent compared against the state-of-the-art general purpose approach. Consequently, the proposed approach could be employed as a useful supplement to existing approaches to expand parameter abbreviations.

Developer Activity Motivated Bug Triaging: Via Convolutional Neural Network

As bugs become prevalent in software development, bug triaging has become one of the most important activities in software maintenance. To decrease the time cost in manual work, text classification techniques have been applied in automatic bug triaging. In this paper, we present a new automatic bug triaging approach which is based on convolution neural network (CNN) and developer activities. Firstly, we implement the word vector representation of the text features in bug report by using Word2vec. Then, we combine CNN with batch normalization, pooling and full connection approach to learn from the word vector representation of bug report with known fixers. In addition, we also study the recent activities of the developers which can effectively distinguish similar bug reports and get a more suitable developer recommendation list. We empirically investigate the accuracy of automatic bug triaging on three large open source projects, namely Eclipse, Mozilla and NetBeans. The experimental results show that our approach can effectively improve the performance of automatic bug triaging.