Features In Source Code Research Articles

Early Detection of Phishing Sites with Enhanced Neural Network Models

Phishing is a digital crime committed with the aim of obtaining personal data by creating a link or website that resembles the original. This form of cyber attack is caused by a notification in a text message, email, or phone call. A common anti-phishing countermeasure technique is to perform early detection of potentially phishing sites, primarily according to the source code features, which are required to traverse web page content, as well as third parties that slow down the process of clarifying phishing URLs. Although the latest technology has long been used in phishing early detection, there is still a need for manual feature engineering that is important and reliable enough to detect emerging phishing offenses. One of these involves training a neural network (NN) using a dataset of known phishing URLs and legitimate URLs. The research was conducted using 200 data, Data were separated into training and testing categories. Training was done using 100 and 120 data. Training results on 100 data and 160 data had lower iterations and errors on the tanh activation function compared to the logistic activation function. The number of iterations that occur in logistic activation is as many as 400 iterations, while when using the tanh activation function only 175 iterations are needed.

A Novel Source Code Representation Approach Based on Multi-Head Attention

Code classification and code clone detection are crucial for understanding and maintaining large software systems. Although deep learning surpasses traditional techniques in capturing the features of source code, existing models suffer from low processing power and high complexity. We propose a novel source code representation method based on the multi-head attention mechanism (SCRMHA). SCRMHA captures the vector representation of entire code segments, enabling it to focus on different positions of the input sequence, capture richer semantic information, and simultaneously process different aspects and relationships of the sequence. Moreover, it can calculate multiple attention heads in parallel, speeding up the computational process. We evaluate SCRMHA on both the standard dataset and an actual industrial dataset, and analyze the differences between these two datasets. Experiment results in code classification and clone detection tasks show that SCRMHA consumes less time and reduces complexity by about one-third compared with traditional source code feature representation methods. The results demonstrate that SCRMHA reduces the computational complexity and time consumption of the model while maintaining accuracy.

Optimizing LSTM for Code Smell Detection: The Role of Data Balancing

Code smells are specific patterns or characteristics in software code that indicate potential design or implementation problems. Identifying code smells has gained significant attention in software engineering. It is essential to address code smells to maintain high-quality software systems. Machine learning (ML) models, such as Long Short-Term Memory (LSTM), have been to detect code smells automatically based on source code features. However, the imbalanced distribution of code smells within software projects poses a challenge to the accuracy of these models. This study explores the role of data balancing methods in optimizing the accuracy of the LSTM model for code smell detection. We investigate different techniques for addressing the class imbalance problem, including random oversampling and synthetic minority oversampling techniques (SMOTE). We evaluate the performance of the LSTM model with and without data balancing methods using accuracy, precision, recall, f-measure, Matthew’s correlation coefficient (MCC), and the area under a receiver operating characteristic curve (AUC). Our experimental results, conducted on four code smell datasets (God class, data class, feature envy, and long method) extracted from 74 open-source systems, demonstrate the effectiveness of data balancing methods in improving the accuracy of the LSTM model for code smell detection. The results indicate that the use of data balancing methods had a positive effect on the predictive accuracy of the LSTM model. In addition, we compared our proposed method with state-of-the-art code smell detection approaches. The findings from the comparison indicate that our proposed method performs notably better than existing state-of the-art approaches across the majority of datasets.

Source Code Features and their Dependencies: An Aggregative Statistical Analysis on Open-Source Java Software Systems

Abstract Source code constitutes the static and human-readable component of a software system. It comprises an array of artifacts and features that collectively execute a specific set of tasks. Coding behaviours and patterns are formulated through the orchestrated utilization of distinct features in a specified sequence, fostering inter-dependencies among these features. This study seeks to explore into the presence of specific coding behaviours and patterns within Java, which could potentially unveil the extent to which developers endeavour to leverage the facilities and services that exist in the programming language aggregatively. In pursuit of investigating behaviours and patterns, 436 open-source Java projects are selected, each having more than 150 Java files (Classes and Interfaces), in a semi-randomized manner. For every project, 39 features have been chosen, and the frequency of each individual feature has been independently assessed. By employing linear regression, the interrelationships among all features across the complete array of projects are scrutinized. This analysis intends to uncover the manifestation of distinct coding behaviours and patterns. Based on the selected features, preliminary findings suggest a notable collective incorporation of diverse coding behaviours among programmers, encompassing Encapsulation and Polymorphism. The findings also point to a distinct preference for using a specific commenting mechanism, JavaDoc, and the potential existence of Code-Clone and dead code. Overall, the results indicate a clear tendency among programmers to strongly adhere to the fundamental principles of Object -Oriented programming. However, certain less obvious attributes of object-oriented languages appear to receive relatively less attention from programmers.

A New Smart Contract Anomaly Detection Method by Fusing Opcode and Source Code Features for Blockchain Services

Digital assets involved in smart contracts are on the rise. Security vulnerabilities in smart contracts have resulted in significant losses for the blockchain community. Existing smart contract vulnerability detection techniques have been typically single-purposed and focused only on the source code or opcode of contracts. This paper presents a new smart contract vulnerability detection method, which extracts features from different levels of smart contracts to train machine learning models for effective detection of vulnerabilities. Specifically, we propose to extract 2-gram features from the opcodes of smart contracts and token features from the source code using a pre-trained CodeBERT model, thereby capturing the semantic information of smart contracts at different levels. The 2-gram and token features are separately aggregated and then fused and input into machine-learning models to mine the vulnerability features of contracts. Over 10,266 smart contracts are used to verify the proposed method. Widespread reentrancy, timestamp dependence, and transaction-ordering dependence vulnerabilities are considered. Experiments show the fused features can help significantly improve smart contract vulnerability detection compared to the single-level features. The detection accuracy is as high as 98%, 98% and 94% for the three vulnerabilities, respectively. The average detection time is 0.99 second per contract, indicating the proposed method is suitable for automatic batch detection of vulnerabilities in smart contracts.

Summarizing source code through heterogeneous feature fusion and extraction

Code summarization, which seeks to automatically produce a succinct natural-language description to summarize the functionality of source code, plays an essential role in maintaining the software. Currently, plentiful approaches have been proposed to first encode the source code based on its Abstract Syntax Tree (AST), and then decode it into a textual summary. However, most existing works interpret the AST-based syntax structure as a homogeneous graph, without discriminating the different relations between graph nodes (e.g., the parent–child and sibling relations) in a heterogeneous way. To mitigate this issue, this paper proposes HetCoS to extract the syntactic and sequential features of source code by exploring its inherent heterogeneity for code summarization. Specifically, we first build a Heterogeneous Code Graph (HCG) that fuses the syntax structure and code sequence with eight types of edges/relations designed between graph nodes. Moreover, we present a heterogeneous graph neural network for capturing the diverse relations in HCG. The represented HCG is then fed into a Transformer decoder, followed by a multi-head attention-based copying mechanism to support high-quality summary generation. Extensive experiments on the major Java and Python datasets illustrate the superiority of our approach over sixteen state-of-the-art baselines. To promote reproducibility studies, we make the implementation of HetCoS publicly accessible at https://github.com/GJCEXP/HETCOS.

Incorporating Signal Awareness in Source Code Modeling: An Application to Vulnerability Detection

AI models of code have made significant progress over the past few years. However, many models are actually not learning task-relevant source code features. Instead, they often fit non-relevant but correlated data, leading to a lack of robustness and generalizability, and limiting the subsequent practical use of such models. In this work, we focus on improving the model quality through signal awareness , i.e., learning the relevant signals in the input for making predictions. We do so by leveraging the heterogeneity of code samples in terms of their signal-to-noise content. We perform an end-to-end exploration of model signal awareness, comprising: (i) uncovering the reliance of AI models of code on task-irrelevant signals, via prediction-preserving input minimization; (ii) improving models’ signal awareness by incorporating the notion of code complexity during model training, via curriculum learning; (iii) improving models’ signal awareness by generating simplified signal-preserving programs and augmenting them to the training dataset; and (iv) presenting a novel interpretation of the model learning behavior from the perspective of the dataset, using its code complexity distribution. We propose a new metric to measure model signal awareness, Signal-aware Recall, which captures how much of the model’s performance is attributable to task-relevant signal learning. Using a software vulnerability detection use-case, our model probing approach uncovers a significant lack of signal awareness in the models, across three different neural network architectures and three datasets. Signal-aware Recall is observed to be in the sub-50s for models with traditional Recall in the high 90s, suggesting that the models are presumably picking up a lot of noise or dataset nuances while learning their logic. With our code-complexity-aware model learning enhancement techniques, we are able to assist the models toward more task-relevant learning, recording up-to 4.8× improvement in model signal awareness. Finally, we employ our model learning introspection approach to uncover the aspects of source code where the model is facing difficulty, and we analyze how our learning enhancement techniques alleviate it.

Summarizing source code with Heterogeneous Syntax Graph and dual position

Code summarization attempts to summarize the semantics of source code by automatically producing brief natural-language descriptions. Most existing work proposes to learn from the Abstract Syntax Tree (AST) and plain text of source code for summary generation. However, little attention has been paid to the structural heterogeneity and layout features of source code. In this paper, we present a novel framework titled HetSum to address these issues. Specifically, a Heterogeneous Syntax Graph (HSG) is first built by designing six types of augmented edges in AST, which indicates the heterogeneous structure of source code. Meanwhile, a dual position is designed for each token in the source code by considering the layout information. Moreover, we develop a heterogeneous graph neural network in HetSum to encode the HSG while extracting the code layout features with the Transformer encoder. By assimilating the learned code token vectors into the HSG encoder, HetSum can capture the relations between its two encoders for improved code representation. To facilitate the generation of high-quality summaries, we integrate a copying mechanism into the decoding procedure while expanding the Transformer decoding sublayer. Extensive experiments on the Java and Python datasets prove that HetSum is superior to seventeen state-of-the-art baselines. To promote reproducibility studies, we make the implementation of HetSum available at https://github.com/GJCEXP/HETSUM.

BiTCN_DRSN: An effective software vulnerability detection model based on an improved temporal convolutional network

The detection of software vulnerabilities is a challenging task in the field of security. With the increasing scale of software and the rapid development of artificial intelligence technology, deep learning has been extensively applied to automatic vulnerability detection. Temporal Convolutional Networks (TCNs) have been shown to perform well in tasks that can be processed in parallel; they can adaptively learn complex structures (including in-time series data); and they have exhibited stable gradients — they are relatively easier to train, and can quickly converge to an optimal solution. However, TCNs cannot simultaneously capture the bidirectional semantics of the source code, since they do not have a bidirectional network structure. Furthermore, because of the weak noise resistance of residual TCN connections, TCNs are also susceptible to learning features that are not related to vulnerabilities when learning the source code features. To overcome the limitations of the traditional TCN, we propose a bidirectional TCN model based on the Deep Residual Shrinkage Network (DRSN), namely BiTCN_DRSN. BiTCN_DRSN combines TCN and DRSN to enhance the noise immunity and make the network model more attentive to the features associated with vulnerabilities. In addition, addressing the limitation that the TCN is a unidirectional network structure, the forward and backward sequences are utilized for bidirectional source-code feature learning. The experimental results show that the proposed BiTCN_DRSN model can effectively improve the accuracy of source-code vulnerability detection, compared with some existing neural-network models. Compared with the traditional TCN, our model increases the accuracy by 4.22%, 2.42% and 2.66% on the BE-ALL, RM-ALL and HY-ALL datasets, respectively. The proposed BiTCN_DRSN model also exhibits improved detection stability.

Towards Modeling Human Attention from Eye Movements for Neural Source Code Summarization

Neural source code summarization is the task of generating natural language descriptions of source code behavior using neural networks. A fundamental component of most neural models is an attention mechanism. The attention mechanism learns to connect features in source code to specific words to use when generating natural language descriptions. Humans also pay attention to some features in code more than others. This human attention reflects experience and high-level cognition well beyond the capability of any current neural model. In this paper, we use data from published eye-tracking experiments to create a model of this human attention. The model predicts which words in source code are the most important for code summarization. Next, we augment a baseline neural code summarization approach using our model of human attention. We observe an improvement in prediction performance of the augmented approach in line with other bio-inspired neural models.

GA-SCS: Graph-Augmented Source Code Summarization

Automatic source code summarization system aims to generate a valuable natural language description for a program, which can facilitate software development and maintenance, code categorization, and retrieval. However, previous sequence-based research did not consider the long-distance dependence and highly structured characteristics of source code simultaneously. In this article, we present a Transformer-based Graph-Augmented Source Code Summarization (GA-SCS), which can effectively incorporate inherent structural and textual features of source code to generate an effective code description. Specifically, we develop a graph-based structure feature extraction scheme leveraging abstract syntax tree and graph attention networks to mine global syntactic information. And then, to take full advantage of the lexical and syntactic information of code snippets, we extend the original attention to a syntax-informed self-attention mechanism in our encoder. In the training process, we also adopt a reinforcement learning strategy to enhance the readability and informativity of generated code summaries. We utilize the Java dataset and Python dataset to evaluate the performance of different models. Experimental results demonstrate that our GA-SCS model outperforms all competitive methods on BLEU, METEOR, ROUGE, and human evaluations.

A Feature-Based Robust Method for Abnormal Contracts Detection in Ethereum Blockchain

Blockchain technology has allowed many abnormal schemes to hide behind smart contracts. This causes serious financial losses, which adversely affects the blockchain. Machine learning technology has mainly been utilized to enable automatic detection of abnormal contract accounts in recent years. In spite of this, previous machine learning methods have suffered from a number of disadvantages: first, it is extremely difficult to identify features that enable accurate detection of abnormal contracts, and based on these features, statistical analysis is also ineffective. Second, they ignore the imbalances and repeatability of smart contract accounts, which often results in overfitting of the model. In this paper, we propose a data-driven robust method for detecting abnormal contract accounts over the Ethereum Blockchain. This method comprises hybrid features set by integrating opcode n-grams, transaction features, and term frequency-inverse document frequency source code features to train an ensemble classifier. The extra-trees and gradient boosting algorithms based on weighted soft voting are used to create an ensemble classifier that balances the weaknesses of individual classifiers in a given dataset. The abnormal and normal contract data are collected by analyzing the open source etherscan.io, and the problem of the imbalanced dataset is solved by performing the adaptive synthetic sampling. The empirical results demonstrate that the proposed individual feature sets are useful for detecting abnormal contract accounts. Meanwhile, combining all the features enhances the detection of abnormal contracts with significant accuracy. The experimental and comparative results show that the proposed method can distinguish abnormal contract accounts for the data-driven security of blockchain Ethereum with satisfactory performance metrics.

Deep Learning-Based Software Defect Prediction via Semantic Key Features of Source Code—Systematic Survey

Software defect prediction (SDP) methodology could enhance software’s reliability through predicting any suspicious defects in its source code. However, developing defect prediction models is a difficult task, as has been demonstrated recently. Several research techniques have been proposed over time to predict source code defects. However, most of the previous studies focus on conventional feature extraction and modeling. Such traditional methodologies often fail to find the contextual information of the source code files, which is necessary for building reliable prediction deep learning models. Alternatively, the semantic feature strategies of defect prediction have recently evolved and developed. Such strategies could automatically extract the contextual information from the source code files and use them to directly predict the suspicious defects. In this study, a comprehensive survey is conducted to systematically show recent software defect prediction techniques based on the source code’s key features. The most recent studies on this topic are critically reviewed through analyzing the semantic feature methods based on the source codes, the domain’s critical problems and challenges are described, and the recent and current progress in this domain are discussed. Such a comprehensive survey could enable research communities to identify the current challenges and future research directions. An in-depth literature review of 283 articles on software defect prediction and related work was performed, of which 90 are referenced.

Hierarchical semantic-aware neural code representation

Code representation is a fundamental problem in many software engineering tasks. Despite the effort made by many researchers, it is still hard for existing methods to fully extract syntactic, structural and sequential features of source code, which form the hierarchical semantics of the program and are necessary to achieve a deeper code understanding. To alleviate this difficulty, we propose a new supervised approach based on the novel use of Tree-LSTM to incorporate the sequential and the global semantic features of programs explicitly into the representation model. Unlike previous techniques, our proposed model can not only learn low-level syntactic information within each statement but also the high-level semantic information between statements over the constructed semantic graph. Besides, considering that the sequential semantics is also critical for developers to understand the dependency path and data flow transmission, we propose a DFS-based method to generate the topological order of statements being processed, and then feed them as well as their in-neighboring information and syntactic embeddings into the proposed model to learn richer statement-level semantic features. Extensive experiments on multiple program comprehension tasks, e.g., code clone detection, demonstrate that our method achieves promising performance compared with other existing baselines.

Hunger Search Optimization with Hybrid Deep Learning Enabled Phishing Detection and Classification Model

Phishing is one of the simplest ways in cybercrime to hack the reliable data of users such as passwords, account identifiers, bank details, etc. In general, these kinds of cyberattacks are made at users through phone calls, emails, or instant messages. The anti-phishing techniques, currently under use, are mainly based on source code features that need to scrape the webpage content. In third party services, these techniques check the classification procedure of phishing Uniform Resource Locators (URLs). Even though Machine Learning (ML) techniques have been lately utilized in the identification of phishing, they still need to undergo feature engineering since the techniques are not well-versed in identifying phishing offenses. The tremendous growth and evolution of Deep Learning (DL) techniques paved the way for increasing the accuracy of classification process. In this background, the current research article presents a Hunger Search Optimization with Hybrid Deep Learning enabled Phishing Detection and Classification (HSOHDL-PDC) model. The presented HSOHDL-PDC model focuses on effective recognition and classification of phishing based on website URLs. In addition, SOHDL-PDC model uses character-level embedding instead of word-level embedding since the URLs generally utilize words with no importance. Moreover, a hybrid Convolutional Neural Network-Long Short Term Memory (HCNN-LSTM) technique is also applied for identification and classification of phishing. The hyperparameters involved in HCNN-LSTM model are optimized with the help of HSO algorithm which in turn produced improved outcomes. The performance of the proposed HSOHDL-PDC model was validated using different datasets and the outcomes confirmed the supremacy of the proposed model over other recent approaches.

Explainable source code authorship attribution algorithm

Source Code Authorship Attribution is a problem that is lately studied more often due improvements in Deep Learning techniques. Among existing solutions, two common issues are inability to add new authors without retraining and lack of interpretability. We address both these problem. In our experiments, we were able to correctly classify 75% of authors for diferent programming languages. Additionally, we applied techniques of explainable AI (XAI) and found that our model seems to pay attention to distinctive features of source code.

Software defect prediction using hybrid model (CBIL) of convolutional neural network (CNN) and bidirectional long short-term memory (Bi-LSTM).

In recent years, the software industry has invested substantial effort to improve software quality in organizations. Applying proactive software defect prediction will help developers and white box testers to find the defects earlier, and this will reduce the time and effort. Traditional software defect prediction models concentrate on traditional features of source code including code complexity, lines of code, etc. However, these features fail to extract the semantics of source code. In this research, we propose a hybrid model that is called CBIL. CBIL can predict the defective areas of source code. It extracts Syntax Tree (AST) tokens as vectors from source code. Mapping and word embedding turn integer vectors into dense vectors. Then, Convolutional Neural Network (CNN) extracts the semantics of AST tokens. After that, Bidirectional Long Short-Term Memory (Bi-LSTM) keeps key features and ignores other features in order to enhance the accuracy of software defect prediction. The proposed model CBIL is evaluated on a sample of seven open-source Java projects of the PROMISE dataset. CBIL is evaluated by applying the following evaluation metrics: F-measure and area under the curve (AUC). The results display that CBIL model improves the average of F-measure by 25% compared to CNN, as CNN accomplishes the top performance among the selected baseline models. In average of AUC, CBIL model improves AUC by 18% compared to Recurrent Neural Network (RNN), as RNN accomplishes the top performance among the selected baseline models used in the experiments.

Predicting unstable software benchmarks using static source code features

Software benchmarks are only as good as the performance measurements they yield. Unstable benchmarks show high variability among repeated measurements, which causes uncertainty about the actual performance and complicates reliable change assessment. However, if a benchmark is stable or unstable only becomes evident after it has been executed and its results are available. In this paper, we introduce a machine-learning-based approach to predict a benchmark’s stability without having to execute it. Our approach relies on 58 statically-computed source code features, extracted for benchmark code and code called by a benchmark, related to (1) meta information, e.g., lines of code (LOC), (2) programming language elements, e.g., conditionals or loops, and (3) potentially performance-impacting standard library calls, e.g., file and network input/output (I/O). To assess our approach’s effectiveness, we perform a large-scale experiment on 4,461 Go benchmarks coming from 230 open-source software (OSS) projects. First, we assess the prediction performance of our machine learning models using 11 binary classification algorithms. We find that Random Forest performs best with good prediction performance from 0.79 to 0.90, and 0.43 to 0.68, in terms of AUC and MCC, respectively. Second, we perform feature importance analyses for individual features and feature categories. We find that 7 features related to meta-information, slice usage, nested loops, and synchronization application programming interfaces (APIs) are individually important for good predictions; and that the combination of all features of the called source code is paramount for our model, while the combination of features of the benchmark itself is less important. Our results show that although benchmark stability is affected by more than just the source code, we can effectively utilize machine learning models to predict whether a benchmark will be stable or not ahead of execution. This enables spending precious testing time on reliable benchmarks, supporting developers to identify unstable benchmarks during development, allowing unstable benchmarks to be repeated more often, estimating stability in scenarios where repeated benchmark execution is infeasible or impossible, and warning developers if new benchmarks or existing benchmarks executed in new environments will be unstable.

B2SMatcher: fine-Grained version identification of open-Source software in binary files

Codes of Open Source Software (OSS) are widely reused during software development nowadays. However, reusing some specific versions of OSS introduces 1-day vulnerabilities of which details are publicly available, which may be exploited and lead to serious security issues. Existing state-of-the-art OSS reuse detection work can not identify the specific versions of reused OSS well. The features they selected are not distinguishable enough for version detection and the matching scores are only based on similarity.This paper presents B2SMatcher, a fine-grained version identification tool for OSS in commercial off-the-shelf (COTS) software. We first discuss five kinds of version-sensitive code features that are trackable in both binary and source code. We categorize these features into program-level features and function-level features and propose a two-stage version identification approach based on the two levels of code features. B2SMatcher also identifies different types of OSS version reuse based on matching scores and matched feature instances. In order to extract source code features as accurately as possible, B2SMatcher innovatively uses machine learning methods to obtain the source files involved in the compilation and uses function abstraction and normalization methods to eliminate the comparison costs on redundant functions across versions. We have evaluated B2SMatcher using 6351 candidate OSS versions and 585 binaries. The result shows that B2SMatcher achieves a high precision up to 89.2% and outperforms state-of-the-art tools. Finally, we show how B2SMatcher can be used to evaluate real-world software and find some security risks in practice.

Code Clone Detection with Hierarchical Attentive Graph Embedding

Code clone serves as a typical programming manner that reuses the existing code to solve similar programming problems, which greatly facilitates software development but recurs program bugs and maintenance costs. Recently, deep learning-based detection approaches gradually present their effectiveness on feature representation and detection performance. Among them, deep learning approaches based on abstract syntax tree (AST) construct models relying on the node embedding technique. In AST, the semantic of nodes is obviously hierarchical, and the importance of nodes is quite different to determine whether the two code fragments are cloned or not. However, some approaches do not fully consider the hierarchical structure information of source code. Some approaches ignore the different importance of nodes when generating the features of source code. Thirdly, when the tree is very large and deep, many approaches are vulnerable to the gradient vanishing problem during training. In order to properly address these challenges, we propose a hierarchical attentive graph neural network embedding model-HAG for the code clone detection. Firstly, the attention mechanism is applied on nodes in AST to distinguish the importance of different nodes during the model training. In addition, the HAG adopts graph convolutional network (GCN) to propagate the code message on AST graph and then exploits a hierarchical differential pooling GCN to sufficiently capture the code semantics at different structure level. To evaluate the effectiveness of HAG, we conducted extensive experiments on public clone dataset and compared it with seven state-of-the-art clone detection models. The experimental results demonstrate that the HAG achieves superior detection performance compared with baseline models. Especially, in the detection of moderately Type-3 or Type-4 clones, the HAG particularly outperforms baselines, indicating the strong detection capability of HAG for semantic clones. Apart from that, the impacts of the hierarchical pooling, attention mechanism and critical model parameters are systematically discussed.