Projective Adaptive Resonance Theory Research Articles

Empirical Analysis of Data Streaming and Batch Learning Models for Network Intrusion Detection

Network intrusion, such as denial of service, probing attacks, and phishing, comprises some of the complex threats that have put the online community at risk. The increase in the number of these attacks has given rise to a serious interest in the research community to curb the menace. One of the research efforts is to have an intrusion detection mechanism in place. Batch learning and data streaming are approaches used for processing the huge amount of data required for proper intrusion detection. Batch learning, despite its advantages, has been faulted for poor scalability due to the constant re-training of new training instances. Hence, this paper seeks to conduct a comparative study using selected batch learning and data streaming algorithms. The batch learning and data streaming algorithms considered are J48, projective adaptive resonance theory (PART), Hoeffding tree (HT) and OzaBagAdwin (OBA). Furthermore, binary and multiclass classification problems are considered for the tested algorithms. Experimental results show that data streaming algorithms achieved considerably higher performance in binary classification problems when compared with batch learning algorithms. Specifically, binary classification produced J48 (94.73), PART (92.83), HT (98.38), and OBA (99.67), and multiclass classification produced J48 (87.66), PART (87.05), HT (71.98), OBA (82.80) based on accuracy. Hence, the use of data streaming algorithms to solve the scalability issue and allow real-time detection of network intrusion is highly recommended.

An ensemble deep learning model for cyber threat hunting in industrial internet of things

By the emergence of the fourth industrial revolution, interconnected devices and sensors generate large-scale, dynamic, and inharmonious data in Industrial Internet of Things (IIoT) platforms. Such vast heterogeneous data increase the challenges of security risks and data analysis procedures. As IIoT grows, cyber-attacks become more diverse and complex, making existing anomaly detection models less effective to operate. In this paper, an ensemble deep learning model that uses the benefits of the Long Short-Term Memory (LSTM) and the Auto-Encoder (AE) architecture to identify out-of-norm activities for cyber threat hunting in IIoT is proposed. In this model, the LSTM is applied to create a model on normal time series of data (past and present data) to learn normal data patterns and the important features of data are identified by AE to reduce data dimension. In addition, the imbalanced nature of IIoT datasets has not been considered in most of the previous literature, affecting low accuracy and performance. To solve this problem, the proposed model extracts new balanced data from the imbalanced datasets, and these new balanced data are fed into the deep LSTM AE anomaly detection model. In this paper, the proposed model is evaluated on two real IIoT datasets -Gas Pipeline (GP) and Secure Water Treatment (SWaT) that are imbalanced and consist of long-term and short-term dependency on data. The results are compared with conventional machine learning classifiers, Random Forest (RF), Multi-Layer Perceptron (MLP), Decision Tree (DT), and Super Vector Machines (SVM), in which higher performance in terms of accuracy is obtained, 99.3% and 99.7% based on GP and SWaT datasets, respectively. Moreover, the proposed ensemble model is compared with advanced related models, including Stacked Auto-Encoders (SAE), Naive Bayes (NB), Projective Adaptive Resonance Theory (PART), Convolutional Auto-Encoder (C-AE), and Package Signatures (PS) based LSTM (PS-LSTM) model.

Severity classification of software code smells using machine learning techniques: A comparative study

AbstractCode smell is a software characteristic that indicates bad symptoms in code design which causes problems related to software quality. The severity of code smells must be measured because it will help the developers when determining the priority of refactoring efforts. Recently, several studies focused on the prediction of design patterns errors using different detection tools. Nowadays, there is a lack of empirical studies regarding how to measure severity of code smells and which learning model is best to detect the severity of code smells. To overcome such gap, this paper focuses on measuring the severity classification of code smells depending on several machine learning models such as regression models, multinominal models, and ordinal classification models. The Local Interpretable Model Agnostic Explanations (LIME) algorithm was further used to explain the machine learning model's predictions and interpretability. On the other side, we extract the prediction rules generated by the Projective Adaptive Resonance Theory (PART) algorithm in order to study the effectiveness of using software metrics to predict code smells. The results of the experiments have shown that the accuracy of severity classification model is enhanced than baseline and ranking correlation between the predicted and actual model reaches 0.92–0.97 by using Spearman's correlation measure.

Incremental Ant-Miner Classifier for Online Big Data Analytics.

Internet of Things (IoT) environments produce large amounts of data that are challenging to analyze. The most challenging aspect is reducing the quantity of consumed resources and time required to retrain a machine learning model as new data records arrive. Therefore, for big data analytics in IoT environments where datasets are highly dynamic, evolving over time, it is highly advised to adopt an online (also called incremental) machine learning model that can analyze incoming data instantaneously, rather than an offline model (also called static), that should be retrained on the entire dataset as new records arrive. The main contribution of this paper is to introduce the Incremental Ant-Miner (IAM), a machine learning algorithm for online prediction based on one of the most well-established machine learning algorithms, Ant-Miner. IAM classifier tackles the challenge of reducing the time and space overheads associated with the classic offline classifiers, when used for online prediction. IAM can be exploited in managing dynamic environments to ensure timely and space-efficient prediction, achieving high accuracy, precision, recall, and F-measure scores. To show its effectiveness, the proposed IAM was run on six different datasets from different domains, namely horse colic, credit cards, flags, ionosphere, and two breast cancer datasets. The performance of the proposed model was compared to ten state-of-the-art classifiers: naive Bayes, logistic regression, multilayer perceptron, support vector machine, K*, adaptive boosting (AdaBoost), bagging, Projective Adaptive Resonance Theory (PART), decision tree (C4.5), and random forest. The experimental results illustrate the superiority of IAM as it outperformed all the benchmarks in nearly all performance measures. Additionally, IAM only needs to be rerun on the new data increment rather than the entire big dataset on the arrival of new data records, which makes IAM better in time- and resource-saving. These results demonstrate the strong potential and efficiency of the IAM classifier for big data analytics in various areas.

SuPART: supervised projective adapted resonance theory for automatic quality assurance approval of radiotherapy treatment plans

Radiotherapy is a common treatment modality for the treatment of cancer, where treatments must be carefully designed to deliver appropriate dose to targets while avoiding healthy organs. The comprehensive multi-disciplinary quality assurance (QA) process in radiotherapy is designed to ensure safe and effective treatment plans are delivered to patients. However, the plan QA process is expensive, often time-intensive, and requires review of large quantities of complex data, potentially leading to human error in QA assessment. We therefore develop an automated machine learning algorithm to identify ‘acceptable’ plans (plans that are similar to historically approved plans) and ‘unacceptable’ plans (plans that are dissimilar to historically approved plans). This algorithm is a supervised extension of projective adaptive resonance theory, called SuPART, that learns a set of distinctive features, and considers deviations from them indications of unacceptable plans. We test SuPART on breast and prostate radiotherapy datasets from our institution, and find that SuPART outperforms common classification algorithms in several measures of accuracy. When no falsely approved plans are allowed, SuPART can correctly auto-approve 34% of the acceptable breast and 32% of the acceptable prostate plans, and can also correctly reject 53% of the unacceptable breast and 56% of the unacceptable prostate plans. Thus, usage of SuPART to aid in QA could potentially yield significant time savings.

Clustering of genes from microarray data using hierarchical projective adaptive resonance theory: a case study of tuberculosis

We propose the hierarchical Projective Adaptive Resonance Theory (PART) algorithm for classification of gene expression data. This algorithm is realized by combing transposed quasi-supervised PART and unsupervised PART. We develop the corresponding validation statistics for each process and compare it with other clustering algorithms in a case study of tuberculosis (TB). First, we use sample-based transposed quasi-supervised PART to obtain optimal clustering results of samples distinguished by time post-infection and the representative genes for each cluster including up-regulated, down-regulated and stable genes. The up- and down-regulated genes show more than 90% similarity to the result derived from Linear Models for Microarray Data and are verified by weighted k-nearest neighbor model on TB projection. Second, we use gene-based unsupervised PART algorithm to cluster these representative genes where functional enrichment analysis is conducted in each cluster. We further confirm the main immune response of human macrophage-like THP-1 cells against TB within 2 days is type I interferon-mediated innate immunity. This study demonstrates how hierarchical PART algorithm analyzes microarray data. The sample-based quasi-supervised PART extracts representative genes and narrows down the shortlist of disease-relevant genes and gene-based unsupervised PART classifies representative genes that help to interpret immune response against TB.

Protecting sensitive information utilizing an efficient association representative rule concealing algorithm for imbalance dataset

<span>In data mining, discrimination is the detrimental behavior of the people which is extensively studied in human society and economical science. However, there are negative perceptions about the data mining. Discrimination has two categories; one is direct, and another is indirect. The decisions depend on sensitive information attributes are named as direct discrimination, and the decisions which depend on non-sensitive information attributes are called as indirect discrimination which is strongly related with biased sensitive ones. Privacy protection has become another one of the most important problems in data mining investigation. To overcome the above issues, an Efficient Association Representative Rule Concealing (EARRC) algorithm is proposed to protect sensitive information or knowledge and offer privacy protection with the classification of the sensitive data. Representative rule concealing is one kind of the privacy-preserving mechanisms to hide sensitive association rules. The objective of this paper is to reduce the alternation of the original database and perceive that there is no sensitive association rule is obtained. The proposed method hides the sensitive information by altering the database without modifying the support of the sensitive item. The EARRC is a type of association classification approach which integrates the benefits of both associative classification and rule-based PART (Projective Adaptive Resonance Theory) classification. Based on Experimental computations, proposed EARRC+PART classifier improves 1.06 NMI and 5.66 Accuracy compared than existing methodologies.</span>

Socially Aware Caching in D2D Enabled Fog Radio Access Networks

As a promising architecture in the future networks, fog radio access network (F-RAN) can effectively improve network performance due to the advantages in both communication and computation aspects. In this paper, the data caching scheme is studied in a device-to-device (D2D) enabled the F-RAN from the social point of view. First, we help each access point (AP) to ascertain its best belonging community in terms of the content preference based on the projective adaptive resonance theory neural network (PART NN). The APs within identical community cache data cooperatively, which can reduce the caching redundancy. Considering the content preference of the AP may change dynamically, a community update scheme is also studied after the community construction is completed. In order to further reduce the content delivery delay, the D2D technology is resorted to and the most appropriate user equipment (UE) is selected to cache data for other UEs within each AP’s coverage. After that, an ant colony optimization scheme is proposed to find out the most requested contents needed to be cached by such most appropriate UE. The effectiveness of the proposed scheme is finally validated by the simulation experiments.

Early hospital mortality prediction of intensive care unit patients using an ensemble learning approach

BackgroundMortality prediction of hospitalized patients is an important problem. Over the past few decades, several severity scoring systems and machine learning mortality prediction models have been developed for predicting hospital mortality. By contrast, early mortality prediction for intensive care unit patients remains an open challenge. Most research has focused on severity of illness scoring systems or data mining (DM) models designed for risk estimation at least 24 or 48h after ICU admission. ObjectivesThis study highlights the main data challenges in early mortality prediction in ICU patients and introduces a new machine learning based framework for Early Mortality Prediction for Intensive Care Unit patients (EMPICU). Materials and methodsThe proposed method is evaluated on the Multiparameter Intelligent Monitoring in Intensive Care II (MIMIC-II) database. Mortality prediction models are developed for patients at the age of 16 or above in Medical ICU (MICU), Surgical ICU (SICU) or Cardiac Surgery Recovery Unit (CSRU). We employ the ensemble learning Random Forest (RF), the predictive Decision Trees (DT), the probabilistic Naive Bayes (NB) and the rule-based Projective Adaptive Resonance Theory (PART) models. The primary outcome was hospital mortality. The explanatory variables included demographic, physiological, vital signs and laboratory test variables. Performance measures were calculated using cross-validated area under the receiver operating characteristic curve (AUROC) to minimize bias. 11,722 patients with single ICU stays are considered. Only patients at the age of 16 years old and above in Medical ICU (MICU), Surgical ICU (SICU) or Cardiac Surgery Recovery Unit (CSRU) are considered in this study. ResultsThe proposed EMPICU framework outperformed standard scoring systems (SOFA, SAPS-I, APACHE-II, NEWS and qSOFA) in terms of AUROC and time (i.e. at 6h compared to 48h or more after admission). Discussion and conclusionThe results show that although there are many values missing in the first few hour of ICU admission, there is enough signal to effectively predict mortality during the first 6h of admission. The proposed framework, in particular the one that uses the ensemble learning approach – EMPICU Random Forest (EMPICU-RF) offers a base to construct an effective and novel mortality prediction model in the early hours of an ICU patient admission, with an improved performance profile.

A Logical Approach for Empirical Risk Minimization in Machine Learning for Data Stratification

The data-driven methods capable of understanding, mimicking and aiding the information processing tasks of Machine Learning (ML) have been applied in an increasing range over the past years in diverse areas at a very high rate, and had achieved great success in predicting and stratifying given data instances of a problem domain. There has been generalization on the performance of the classifier to be the optimal based on the existing performance benchmarks such as accuracy, speed, time to learn, number of features, comprehensibility, robustness, scalability and interpretability. However, these benchmarks alone do not guarantee the successful adoption of an algorithm for prediction and stratification since there may be an incurring risk in its adoption. Therefore, this paper aims at developing a logical approach for using Empirical Risk Minimization (ERM) technique to determine the machine learning classifier with the minimum risk function for data stratification. The generalization on the performance of optimal algorithm was tested on BayesNet, Multilayered perceptron, Projective Adaptive Resonance Theory (PART) and Logistic Model Trees algorithms based on existing performance benchmarks such as correctly classified instances, time to build, kappa statistics, sensitivity and specificity to determine the algorithms with great performances. The study showed that PART and Logistic Model Trees algorithms perform well than others. Hence, a logical approach to apply Empirical Risk Minimization technique on PART and Logistic Model Trees algorithms is shown to give a detailed procedure of determining their empirical risk function to aid the decision of choosing an algorithm to be the best fit classifier for data stratification. This therefore serves as a benchmark for selecting an optimal algorithm for stratification and prediction alongside other benchmarks.

An application of PART to the Football Manager data for players clusters analyses to inform club team formation

We aim to show how a neural network based machine learning projective clustering algorithm, Projective Adaptive Resonance Theory (PART), can be effectively used to provide data-informed sports decisions. We illustrate this data-driven decision recommendation for AS Roma player market in the Summer 2018 season, using the two separate databases of fourty-seven attributes taken from Football Manager 2018 for each of the twenty-four soccer player, with the first including players of the AS Roma squad 2017-18, and the second consisting of all players linked with transfer moves to AS Roma. This is high dimensional data as players should be grouped only in terms of their performance with respect to a small subset of attributes. Projective clustering analyses provide a purely data-driven analysis to identify critical attributes and attribute characteristics for a group of players to form a natural cluster (in lower dimensional data space) in an unsupervised way. By merging the two databases, our unsupervised clustering analysis provides evidence-based recommendations about the club team formation, and in particular, the decision to buy and sell players within the same clusters, under different scenarios including financial constraints.

An ANN Approach for Network Intrusion Detection using Entropy based Feature Selection

With the increase in Internet users the number of malicious users are also growing day-by-day posing a serious problem in distinguishing between normal and abnormal behavior of users in the network. This has led to the research area of intrusion detection which essentially analyzes the network traffic and tries to determine normal and abnormal patterns of behavior.In this paper, we have analyzed the standard NSL-KDD intrusion dataset using some neural network based techniques for predicting possible intrusions. Four most effective classification methods, namely, Radial Basis Function Network, SelfOrganizing Map, Sequential Minimal Optimization, and Projective Adaptive Resonance Theory have been applied. In order to enhance the performance of the classifiers, three entropy based feature selection methods have been applied as preprocessing of data. Performances of different combinations of classifiers and attribute reduction methods have also been compared.

Automating construction of a domain ontology using a projective adaptive resonance theory neural network and Bayesian network

Abstract: Research on semantic webs has become increasingly widespread in the computer science community. The core technology of a semantic web is an artefact called an ontology. The major problem in constructing an ontology is the long period of time required. Another problem is the large number of possible meanings for the knowledge in the ontology. In this paper, we present a novel ontology construction based on artificial neural networks and a Bayesian network. First, we collected web pages related to the problem domain using search engines. The system then used the labels of the HTML tags to select keywords, and used WordNet to determine the meaningful keywords, called terms. Next, it calculated the entropy value to determine the weight of the terms. After the above steps, the projective adaptive resonance theory neural network clustered the collected web pages and found the representative term of each cluster of web pages using the entropy value. The system then used a Bayesian network to insert the terms and complete the hierarchy of the ontology. Finally, the system used a resource description framework to store and express the ontology results.

Projective ART with buffers for the high dimensional space clustering and an application to discover stock associations

Unlike to traditional hierarchical and partitional clustering algorithms which always fail to deal with very large databases, a neural network architecture, projective adaptive resonance theory (PART), is developed for the high dimensional space clustering. However, the success of the PART algorithm depends on both accurate parameters and satisfied orders of input data sets. These disadvantages prevent PART from being applied to realtime databases. In this paper, we propose an improved method, PART with buffer management, to overcome these disadvantages. The major contributions of our method are introducing a buffer management and a new similar degree function and buffer checkout process. The buffer management mechanism allows data sets not to be immediately clustered to one cluster. The purpose of the average similar degree is to successfully work with high similar noise data sets and partly achieve an order-independent objective without correct parameters. And the average similar degree has a good attribute, the parameter-tolerance. Namely, the clustering result does not depend on the precise choice of input parameters, and different parameter values have close clustering results including dimensions associated with clusters. The buffer checkout process can handle a huge amount of input data sets by a small buffer space. Also, simulations and comparisons in high dimensional spaces are reported, and an application by using our algorithm to find stock concurrence association rules is given finally.

Proposal of new gene filtering method, BagPART, for gene expression analysis with small sample

A significant problem in gene expression analysis is that the sample size is substantially lower than the number of genes. Bagging is an effective method of solving this problem in the case of small sample datasets. We have devised a combination method, called the BagPART filtering method, that uses the projective adaptive resonance theory (PART) to select important genes and achieve a binary classification more accurately (p<10(-10)) than conventional methods, particularly when the sample size is small.

Cancer diagnosis marker extraction for soft tissue sarcomas based on gene expression profiling data by using projective adaptive resonance theory (PART) filtering method.

BackgroundRecent advances in genome technologies have provided an excellent opportunity to determine the complete biological characteristics of neoplastic tissues, resulting in improved diagnosis and selection of treatment. To accomplish this objective, it is important to establish a sophisticated algorithm that can deal with large quantities of data such as gene expression profiles obtained by DNA microarray analysis.ResultsPreviously, we developed the projective adaptive resonance theory (PART) filtering method as a gene filtering method. This is one of the clustering methods that can select specific genes for each subtype. In this study, we applied the PART filtering method to analyze microarray data that were obtained from soft tissue sarcoma (STS) patients for the extraction of subtype-specific genes. The performance of the filtering method was evaluated by comparison with other widely used methods, such as signal-to-noise, significance analysis of microarrays, and nearest shrunken centroids. In addition, various combinations of filtering and modeling methods were used to extract essential subtype-specific genes. The combination of the PART filtering method and boosting – the PART-BFCS method – showed the highest accuracy. Seven genes among the 15 genes that are frequently selected by this method – MIF, CYFIP2, HSPCB, TIMP3, LDHA, ABR, and RGS3 – are known prognostic marker genes for other tumors. These genes are candidate marker genes for the diagnosis of STS. Correlation analysis was performed to extract marker genes that were not selected by PART-BFCS. Sixteen genes among those extracted are also known prognostic marker genes for other tumors, and they could be candidate marker genes for the diagnosis of STS.ConclusionThe procedure that consisted of two steps, such as the PART-BFCS and the correlation analysis, was proposed. The results suggest that novel diagnostic and therapeutic targets for STS can be extracted by a procedure that includes the PART filtering method.

New cancer diagnosis modeling using boosting and projective adaptive resonance theory with improved reliable index

An optimal and individualized treatment protocol based on accurate diagnosis is urgently required for the adequate treatment of patients. For this purpose, it is important to develop a sophisticated algorithm that can manage large amount of data, such as gene expression data from DNA microarray, for optimal and individualized diagnosis. Especially, marker gene selection is essential in the analysis of gene expression data. In the present study, we developed the combination method of projective adaptive resonance theory and boosted fuzzy classifier with SWEEP operator method for model construction and marker selection. And we applied this method to microarray data of acute leukemia and brain tumor. The method enabled the selection of 14 important genes related to the prognosis of the tumor. In addition, we proposed improved reliability index for cancer diagnostic prediction of blinded subjects. Based on the index, the discriminated group with over 90% prediction accuracy was separated from the others. PART-BFCS with improved RI BFCS method does not only show high performance, but also has the feature of reliable prediction further. This result suggests that PART-BFCS with improved RI BFCS method has the potential to function as a new method of class prediction for diagnosis of patients.

Classification of intramural metastases and lymph node metastases of esophageal cancer from gene expression based on boosting and projective adaptive resonance theory

Esophageal cancer is a well-known cancer with poorer prognosis than other cancers. An optimal and individualized treatment protocol based on accurate diagnosis is urgently needed to improve the treatment of cancer patients. For this purpose, it is important to develop a sophisticated algorithm that can manage a large amount of data, such as gene expression data from DNA microarrays, for optimal and individualized diagnosis. Marker gene selection is essential in the analysis of gene expression data. We have already developed a combination method of the use of the projective adaptive resonance theory and that of a boosted fuzzy classifier with the SWEEP operator denoted PART-BFCS. This method is superior to other methods, and has four features, namely fast calculation, accurate prediction, reliable prediction, and rule extraction. In this study, we applied this method to analyze microarray data obtained from esophageal cancer patients. A combination method of PART-BFCS and the U-test was also investigated. It was necessary to use a specific type of BFCS, namely, BFCS-1,2, because the esophageal cancer data were very complexity. PART-BFCS and PART-BFCS with the U-test models showed higher performances than two conventional methods, namely, k-nearest neighbor (kNN) and weighted voting (WV). The genes including CDK6 could be found by our methods and excellent IF-THEN rules could be extracted. The genes selected in this study have a high potential as new diagnosis markers for esophageal cancer. These results indicate that the new methods can be used in marker gene selection for the diagnosis of cancer patients.

Modified signal-to-noise: a new simple and practical gene filtering approach based on the concept of projective adaptive resonance theory (PART) filtering method

Considering the recent advances in and the benefits of DNA microarray technologies, many gene filtering approaches have been employed for the diagnosis and prognosis of diseases. In our previous study, we developed a new filtering method, namely, the projective adaptive resonance theory (PART) filtering method. This method was effective in subclass discrimination. In the PART algorithm, the genes with a low variance in gene expression in either class, not both classes, were selected as important genes for modeling. Based on this concept, we developed novel simple filtering methods such as modified signal-to-noise (S2N') in the present study. The discrimination model constructed using these methods showed higher accuracy with higher reproducibility as compared with many conventional filtering methods, including the t-test, S2N, NSC and SAM. The reproducibility of prediction was evaluated based on the correlation between the sets of U-test p-values on randomly divided datasets. With respect to leukemia, lymphoma and breast cancer, the correlation was high; a difference of >0.13 was obtained by the constructed model by using <50 genes selected by S2N'. Improvement was higher in the smaller genes and such higher correlation was observed when t-test, NSC and SAM were used. These results suggest that these modified methods, such as S2N', have high potential to function as new methods for marker gene selection in cancer diagnosis using DNA microarray data. Software is available upon request.

Lymphoma Prognostication from Expression Profiling Using a Combination Method of Boosting and Projective Adaptive Resonance Theory

In the present study, we developed the PART-robustBFCS method modified from PART-BFCS. This modeling was performed by using a bagging algorithm. In this algorithm, boosting result was assessed by using the data except one for model construction in order to repress the overfitting by modeling. We applied this method to the analysis of microarray data for the subclass identification of diffuse large B-cell lymphoma (DLBCL) patients. The results of our methods were superior to those of various other methods. The prediction accuracies were 75% for PART-BFCS and 79% for PART-robustBFCS.