Two-level Learning Research Articles

A smart heuristic scanner for an intrusion detection system using two-stage machine learning techniques

Building intrusion detection system (IDS) for an enterprise is a complex and challenging task as the attack types are growing day by day. Hence, there is a need for a smart heuristic scanner in an IDS to perform deep packet inspection in order to detect newer form of attacks and decisively declare a source as trusted or un-trusted. To perform deep packet inspection, packet headers at transport and network layers are processed through a two-level machine learning classifiers. Naive Bayes is applied in the first stage on TCP level heuristics. The output of the first stage classifier and IP heuristics are given as input to the k-nearest neighbours (KNNs) classifier in the second stage. At the end of the second stage classification, results are rendered as trusted or un-trusted. The experimental results showed that the proposed approach is efficient in terms of detection rate and false alarms.

An adaptive unimodal subclass decomposition (AUSD) learning system for land use and land cover classification using high-resolution remote sensing

Land use and land cover classification is an important application of remote-sensing images. The performances of most classification models are largely limited by the incompleteness of the calibration set and the complexity of spectral features. It is difficult for models to realize continuous learning when the study area is transferred or enlarged. This paper proposed an adaptive unimodal subclass decomposition (AUSD) learning system, which comprises two-level iterative learning controls: The inner loop separates each class into several unimodal Gaussian subclasses; the outer loop utilizes transfer learning to extend the model to adapt to supplementary calibration set collected from enlarged study areas. The proposed model can be efficiently adjusted according to the variability of spectral signatures caused by the increasingly high-resolution imagery. The classification result can be obtained using the Gaussian mixture model by Bayesian decision theory. This AUSD learning system was validated using simulated data with the Gaussian distribution and multi-area SPOT-5 high-resolution images with 2.5-m resolution. The experimental results on numerical data demonstrated the ability of continuous learning. The proposed method achieved an overall accuracy of over 90% in all the experiments, validating the effectiveness as well as its superiority over several widely used classification methods.

Discriminative Link Prediction using Local, Community, and Global Signals

Predicting plausible links that may emerge between pairs of nodes is an important task in social network analysis, with over a decade of active research. Here, we propose a novel framework for link prediction. It integrates signals from node features, the existing local link neighborhood of a node pair, community-level link density, and global graph properties. Our framework uses a stacked two-level learning paradigm. At the lower level, the first two kinds of features are processed by a novel local learner. Its outputs are then integrated with the last two kinds of features by a conventional discriminative learner at the upper-level. We also propose a new stratified sampling scheme for evaluating link prediction algorithms in the face of an extremely large number of potential edges, out of which very few will ever materialize. It is not tied to a specific application of link prediction, but robust to a range of application requirements. We report on extensive experiments with seven benchmark datasets and over five competitive baseline systems. The system we present consistently shows at least 10 percent accuracy improvement over state-of-the-art, and over 30 percent improvement in some cases. We also demonstrate, through ablation, that our features are complementary in terms of the signals and accuracy benefits they provide.

Aging Face Recognition: A Hierarchical Learning Model Based on Local Patterns Selection.

Aging face recognition refers to matching the same person's faces across different ages, e.g., matching a person's older face to his (or her) younger one, which has many important practical applications, such as finding missing children. The major challenge of this task is that facial appearance is subject to significant change during the aging process. In this paper, we propose to solve the problem with a hierarchical model based on two-level learning. At the first level, effective features are learned from low-level microstructures, based on our new feature descriptor called local pattern selection (LPS). The proposed LPS descriptor greedily selects low-level discriminant patterns in a way, such that intra-user dissimilarity is minimized. At the second level, higher level visual information is further refined based on the output from the first level. To evaluate the performance of our new method, we conduct extensive experiments on the MORPH data set (the largest face aging data set available in the public domain), which show a significant improvement in accuracy over the state-of-the-art methods.

ARTgrid: A Two-Level Learning Architecture Based on Adaptive Resonance Theory

This paper proposes a novel neural network architecture based on adaptive resonance theory (ART) called ARTgrid that can perform both online and offline clustering of 2D object structures. The main novelty of the proposed architecture is a two-level categorization and search mechanism that can enhance computation speed while maintaining high performance in cases of higher vigilance values. ARTgrid is developed for specific robotic applications for work in unstructured environments with diverse work objects. For that reason simulations are conducted on random generated data which represents actual manipulation objects, that is, their respective 2D structures. ARTgrid verification is done through comparison in clustering speed with the fuzzy ART algorithm and Adaptive Fuzzy Shadow (AFS) network. Simulation results show that by applying higher vigilance values (ρ>0.85) clustering performance of ARTgrid is considerably better, while lower vigilance values produce comparable results with the original fuzzy ART algorithm.

A two-level on-line learning algorithm of Artificial Neural Network with forward connections

An Artificial Neural Network with cross-connection is one of the most popular network structures. The structure contains: an input layer, at least one hidden layer and an output layer. Analysing and describing an ANN structure, one usually finds that the first parameter is the number of ANN’s layers. A hierarchical structure is a default and accepted way of describing the network. Using this assumption, the network structure can be described from a different point of view. A set of concepts and models can be used to describe the complexity of ANN’s structure in addition to using a two-level learning algorithm. Implementing the hierarchical structure to the learning algorithm, an ANN structure is divided into sub-networks. Every sub-network is responsible for finding the optimal value of its weight coefficients using a local target function to minimise the learning error. The second coordination level of the learning algorithm is responsible for coordinating the local solutions and finding the minimum of the global target function. In the article a special emphasis is placed on the coordinator’s role in the learning algorithm and its target function. In each iteration the coordinator has to send coordination parameters into the first level of sub-networks. Using the input X and the teaching ?? vectors, the local procedures are working and finding their weight coefficients. At the same step the feedback information is calculated and sent to the coordinator. The process is being repeated until the minimum of local target functions is achieved. As an example, a two-level learning algorithm is used to implement an ANN in the underwriting process for classifying the category of health in a life insurance company.

Design of Radial Basis Function Network Using Adaptive Particle Swarm Optimization and Orthogonal Least Squares

This paper presents a two-level learning method for designing an optimal Radial Basis Function Network (RBFN) using Adaptive Velocity Update Relaxation Particle Swarm Optimization algorithm (AVURPSO) and Orthogonal Least Squares algorithm (OLS) called as OLS-AVURPSO method. The novelty is to develop an AVURPSO algorithm to form the hybrid OLS-AVURPSO method for designing an optimal RBFN. The proposed method at the upper level finds the global optimum of the spread factor parameter using AVURPSO while at the lower level automatically constructs the RBFN using OLS algorithm. Simulation results confirm that the RBFN is superior to Multilayered Perceptron Network (MLPN) in terms of network size and computing time. To demonstrate the effectiveness of proposed OLS-AVURPSO in the design of RBFN, the Mackey-Glass Chaotic Time-Series as an example is modeled by both MLPN and RBFN.

Intelligent Mutation Rate Control in an Economic Application of Genetic Algorithms

This work represents the first application of two-level learning in genetic algorithms in an economic environment in which the fitness value of potential rules are complementary across individuals. Two-level learning, or self-adaptation, incorporates certain strategy parameters into the representation of each individual. In this work, these strategy parameters provide the likelihood of mutation for the individual. These strategy parameters evolve by means of mutation and recombination, just as the object variables do. It is argued that self-adaptation over the parameter governing mutation can replace the election operator proposed by Arifovic (1994) in order to attain convergence to the rational expectations equilibrium. While both adaptive mutation and the election operator are sufficient for convergence, self-adaptation may be more appropriate when being compared with real-world or experimental economic data. Through analysis of a static environment it is shown that this convergence, however, will require a strong selective pressure only attained through a transformation of the baseline fitness function.

A Two-Level Approach to Fault-Tolerant Control of Distributed Systems based on the Sliding Mode

A new approach to Fault-Tolerant Control (FTC) of distributed and interconnected systems is proposed based on applying sliding mode control (SMC) to the subsystems of a level de-centralised and hierarchical scheme. The SMC approach involves a new optimal control strategy for the design of local sliding functions for the subsystem controllers, replacing the conventional approach based on constrained locally linear LQ receding horizon control. The linear SMC gains handle the reachability for the sliding surfaces and the interaction effects from subsystem interconnections as well as small fault effects (i.e. giving passive fault-tolerance), whilst the non-linear (discontinuous) SMC gains facilitate a powerful way of accounting for larger but bounded system non-linearities and faults and can fulfill the role of an active FTC scheme. The local and global performance constraints are retained and implemented under autonomous learning supervision via the interaction-prediction principle. The scheme for an Autonomous Control and Supervision System (ACSS) is described that is capable of learning its coordination function and carrying out fault-tolerant balancing of the distributed system. The paper describes how the two-level learning strategy offers advantages over single-level FTC distributed SMC. The design concepts are illustrated using a non-linear 3-tank liquid level and heating control system with component faults

Two-Level Automatic Adaptation of a Distributed User Profile for Personalized News Content Delivery

This paper presents a distributed client-server architecture for the personalized delivery of textual news content to mobile users. The user profile consists of two separate models, that is, the long-term interests are stored in a skeleton profile on the server and the short-term interests in a detailed profile in the handset. The user profile enables a high-level filtering of available news content on the server, followed by matching of detailed user preferences in the handset. The highest rated items are recommended to the user, by employing an efficient ranking process. The paper focuses on a two-level learning process, which is employed on the client side in order to automatically update both user profile models. It involves the use of machine learning algorithms applied to the implicit and explicit user feedback. The system's learning performance has been systematically evaluated based on data collected from regular system users.

Generalized Radial Basis Function Networks Trained with Instance Based Learning for Data Mining of Symbolic Data

The application of the Radial Basis Function neural networks in domains involving prediction and classification of symbolic data requires a reconsideration and a careful definition of the concept of distance between patterns. This distance in addition to providing information about the proximity of patterns should also obey some mathematical criteria in order to be applicable. Traditional distances are inadequate to access the differences between symbolic patterns. This work proposes the utilization of a statistically extracted distance measure for Generalized Radial Basis Function (GRBF) networks. The main properties of these networks are retained in the new metric space. Especially, their regularization potential can be realized with this type of distance. However, the examples of the training set for applications involving symbolic patterns are not all of the same importance and reliability. Therefore, the construction of effective decision boundaries should consider the numerous exceptions to the general motifs of classification that are frequently encountered in data mining applications. The paper supports that heuristic Instance Based Learning (IBL) training approaches can uncover information within the uneven structure of the training set. This information is exploited for the estimation of an adequate subset of the training patterns serving as RBF centers and for the estimation of effective parameter settings for those centers. The IBL learning steps are applicable to both the traditional and the statistical distance metric spaces and improve significantly the performance in both cases. The obtained results with this two-level learning method are significantly better than the traditional nearest neighbour schemes in many data mining problems.

Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks

The paper presents a two-level learning method for radial basis function (RBF) networks. A regularized orthogonal least squares (ROLS) algorithm is employed at the lower level to construct RBF networks while the two key learning parameters, the regularization parameter and the RBF width, are optimized using a genetic algorithm (GA) at the upper level. Nonlinear time series modeling and prediction is used as an example to demonstrate the effectiveness of this hierarchical learning approach.

Tracking Context Changes through Meta-Learning

The article deals with the problem of learning incrementally (‘on-line’) in domains where the target concepts are context-dependent, so that changes in context can produce more or less radical changes in the associated concepts. In particular, we concentrate on a class of learning tasks where the domain provides explicit clues as to the current context (e.g., attributes with characteristic values). A general two-level learning model is presented that effectively adjusts to changing contexts by trying to detect (via ‘meta-learning’) contextual clues and using this information to focus the learning process. Context learning and detection occur during regular on-line learning, without separate training phases for context recognition. Two operational systems based on this model are presented that differ in the underlying learning algorithm and in the way they use contextual information: METAL(B) combines meta-learning with a Bayesian classifier, while METAL(IB) is based on an instance-based learning algorithm. Experiments with synthetic domains as well as a number of ‘real-world’ problems show that the algorithms are robust in a variety of dimensions, and that meta-learning can produce substantial increases in accuracy over simple object-level learning in situations with changing contexts.