Neural Tree Research Articles

Afrikaans Syllabification Patterns

In contrast to English, automatic hyphenation by computer of Afrikaans words is a problem that still needs to be addressed, since errors are still often encountered in printed text. An initial step in this task is the ability to automatically syllabify words. Since new words are created continuously by joining words, it is necessary to develop an “intelligent” technique for syllabification. As a first phase of the research, we consider only the orthographic information of words, and disregard both syntactic and morphological information. This approach allows us to use machine-learning techniques such as artificial neural networks and decision trees that are known for their pattern recognition abilities. Both these techniques are trained with isolated patterns consisting of input patterns and corresponding outputs (or targets) that indicate whether the input pattern should be split at a certain position, or not. In the process of compiling a list of syllabified words from which to generate training data for the syllabification problem, irregular patterns were identified. The same letter patterns are split differently in different words and complete words that are spelled identically are split differently due to meaning. We also identified irregularities in and between the different dictionaries that we used. We examined the influence range of letters that are involved in irregularities. For example, for their in agter-ente and vaste-rente we have to consider three letters to the left of r to be certain where the hyphen should be inserted. The influence range of the k in verstek-waarde and kleinste-kwadrate is four to the left and three to the right. In an analysis of letter patterns in Afrikaans words we found that the letter e has the highest frequency overall (16,2% of all letters in the word list). The frequency of words starting with s is the highest, while the frequency of words ending with e is the highest. It is important to note that the frequency of words ending with s is even higher than for words starting with s. The two and three letter patterns that occur most are er (10% of all two letter patterns) and ing (4% of all three letter patterns). In an analysis of syllables in Afrikaans words, we found that (as for complete words) syllables most often start with the letter s and end with e, while the frequency of syllables ending with s is almost as high as the frequency of syllables starting with s. This indicates that problems with hyphenation can be expected around the letter s. The two and three letter syllables that occur most often are -ge- and -ver-, respectively.In an attempt to decide on the window length to use to generate training data for machine-learning techniques we also analysed the length of syllables. The results show that two and three letter syllables occur most often, but that four letter syllables have the most unique instances. We also analysed a spectrum of window configurations and found that the ideal configuration will have to be determined empirically. One major problem we identified in this study is that irregular syllabification often occurs where letter patterns include the letter s. The reasons being (i) the use of the combining s when joining words, (ii) almost equal frequencies of syllables starting and ending with s and (iii) vague hyphe- nation rules for letter combinations containing s. To effectively address automatic syllabification in Afrikaans, it is necessary to develop more sophisticated methods to handle vagueness around the letter s.

Development of an enhanced oil recovery screening system

Selecting a suitable enhanced oil recovery (EOR) process for a given reservoir is a challenge for reservoir engineers because he or she has to compare pros and cons of all the available EOR methods in the context of each selected reservoir. The task becomes even more complex if they have to go through the details of hundreds of mature and depleted reservoirs to come up with a shortlist of EOR candidates for more detailed investigation. Conducting such a screening study could be daunting unless it is approached in a systematic manner. In this paper we describe development of an EOR screening system, which will enable engineers and scientists to sieve through some commonly available reservoir parameters for dozens or hundreds of reservoirs at a time, in a relatively efficient manner without compromising the quality of the outcome. Prior to the development of the screening system, an extensive literature review was conducted to develop the main EOR database and to narrow down the few essential criteria that would be sufficient for judging a reservoir’s candidacy for one or more known EOR techniques. Multilayer feed forward neural network and classification tree methods have been selected for the proposed screening system using the data mining software called XLMiner. The developed EOR screening system was tested by conducting some sensitivity analysis and screening tests. The results generated by the proposed EOR screening system were found satisfactory.

An Improved Intrusion Detection Technique based on two Strategies Using Decision Tree and Neural Network

In this paper we enhance the notion of anomaly detection and use both neural network (NN) and decision tree (DT) for intrusion detection. While DTs are highly successful in detecting known attacks, NNs are more interesting to detect new attacks. In our method we proposed a new approach to design the system using both DT and combination of unsupervised and supervised NN for Intrusion Detection System (IDS). By applying DT known attacks would be recognized with a quick execution time. Unknown attacks would be detected by applying the unsupervised NN based on hybrid of Self Organizing Map (SOM) for clustering attacks into smaller categories and supervised NN based on Backpropagation for detailed clustering.

A novel classification method using the combination of FDPS and flexible neural tree

The combination of Further Division of Partition Space (FDPS) and Flexible Neural Tree (FNT) is proposed to improve the neural network classification performance. FDPS, which divides partition space into many partitions that will attach to different classes automatically, is a novel technique for neural network classification. FNT is a neural network's structure which uses flexible tree model. The proposed method combines FDPS and FNT to overcome their respective problems by using the other's merit. In order to evaluate the performance of this method, four well-known data sets are used for classification test. Experiment results have shown that this method has favorable performance.

Tutorial: Pattern recognition in acoustic signal processing.

This tutorial presents a framework for understanding and comparing applications of pattern recognition in acoustic signal processing. Representative examples will be delimited by two binary features: (1) regression versus classification (inferred variables are continuous versus discrete); (2) instantaneous versus dynamic (inference algorithms consider only an instantaneous observation vector versus inference algorithms integrate observations with knowledge of system dynamics). (1. Regression) problems include imaging and sound source tracking using a device with unknown properties and inverse problems, e.g., articulatory estimation from speech audio. (2. Classification) problems include, e.g., classification of animal and human vocalizations and nonspeech audio events. Instantaneous classification and regression are performed using a universal approximator (neural network, Gaussian mixture, classification, and regression tree), regularized, if necessary, to reduce generalization error (resulting in a support vector machine, regularized neural net, pruned classification tree, or AdaBoost). Dynamic classification and regression are done by imposing a prior to characterize system dynamics. Depending on the prior, the resulting model may be called a hidden Markov model, finite state transducer, dynamic Bayesian network, or conditional random field (dynamic classification), or a Kalman filter, extended Kalman filter, or switching Kalman filter (dynamic regression).

Evolving comprehensible neural network trees using genetic algorithms

This paper presents a comprehensible neural network tree (CNNTREE). CNNTREE is a proposed general modular neural network structure, where each node in this tree is a comprehensible expert neural network (CENN). One advantage of using CNNTREE is that it is a “gray box”; because it can be interpreted easily for symbolic systems; where each node in the CNNTREE is equivalent for symbolic operator in the symbolic system. Another advantage of CNNTREE is that it can be trained as any normal multi layer feed forward neural network. An evolutionary algorithm is given for designing the CNNTREE. Back propagation is also checked as local learning algorithm that fits for real time learning constraints. The tree generalization and training performance are examined using experiments with a digit recognition problem.

Modeling of Fluid Industry Based on FlexibleNeural Tree

Realizing optimal control of fluid industry is a difficult problem due to its features of complexity, strong correlation, non-linear and uncertainty. For solving the problem, we propose build the whole model for it. Cement production process is taken as an instance, and its production process model is gotten by evolving flexible neural tree (FNT). The FNT model's structure and parameters are optimized by probabilistic incremental program evolution (PIPE) and simulation annealing (SA) respectively. The result demonstrates that the put forward method is effective and feasible for solving the problem.

A Data Mining Approach to Rainfall Intensity Classification Using TRMM/TMI Data

The systematic approach we propose for classifying oceanic rainfall intensity during the typhoon season consists of two major steps – 1) identifying the rain areas and 2) classifying rainfall intensity intonormalandheavyfor these areas. The heterogeneous hierarchical classifier (HHC), an ensemble model we developed for accurately identifying heavy rainfall events, consists of a set of base classifiers. The base classifiers are independently constructed through heterogeneous data mining approaches such as artificial neural networks, decision trees, and self-organizing maps. The meteorological satellite Tropical Rainfall Measuring Mission (TRMM) microwave imager (TMI) data from 2000 to 2005 are used to create the classification models. TRMM precipitation radar (PR) data and rain gauge data from Automatic Rainfall and Meteorological Telemetry System (ARMTS) measurement are used as ground truth data to evaluate models. Two thirds of the dataset is used for model training and one third for testing. Experimental results show that the proposed model classifies rainfall intensity highly accurately and outperforms previously published methods.

Online hybrid traffic classifier for Peer-to-Peer systems based on network processors

It is estimated that 70% or more of broadband bandwidth is consumed by transmitting music, games, video and other content through Peer-to-Peer (P2P) clients. In order to detect, identify, and manage P2P traffic, some port, payload and transport layer feature based methods were proposed. Most of them were applied to offline traffic classification mainly due to the performance reason. In this paper, a network processors (NPs) based online hybrid traffic classifier is proposed. The designed hardware classifier is able to classify P2P traffic based on the static characteristic namely on line speed, and the Flexible Neural Tree(FNT) based software classifier helps learning and selecting P2P traffic attributes from the statistical characteristics of the P2P traffic. Experiment results illustrate that the hybrid classifier performs well for online classification of P2P traffic from gigabit network. The proposed framework also depicts good expansion capabilities to add new P2P features and to adapt to new P2P applications online.

Pattern discrimination method with a boosting approach using hierarchical neural trees

This paper proposes a new pattern classification method using probabilistic neural networks based on a boosting approach. In this method, a log-linearized Gaussian mixture network is used as a weak classifier. The method proposed automatically constructs a suitable classification network from given data. Validity of the proposed method is shown by discrimination results of artificial data and hand shape. The application is confirmed of the proposed method to human interface controlling of home electric appliances using hand shapes.

Classification Methods in the Detection of New Suspicious Emails

Email has become one of the fastest and most economical forms of communication. However, the increase of email users has resulted in the dramatic increase of suspicious emails during the past few years. This paper proposes to apply classification data mining for the task of suspicious email detection based on deception theory. In this paper, email data was classified using four different classifiers (Neural Network, SVM, Naïve Bayesian and Decision Tree). The experiment was performed using weka on the basis of different data size by which the suspicious emails are detected from the email corpus. Experimental results show that simple ID3 classifier which make a binary tree, will give a promising detection rates.

Adaptive Fuzzy Neural Tree Network

The objective of this article is to combine different adaptive methods, such as: Adaptive Decision Trees,Artificial Neural Networks and Fuzzy Logic for data mining purposes. The expected result is to maximize the advantages of different methods in a new model called: Adaptive Neural Tree Network.

Demonstration of data mining approaches in credit risk evaluation

Ratios from financial statements provide useful information to describe credit conditions from various perspectives, such as financial conditions and credit status. A good quantitative model is crucial in scoring the company credit status. This study demonstrates the utilisation of several data mining algorithms, i.e., Backpropagation Neural Network (BPNN), decision tree, logistic regression and Monte Carlo simulation in credit-scoring problem. Results are favourable in this case study.

Fast training of neural trees by adaptive splitting based on cubature

In this paper we prove that any affine function defined on a d-simplex in R d can be uniformly approximated by a single-layer neural network having only two neurons irrespective of d. The weights of this network are obtained in a closed analytical form, without training. This fact gives a correspondence rule that allows to transform mathematical approximants based on piecewise affine functions, into neural networks. We introduce such an approximant, adaptive splitting based on cubature (ASBC), for the efficient approximation of continuous functions. Using ASBC and the above correspondence rule, we obtain a neural tree. Numerical experiments on learning the function distance from a variable point to a geometric body in two and three dimensions show fast learning speed and high accuracy when compared with single-hidden layer feedforward networks trained by a trust region method based on the interior-reflective Newton algorithm.

Towards the experimental evaluation of novel supervised fuzzy adaptive resonance theory for pattern classification

This paper presents a comparative analysis of novel supervised fuzzy adaptive resonance theory (SF-ART), multilayer perceptron (MLP) and competitive neural trees (CNeT) Networks over three pattern recognition problems. We have used two well-known patterns (IRIS and Vowel data) and a biological data (hydrogen data) to evaluate and check SF-ART stability, reliability, learning speed and computational load. The comparative tests with IRIS, Vowels and H 2 data indicate that the SF-ART is capable to perform with a high classification performance, high learning speed (elapsed time for learning around half second), and very low computational load compared to the well-known neural networks such as MLP and CNeT which need minutes and seconds respectively to learn the training material.

Hybrid flexible neural-tree-based intrusion detection systems

An intrusion is defined as a violation of the security policy of the system, and, hence, intrusion detection mainly refers to the mechanisms that are developed to detect violations of system security policy. Current intrusion detection systems ~IDS! examine all data features to detect intrusion or misuse patterns. Some of the features may be redundant or contribute little ~if anything! to the detection process. The purpose of this study is to identify important input features in building an IDS that is computationally efficient and effective. This article proposes an IDS model based on a general and enhanced flexible neural tree ~FNT!. Based on the predefined instruction/operator sets, a flexible neural tree model can be created and evolved. This framework allows input variables selection, overlayer connections, and different activation functions for the various nodes involved. The FNT structure is developed using an evolutionary algorithm, and the parameters are optimized by a particle swarm optimization algorithm. Empirical results indicate that the proposed method is efficient. © 2007 Wiley Periodicals, Inc.

Self-generating prototypes for pattern classification

Prototype classifiers are a type of pattern classifiers, whereby a number of prototypes are designed for each class so as they act as representatives of the patterns of the class. Prototype classifiers are considered among the simplest and best performers in classification problems. However, they need careful positioning of prototypes to capture the distribution of each class region and/or to define the class boundaries. Standard methods, such as learning vector quantization (LVQ), are sensitive to the initial choice of the number and the locations of the prototypes and the learning rate. In this article, a new prototype classification method is proposed, namely self-generating prototypes (SGP). The main advantage of this method is that both the number of prototypes and their locations are learned from the training set without much human intervention. The proposed method is compared with other prototype classifiers such as LVQ, self-generating neural tree (SGNT) and K-nearest neighbor ( K-NN) as well as Gaussian mixture model (GMM) classifiers. In our experiments, SGP achieved the best performance in many measures of performance, such as training speed, and test or classification speed. Concerning number of prototypes, and test classification accuracy, it was considerably better than the other methods, but about equal on average to the GMM classifiers. We also implemented the SGP method on the well-known STATLOG benchmark, and it beat all other 21 methods (prototype methods and non-prototype methods) in classification accuracy.

Flexible neural trees ensemble for stock index modeling

The use of intelligent systems for stock market predictions has been widely established. In this paper, we investigate how the seemingly chaotic behavior of stock markets could be well represented using flexible neural tree (FNT) ensemble technique. We considered the Nasdaq-100 index of Nasdaq Stock Market SM and the S&P CNX NIFTY stock index. We analyzed 7-year Nasdaq-100 main index values and 4-year NIFTY index values. This paper investigates the development of novel reliable and efficient techniques to model the seemingly chaotic behavior of stock markets. The structure and parameters of FNT are optimized using genetic programming (GP) like tree structure-based evolutionary algorithm and particle swarm optimization (PSO) algorithms, respectively. A good ensemble model is formulated by the local weighted polynomial regression (LWPR). This paper investigates whether the proposed method can provide the required level of performance, which is sufficiently good and robust so as to provide a reliable forecast model for stock market indices. Experimental results show that the model considered could represent the stock indices behavior very accurately.

A novel intrusion detection mode based on understandable neural network trees

Several data mining techniques such as Hidden Markov Model (HMM), artificial neural network, statistical techniques and expert systems are used to model network packets in the field of intrusion detection. In this paper a novel intrusion detection mode based on understandable Neural Network Tree (NNTree) is presented. NNTree is a modular neural network with the overall structure being a Decision Tree (DT), and each non-terminal node being an Expert Neural Network (ENN). One crucial advantage of using NNTrees is that they keep the non-symbolic model ENN’s capability of learning in changing environments. Another potential advantage of using NNTrees is that they are actually “gray boxes” as they can be interpreted easily if the number of inputs for each ENN is limited. We showed through experiments that the trained NNTree achieved a simple ENN at each non-terminal node as well as a satisfying recognition rate of the network packets dataset. We also compared the performance with that of a three-layer backpropagation neural network. Experimental results indicated that the NNTree based intrusion detection model achieved better performance than the neural network based intrusion detection model.

Feature selection and classification using flexible neural tree

The purpose of this research is to develop effective machine learning or data mining techniques based on flexible neural tree FNT. Based on the pre-defined instruction/operator sets, a flexible neural tree model can be created and evolved. This framework allows input variables selection, over-layer connections and different activation functions for the various nodes involved. The FNT structure is developed using genetic programming (GP) and the parameters are optimized by a memetic algorithm (MA). The proposed approach was applied for two real-world problems involving designing intrusion detection system (IDS) and for breast cancer classification. The IDS data has 41 inputs/features and the breast cancer classification problem has 30 inputs/features. Empirical results indicate that the proposed method is efficient for both input feature selection and improved classification rate.