Fisher's Iris Research Articles

Application of a noisy data classification technique to determine the occurrence of flashover in compartment fires

This paper presents a hybrid Artificial Neural Network (ANN) model that is developed for noisy data classification. The model, named GRNNFA, is a fusion of the Fuzzy Adaptive Resonance Theory (FA) model and the General Regression Neural Network (GRNN) model. The GRNNFA model not only retains the important features of the parent models, which include stable learning, fast training, and an incremental growth network structure, but also facilitates the removal of noise that is embedded in training samples. The robustness of the GRNNFA model is demonstrated by the Noisy Two Intertwined Spirals problem and other benchmarking problems. The model is then applied to Fisher's Iris Data, which is a real-world classification problem. The results show that the percentage of correct predictions is statistically higher than in variant models of the adaptive resonance theory. The GRNNFA is further employed in a new application area of soft computing—fire dynamics, which is highly non-linear in nature. Flashover is the most dangerous scenrio in the development of a compartment fire, during which, any unburned combustible material, including the unburned soot particles inside the compartment, is ignited spontaneously and all combustible material is then simultaneously involved in the burning process. The GRNNFA model is applied to predict the occurrence of the flashover in compartment fires based on the fire size and the geometry of the fire compartment. The performance of the GRNNFA is compared with other published results, and it is shown to be statistically superior to other ANN models.

GENETIC PROGRAMMING APPROACH FOR MULTI-CATEGORY PATTERN CLASSIFICATION APPLIED TO NETWORK INTRUSIONS DETECTION

The present paper describes a new approach of classification using genetic programming. The proposed technique consists of genetically co-evolve a population of nonlinear transformations on the input data to be classified, and map them to a new space with reduced dimension in order to get a maximum inter-classes discrimination. It is much easier to classify the new samples from the transformed data. Contrary to the existing GP-classification techniques, the proposed one uses a dynamic repartition of the transformed data in separated intervals, the efficiency of a given intervals repartition is handled by the fitness criterion, with a maximum classes discrimination. Experiments were performed using the Fisher's Iris dataset. After that, the KDD'99 Cup dataset was used to study the intrusion detection and classification problem. The results demonstrate that the proposed genetic approach outperforms the existing GP-classification methods, and provides improved results compared to other existing techniques.

Influential rule search scheme (IRSS) - a new fuzzy pattern classifier

Automatic generation of fuzzy rule base and membership functions from an input-output data set, for reliable construction of an adaptive fuzzy inference system, has become an important area of research interest. We propose a new robust, fast acting adaptive fuzzy pattern classification scheme, named influential rule search scheme (IRSS). In IRSS, rules which are most influential in contributing to the error produced by the adaptive fuzzy system are identified at the end of each epoch and subsequently modified for satisfactory performance. This fuzzy rule base adjustment scheme is accompanied by an output membership function adaptation scheme for fine tuning the fuzzy system architecture. This iterative method has shown a relatively high speed of convergence. Performance of the proposed IRSS is compared with other existing pattern classification schemes by implementing it for Fisher's iris data problem and Wisconsin breast cancer data problems.

Integrated Feature and Architecture Selection for Radial Basis Neural Networks

In this paper, we present a new integrated architecture and feature selection algorithm for radial basis neural networks (RBNNs). The objective is to apply the network in an iterative fashion to determine the final architecture and feature set used for the purpose of classification. Numerical experiments were conducted to compare the performance of the new algorithm to that of discriminant analysis (DA) and feed-forward neural networks (FFNNs) on four problems: the University of Wisconsin breast cancer data, a noise-corrupted version of Fisher's iris problem and two problems of higher dimensionality taken from UCI Repository of Machine Learning Databases and Domain Theories (http://www.mat.univie.ac.at/∼neum/contrib/class.data). Preliminary numerical results indicate great promise for the new algorithm as a general classification tool for a wide range of problems.

주성분 자기조직화 지도 PC-SOM

자기조직화 지도(SOM)은 T. 코호넨의 주도하에 개발된 비지도 학습 신경망 모형이다. 그 동안 패턴인식과 문서검색 분야에 주로 응용되어 왔기 때문에 통계학 분야에서는 덜 알려졌으나, 최근 K-평균 군집화에 대한 대안적 데이터 마이닝 기법으로 활용되기 시작하였다. 본 연구에서는 SOM의 한 버전인 PC-SOM(주성분 자기조직화 지도)을 제안하고 활용 예를 제시하고자 한다. PC-SOM은 1차원적 SOM 알고리즘을 반복 수행하여 2차원, 3차원 등의 SOM을 얻는 방법이기 때문에 기존 SOM과는 달리 사전 Map의 크기를 확정할 필요가 없다. 또한, 기존 SOM에 비하여 향상된 시각화를 가능하게 한다. Self-organizing map (SOM), a unsupervised learning neural network, has been developed by T. Kohonen since 1980's. Main application areas were pattern recognition and text retrieval. Because of that, it has not been spread to statisticians until late. Recently, SOM's are frequently drawn in data mining fields. Kohonen's SOM, however, needs improvements to become a statistician's standard tool. First, there should be a good guideline as for the size of map. Second, an enhanced visualization mode is wanted. In this study, principal components self-organizing map (PC-SOM), a modification of Kohonen's SOM, is proposed to meet such needs. PC-SOM performs one-dimensional SOM during the first stage to decompose input units into node weights and residuals. At the second stage, another one-dimensional SOM is applied to the residuals of the first stage. Finally, by putting together two stages, one obtains two-dimensional SOM. Such procedure can be easily expanded to construct three or more dimensional maps. The number of grid lines along the second axis is determined automatically, once that of the first axis is given by the data analyst. Furthermore, PC-SOM provides easily interpretable map axes. Such merits of PC-SOM are demonstrated with well-known Fisher's iris data and a simulated data set.

Constructing a fuzzy rule-based system using the ILFN network and Genetic Algorithm.

In this paper, a method for automatic construction of a fuzzy rule-based system from numerical data using the Incremental Learning Fuzzy Neural (ILFN) network and the Genetic Algorithm is presented. The ILFN network was developed for pattern classification applications. The ILFN network, which employed fuzzy sets and neural network theory, equips with a fast, one-pass, on-line, and incremental learning algorithm. After trained, the ILFN network stored numerical knowledge in hidden units, which can then be directly interpreted into if then rule bases. However, the rules extracted from the ILFN network are not in an optimized fuzzy linguistic form. In this paper, a knowledge base for fuzzy expert system is extracted from the hidden units of the ILFN classifier. A genetic algorithm is then invoked, in an iterative manner, to reduce number of rules and select only discriminate features from input patterns needed to provide a fuzzy rule-based system. Three computer simulations using a simulated 2-D 3-class data, the well-known Fisher's Iris data set, and the Wisconsin breast cancer data set were performed. The fuzzy rule-based system derived from the proposed method achieved 100% and 97.33% correct classification on the 75 patterns for training set and 75 patterns for test set, respectively. For the Wisconsin breast cancer data set, using 400 patterns for training and 299 patterns for testing, the derived fuzzy rule-based system achieved 99.5% and 98.33% correct classification on the training set and the test set, respectively.

A novel approach to the derivation of fuzzy membership functions using the Falcon-MART architecture

A fuzzy neural network, Falcon-MART, is proposed in this paper. This is a modification of the original Falcon-ART architecture. Both Falcon-ART and Falcon-MART are fuzzy neural networks that can be used as fuzzy controllers or applied to areas such as forgery detection, pattern recognition and data analysis. They constitute a group of hybrid systems that incorporate fuzzy logic into neural networks. In this way, the structure of these hybrid networks become transparent as high level IF-THEN human-like reasoning is used to interpret the network connections. In addition, the hybrid networks automatically derive the fuzzy rules (knowledge base) of the problem domain using neural network techniques and hence avoid the pitfalls of traditional fuzzy systems. The main problem in designing a fuzzy neural network is how to formulate the fuzzy rule base. Most proposed fuzzy neural networks in the literature could be classified into two categories. The first group assumes the existence of a preliminary rule base and uses neural techniques to tune the parameters to obtain the final set of fuzzy rules. The second group assumes no knowledge of any fuzzy rules and performs a cluster analysis on the numerical training data before formulating the rules from the computed clusters. Falcon-ART attempts to overcome the constraints faced by these two groups of fuzzy neural networks by using the fuzzy ART technique to partition the training data set. However, there are several shortcomings in the Falcon-ART network. They are: 1. Poor network performances when the classes of input data are closely similar to each other; 2. Weak resistance to noisy/spurious training data; 3. Termination of network training process depends heavily on a preset error parameter; and 4. Learning efficiency may deteriorate as a result of using complementary coded training data. Falcon-MART has been developed to address these shortcomings. To evaluate the effectiveness of Falcon-MART, three different sets of experiments are conducted. The first experiment demonstrates the efficiency of Falcon-MART over Falcon-ART using the Fisher's Iris data set. The second experiment evaluates the modeling capability of Falcon-MART against the classical multi-layered perceptron (MLP) network using a set of traffic flow data. The last experiment uses a set of phoneme data to demonstrate the clustering ability of Falcon-MART against the traditional K-nearest-neighbor ( K-NN) classifier. The results obtained are encouraging.

A resource limited artificial immune system for data analysis

This paper presents a resource limited artificial immune system (RLAIS) for data analysis. The work presented here builds upon previous work on artificial immune systems (AIS) for data analysis. A population control mechanism, inspired by the natural immune system, has been introduced to control population growth and allow termination of the learning algorithm. The new algorithm is presented, along with the immunological metaphors used as inspiration. Results are presented for Fisher Iris data set, where very successful results are obtained in identifying clusters within the data set. It is argued that this new resource-based mechanism is a large step forward in making AISs a viable contender for effective unsupervised machine learning and allows for not just a one shot learning mechanism, but a continual learning model to be developed.

An effective neuro-fuzzy paradigm for machinery condition health monitoring

An innovative neuro-fuzzy network appropriate for fault detection and classification in a machinery condition health monitoring environment is proposed. The network, called an incremental learning fuzzy neural (ILFN) network, uses localized neurons to represent the distributions of the input space and is trained using a one-pass, on-line, and incremental learning algorithm that is fast and can operate in real time. The ILFN network employs a hybrid supervised and unsupervised learning scheme to generate its prototypes. The network is a self-organized structure with the ability to adaptively learn new classes of failure modes and update its parameters continuously while monitoring a system. To demonstrate the feasibility and effectiveness of the proposed network, numerical simulations have been performed using some well-known benchmark data sets, such as the Fisher's Iris data and the Deterding vowel data set. Comparison studies with other well-known classifiers were performed and the ILFN network was found competitive with or even superior to many existing classifiers. The ILFN network was applied on the vibration data known as Westland data set collected from a U.S. Navy CH-46E helicopter test stand, in order to assess its efficiency in machinery condition health monitoring. Using a simple fast Fourier transform (FFT) technique for feature extraction, the ILFN network has shown promising results. With various torque levels for training the network, 100% correct classification was achieved for the same torque Levels of the test data.

Pattern classification by a neurofuzzy network: application to vibration monitoring

An innovative neurofuzzy network is proposed herein for pattern classification applications, specifically for vibration monitoring. A fuzzy set interpretation is incorporated into the network design to handle imprecise information. A neural network architecture is used to automatically deduce fuzzy if-then rules based on a hybrid supervised learning scheme. The neurofuzzy classifier proposed is equipped with a one-pass, on-line, and incremental learning algorithm. This network can be considered a self-organized classifier with the ability to adaptively learn new information without forgetting old knowledge. The classification performance of the proposed neurofuzzy network is validated on the Fisher's Iris data, which is a well-known benchmark data set. For the generalization capability, the neurofuzzy network can achieve 97.33% correct classification. In addition, to demonstrate the efficiency and effectiveness of the proposed neurofuzzy paradigm, numerical simulations have been performed using the Westland data set. The Westland data set consists of vibration data collected from a US Navy CH-46E helicopter test stand. Using a simple fast Fourier transform technique for feature extraction, the proposed neurofuzzy network has shown promising results. Using various torque levels for training and testing, the network achieved 100% correct classification.

Feature screening using signal-to-noise ratios

We present an approach for identifying important input features in multi-layer perceptron feedforward artificial neural networks (ANN) trained via backpropagation. Specifically, we propose a new saliency measure and demonstrate its use in a methodology that holds the promise of potentially identifying and removing noisy input features in a single training run. First, we propose a signal-to-noise ratio (SNR) saliency measure, which determines the saliency of a feature by comparing it to that of an injected noise feature. Next, we propose a feature screening method that utilizes the SNR saliency measure to select a parsimonious set of salient features. The SNR screening method is demonstrated using Fisher's Iris problem. Confidence in the SNR saliency measure and screening method are bolstered by comparisons to Setiono and Liu's neural network feature selector and a principal component analysis (PCA) approach on three real-world applications: the University of Wisconsin Breast Cancer Diagnosis problem, the US Congressional voting records problem, and the Pima Indians Diabetes problem.

A New Neuro-Fuzzy Classifier with Application to On-Line Face Detection and Recognition

In this paper, we propose a neuro-fuzzy classifier (NEFCAR) that utilizes positive and negative rules with different rule importances to create the decision boundaries between different classes. The locally unsupervised and globally supervised training technique is adopted. The decision-based and approximation-based strategies are combined to provide a suitable amount of training for each training pattern. The reinforced and anti-reinforced learning rules are given with different weighting so that the training can be efficient and can reach convergence quickly. Moreover, NEFCAR can easily provide the confidence measure of each classification decision. Therefore, the rejection algorithm can be implemented in a straightforward manner. Noise tolerant training is conducted to improve the generalization performance and the confidence measure is adopted to avoid overtraining. The proposed classifier is applied to two applications. The first one is the Fisher iris data classification, and the second one is an on-line face detection and recognition application. Good classification results are obtained in both applications. In the on-line face detection and recognition system, two NEFCAR's are utilized: a two-class and a multi-class NEFCAR's are adopted to detect the face and recognize the face, respectively. The color of skin and the motion information are taken into consideration heuristically to improve the effectiveness of the face location algorithm.

A network model using distance-based cosine elements

In this paper, we propose a new network element, “distance-based cosine element,” for neural networks. We also derive a learning algorithm based on the backpropagation algorithms for multilayer networks, The distance-based cosine element inputs a squared distance between an input pattern vector and its weight vector, and uses an affine transformation of cosine function as its output function. The proposed distance-based cosine network is able to improve its learning speed as well as convergence rate because its output function does not have any saturated regions which cause slow learning speed of the backpropagation learning using sigmoid elements. We demonstrate the advantages of the proposed network by solving N-bit parity problems and Fisher's iris classification problem. Experimental results indicate that our distance-based cosine network consistently obtains better results than the conventional sigmoid network in terms of both the learning speed and the convergence rate. © 1999 Scripta Technica, Electr Eng Jpn, 129(4): 87–95, 1999

AN INCREMENTAL LEARNING NEURAL NETWORK FOR PATTERN CLASSIFICATION

A neural network architecture that incorporates a supervised mechanism into a fuzzy adaptive Hamming net (FAHN) is presented. The FAHN constructs hyper-rectangles that represent template weights in an unsupervised learning paradigm. Learning in the FAHN consists of creating and adjusting hyper-rectangles in feature space. By aggregating multiple hyper-rectangles into a single class, we can build a classifier, to be henceforth termed as a supervised fuzzy adaptive Hamming net (SFAHN), that discriminates between nonconvex and even discontinuous classes. The SFAHN can operate at a fast-learning rate in online (incremental) or offline (batch) applications, without becoming unstable. The performance of the SFAHN is tested on the Fisher iris data and on an online character recognition problem.

A fuzzy classifier with ellipsoidal regions for diagnosis problems

In our previous work, we developed a fuzzy classifier with ellipsoidal regions that has a training capability. In this paper, we extend the fuzzy classifier to diagnosis problems, in which the training data belonging to abnormal classes are difficult to obtain while the training data belonging to normal classes are easily obtained. Assuming that there are no data belonging to abnormal classes, we first train the fuzzy classifier with only the data belonging to normal classes. We then introduce the threshold of the minimum-weighted distance from the centers of the clusters for the data belonging to normal classes. If the unknown data is within the threshold, we classify the data into normal classes and, if not, abnormal classes. The operator checks whether the diagnosis is correct. If the incoming data is classified into the same normal class both by the classifier and the operator, nothing is done. But if the input data is classified into the different normal classes by the classifier and the operator, or if the incoming data is classified into an abnormal class, but the operator classified it into a normal class, the slopes of the membership functions of the fuzzy rules are tuned. If the operator classifies the data into an abnormal class, the classifier is retrained adding the newly obtained data irrespective of the classifier's classification result. The online training is continued until a sufficient number of the data belonging to abnormal classes are obtained. Then the threshold is optimized using the data belonging to both normal and abnormal classes. We evaluate our method using the Fisher iris data, blood cell data, and thyroid data, assuming some of the classes are abnormal.

Structured discriminant analysis

Statistical discriminant analysis is a most common procedure, widely used for statistical decision applications. One reason for this popularity may be that classification is a perfect model for decision making. However, the available a priori knowledge may have other forms than strict class belonging : more or less precise. Structured Discriminant Analysis (SDA) offers a general framework for a wide variety of prediction problems. As Factorial Discriminant Analysis may be seen as Contiguity Analysis on a partition graph, a natural extension of the former is derived by representing a priori knowledge through interindividual similarity or agreement graphs. Fisher's Iris problem with various fictitious misclassification co. structures, illustrates the kind of facilities SDA brings to users. Implementation of other discrimination situations are described, and simulation results are reported. It appears that SDA also shows computational qualities, although its main purpose is allowing for a better attention to specificity of each prediction problem.

A Bayesian Evaluation of Fuzzy Logic in a Classification Problem

Models based on fuzzy sets are increasingly being used in academic and industrial research and applications, primarily for data analysis and systems control. The possible consequences of their popular use necessitate their evaluation from a probabilistic viewpoint. In this paper, a simple fuzzy logic model is translated into a Bayesian setting and used to address a discrimination and attribution (or classification) problem. Probabilistic tools and concepts are then employed to evaluate the fuzzy-set paradigm. The translation suggests that applications of fuzzy-set theory may suffer from many serious problems, primarily due to the unobservability of the fuzzy-set membership vector and to the poor structure of the fuzzy-set model. These difficulties are illustrated using the well-known Fisher iris data. For example, Table 11 suggests that fuzzy-set models may find probabilistic mixtures only when the evidence is thoroughly overwhelming in Bayesian models

An approach of clustering data with noisy or imprecise feature measurement

In this paper we considered clustering of data corrupted by noise or suffering from imprecision due to finite resolution of the feature measuring device. Our work is motivated by the fact that no measurement can be made perfect and addition of noise is not an uncommon phenomenon for telemetric data. Here we tried to show how the classical k-means algorithm should be modified to take care of the noise/imprecision. Experimental results on Fisher's Iris data and a Nutrition data are demonstrated.

CARVE-a constructive algorithm for real-valued examples

A constructive neural-network algorithm is presented. For any consistent classification task on real-valued training vectors, the algorithm constructs a feedforward network with a single hidden layer of threshold units which implements the task. The algorithm, which we call CARVE, extends the "sequential learning" algorithm of Marchand et al. from Boolean inputs to the real-valued input case, and uses convex hull methods for the determination of the network weights. The algorithm is an efficient training scheme for producing near-minimal network solutions for arbitrary classification tasks. The algorithm is applied to a number of benchmark problems including Gorman and Sejnowski's sonar data, the Monks problems and Fisher's iris data. A significant application of the constructive algorithm is in providing an initial network topology and initial weights for other neural-network training schemes and this is demonstrated by application to backpropagation.

A fuzzy classifier with ellipsoidal regions

In this paper, we discuss a fuzzy classifier with ellipsoidal regions which has a learning capability. First, we divide the training data for each class into several clusters. Then, for each cluster, we define a fuzzy rule with an ellipsoidal region around a cluster center. Using the training data for each cluster, we calculate the center and the covariance matrix of the ellipsoidal region for the cluster. Then we tune the fuzzy rules, i.e., the slopes of the membership functions, successively until there is no improvement in the recognition rate of the training data. We evaluate our method using the Fisher iris data, numeral data of vehicle license plates, thyroid data, and blood cell data. The recognition rates (except for the thyroid data) of our classifier are comparable to the maximum recognition rates of the multilayered neural network classifier and the training times (except for the iris data) are two to three orders of magnitude shorter.