Fisher's Iris Research Articles

An Oscillator-Synchronization-Based Off-Line Learning Algorithm, With On-Chip Inference on an Array of Spin Hall Nano-Oscillators

Learning algorithms based on synchronization of oscillators are currently being pursued as an alternative to standard neural-network algorithms for data classification. For such oscillator-based algorithms, while learning can be implemented off-line on a computer, on-chip inference/ testing can be implemented on an array of uniform-mode spin Hall nano-oscillators (SHNOs), leading to a scalable and energy-efficient technology. Here, we propose a modification to an existing oscillator-based off-line learning algorithm for binary classification: unlike in the previous version of the algorithm, here the difference between natural frequencies of the output oscillators is kept constant throughout the learning process. This helps in preservation of the shape of the synchronization region and leads to higher classification accuracy, as we show for binary-classification tasks using two popular data sets: Fisher's Iris and MNIST. Next, in this paper, we show how a synchronization pattern obtained after such training, using our proposed algorithm, can be implemented on an array of dipole-coupled uniform-mode SHNOs for on-chip inference. We model the SHNO system through the macro-spin model, which is computationally much less resource-intensive to simulate compared to the micromagnetic model used previously for this kind of study. We also extend our algorithm to multi-class classification and thereby discuss scaling of our system.

Agent-oriented data clustering for medical monitoring

Medical data processing is one of the priority machine learning areas. Usually, data obtained in the process of medical patient monitoring are complex and have a different nature. Solving the problem of clustering, classification, or forecasting problem these data requires the creation of new methods or improvement of existing methods to improve the decision accuracy and effectiveness. The classical clustering approaches and the c-means fuzzy clustering method were analyzed. Based on the multiagent systems theory, it is proposed to use in the c-means method the separate rules for selecting elites when forming clusters and selecting the best of them in accordance with the chosen intra-cluster distance measures. The result of solving such a problem is the number of clusters, as well as the number of elements in them. The method quality was tested on Fisher iris data set using three measures of intra-cluster distance: Mahalanobis distance, Mahalanobis distance considering the membership function, and Kullbak-Leibler entropy. The highest accuracy of 98% was obtained for the distance measured by the Kullbak-Leibler entropy. Therefore, this measure was chosen to solve the clustering problem of medical monitoring data for prostate disease. Medical monitoring data were divided into four classes of patient states: “healthy persons”, “non-metastatic patients”, “metastatic patients” and “hormone-resistant patients”. The accuracy of clustering according to medical data was 95,6%. In addition to accuracy, the confusion matrix, ROC- and LF-curves were used to assess the method quality. The minimum value of the ROC-curve was 0.96 for Fisher's irises and 0.95 for medical monitoring data, which characterizes the high quality of the proposed clustering method. The loss function value is also quite small (-0.056 and -0.0176 for each considered data set), which means that the optimal cluster number and the distribution of data over them are obtained. Based on the obtained results analysis, the proposed method can be recommended for use in medical information and diagnostic decision support systems for clustering monitoring data.

Elucidating the solution structure of the K-means cost function using energy landscape theory.

The K-means algorithm, routinely used in many scientific fields, generates clustering solutions that depend on the initial cluster coordinates. The number of solutions may be large, which can make locating the global minimum challenging. Hence, the topography of the cost function surface is crucial to understanding the performance of the algorithm. Here, we employ the energy landscape approach to elucidate the topography of the K-means cost function surface for Fisher's Iris dataset. For any number of clusters, we find that the solution landscapes have a funneled structure that is usually associated with efficient global optimization. An analysis of the barriers between clustering solutions shows that the funneled structures result from remarkably small barriers between almost all clustering solutions. The funneled structure becomes less well-defined as the number of clusters increases, and we analyze kinetic analogs to quantify the increased difficulty in locating the global minimum for these different landscapes.

Bayesian Inference on Multivariate Medians and Quantiles

In this paper, we consider Bayesian inference on a class of multivariate median and the multivariate quantile functionals of a joint distribution using a Dirichlet process prior. Since, unlike univariate quantiles, the exact posterior distribution of multivariate median and multivariate quantiles are not obtainable explicitly, we study these distributions asymptotically. We derive a Bernstein-von Mises theorem for the multivariate $\ell_1$-median with respect to general $\ell_p$-norm, which in particular shows that its posterior concentrates around its true value at $n^{-1/2}$-rate and its credible sets have asymptotically correct frequentist coverage. In particular, asymptotic normality results for the empirical multivariate median with general $\ell_p$-norm is also derived in the course of the proof which extends the results from the case $p=2$ in the literature to a general $p$. The technique involves approximating the posterior Dirichlet process by a Bayesian bootstrap process and deriving a conditional Donsker theorem. We also obtain analogous results for an affine equivariant version of the multivariate $\ell_1$-median based on an adaptive transformation and re-transformation technique. The results are extended to a joint distribution of multivariate quantiles. The accuracy of the asymptotic result is confirmed by a simulation study. We also use the results to obtain Bayesian credible regions for multivariate medians for Fisher's iris data, which consists of four features measured for each of three plant species.

A Single-Node Classifier Implementation on Chua Oscillator within a Physical Reservoir Computing Framework

The study lies in the field of physical reservoir computing and aims to develop a classifier using Fisher Iris dataset for benchmark tasks. Single Chua chaotic oscillator acts as a physical reservoir. The study was performed using computer simulation. The features of Iris flowers are represented as the consequence of short pulses at a constant level of a control parameter, which is fed to the oscillator, changing its dynamics. During the classification of flowers, the oscillator works without being reset, so each pulse on the input changes the phase trajectory and makes it unique for each Iris flower. Finally, the estimation of the symmetry of an attractor makes it possible to connect each species of Iris with the properties of the attractor. The resulting architecture of the classifier includes a single-node externally-driven Chua oscillator with time-delayed input. The classifier shows two mistakes in classifying the dataset with 75 samples working in chaotic mode.

Harnessing the power of intersection for pattern recognition: a novel unsupervised learning method and its application to financial engineering

AbstractIn the present paper a data‐driven hard cluster analysis derived from a novel similarity measure is proposed to support financial investors in their portfolio management decision‐making process. The main objective of the proposed method is to provide a less arbitrary learning procedure to quantify similarity levels between investment alternatives (pairwise) as well as revealing clustering patterns (whole sample). This is especially useful during periods of high volatility, when investment alternatives tend to become more similar and, therefore, harder to distinguish between themselves. The method dynamics may be readily interpreted through a clear data visualisation. The advantages and caveats of the proposed method is compared to the most popular class of cluster analysis, applied to the well‐known Fisher's Iris dataset. Such results show a slightly superior performance of the proposed method but, most importantly, through remarkably different clustering allocation approaches. Moreover, further empirical results applied to daily data reflecting a period of 15 years of time series of economies/stock markets of the Group of Seven (G7) illustrate the potential practical usefulness of the proposed unsupervised learning method, particularly, for portfolio strategy, asset allocation, and investment diversification.

An Improved Negative Selection Algorithm-Based Fault Detection Method

Fault detection has been an active research field and increasingly important for the safety of technical processes and systems. A variety of fault detection methods have been developed that are generally designed for a specific system; therefore, they target limited types of fault. An efficient approach that could monitor the degradation conditions and be less problem-specific is necessary for improved reliability and efficiency. This paper presents an improved negative selection algorithm using specialized detectors that is model-free and independent of prior knowledge about fault types. An artificial immune system employs a negative selection algorithm and requires only normal (self) patterns for detector generation. In the training phase, uncovered gaps are identified and covered with new detectors to improve nonself (faulty) space coverage. Moreover, to alleviate the online detection, cost reshaping process of nonself space with detector clusters is performed. Theoretical analysis shows that the nonself space coverage with a specialized detector is effectively improved. In experimental analysis, three data sets are used for training and testing, including two-dimensional patterns (cross-mid, ring-mid), Fisher Iris, and KDD CUP99. The experimental results show that the accuracy of the developed method is better than that of the standard negative selection algorithm and other established machine learning algorithms with reduced online anomaly detection time.

Improved N-dimensional data visualization from hyper-radial values

Higher-dimensional data, which is becoming common in many disciplines due to big data problems, are inherently difficult to visualize in a meaningful way. While many visualization methods exist, they are often difficult to interpret, involve multiple plots and overlaid points, or require simultaneous interpretations. This research adapts and extends hyper-radial visualization, a technique used to visualize Pareto fronts in multi-objective optimizations, to become an n-dimensional visualization tool. Hyper-radial visualization is seen to offer many advantages by presenting a low-dimensionality representation of data through easily understood calculations. First, hyper-radial visualization is extended for use with general multivariate data. Second, a method is developed by which to optimally determine groupings of the data for use in hyper-radial visualization to create a meaningful visualization based on class separation and geometric properties. Finally, this optimal visualization is expanded from two to three dimensions in order to support even higher-dimensional data. The utility of this work is illustrated by examples using seven datasets of varying sizes, ranging in dimensionality from Fisher Iris with 150 observations, 4 features, and 3 classes to the Mixed National Institute of Standards and Technology data with 60,000 observations, 717 non-zero features, and 10 classes.

On a scale as a sum of manifest variables

The most common approach for a scale construction is to create a scale as a sum of manifest variables (a “sum scale”). When we use the sum scale for analysis, we implicitly assume that there is a one-dimensional latent structure representing the manifest data on a multidimensional space. In this commentary, we review basics of identifying a latent structure using measured variables with a minimum linear algebra. We demonstrate the technique using Fisher's iris data as an illustration. We examine the relationships between resulting latent variables and the sum scale to evaluate goodness of the sum scale. As a practical solution, in general, we could create a sum scale using a set of positively and highly correlated measured variables. More care is needed when the data are not unidimensional.

OPTIMIZED FUZZY MIN-MAX NEURAL NETWORK: AN EFFICIENT APPROACH FOR SUPERVISED OUTLIER DETECTION

Fuzzy min-max neural network (FMN) proposed by Simpson [18] is a well-known supervised fuzzy-neural classifier that has been successfully used by many researchers for pattern recognition. However, the FMN represents the learned knowledge with exhaustive details in ‘fine-grained’ manner that reduce its performance for pattern recognition in terms of recall time per pattern. In this paper, we adapt the basic architecture of FMN to represent the learned knowledge in a compact way, i.e., in a ‘coarse-grained’ manner which is closed to human thinking. Working of the proposed method, that is fuzzy min-max neural network with knowledge compaction (FMN_KC), is described using Fisher Iris dataset. The potential of using FMN_KC for supervised outlier detection is demonstrated using a time-series disk defect dataset published by NASA and KDD cup99 dataset available in UCI repository. The proposed method has achieved around 50% gain in the recall time compared to the original FMN and the recognition rate is also comparable. We strongly recommend using the proposed architecture called FMN_KC for supervised outlier detection in real time applications where recall time per pattern is one of the key parameters.

To the role of the choice of the neuron model in spiking network learning on base of Spike-Timing-Dependent Plasticity

The goal of this work is to study the influence of the neuron model choice on the results of STDP learning on base of simple toy tasks. As shown, the resulting mean output firing rate after STDP learning with restricted symmetric spike pairing scheme does not depend on the mean input rates for such neuron models as Leaky Integrate-and-Fire, Traub, and static neuron. Then this effect, being used to solve a typical classification task of Fishers Iris, demonstrates that the classification accuracy does not depend significantly on the choice of the neuron model. Thus, the independence of learning results on the neuron model gives the possibility to use simpler neuron models in further investigations.

A Learning Method for Block‐Based Neural Networks with Structure Search Based on the Least Number of Routes

SUMMARYIn recent years, a study of evolvable hardware (EHW) which can adapt to new an unknown environment attracts much attention among hardware designers. EHW is reconfigurable hardware and can be implemented combining reconfigurable devices such as FPGA (Field Programmable Gate Array) and evolutionary computation such as Genetic Algorithms (GAs). As such research of EHW, Block‐Based Neural Networks (BBNNs) have been proposed. BBNNs have simplified network structures and their weights and network structure can be optimized at the same time using GAs. The learning of BBNNs without constraint of network structure is, however, not efficient because the degree of difficulty of learning depends on network structures. In this paper, we proposed a new evaluation index of network structures for BBNNs based on the least number of routes which are from inputs to outputs, and apply it to the structure search. The learning of BBNNs is efficiently executed with structure constraint condition based on the proposed index because the network structures which are difficult to learn are excluded. In order to evaluate the proposed method, we apply it to XOR, 3 bit‐parity, square function approximation, contact lenses fitting, Fisher's iris classification, and Wine classification. Results of computational experiments indicate the validity of the proposed method.

Finding Ultrametricity in Data using Topology

The topological ultrametricity index can be approximated by the expected survival time of a dataset in the state of being ultrametric while only distances up to a given value are considered. It is observed that the quotient of the number of connected components by the number of maximal cliques in the Vietoris-Rips graph initially is the survival function of a Weibull distribution. This is shown for some codings of Fisher's Iris data as well as for random samples in the Euclidean hypercube.

K-NN Based Outlier Detection Technique on Intrusion Dataset

Outliers in the database are the objects that deviate from the rest of the dataset by some measure. The Nearest Neighbor Outlier Factor is considering to measure the degree of outlier-ness of the object in the dataset. Unlike the other methods like Local Outlier Factor, this approach shows the interest of a point from both neighbors and reverse neighbors, and after that, an object comes into consideration. We have observed that in GBBK algorithm that based on K-NN, used quick sort to find k nearest neighbors that take O (N log N) time. However, in proposed method, the time required for searching on K times which complete in O (KN) time to find k nearest neighbors (k < < log N). As a result, the proposed method improves the time complexity. The NSL-KDD and Fisher iris dataset is used, and experimental results compared with the GBBK method. The result is same in both the methods, but the proposed method takes less time for computation.

Clustering and Curve Fitting by Line Segments

Nonlinear nonparametric statistics (NNS) algorithm offers new tools for curve fitting. A surprising relationship between k-means clustering and NNS regression points is explored with graphics showing a perfect fit in the limit. The goal of this paper is to demonstrate NNS as a form of unsupervised learning, and supply a proof of its limit condition. The procedural similarity NNS shares with vector quantization is also documented, along with identical outputs for NNS and a k nearest neighbors (kNN) classification algorithm under a specific NNS setting. Fisher's iris data and artificial data are used. Even though a perfect fit should obviously be reserved for instances of high signal to noise ratios, it permits greater flexibility by offering a large spectrum of possible fits from linear to perfect.

A new test for multivariate normality by combining extreme and nonextreme BHEP tests

ABSTRACTIn this article, we propose a new multiple test procedure for assessing multivariate normality, which combines BHEP (Baringhaus–Henze–Epps–Pulley) tests by considering extreme and nonextreme choices of the tuning parameter in the definition of the BHEP test statistic. Monte Carlo power comparisons indicate that the new test presents a reasonable power against a wide range of alternative distributions, showing itself to be competitive against the most recommended procedures for testing a multivariate hypothesis of normality. We further illustrate the use of the new test for the Fisher Iris dataset.

Data Mining using Modified GFMM Neural Network

The fuzzy neural networks are adaptive, learns quickly and are highly suitable in decision making where uncertainty is involved. In this paper the Modified General Fuzzy Min-Max Neural Network (MGFMMNN) is described which is experimented for the data mining tasks such as classification and clustering. The MGFMMNN utilizes fuzzy sets as pattern classes in which each fuzzy set is a union of fuzzy set hyperboxes. It is an extension of the general fuzzy min-max (GFMM) neural network proposed by Gabrys and Bargiala. The data mining tasks such as classification and clustering have been studied using MGFMMNN and Fisher Iris data set. Further, MGFMMNN is trained using Hepatitis Data Set to verify its classification and recognition ability. The results obtained are awfully persuading and confirms the effectiveness of the proposed system. The technique proposed is quick and reliably deployable in the applications that need classification and clustering. General Terms Data Mining, Fuzzy Systems, Neural Networks, Pattern Recognition.

How many separable sources? Model selection in independent components analysis.

Unlike mixtures consisting solely of non-Gaussian sources, mixtures including two or more Gaussian components cannot be separated using standard independent components analysis methods that are based on higher order statistics and independent observations. The mixed Independent Components Analysis/Principal Components Analysis (mixed ICA/PCA) model described here accommodates one or more Gaussian components in the independent components analysis model and uses principal components analysis to characterize contributions from this inseparable Gaussian subspace. Information theory can then be used to select from among potential model categories with differing numbers of Gaussian components. Based on simulation studies, the assumptions and approximations underlying the Akaike Information Criterion do not hold in this setting, even with a very large number of observations. Cross-validation is a suitable, though computationally intensive alternative for model selection. Application of the algorithm is illustrated using Fisher's iris data set and Howells' craniometric data set. Mixed ICA/PCA is of potential interest in any field of scientific investigation where the authenticity of blindly separated non-Gaussian sources might otherwise be questionable. Failure of the Akaike Information Criterion in model selection also has relevance in traditional independent components analysis where all sources are assumed non-Gaussian.

Use of biplot technique for the comparison of the missing value imputation methods

This study was performed to assess the effects of different imputation methods on the performance of a biplot technique. We selected the Fisher's iris data as our reference dataset. Some elements of the Iris data were deleted in different rates under missing at random MAR assumption to generate incomplete datasets which had 3.5%, 7%, 15%, 20% missing value. Datasets with missing values were completed by four imputation methods [mean imputation, regression imputation, expectation maximisation EM algorithm, multiple imputation MI]. The new imputed datasets were analysed by biplot technique and their results were compared with original complete biplot of the data. The results of biplot analysis were similar in all the imputation methods when missing rate is low under MAR assumption. Even when the missing rate was greater than 10%, results of EM and MI methods were similar to real values and graphical representation of original data. For multivariate methods, we also propose filling in the missing value with the arithmetic mean of the imputed estimates which are obtained with multiple imputation. This paper also indicates that the use of biplot technique for the comparison of the missing value imputation methods provides a useful visual tool.

On the type II error in SIMCA method

A novel method for theoretical calculation of the type II (β) error in soft independent modeling by class analogy is proposed. It can be used to compare tentatively predicted and empirically observed results of classification. Such an approach can better characterize model quality and thus improve its validation. Method efficiency is demonstrated on the famous Fisher Iris dataset and on a real‐world example of quality control of packed raw materials in pharmaceutical industry. Copyright © 2014 John Wiley & Sons, Ltd.