Nonlinear Pattern Classification Research Articles

Simplification of ANFIS based on importance-confidence-similarity measures

Adaptive-network-based fuzzy inference system (ANFIS) is a well-known neuro-fuzzy system, which is widely implemented in nonlinear system approximation, prediction, control, and pattern classification. However, either too many fuzzy sets or fuzzy rules are indispensable to establish an accurate ANFIS for a high-dimensional system, which increases the complexity and limits the widespread application of the ANFIS. In this paper, an approach to simplify the ANFIS is developed based on importance-confidence-similarity (ICS) measures. The simplification process consists of selection of fuzzy rules with large importance-confidence (IC) values and reduction of similar fuzzy rules. The importance is defined as the average normalized firing strength of the rule. The confidence which reflects the approximation performance of a T-S fuzzy rule is estimated by the weighted orthogonal distance from the output data to the rule hyperplane (the consequent of a linear T-S fuzzy rule). The IC measures provide a comprehensive basis for the selection of the appropriate fuzzy rules. Furthermore, to the best of our knowledge, an approach to measure similarities of the T-S fuzzy rules including the rule premises and consequents is first proposed in this paper. We also develop a direct and simple method to remove the similar fuzzy rules based on a hierarchical tree and the IC measures. The proposed simplification approach is applied to Mackey-Glass chaotic time series prediction problem and the strip temperature increment prediction problem of the indirect-fired furnace of the annealing heating line. The results show that our proposed method achieves a balance between simplicity and accuracy of the ANFIS for the high-dimensional system.

Adaptive graph regularized and label embedded dictionary learning for pattern classification

During the last decade, graph regularized dictionary learning (DL) models have obtained a lot of attention due to their flexible and discriminative ability in nonlinear pattern classification. However, the conventional graph-regularized methods construct a fixed affinity matrix for nearby samples in high-dimensional data space, which is vulnerable to noisy and redundant sample features. Furthermore, the discrimination of the graph regularized representation is not fully explored with the supervised classifier learning framework. To remedy these limitations, we propose an adaptive graph-regularized and label-embedded DL model for pattern classification. Especially, the affinity graph construction in low-dimensional representation space and the discriminative sparse representation is simultaneously learned in a unified framework for mutual promotion. More concretely, we iteratively update the sample similarity weight matrix in representation space to enhance the model robustness and further impose a supervised label-embedding term on sparse representation to enhancing its discriminative capability for classification. The dictionary orthonormal constraint is also considered to eliminate the redundant atoms and enhance the model discrimination. An efficient alternating direction solution with guaranteed convergence is proposed for the nonconvex and unsmooth model. Experimental results on five benchmark datasets verify the effectiveness of the proposed model.

Supervised Learning Algorithm for Multilayer Spiking Neural Networks with Long-Term Memory Spike Response Model.

As a new brain-inspired computational model of artificial neural networks, spiking neural networks transmit and process information via precisely timed spike trains. Constructing efficient learning methods is a significant research field in spiking neural networks. In this paper, we present a supervised learning algorithm for multilayer feedforward spiking neural networks; all neurons can fire multiple spikes in all layers. The feedforward network consists of spiking neurons governed by biologically plausible long-term memory spike response model, in which the effect of earlier spikes on the refractoriness is not neglected to incorporate adaptation effects. The gradient descent method is employed to derive synaptic weight updating rule for learning spike trains. The proposed algorithm is tested and verified on spatiotemporal pattern learning problems, including a set of spike train learning tasks and nonlinear pattern classification problems on four UCI datasets. Simulation results indicate that the proposed algorithm can improve learning accuracy in comparison with other supervised learning algorithms.

Effects of synaptic integration on the dynamics and computational performance of spiking neural network

Neurons in the brain receive thousands of synaptic inputs from other neurons. This afferent information is processed by neurons through synaptic integration, which is an important information processing mechanism in biological neural networks. Synaptic currents integrated from spiking trains of presynaptic neurons have complex nonlinear dynamics which endow neurons with significant computational abilities. However, in many computational studies of neural networks, external input currents are often simply taken as a direct current that is static. In this paper, the influences of synaptic and noise external currents on the dynamics of spiking neural network and its computational capability have been investigated in detail. Our results show that due to the nonlinear synaptic integration, both of fast and slow excitatory synaptic currents have much more complex and oscillatory fluctuations than the noise current with the same average intensity. Thus network driven by synaptic external current exhibits remarkably more complex dynamics than that driven by noise external current. Interestingly, the enhancement of network activity is beneficial for information transmission, which is further supported by two computational tasks conducted on the liquid state machine (LSM) network. LSM with synaptic external current displays considerably better performance in both nonlinear fitting and pattern classification than that with noise external current. Synaptic integration can significantly enhance the entropy of activity patterns and computational performance of LSM. Our results demonstrate that the complex dynamics of nonlinear synaptic integration play a critical role in the computational abilities of neural networks and should be more broadly considered in the modelling studies of spiking neural networks.

Colorectal Cancer Diagnostic Algorithm Based on Sub-Patch Weight Color Histogram in Combination of Improved Least Squares Support Vector Machine for Pathological Image

In order to improve the diagnostic accuracy of colon cancer, a novel classification algorithm based on sub-patch weight color histogram and improved SVM is proposed, which has good approximation ability for complex pathological image. Our proposed algorithm combines wavelet kernel SVM with color histogram to classify pathological image. Firstly, the pathological image is divided into non-overlapping sub-patches, and the features of sub-patch histogram are extracted. Then, the global and local features are fused by the sub-patch weighting algorithm. Then, the RelicfF based forward selection algorithm is used to integrate color features and texture features so as to enhance the characterization capabilities of the tumor cell. Finally, Morlet wavelet kernel-based least squares support vector machine method is adopted to enhance the generalization ability of the model for small sample with non-linear and high-dimensional pattern classification problems. Experimental results show that the proposed pathological diagnostic algorithm can gain higher accuracy compared with existing comparison algorithms.

A Novel Fractional Gradient-Based Learning Algorithm for Recurrent Neural Networks

In this research, we propose a novel algorithm for learning of the recurrent neural networks called as the fractional back-propagation through time (FBPTT). Considering the potential of the fractional calculus, we propose to use the fractional calculus-based gradient descent method to derive the FBPTT algorithm. The proposed FBPTT method is shown to outperform the conventional back-propagation through time algorithm on three major problems of estimation namely nonlinear system identification, pattern classification and Mackey–Glass chaotic time series prediction.

A Unified Theory of Neuro-MRI Data Shows Scale-Free Nature of Connectivity Modes.

A primary goal of many neuroimaging studies that use magnetic resonance imaging (MRI) is to deduce the structure-function relationships in the human brain using data from the three major neuro-MRI modalities: high-resolution anatomical, diffusion tensor imaging, and functional MRI. To date, the general procedure for analyzing these data is to combine the results derived independently from each of these modalities. In this article, we develop a new theoretical and computational approach for combining these different MRI modalities into a powerful and versatile framework that combines our recently developed methods for morphological shape analysis and segmentation, simultaneous local diffusion estimation and global tractography, and nonlinear and nongaussian spatial-temporal activation pattern classification and ranking, as well as our fast and accurate approach for nonlinear registration between modalities. This joint analysis method is capable of extracting new levels of information that is not achievable from any of those single modalities alone. A theoretical probabilistic framework based on a reformulation of prior information and available interdependencies between modalities through a joint coupling matrix and an efficient computational implementation allows construction of quantitative functional, structural, and effective brain connectivity modes and parcellation. This new method provides an overall increase of resolution, accuracy, level of detail, and information content and has the potential to be instrumental in the clinical adaptation of neuro-MRI modalities, which, when jointly analyzed, provide a more comprehensive view of a subject's structure-function relations, while the current standard, wherein single-modality methods are analyzed separately, leaves a critical gap in an integrated view of a subject's neuorphysiological state. As one example of this increased sensitivity, we demonstrate that the jointly estimated structural and functional dependencies of mode power follow the same power law decay with the same exponent.

Supervised learning in multilayer spiking neural networks with inner products of spike trains

Recent advances in neurosciences have revealed that neural information in the brain is encoded through precisely timed spike trains, not only through the neural firing rate. This paper presents a new supervised, multi-spike learning algorithm for multilayer spiking neural networks, which can implement the complex spatio-temporal pattern learning of spike trains. The proposed algorithm firstly defines inner product operators to mathematically describe and manipulate spike trains, and then solves the problems of error function construction and backpropagation among multiple output spikes during learning. The algorithm is successfully applied to different temporal tasks, such as learning sequences of spikes and nonlinear pattern classification problems. The experimental results show that the proposed algorithm has higher learning accuracy and efficiency than the Multi-ReSuMe learning algorithm. It is effective for solving complex spatio-temporal pattern learning problems.

A Novel Adaptive Kernel for the RBF Neural Networks

In this paper, we propose a novel adaptive kernel for the radial basis function (RBF) neural networks. The proposed kernel adaptively fuses the Euclidean and cosine distance measures to exploit the reciprocating properties of the two. The proposed framework dynamically adapts the weights of the participating kernels using the gradient descent method thereby alleviating the need for predetermined weights. The proposed method is shown to outperform the manual fusion of the kernels on three major problems of estimation namely nonlinear system identification, pattern classification and function approximation.

AUTONOMOUS ROBOTIC INSPECTION IN TUNNELS

In this paper, an automatic robotic inspector for tunnel assessment is presented. The proposed platform is able to autonomously navigate within the civil infrastructures, grab stereo images and process/analyse them, in order to identify defect types. At first, there is the crack detection via deep learning approaches. Then, a detailed 3D model of the cracked area is created, utilizing photogrammetric methods. Finally, a laser profiling of the tunnel’s lining, for a narrow region close to detected crack is performed; allowing for the deduction of potential deformations. The robotic platform consists of an autonomous mobile vehicle; a crane arm, guided by the computer vision-based crack detector, carrying ultrasound sensors, the stereo cameras and the laser scanner. Visual inspection is based on convolutional neural networks, which support the creation of high-level discriminative features for complex non-linear pattern classification. Then, real-time 3D information is accurately calculated and the crack position and orientation is passed to the robotic platform. The entire system has been evaluated in railway and road tunnels, i.e. in Egnatia Highway and London underground infrastructure.

AUTONOMOUS ROBOTIC INSPECTION IN TUNNELS

Abstract. In this paper, an automatic robotic inspector for tunnel assessment is presented. The proposed platform is able to autonomously navigate within the civil infrastructures, grab stereo images and process/analyse them, in order to identify defect types. At first, there is the crack detection via deep learning approaches. Then, a detailed 3D model of the cracked area is created, utilizing photogrammetric methods. Finally, a laser profiling of the tunnel’s lining, for a narrow region close to detected crack is performed; allowing for the deduction of potential deformations. The robotic platform consists of an autonomous mobile vehicle; a crane arm, guided by the computer vision-based crack detector, carrying ultrasound sensors, the stereo cameras and the laser scanner. Visual inspection is based on convolutional neural networks, which support the creation of high-level discriminative features for complex non-linear pattern classification. Then, real-time 3D information is accurately calculated and the crack position and orientation is passed to the robotic platform. The entire system has been evaluated in railway and road tunnels, i.e. in Egnatia Highway and London underground infrastructure.

Designing Artificial Neural Networks Using Particle Swarm Optimization Algorithms.

Artificial Neural Network (ANN) design is a complex task because its performance depends on the architecture, the selected transfer function, and the learning algorithm used to train the set of synaptic weights. In this paper we present a methodology that automatically designs an ANN using particle swarm optimization algorithms such as Basic Particle Swarm Optimization (PSO), Second Generation of Particle Swarm Optimization (SGPSO), and a New Model of PSO called NMPSO. The aim of these algorithms is to evolve, at the same time, the three principal components of an ANN: the set of synaptic weights, the connections or architecture, and the transfer functions for each neuron. Eight different fitness functions were proposed to evaluate the fitness of each solution and find the best design. These functions are based on the mean square error (MSE) and the classification error (CER) and implement a strategy to avoid overtraining and to reduce the number of connections in the ANN. In addition, the ANN designed with the proposed methodology is compared with those designed manually using the well-known Back-Propagation and Levenberg-Marquardt Learning Algorithms. Finally, the accuracy of the method is tested with different nonlinear pattern classification problems.

Non-linear molecular pattern classification using molecular beacons with multiple targets

In vitro pattern classification has been highlighted as an important future application of DNA computing. Previous work has demonstrated the feasibility of linear classifiers using DNA-based molecular computing. However, complex tasks require non-linear classification capability. Here we design a molecular beacon that can interact with multiple targets and experimentally shows that its fluorescent signals form a complex radial-basis function, enabling it to be used as a building block for non-linear molecular classification in vitro. The proposed method was successfully applied to solving artificial and real-world classification problems: XOR and microRNA expression patterns.

System-on-programmable-chip implementation of diminishing learning based pattern recognition system

Support vector machine (SVM) is one of the state-of-the-art tools for linear and nonlinear pattern classification. One of the design issues in SVM classifier is reducing the number of support vectors without compromising the classification accuracy. A technique denoted as diminishing learning (DL) is already proposed in literature for an SVM based multi-class isolated digit recognition system using speaker dependent TI46 database of isolated digits. In this paper, the computational complexity for SVM and SVM-DL based isolated digit recognition system is studied and the computation time for both the classifiers is evaluated by system-on-programmable-chip (SOPC) implementation of the recognition system onto an Altera Cyclone II Series FPGA using Nios II Soft-core processor. The number of support vectors is reduced by 38.28–90.25 % on using SVM-DL for isolated digit recognition problem. This in turn reduces the classification time for SVM-DL by 31.45–91.78 % over SVM. Recognition accuracies of 97 and 98 % are achieved for SVM classifier with and without DL technique, respectively. The study confirms the effect of, the order in which the classes are classified, on the recognition accuracy. For the TI46 database, about 2–4 % increase in recognition accuracy is obtained by choosing the optimum order for SVM-DL classifier. The proposed SOPC implementation of SVM-DL based recognition system can be employed for various other pattern recognition applications too such as face recognition, character recognition and target recognition.

Fault Diagnosis of Rolling Bearing Based on Rough Set and Neural Network

Artificial neural network was one of the most important methods in intelligent fault diagnosis because it has the performance of nonlinear pattern classification and the capacity of self-learning and self-organization, but it can not judge redundancy and usefulness of information. Rough set can reduce the knowledge of information system and dislodge redundant information. In this paper, fault data of rolling bearing was reduced by the greedy algorithm of rough set. Training data and test data of BP neural network had been reduced by rough set. By comparison of two test result about simply data and original data, it was indicated that resolving power was unchanged and database was simply.

Mine Fan Fault Diagnosis Based on EMD and SVM

In this research, a new method based on EMD and SVM for mine fan fault diagnosis is introduced. With EMD, fault feature can be extracted quickly and accurately, and taken as the input samples for SVM with the outstanding non-linear pattern classification performances. 5 two-class SVM classifiers are designed in order to classify and diagnosis 5 typical fault patterns of mine fan. The result of this research shows that the integrative method of EMD and SVM is very fit for the intelligent diagnosis and fault patterns recognition, and it will lead to the possible development of an automated and on-line mine fan condition monitoring and diagnostic system.

Study and evaluation of a multi-class SVM classifier using diminishing learning technique

Support vector machine (SVM) is one of the state-of-the-art tools for linear and non-linear pattern classification. One of the design objectives of an SVM classifier is reducing the number of support vectors without compromising the classification accuracy. For this purpose, a novel technique referred to as diminishing learning (DL) technique is proposed in this paper for a multiclass SVM classifier. In this technique, a sequential classifier is proposed wherein the classes which require stringent boundaries are tested one by one and once the tests for these classes fail, the stringency of the classifier is increasingly relaxed. An automated procedure is also proposed to obtain the optimum classification order for SVM–DL classifier in order to improve the recognition accuracy. The proposed technique is applied for SVM based isolated digit recognition system and is studied using speaker dependent and multispeaker dependent TI46 database of isolated digits. Both LPC and MFCC are used for feature extraction. The features extracted are mapped using self-organized feature maps (SOFM) for dimensionality reduction and the mapped features are used by SVM classifier to evaluate the recognition accuracy using various kernels. The performance of the system using the proposed SVM–DL classifier is compared with those using other techniques: one-against-all (OAA), half-against-half (HAH) and directed acyclic graph (DAG). SVM–DL classifier results in 1–2% increase in recognition accuracy compared to HAH classifier for some of the kernels with both LPC and MFCC feature inputs. For MFCC feature inputs, both HAH and SVM–DL classifiers have 100% recognition accuracy for some of the kernels. The total number of support vectors required is the least for HAH classifier followed by the SVM–DL classifier. The proposed diminishing learning technique is applicable for a number of pattern recognition applications.

Multiparametric Tissue Characterization of Brain Neoplasms and Their Recurrence Using Pattern Classification of MR Images

Treatment of brain neoplasms can greatly benefit from better delineation of bulk neoplasm boundary and the extent and degree of more subtle neoplastic infiltration. Magnetic resonance imaging (MRI) is the primary imaging modality for evaluation before and after therapy, typically combining conventional sequences with more advanced techniques such as perfusion-weighted imaging and diffusion tensor imaging (DTI). The purpose of this study is to quantify the multiparametric imaging profile of neoplasms by integrating structural MRI and DTI via statistical image analysis methods to potentially capture complex and subtle tissue characteristics that are not obvious from any individual image or parameter. Five structural MRI sequences, namely, B0, diffusion-weighted images, fluid-attenuated inversion recovery, T1-weighted, and gadolinium-enhanced T1-weighted, and two scalar maps computed from DTI (ie, fractional anisotropy and apparent diffusion coefficient) are used to create an intensity-based tissue profile. This is incorporated into a nonlinear pattern classification technique to create a multiparametric probabilistic tissue characterization, which is applied to data from 14 patients with newly diagnosed primary high-grade neoplasms who have not received any therapy before imaging. Preliminary results demonstrate that this multiparametric tissue characterization helps to better differentiate among neoplasm, edema, and healthy tissue, and to identify tissue that is likely to progress to neoplasm in the future. This has been validated on expert assessed tissue. This approach has potential applications in treatment, aiding computer-assisted surgery by determining the spatial distributions of healthy and neoplastic tissue, as well as in identifying tissue that is relatively more prone to tumor recurrence.

Nonlinear transient computation as a potential “kernel trick” in cortical processing

Evidence has been found for the presence of chaotic dynamics at all levels of the mammalian brain. This has led to some searching questions about the potential role that nonlinear dynamics may have in neural information processing. We propose that chaos equips the brain with the equivalent of a kernel trick for solving hard nonlinear problems. The approach presented, which is described as nonlinear transient computation, uses the dynamics of a well known chaotic attractor. The paper provides experimental results to show that this approach can be used to solve some challenging pattern recognition tasks. The paper also offers evidence to suggest that the efficacy of nonlinear transient computation for nonlinear pattern classification is dependent only on the generic properties of chaotic attractors and is not sensitive to the particular dynamics of specific sub-regions of chaotic phase space. If, as this work suggests, nonlinear transient computation is independent of the particulars of any given chaotic attractor, then it could be offered as a possible explanation of how the chaotic dynamics that have been observed in brain structures contribute to neural information processing tasks.

Whole-Brain Morphometric Study of Schizophrenia Revealing a Spatially Complex Set of Focal Abnormalities

Neuroanatomic abnormalities in schizophrenia may underlie behavioral manifestations. Characterization of such abnormalities is required for interpreting functional data. Frontotemporal abnormalities have been documented by using predetermined region-of-interest approaches, but deformation-based morphometry permits examination of the entire brain. To perform whole-brain analyses of structural differences between patients with schizophrenia and controls, to examine sex and medication effects, and to apply a high-dimensional nonlinear pattern classification technique to quantify the degree of separation between patients and controls, thereby testing the potential of this new technique as an aid to diagnostic procedures. Whole-brain morphologic analysis using high-dimensional shape transformations. Schizophrenia Research Center, University of Pennsylvania Medical Center. Neuroleptic-naïve and previously treated patients with DSM-IV schizophrenia (n = 69) and sociodemographically matched controls (n = 79). Gray matter, white matter, and ventricular cerebrospinal fluid volumes in the brain. Magnetic resonance images showed reduced gray matter and increased ventricular cerebrospinal fluid volumes in patients with schizophrenia in the whole brain and in specific foci: the hippocampus and adjacent white matter, the cingulate and orbitofrontal cortex, the frontotemporal and parietotemporal areas, and the occipital areas near the lingual gyrus. The classifier had a mean classification accuracy of 81.1% for men and women combined (82% for women and 85% for men, when each group was treated separately), determined via cross-validation. This study confirms previous findings of reduced frontotemporal volumes and suggests new hypotheses, especially involving occipital association and speech production areas. It also suggests finer localization of volume reduction in the hippocampus and other limbic structures and in the frontal lobe. Pattern classification showed high sensitivity and specificity for the diagnosis of schizophrenia, suggesting the potential utility of magnetic resonance imaging as a diagnostic aid.