Nonlinear Classification Research Articles

Tactile Perception for Teleoperated Robotic Exploration within Granular Media

The sense of touch is essential for locating buried objects when vision-based approaches are limited. We present an approach for tactile perception when sensorized robot fingertips are used to directly interact with granular media particles in teleoperated systems. We evaluate the effects of linear and nonlinear classifier model architectures and three tactile sensor modalities (vibration, internal fluid pressure, fingerpad deformation) on the accuracy of estimates of fingertip contact state. We propose an architecture called the Sparse-Fusion Recurrent Neural Network (SF-RNN) in which sparse features are autonomously extracted prior to fusing multimodal tactile data in a fully connected RNN input layer. The multimodal SF-RNN model achieved 98.7% test accuracy and was robust to modest variations in granular media type and particle size, fingertip orientation, fingertip speed, and object location. Fingerpad deformation was the most informative modality for haptic exploration within granular media while vibration and internal fluid pressure provided additional information with appropriate signal processing. We introduce a real-time visualization of tactile percepts for remote exploration by constructing a belief map that combines probabilistic contact state estimates and fingertip location. The belief map visualizes the probability of an object being buried in the search region and could be used for planning.

Risk Identification of Heavy Rain-induced Muck soil Landslide

Based on Monte Carlo method, this paper calculates landslide probability of muck soil slopes under different rainfall time, rainfall intensity, soil permeability coefficient and slope angle, thus obtaining the probability samples of muck soil landslides. On this basis, logistic regression method of nonlinear classification is used for data fitting and analysis, thus establishing nonlinear function . Function expression is derived by data fitting, and a landslide probability evaluation model is constructed. Based on analysis of engineering examples, the error between this method and the numerical calculation results is within 10%, and the evaluation results are reasonable. It provides theoretical support for rapid identification of muck soil landslide risk under heavy rain conditions.

Triply stochastic gradient method for large-scale nonlinear similar unlabeled classification

Similar unlabeled (SU) classification is pervasive in many real-world applications, where only similar data pairs (two data points have the same label) and unlabeled data points are available to train a classifier. Recent work has identified a practical SU formulation and has derived the corresponding estimation error bound. It evaluated SU learning with linear classifiers on medium-sized datasets. However, in practice, we often need to learn nonlinear classifiers on large-scale datasets for superior predictive performance. How this could be done in an efficient manner is still an open problem for SU classification. In this paper, we propose a scalable kernel learning algorithm for SU classification using a triply stochastic optimization framework, called TSGSU. Specifically, in each iteration, our method randomly samples an instance from the similar pairs set, an instance from the unlabeled set, and their random features to calculate the stochastic functional gradient for the model update. Theoretically, we prove that our method can converge to a stationary point at the rate of \(O(1/\sqrt{T})\) after T iterations. Experiments on various benchmark datasets and high-dimensional datasets not only demonstrate the scalability of TSGSU but also show the efficiency of TSGSU compared with existing SU learning algorithms while retaining similar generalization performance.

Capped [formula omitted]-norm metric based robust least squares twin support vector machine for pattern classification

Least squares twin support vector machine (LSTSVM) is an effective and efficient learning algorithm for pattern classification. However, the distance in LSTSVM is measured by squared L2-norm metric that may magnify the influence of outliers. In this paper, a novel robust least squares twin support vector machine framework is proposed for binary classification, termed as CL2,p-LSTSVM, which utilizes capped L2,p-norm distance metric to reduce the influence of noise and outliers. The goal of CL2,p-LSTSVM is to minimize the capped L2,p-norm intra-class distance dispersion, and eliminate the influence of outliers during training process, where the value of the metric is controlled by the capped parameter, which can ensure better robustness. The proposed metric includes and extends the traditional metrics by setting appropriate values of p and capped parameter. This strategy not only retains the advantages of LSTSVM, but also improves the robustness in solving a binary classification problem with outliers. However, the nonconvexity of metric makes it difficult to optimize. We design an effective iterative algorithm to solve the CL2,p-LSTSVM. In each iteration, two systems of linear equations are solved. Simultaneously, we present some insightful analyses on the computational complexity and convergence of algorithm. Moreover, we extend the CL2,p-LSTSVM to nonlinear classifier and semi-supervised classification. Experiments are conducted on artificial datasets, UCI benchmark datasets, and image datasets to evaluate our method. Under different noise settings and different evaluation criteria, the experiment results show that the CL2,p-LSTSVM has better robustness than state-of-the-art approaches in most cases, which demonstrates the feasibility and effectiveness of the proposed method.

MCI Detection Using Kernel Eigen-Relative-Power Features of EEG Signals

Classification between individuals with mild cognitive impairment (MCI) and healthy controls (HC) based on electroencephalography (EEG) has been considered a challenging task to be addressed for the purpose of its early detection. In this study, we proposed a novel EEG feature, the kernel eigen-relative-power (KERP) feature, for achieving high classification accuracy of MCI versus HC. First, we introduced the relative powers (RPs) between pairs of electrodes across 21 different subbands of 2-Hz width as the features, which have not yet been used in previous MCI-HC classification studies. Next, the Fisher’s class separability criterion was applied to determine the best electrode pairs (five electrodes) as well as the frequency subbands for extracting the most sensitive RP features. The kernel principal component analysis (kernel PCA) algorithm was further performed to extract a few more discriminating nonlinear principal components from the optimal RPs, and these components form a KERP feature vector. Results carried out on 51 participants (24 MCI and 27 HC) show that the newly introduced subband RP feature showed superior classification performance to commonly used spectral power features, including the band power, single-electrode relative power, and also the RP based on the conventional frequency bands. A high leave-one-participant-out cross-validation (LOPO-CV) classification accuracy 86.27% was achieved by the RP feature, using a simple linear discriminant analysis (LDA) classifier. Moreover, with the same classifier, the proposed KERP further improved the accuracy to 88.24%. Finally, cascading the KERP feature to a nonlinear classifier, the support vector machine (SVM), yields a high MCI-HC classification accuracy of 90.20% (sensitivity = 87.50% and specificity = 92.59%). The proposed method demonstrated a high accuracy and a high usability (only five electrodes are required), and therefore, has great potential to further develop an EEG-based computer-aided diagnosis system that can be applied for the early detection of MCI.

Infrared small target detection based on divergence operator and nonlinear classifier

Infrared small target detection based on divergence operator and nonlinear classifier

Redundancy between spectral and higher-order texture statistics for natural image segmentation

Visual texture, defined by local image statistics, provides important information to the human visual system for perceptual segmentation. Second-order or spectral statistics (equivalent to the Fourier power spectrum) are a well-studied segmentation cue. However, the role of higher-order statistics (HOS) in segmentation remains unclear, particularly for natural images. Recent experiments indicate that, in peripheral vision, the HOS of the widely adopted Portilla-Simoncelli texture model are a weak segmentation cue compared to spectral statistics, despite the fact that both are necessary to explain other perceptual phenomena and to support high-quality texture synthesis. Here we test whether this discrepancy reflects a property of natural image statistics. First, we observe that differences in spectral statistics across segments of natural images are redundant with differences in HOS. Second, using linear and nonlinear classifiers, we show that each set of statistics individually affords high performance in natural scenes and texture segmentation tasks, but combining spectral statistics and HOS produces relatively small improvements. Third, we find that HOS improve segmentation for a subset of images, although these images are difficult to identify. We also find that different subsets of HOS improve segmentation to a different extent, in agreement with previous physiological and perceptual work. These results show that the HOS add modestly to spectral statistics for natural image segmentation. We speculate that tuning to natural image statistics under resource constraints could explain the weak contribution of HOS to perceptual segmentation in human peripheral vision.

EEG-Based Demarcation of Yogic and Non-Yogic Sleep Patterns Using Power Spectral Analysis

Electroencephalogram (EEG) signals resulting from recordings of polysomnography play a significant role in determining the changes in physiology and behavior during sleep. This study aims at demarcating the sleep patterns of yogic and non-yogic subjects. Frequency domain features based on power spectral density methods were explored in this study. The EEG recordings were segmented into 1s and 0.5s. EEG patterns with four windowing scheme overlaps (0%, 50%, 60% and 75%) to ensure stationarity of the signal in order to investigate the effect of the pre-processing stage. In order to recognize the yoga and non-yoga group through N3 sleep stage, non-linear KNN classifier was introduced and performance was evaluated in terms of sensitivity and specificity. The experimental results show that modified covariance PSD estimate is the best method in classifying the sleep stage N3 of yogic and non-yogic subjects with 95% confidence interval, sensitivity, specificity and accuracy of 97.3%, 98% and 97%, respectively.

A Nanoradiomics Approach for Differentiation of Tumors Based on Tumor-Associated Macrophage Burden.

Objective Tumor-associated macrophages (TAMs) within the tumor immune microenvironment (TiME) of solid tumors play an important role in treatment resistance and disease recurrence. The purpose of this study was to investigate if nanoradiomics (radiomic analysis of nanoparticle contrast-enhanced images) can differentiate tumors based on TAM burden. Materials and Methods In vivo studies were performed in transgenic mouse models of neuroblastoma with low (N = 11) and high (N = 10) tumor-associated macrophage (TAM) burden. Animals underwent delayed nanoparticle contrast-enhanced CT (n-CECT) imaging at 4 days after intravenous administration of liposomal-iodine agent (1.1 g/kg). CT imaging-derived conventional tumor metrics (tumor volume and CT attenuation) were computed for segmented tumor CT datasets. Nanoradiomic analysis was performed using a PyRadiomics workflow implemented in the quantitative image feature pipeline (QIFP) server containing 900 radiomic features (RFs). RF selection was performed under supervised machine learning using a nonparametric neighborhood component method. A 5-fold validation was performed using a set of linear and nonlinear classifiers for group separation. Statistical analysis was performed using the Kruskal–Wallis test. Results N-CECT imaging demonstrated heterogeneous patterns of signal enhancement in low and high TAM tumors. CT imaging-derived conventional tumor metrics showed no significant differences (p > 0.05) in tumor volume between low and high TAM tumors. Tumor CT attenuation was not significantly different (p > 0.05) between low and high TAM tumors. Machine learning-augmented nanoradiomic analysis revealed two RFs that differentiated (p < 0.002) low TAM and high TAM tumors. The RFs were used to build a linear classifier that demonstrated very high accuracy and further confirmed by 5-fold cross-validation. Conclusions Imaging-derived conventional tumor metrics were unable to differentiate tumors with varying TAM burden; however, nanoradiomic analysis revealed texture differences and enabled differentiation of low and high TAM tumors.

Temporally delayed linear modelling (TDLM) measures replay in both animals and humans.

There are rich structures in off-task neural activity which are hypothesized to reflect fundamental computations across a broad spectrum of cognitive functions. Here, we develop an analysis toolkit - temporal delayed linear modelling (TDLM) - for analysing such activity. TDLM is a domain-general method for finding neural sequences that respect a pre-specified transition graph. It combines nonlinear classification and linear temporal modelling to test for statistical regularities in sequences of task-related reactivations. TDLM is developed on the non-invasive neuroimaging data and is designed to take care of confounds and maximize sequence detection ability. Notably, as a linear framework, TDLM can be easily extended, without loss of generality, to capture rodent replay in electrophysiology, including in continuous spaces, as well as addressing second-order inference questions, for example, its temporal and spatial varying pattern. We hope TDLM will advance a deeper understanding of neural computation and promote a richer convergence between animal and human neuroscience.

Nonlinear Signal Classification based on Wavelet Transform and Deep Belief Network

In recent years, the analysis of EMI (Electromagnetic interference) signals has become a hot research topic in the signal processing field. Particularly, EMI signal classification has attracted more and more attention. Conventional signal classification methods usually select features by experience or implicitly by the shallow artificial neural network, and always results in bad performance on high-dimensional and nonlinear EMI signals. This paper proposed a novel classification method based on wavelet transform and deep belief network, while wavelet transform method is used to reduce the dimension of high-dimensional signals and deep belief network can extract nonlinear features from EMI signals. Then we apply BP neural network as the classifier. Results on the benchmark dataset have shown the superiority of the proposed method compared with the state-of-the-art approaches.

Prediction of Cardiovascular Diseases based on Machine Learning

With the rapid development of artificial intelligence, it is very important to find the pattern of the data from the observed data and the functional dependency relationship between the data. By finding the existing functional dependencies, we can classify and predict them. At present, cardiovascular disease has become a major disease harmful to human health. As a disease with high mortality, the prediction problem of cardiovascular disease is becoming more and more urgent. However, some computer methods are mainly used for disease detection rather than prediction. If the computer method can be used to predict cardiovascular disease in advance and treat it as early as possible, then the consequences of the disease can be reduced to a certain extent. Diseases can be predicted by mechanical methods. Support vector machine (SVM) has strict mathematical theory support, and can deal with nonlinear classification after using kernel techniques. Therefore, support vector machine can be used to predict cardiovascular disease. On the other hand, we also use logical regression and random forest to predict cardiovascular disease. This paper mainly uses the method of machine learning to predict whether the population is sick or not. First of all, we preprocess the obtained data to improve the quality of the data, and then use svm and logical regression to predict, so as to provide reference for the prevention and treatment of cardiovascular diseases.

Recent Approaches on Classification and Feature Extraction of EEG Signal: A Review

AbstractObjective:Electroencephalography (EEG) has an influential role in neuroscience and commercial applications. Most of the tools available for EEG signal analysis use machine learning to extract the required information. So, the study of robust techniques for feature extraction and classification is an important thing to understand the practical use of EEG. The paper aims that if there is any special tool for a particular task. Which feature domain or classifier has a significant role in EEG signal analysis?Approach:It presents a detailed report of the current trend for bio-electrical signals classification focusing on various classifiers’ advantages and disadvantages. This study includes literature from 2000 to 2021 with a brief description of EEG signal origin and advancement in classification techniques.Results:Randomly used classifiers for EEG signal can be categorized into five classes, namely Linear Classifiers, Nearest Neighbor Classifiers, Nonlinear Bayesian Classifiers, Neural Networks, and Combinations of Classifiers. Approximately 40% of studies use Support Vector Machine, Nearest Neighbor, and their combination with others. For specific tasks, particular classifiers are recommended in the survey. Features can be defined into four categories, namely TDFs, FDFs, TFDFs, and statistical features, where 39% of studies used TFDFs. Multi-domains features are preferred when the required information cannot be obtained from one domain.Significance:The paper summarizes the recent approaches for feature extraction and classification of EEG signals. It describes the brain waves with their classification, related behavior, and task with the physiological correlation. The comparative analysis of different classifiers, toolbox, the channel used, accuracy, and the number of subjects from various studies can help the practitioners choose a suitable classifier. Furthermore, future directions can cope up with the relevant problems and can lead to accurate classification.

Mixture of robust Gaussian processes and its hard-cut EM algorithm with variational bounding approximation

The Gaussian process is a powerful statistical learning model and has been applied widely in nonlinear regression and classification. However, it fails to model multi-modal data from a non-stationary source since a prior Gaussian process is generally stationary. Based on the idea of the mixture of experts, the mixture of Gaussian processes was established to increase the model flexibility. On the other hand, the Gaussian process is also sensitive to outliers and thus robust Gaussian processes have been suggested to own the heavy-tailed property. In practical applications, the datasets may be multi-modal and contain outliers at the same time. In order to overcome these two difficulties together, we propose a mixture of robust Gaussian processes (MRGP) model and establish a precise hard-cut EM algorithm for learning its parameters. Since the exact solving process is intractable due to the fact that non-Gaussian probability density functions of the noises are adopted into the likelihood of the proposed model on the dataset, we employ a variational bounding method to approximate the marginal likelihood functions so that the hard-cut EM algorithm can be implemented effectively. Moreover, we conduct various experiments on both synthetic and real-world datasets to evaluate and compare our proposed MRGP method with several competitive nonlinear regression methods. The experimental results demonstrate that our MRGP model with the hard-cut EM algorithm is much more effective and robust than the competitive nonlinear regression models.

Deep proximal support vector machine classifiers for hyperspectral images classification

In this work, an effective classification of hyperspectral images is modelled and simulated with the proximal support vector machine (PSVM) by integrating them with the deep learning approach. The modelled new deep proximal support vector machines are designed in a manner to handle the existing complexity, discrepancies and irregularities in the traditional hyperspectral image classifiers. This paper investigates the applicability of the new deep linear and nonlinear proximal support vector machines as applied for hyperspectral image classification. In respect of the new deep PSVM classifier, it is modelled for deep linear PSVM and deep nonlinear PSVM to perform classification of spectral images so as to bring out the best classifier model. To test and validate the proposed deep PSVM classifiers University of Pavia datasets, Indian Pine datasets and Kennedy Space Centre datasets are employed as test beds and results are attained. The developed new deep PSVM classifiers are developed with varied kernel functions to do the classification process. The deep learning technique enhances the linear and nonlinear PSVM classifier models to perform more effectively during the learning process and carry out the classification using auto-encoders and decoders. Results attained during the process infer that the developed new deep PSVM (linear and nonlinear) has come out with better classification accuracy in comparison with that of the other techniques from literature for the same datasets. Statistical analysis validates the randomness that occurs in the proposed deep learning techniques as applied for spectral image classification.

Disambiguation of author entities in ADS using supervised learning and graph theory methods

Disambiguation of authors in digital libraries is essential for many tasks, including efficient bibliographical searches and scientometric analyses to the level of individuals. The question of how to link documents written by the same person has been given much attention by academic publishers and information retrieval researchers alike. Usual approaches rely on publications’ metadata such as affiliations, email addresses, co-authors, or scholarly topics. Lack of homogeneity in the structure of bibliographic collections and discipline-specific dissimilarities between them make the creation of general-purpose disambiguators arduous. We present an algorithm to disambiguate authorships in the Astrophysics Data System (ADS) following an established semi-supervised approach of training a classifier on authorship pairs and clustering the resulting graphs. Due to the lack of high-signal features such as email addresses and citations, we engineer additional content- and location-based features via text embeddings and named-entity recognition. We train various nonlinear tree-based classifiers and detect communities from the resulting weighted graphs through label propagation, a fast yet efficient algorithm that requires no tuning. The resulting procedure reaches reasonable complexity and offers possibilities for interpretation. We apply our method to the creation of author entities in a recent ADS snapshot. The algorithm is evaluated on 39 manually-labeled author blocks comprising 9545 authorships from 562 author profiles. Our best approach utilizes the Random Forest classifier and yields a micro- and macro-averaged BCubed mathrm {F}_1 score of 0.95 and 0.87, respectively. We release our code and labeled data publicly to foster the development of further disambiguation procedures for ADS.

Potential of VIS/NIR spectroscopy to detect and predict bitter pit in ‘Golden Smoothee’ apples

Aim of study: A portable VIS/NIR spectrometer and chemometric techniques were combined to identify bitter pit (BP) in Golden apples.Area of study: WorldwideMaterial and methods: Three different classification algorithms – linear discriminant analysis (LDA), quadratic discriminant analysis (QDA) and support-vector machine (SVM) –were used in two experiments. In experiment #1, VIS/NIR measurements were carried out at postharvest on apples previously classified according to 3 classes (class 1: non-BP; class 2: slight symptoms; class 3: severe symptoms). In experiment #2, VIS/NIR measurements were carried out on healthy apples collected before harvest to determinate the capacity of the classification algorithms for detecting BP prior to the appearance of symptoms.Main results: In the experiement #1, VIS/NIR spectroscopy showed great potential in pitted apples detection with visibly symptoms (accuracies of 75–81%). The linear classifier LDA performed better than the multivariate non-linear QDA and SVM classifiers in discriminating between healthy and bitter pitted apples. In the experiment #2, the accuracy to predict bitter pit prior to the appearance of visible symptoms decreased to 44–57%.Research highlights: The identification of apples with bitter pit through VIS/NIR spectroscopy may be due to chlorophyll degradation and/or changes in intercellular water in fruit tissue.

Stochastic gradient support vector machine with local structural information for pattern recognition

Structural information is very important for improving the classification performance of classifiers. In order to increase the generalization performance of support vector machine (SVM) directly, several kinds of structured SVMs have been proposed. These algorithms with structural information only simply embed the global structural information or the local within-class information into SVM model. Thus, they sometimes are not suitable for real-world problems. To overcome the drawbacks, we firstly propose a novel SVM with local structural information (LSI-SVM) in this paper. In the LSI-SVM, the K-nearest neighbor (KNN) method is adopted. Applying the KNN method, the farthest neighbors set intra-class and the nearest neighbors set inter-class of the overall samples are obtained. It is more reasonable to maximize the margin between the nearest neighbors and the farthest neighbors. Both the global and local data structures are added into the optimization problem, making the LSI-SVM can fully utilize the underlying structural information and yield better performance. Furthermore, for nonlinear classification, the reproducing kernel Hilbert space theory is introduced and the kernel-based LSI-SVM is generated. What’s more, in order to accelerate the training speed of LSI-SVM, a stochastic gradient LSI-SVM (LSI-SVM +) is constructed using the stochastic gradient descent (SGD) solver. Lastly, experimental results on regular-scale datasets, steel surface defects datasets, ORL face dataset and large-scale datasets demonstrate that both LSI-SVM and LSI-SVM + outperform other state-of-the-art algorithms on accuracy. In the meanwhile, our LSI-SVM + has high efficiency.

Machine Learning and Feature Selection Methods for EGFR Mutation Status Prediction in Lung Cancer

The evolution of personalized medicine has changed the therapeutic strategy from classical chemotherapy and radiotherapy to a genetic modification targeted therapy, and although biopsy is the traditional method to genetically characterize lung cancer tumor, it is an invasive and painful procedure for the patient. Nodule image features extracted from computed tomography (CT) scans have been used to create machine learning models that predict gene mutation status in a noninvasive, fast, and easy-to-use manner. However, recent studies have shown that radiomic features extracted from an extended region of interest (ROI) beyond the tumor, might be more relevant to predict the mutation status in lung cancer, and consequently may be used to significantly decrease the mortality rate of patients battling this condition. In this work, we investigated the relation between image phenotypes and the mutation status of Epidermal Growth Factor Receptor (EGFR), the most frequently mutated gene in lung cancer with several approved targeted-therapies, using radiomic features extracted from the lung containing the nodule. A variety of linear, nonlinear, and ensemble predictive classification models, along with several feature selection methods, were used to classify the binary outcome of wild-type or mutant EGFR mutation status. The results show that a comprehensive approach using a ROI that included the lung with nodule can capture relevant information and successfully predict the EGFR mutation status with increased performance compared to local nodule analyses. Linear Support Vector Machine, Elastic Net, and Logistic Regression, combined with the Principal Component Analysis feature selection method implemented with 70% of variance in the feature set, were the best-performing classifiers, reaching Area Under the Curve (AUC) values ranging from 0.725 to 0.737. This approach that exploits a holistic analysis indicates that information from more extensive regions of the lung containing the nodule allows a more complete lung cancer characterization and should be considered in future radiogenomic studies.

Physical Deep Learning Based on Optimal Control of Dynamical Systems

Deep learning is the backbone of artificial intelligence technologies, and it can be regarded as a kind of multilayer feedforward neural network. An essence of deep learning is information propagation through layers. This suggests that there is a connection between deep neural networks and dynamical systems in the sense that information propagation is explicitly modeled by the time-evolution of dynamical systems. In this study, we perform pattern recognition based on the optimal control of continuous-time dynamical systems, which is suitable for physical hardware implementation. The learning is based on the adjoint method to optimally control dynamical systems, and the deep (virtual) network structures based on the time evolution of the systems are used for processing input information. As a key example, we apply the dynamics-based recognition approach to an optoelectronic delay system and demonstrate that the use of the delay system allows for image recognition and nonlinear classifications using only a few control signals. This is in contrast to conventional multilayer neural networks, which require a large number of weight parameters to be trained. The proposed approach provides insight into the mechanisms of deep network processing in the framework of an optimal control problem and presents a pathway for realizing physical computing hardware.