Nonlinear Classification Research Articles

Analytical learning classifier based on predefined evenly-distributed class centroids

For the pattern recognition, most classification models are solved iteratively, except for Linear LDA, KLDA and ELM etc. In this paper, a nonlinear classification network model based on predefined evenly-distributed class centroids (PEDCC) is proposed. Its analytical solution can be obtained and has good interpretability. Using the characteristics of maximizing the inter-class distance of PEDCC and derivative weighted minimum mean square error loss function to minimize the intra-class distance, we can not only realize the effective nonlinearity of the network, but also obtain the analytical solution of the network weight. Then, the sample is classified based on GDA. In order to further improve the performance of classification, PCA is used to reduces the dimensionality of the original sample, meanwhile, the CReLU activation function are adopted to enhances the expression ability of the features. The network transforms the samples into the higher dimensional feature space through the weighted minimum mean square error, so as to find a better separation hyperplane. In experiments, the feasibility of the network structure is verified from pure linear 𝑾, 𝑾+Tanh, and PCA+𝑾+Tanh respectively on many small data sets and large data sets, and compared with SVM and ELM in terms of training speed and recognition rate. The results show that, in general, this model has advantages on small data set both in recognition accuracy and training speed, while it has advantages in training speed on large data sets. Finally, by introducing a multi-stage network structure based on the latent feature norm, the classifier network can further significantly improve the classification performance, the recognition rate of small data sets is effectively improved and much higher than that of existing methods, while the recognition rate of large data sets is similar to that of SVM.

SGS: SqueezeNet-guided Gaussian-kernel SVM for COVID-19 Diagnosis

AbstractThe ongoing global pandemic has underscored the importance of rapid and reliable identification of COVID-19 cases to enable effective disease management and control. Traditional diagnostic methods, while valuable, often have limitations in terms of time, resources, and accuracy. The approach involved combining the SqueezeNet deep neural network with the Gaussian kernel in support vector machines (SVMs). The model was trained and evaluated on a dataset of CT images, leveraging SqueezeNet for feature extraction and the Gaussian kernel for non-linear classification. The SN-guided Gaussian-Kernel SVM (SGS) model achieved high accuracy and sensitivity in diagnosing COVID-19. It outperformed other models with an impressive accuracy of 96.15% and exhibited robust diagnostic capabilities. The SGS model presents a promising approach for accurate COVID-19 diagnosis. Integrating SqueezeNet and the Gaussian kernel enhances its ability to capture complex relationships and classify COVID-19 cases effectively.

Robust Ranking Kernel Support Vector Machine via Manifold Regularized Matrix Factorization for Multi-Label Classification

Multi-label classification has been extensively researched and utilized for several decades. However, the performance of these methods is highly susceptible to the presence of noisy data samples, resulting in a significant decrease in accuracy when noise levels are high. To address this issue, we propose a robust ranking support vector machine (Rank-SVM) method that incorporates manifold regularized matrix factorization. Unlike traditional Rank-SVM methods, our approach integrates feature selection and multi-label learning into a unified framework. Within this framework, we employ matrix factorization to learn a low-rank robust subspace within the input space, thereby enhancing the robustness of data representation in high-noise conditions. Additionally, we incorporate manifold structure regularization into the framework to preserve manifold relationships among low-rank samples, which further improves the robustness of the low-rank representation. Leveraging on this robust low-rank representation, we extract a resilient low-rank features and employ them to construct a more effective classifier. Finally, the proposed framework is extended to derive a kernelized ranking approach, for the creation of nonlinear multi-label classifiers. To effectively solve this non-convex kernelized method, we employ the augmented Lagrangian multiplier (ALM) and alternating direction method of multipliers (ADMM) techniques to obtain the optimal solution. Experimental evaluations conducted on various datasets demonstrate that our framework achieves superior classification results and significantly enhances performance in high-noise scenarios.

Analysis of genetic diversity in patients with major psychiatric disorders versus healthy controls: A molecular-genetic study of 1698 subjects genotyped for 100 candidate genes (549 SNPs)

BackgroundThis study analyzed the extent to which irregularities in genetic diversity separate psychiatric patients from healthy controls. MethodsGenetic diversity was quantified through multidimensional “gene vectors” assembled from 4 to 8 polymorphic SNPs located within each of 100 candidate genes. The number of different genotypic patterns observed per gene was called the gene's “diversity index”. ResultsThe diversity indices were found to be only weakly correlated with their constituent number of SNPs (20.5 % explained variance), thus suggesting that genetic diversity is an intrinsic gene property that has evolved over the course of evolution. Significant deviations from “normal” diversity values were found for (1) major depression; (2) Alzheimer's disease; and (3) schizoaffective disorders. Almost one third of the genes were correlated with each other, with correlations ranging from 0.0303 to 0.7245.The central finding of this study was the discovery of “singular genes” characterized by distinctive genotypic patterns that appeared exclusively in patients but not in healthy controls. Neural Nets yielded nonlinear classifiers that correctly identified up to 90 % of patients. Overlaps between diagnostic subgroups on the genotype level suggested that (1) diagnoses-crossing vulnerabilities are likely involved in the pathogenesis of major psychiatric disorders; (2) clinically defined diagnoses may not constitute etiological entities. ConclusionDetailed analyses of the variation of genotypic patterns in genes along with the correlation between genes lead to nonlinear classifiers that enable very robust separation between psychiatric patients and healthy controls on the genotype level.

Implementation of a non-linear SVM classification for seizure EEG signal analysis on FPGA

An automatic detection method for disease diagnosis plays an important role in the medical field. A computer-aided diagnosis (CAD) system for seizure detection using electroencephalogram (EEG) data is significant in protecting patients' lives. This paper presents machine learning algorithm-based seizure detection approach implemented in both software and hardware using EEG recordings. The input EEG signal is subjected to pre-processing, decomposition using wavelet transform and classification by a support vector machine (SVM) classifier. The maximum and variance statistical values of the decomposed EEG sub-bands are computed to generate a feature vector, while classification accuracy and computational complexity are taken into account. Gaussian Radial Basis Function (RBF) kernel based Non-Linear SVM (NLSVM) is used in this proposed approach to solve both two-class and three-class classification problems. In three-class classification, the one-against-one (OAO) technique is used. The competency of the proposed technique is evaluated using two distinct datasets provided by Bonn University, Germany & Neurology and Sleep Centre, New Delhi. The adequate parameters of the NLSVM classifier model are determined by using MATLAB software in the training phase and the best performing NLSVM model is selected for implementation on the Field-Programmable Gate Array (FPGA). The proposed seizure detection method is implemented with Verilog code and examined on a Xilinx Zynq-7000 XC7Z020 FPGA board to evaluate its performance. The experimental results show that the proposed method with a minimal number of features achieved a good performance metric for seizure detection and is comparable with that of the state-of-the-art methods.

Ocular biomarkers of cognitive decline based on deep-learning retinal vessel segmentation

BackgroundThe current literature shows a strong relationship between retinal neuronal and vascular alterations in dementia. The purpose of the study was to use NFN+ deep learning models to analyze retinal vessel characteristics for cognitive impairment (CI) recognition.MethodsWe included 908 participants from a community-based cohort followed for over 15 years (the prospective KaiLuan Study) who underwent brain magnetic resonance imaging (MRI) and fundus photography between 2021 and 2022. The cohort consisted of both cognitively healthy individuals (N = 417) and those with cognitive impairment (N = 491). We employed the NFN+ deep learning framework for retinal vessel segmentation and measurement. Associations between Retinal microvascular parameters (RMPs: central retinal arteriolar / venular equivalents, arteriole to venular ratio, fractal dimension) and CI were assessed by Pearson correlation. P < 0.05 was considered statistically significant. The correlation between the CI and RMPs were explored, then the correlation coefficients between CI and RMPs were analyzed. Random Forest nonlinear classification model was used to predict whether one having cognitive decline or not. The assessment criterion was the AUC value derived from the working characteristic curve.ResultsThe fractal dimension (FD) and global vein width were significantly correlated with the CI (P < 0.05). Age (0.193), BMI (0.154), global vein width (0.106), retinal vessel FD (0.099), and CRAE (0.098) were the variables in this model that were ranked in order of feature importance. The AUC values of the model were 0.799.ConclusionsEstablishment of a predictive model based on the extraction of vascular features from fundus images has a high recognizability and predictive power for cognitive function and can be used as a screening method for CI.

Deep IDA: a deep learning approach for integrative discriminant analysis of multi-omics data with feature ranking-an application to COVID-19.

Many diseases are complex heterogeneous conditions that affect multiple organs in the body and depend on the interplay between several factors that include molecular and environmental factors, requiring a holistic approach to better understand disease pathobiology. Most existing methods for integrating data from multiple sources and classifying individuals into one of multiple classes or disease groups have mainly focused on linear relationships despite the complexity of these relationships. On the other hand, methods for nonlinear association and classification studies are limited in their ability to identify variables to aid in our understanding of the complexity of the disease or can be applied to only two data types. We propose Deep Integrative Discriminant Analysis (IDA), a deep learning method to learn complex nonlinear transformations of two or more views such that resulting projections have maximum association and maximum separation. Further, we propose a feature ranking approach based on ensemble learning for interpretable results. We test Deep IDA on both simulated data and two large real-world datasets, including RNA sequencing, metabolomics, and proteomics data pertaining to COVID-19 severity. We identified signatures that better discriminated COVID-19 patient groups, and related to neurological conditions, cancer, and metabolic diseases, corroborating current research findings and heightening the need to study the post sequelae effects of COVID-19 to devise effective treatments and to improve patient care. Our algorithms are implemented in PyTorch and available at: https://github.com/JiuzhouW/DeepIDA.

Exploring the mesoscale connectivity of phytoplankton periodic assemblages' succession in northern Adriatic pelagic habitats

An appropriate model for phytoplankton distribution patterns is critical for understanding biogeochemical cycles and trophic interactions in the oceans and seas. Because phytoplankton dynamics in coastal waters are more complex due to shallow depth and proximity to land, more accurate models applied to the correct spatial and temporal scales are needed. Our study investigates the role of the atmosphere and hydrosphere in pelagic habitat by modelling phytoplankton assemblages at two Long Term Ecological Research sites in the northern Adriatic Sea using niche-forming environmental variables (wind, temperature, salinity, river discharge, rain, and water column stratification). To study the synchronization between the phytoplankton community and these environmental variables at the two LTER sites, we applied current linear and nonlinear numerical methods for ecological modelling. The aim was to use periodic and/or non-periodic properties of the environmental variables to classify the phytoplankton assemblages at one LTER site (Gulf of Trieste - Slovenia) and then predict them at another LTER site 100 km away (Gulf of Venice - Italy). We found that periodicity played a role in the explanatory and predictive power of the environmental variables and that it was more important than non-periodic events in defining the common structure of the two pelagic habitats. The non-linear classification functions of the neural networks further increased the predictive power of these variables. We observed partial synchronization of communities at the mesoscale and differences between the original and predicted assemblages under similar environmental conditions. We conclude that mesoscale connectivity plays an important role in phytoplankton communities in the northern Adriatic. However, the loss of periodicity of niche-forming variables due to more frequent extreme meteorological and hydrological events could loosen these connections and affect the temporal succession of phytoplankton assemblages.

Automatic Detection of Voltage Notches using Support Vector Machine

This paper presents a comprehensive framework for voltage notch analysis and an automatic method for notch detection using a nonlinear support vector machine (SVM) classifier. A comprehensive simulation of the notch disturbance has been conducted to generate a diverse database. Based on domain knowledge and properties of power quality disturbances (PQDs), a set of characteristic features is extracted. After feature extraction, a set of most descriptive features has been selected with decision tree (DT) algorithm, and a nonlinear SVM classifier has been trained. Finally, the detection efficiency of the trained model is presented and discussed

Classification of pleasantness of wind by electroencephalography.

Thermal comfort of humans depends on the surrounding environment and affects their productivity. Several environmental factors, such as air temperature, relative humidity, wind or airflow, and radiation, have considerable influence on the thermal comfort or pleasantness; hence, these are generally controlled by electrical devices. Lately, the development of objective measurement methods for thermal comfort or pleasantness using physiological signals is receiving attention to realize a personalized comfortable environment through the automatic control of electrical devices. In this study, we focused on electroencephalography (EEG) and investigated whether EEG signals contain information related to the pleasantness of ambient airflow reproducing natural wind fluctuations using machine learning methods. In a hot and humid artificial climate chamber, we measured EEG signals while the participants were exposed to airflow at four different velocities. Based on the reported pleasantness levels, we performed within-participant classification from the source activity of the EEG and obtained a classification accuracy higher than the chance level using both linear and nonlinear support vector machine classifiers as well as an artificial neural network. The results of this study showed that EEG is useful in identifying people's transient pleasantness when exposed to wind.

Parametrized Constant-Depth Quantum Neuron.

Quantum computing has been revolutionizing the development of algorithms. However, only noisy intermediate-scale quantum devices are available currently, which imposes several restrictions on the circuit implementation of quantum algorithms. In this article, we propose a framework that builds quantum neurons based on kernel machines, where the quantum neurons differ from each other by their feature space mappings. Besides contemplating previous quantum neurons, our generalized framework has the capacity to instantiate other feature mappings that allow us to solve real problems better. Under that framework, we present a neuron that applies a tensor-product feature mapping to an exponentially larger space. The proposed neuron is implemented by a circuit of constant depth with a linear number of elementary single-qubit gates. The previous quantum neuron applies a phase-based feature mapping with an exponentially expensive circuit implementation, even using multiqubit gates. Additionally, the proposed neuron has parameters that can change its activation function shape. Here, we show the activation function shape of each quantum neuron. It turns out that parametrization allows the proposed neuron to optimally fit underlying patterns that the existing neuron cannot fit, as demonstrated in the nonlinear toy classification problems addressed here. The feasibility of those quantum neuron solutions is also contemplated in the demonstration through executions on a quantum simulator. Finally, we compare those kernel-based quantum neurons in the problem of handwritten digit recognition, where the performances of quantum neurons that implement classical activation functions are also contrasted here. The repeated evidence of the parametrization potential achieved in real-life problems allows concluding that this work provides a quantum neuron with improved discriminative abilities. As a consequence, the generalized framework of quantum neurons can contribute toward practical quantum advantage.

Performance Analysis of Vector Machine Learning Algorithm based Plant Leaf Disease Detection and Soil Type Classification

The objective is to create and evaluate an image processing application designed for identifying leaf diseases and classifying soil types, utilizing a combination of Linear-Non-Linear (L-NL) Classification Techniques and the Vector Machine Learning (VML) Algorithm. We go over the methods created and put to the test for automating the process of classifying soil types and leaf diseases. The majority of methods begin by segmenting the measured signals from the acquired data using a segmentation algorithm. Next, effective extraction techniques are employed to extract the valuable information from these segments. Certain classifiers that are specifically designed to assign classes to these segments are constructed using the measured data and features that were extracted from the collected samples; these classifiers may make use of VML methods. Cone penetration testing collected data was utilised in the traditional approach for categorising subsurface soil, and positive outcomes were attained. In the past, classification systems required a domain expert to observe the trends and size of the segmented signals in addition to any accessible a priori knowledge. The suggested methods for employing image processing to automate this classification process are also included in this paper. Boundary energy approach can be used to extract the prominent aspects of the observed segments from a segmentation algorithm, for example. Dedicated linear and non-linear classifiers assign classes to the observed segments based on the measured data and attributes taken from the gathered data of soil and damaged and healthy plant leaves. Then, VML algorithm classification is applied. In summary, the procedure begins with the application of feature extraction methods in image processing to identify the characteristics of soil samples and leaf images. Subsequently, a database of these sample images is created. The classification of soil types and leaf diseases is then performed based on their vector features using the VML algorithm.

Mechanical Fault Diagnosis With Noisy Multisource Signals via Unified Pinball Loss Intuitionistic Fuzzy Support Tensor Machine

In this paper, a challenging and significant intelligent fault diagnosis task is investigated, in which multisource sensor signals with intense noise and outlier disturbances are jointly analyzed. Such a scenario has hardly been considered in industrial research. To this end, we develop a novel tensor-based nonlinear classifier called unified pinball loss intuitionistic fuzzy support tensor machine (UPIFSTM), which can successfully solve the above tasks and improve the performance of fault diagnosis in practical applications. First, the noisy multisource signals are converted into time-frequency images and reconstructed into tensor samples to mine the time domain, frequency domain features and coupled structure information in the spatial domain. Next, we design two nonlinear forms of nonmembership functions in the tensor space, and obtain an intuitionistic fuzzy score for each training sample to enhance the robustness of the model. Subsequently, pinball loss function is introduced to better handle noise sensitivity and resampling instability problems. Note that, we employ the tensor robust principal component analysis method to accurately recover the low-rank tensors corrupted by sparse noise from the original tensors. Finally, two numerical examples are presented to verify the feasibility and validity of the proposed method.

Physical fitness and motor competence performance characteristics of Chinese elite youth athletes from four track and field throwing disciplines-a cross-sectional study.

Purpose: For systematic athletic training and targeted talent development, it is essential to know the physical fitness and motor competencies of top athletes in detail. However, it can be difficult to identify differences in performance requirements and thus to provide adequate support, especially for sports that at first glance appear to have similar demands-such as track and field throwing disciplines. Therefore, the aim of the study was to examine the physical fitness and motor competence of top athletes from different throwing disciplines and to check whether the athletes' performance parameters match the specific requirements of the respective sport. Methods: The study involved 289 male youth athletes (aged 14-18years) across four distinct throwing disciplines: shot put (n = 101), hammer throw (n = 16), discus throw (n = 63), and javelin throw (n = 109). The performance evaluation comprised three anthropometric measurements and twelve motor performance prerequisites applicable to the throwing disciplines. Discriminant analysis and neural networks (Multilayer Perceptron) were implemented to determine the possibility of distinguishing among athletes from the four sports. Results: The study's findings indicate that in male throwing athletes, disparities in general physical fitness and motor proficiency assessments discern the majority of talented young athletes based on their specific sport (discriminant analysis: 68.2%; multilayer perceptron analysis: 72.2%). This remains applicable irrespective of the classification method employed. Discus throwers possessed a height advantage, while shot putters and hammer throwers exhibited superior arm strength. Javelin throwers displayed better explosive strength and sprinting speed. Except for the hammer throwers, all events demonstrated a high level of explosive power in the medicine ball forward or backward throw test, which was especially crucial for shot put and discus athletes. Conclusion: The significance of physical fitness and motor competence tests in identifying and transferring talented athletes in track and field throwing disciplines has been affirmed. Using linear and non-linear classification methods, most athletes could be assigned to their correct sport. However, this also shows that slightly different training and talent identification is required for each of these sports. Furthermore, non-linear analysis methods can provide useful support for the development processes in junior competitive sports.

Advancing forensic research: An examination of compositional data analysis with an application on petrol fraud detection

In recent years, numerous studies have examined the chemical compounds of petrol and petrol data for forensic research. Standard quantitative methods often assume that the variables or compounds do not have compositional constraints or are not part of a constrained whole, operating within an Euclidean vector space. However, chemical compounds are typically part of a whole, and the appropriate vector space for their analysis is the simplex. Biased and arbitrary results result when statistical analysis are applied on such data without proper pre-processing of such data. Compositional analysis of data has not yet been considered in forensic science. Therefore, we compare classical statistical analysis as applied in forensic research and the new proposed paradigm of compositional data analysis (CoDa). It is demonstrated how such analysis improves the analysis in petrol and forensic science. Our study shows how principal component analysis (PCA) and classification results are affected by the preprocessing steps performed on the raw data.Our results indicate that results from a log ratio analysis provides a better separation between subgroups of the data and leads to an easier interpretation of the results. In addition, with a compositional analysis a higher classification accuracy is obtained. Even a non-linear classification method - in our case a random forest - was shown to perform poorly when applied without using compositional methods. Moreover, normalization of samples due to laboratory/unit-of-measurement effects is no longer necessary, since the composition of an observation is in compositional thinking equivalent to a multiple of it, because the used (log) ratios on raw and log ratio transformed data are equal.Petrol data from different petrol stations in Brazil are used for the demonstration. This data is highly susceptible to counterfeit petrol. Forensic analysis of its chemical elements requires non-biased statistical analysis designed for compositional data to detect fraud.Based on these results, we recommend the use of compositional data methods for gasoline and petrol chemical element analysis and gasoline product characterization, authentication and fraud detection in forensic sciences.

Forecasting failure-prone air pressure systems (FFAPS) in vehicles using machine learning

Vehicles become an inevitable factor in everyone’s life. Sometimes it becomes a threat to human lives and society. For any real-time-based applications, everyone should focus on predicting failure-prone components. A vehicle’s air pressure system (APS) is one of its most important parts. If any system failure happens against APS it leads to core-financial losses, which in turn sometimes leads to loss of human lives. Prediction of APS negligence in a real-time application requires a deep diagnosis and diligent solution. In this study, we developed a machine learning model to predict system failure against APS. A real-time dataset that includes the 170 features and the presence of high-class imbalance data and missing values has been taken and experimentally validated with existing linear and nonlinear classifiers. The performance metrics results show that the Random Forest classifier exceeds other algorithms for training and testing data with an accuracy and F1 score of 99.5 and 99.5 percent respectively.

Realization of a quantum neural network using repeat-until-success circuits in a superconducting quantum processor

Artificial neural networks are becoming an integral part of digital solutions to complex problems. However, employing neural networks on quantum processors faces challenges related to the implementation of non-linear functions using quantum circuits. In this paper, we use repeat-until-success circuits enabled by real-time control-flow feedback to realize quantum neurons with non-linear activation functions. These neurons constitute elementary building blocks that can be arranged in a variety of layouts to carry out deep learning tasks quantum coherently. As an example, we construct a minimal feedforward quantum neural network capable of learning all 2-to-1-bit Boolean functions by optimization of network activation parameters within the supervised-learning paradigm. This model is shown to perform non-linear classification and effectively learns from multiple copies of a single training state consisting of the maximal superposition of all inputs.

Research of methods based on neural networks for the analysis of the tonality of the corps of the texts

The object of the study is methods based on neural networks for analyzing the tonality of a corpus of texts. To achieve the goal set in the work, it is necessary to solve the following tasks: study the theoretical material for learning deep neural networks and their features in relation to natural language processing; study the documentation of the Tensorflow library; develop models of convolutional and recurrent neural networks; to develop the implementation of linear and non-linear classification methods on bag of words and Word2Vec models; to compare the accuracy and other quality indicators of implemented neural network models with classical methods. Tensorboard is used for learning visualization. The work shows the superiority of classifiers based on deep neural networks over classical classification methods, even if the Word2Vec model is used for vector representations of words. The model of recurrent neural network with LSTM blocks has the highest accuracy for this corpus of texts.

Mobile Botnet Detection A Machine Learning Approach using SVM

The security and privacy of smartphone users are seriously threatened by mobile botnets, which allow malevolent actors to carry out a variety of illegal actions, such as DDoS attacks, data theft, and resource exploitation. These mobile botnets are becoming more and more sophisticated, making it difficult to detect them with conventional signature-based and heuristic methods. This study proposes a machine learning-based method for mobile botnet detection that makes use of Support Vector Machines (SVM).The study focuses on behavioural feature extraction from mobile device system-level data and network traffic. The SVM model is used for classification once feature selection techniques have been used to select the most pertinent and discriminative attributes. The SVM model, making use of its capacity to manage nonlinear classification and high-dimensional data.Tests carried out on a variety of network traffic and system behaviour datasets gathered from mobile devices show encouraging outcomes for the detection of botnets. Compared to conventional detection techniques, the SVM classifier outperforms them in identifying mobile botnet activities with a high degree of accuracy, precision, and recall.The suggested SVM-based method improves mobile device security by offering a flexible and successful mobile botnet detection solution. The results of this study open the door to the development of strong and durable mobile security systems by providing insights into the proactive identification and mitigation of mobile botnet threats through the use of machine learning techniques

A Comparative Analysis of Active Learning for Rumor Detection on Social Media Platforms

In recent years, the ubiquity of social networks has transformed them into essential platforms for information dissemination. However, the unmoderated nature of social networks and the advent of advanced machine learning techniques, including generative models such as GPT and diffusion models, have facilitated the propagation of rumors, posing challenges to society. Detecting and countering these rumors to mitigate their adverse effects on individuals and society is imperative. Automatic rumor detection, typically framed as a binary classification problem, predominantly relies on supervised machine learning models, necessitating substantial labeled data; yet, the scarcity of labeled datasets due to the high cost of fact-checking and annotation hinders the application of machine learning for rumor detection. In this study, we address this challenge through active learning. We assess various query strategies across different machine learning models and datasets in order to offer a comparative analysis. Our findings reveal that active learning reduces labeling time and costs while achieving comparable rumor detection performance. Furthermore, we advocate for the use of machine learning models with nonlinear classification boundaries on complex environmental datasets for more effective rumor detection.