Nonlinear Classification Research Articles

Revisiting miRNA Association with Melanoma Recurrence and Metastasis from a Machine Learning Point of View.

The diagnostic and prognostic value of miRNAs in cutaneous melanoma (CM) has been broadly studied and supported by advanced bioinformatics tools. From early studies using miRNA arrays with several limitations, to the recent NGS-derived miRNA expression profiles, an accurate diagnostic panel of a comprehensive pre-specified set of miRNAs that could aid timely identification of specific cancer stages is still elusive, mainly because of the heterogeneity of the approaches and the samples. Herein, we summarize the existing studies that report several miRNAs as important diagnostic and prognostic biomarkers in CM. Using publicly available NGS data, we analyzed the correlation of specific miRNA expression profiles with the expression signatures of known gene targets. Combining network analytics with machine learning, we developed specific non-linear classification models that could successfully predict CM recurrence and metastasis, based on two newly identified miRNA signatures. Subsequent unbiased analyses and independent test sets (i.e., a dataset not used for training, as a validation cohort) using our prediction models resulted in 73.85% and 82.09% accuracy in predicting CM recurrence and metastasis, respectively. Overall, our approach combines detailed analysis of miRNA profiles with heuristic optimization and machine learning, which facilitates dimensionality reduction and optimization of the prediction models. Our approach provides an improved prediction strategy that could serve as an auxiliary tool towards precision treatment.

Hybrid architecture based on two-dimensional memristor crossbar array and CMOS integrated circuit for edge computing

The fabrication of integrated circuits (ICs) employing two-dimensional (2D) materials is a major goal of semiconductor industry for the next decade, as it may allow the extension of the Moore’s law, aids in in-memory computing and enables the fabrication of advanced devices beyond conventional complementary metal-oxide-semiconductor (CMOS) technology. However, most circuital demonstrations so far utilizing 2D materials employ methods such as mechanical exfoliation that are not up-scalable for wafer-level fabrication, and their application could achieve only simple functionalities such as logic gates. Here, we present the fabrication of a crossbar array of memristors using multilayer hexagonal boron nitride (h-BN) as dielectric, that exhibit analog bipolar resistive switching in >96% of devices, which is ideal for the implementation of multi-state memory element in most of the neural networks, edge computing and machine learning applications. Instead of only using this memristive crossbar array to solve a simple logical problem, here we go a step beyond and present the combination of this h-BN crossbar array with CMOS circuitry to implement extreme learning machine (ELM) algorithm. The CMOS circuit is used to design the encoder unit, and a h-BN crossbar array of 2D hexagonal boron nitride (h-BN) based memristors is used to implement the decoder functionality. The proposed hybrid architecture is demonstrated for complex audio, image, and other non-linear classification tasks on real-time datasets.

Kernel-based data transformation model for nonlinear classification of symbolic data

Kernel-based data transformation model for nonlinear classification of symbolic data

Multivariate Classification of Brain Blood-Oxygen Signal Complexity for the Diagnosis of Children with Tourette Syndrome.

Tourette syndrome (TS) is a childhood-onset neuropsychiatric disorder characterized by the presence of multiple motor and vocal tics. Because of its varied clinical expressions and lack of reliable diagnostic biomarker, present TS diagnosis still depends on qualitative descriptions of symptoms. Our study aimed to investigate whether the complexity of resting state brain activity can serve as a potential biomarker for TS diagnosis, since it has been used successfully in various neuropsychiatric disorders, including two common TS comorbidities: attention-deficit hyperactivity disorder (ADHD) and obsessive-compulsive disorder (OCD). In the current study, we used both univariate analysis and multivariate searchlight analysis with both linear and non-linear classification methods to explore the group differences in the complexity of resting state brainblood oxygen level-dependent (BOLD) signals between 25 TS boys without comorbidity and 25 sex, age and educational years matched healthy controls (HCs). We also investigated the relation between symptom severity in TS patients (YGTSS scores) and complexity indices derived from different analysis methods. We found: i) univariate analysis revealed reduced complexity in TS patients in the left cerebellum, left superior frontal gyrus, and left medial frontal gyrus; ii) multivariate analysis with non-linear classification method achieved thehighest performance (accuracy: 0.94, sensitivity: 0.96, specificity: 0.92, AUC: 0.95) in bilateral supplementary motor areas; iii) significant correlationswere found between complexity index derived from multivariate analysis with non-linear classification method and Tic severity (YGTSS scores) in the left cerebellum (r = 0.523, with YGTSS phonic) and in theright supplementary motor area (r = 0.767, with YGTSS motor). Taken together, theseresults suggested that complexity of resting stateBOLD activity is a highly effective index for differentiating TS patients from normal controls. Ithas agood potential to be a quantitative biomarker for TS diagnosis.

An efficient Mahalanobis-Taguchi system for nonlinear multi-class classification problem

Mahalanobis-Taguchi system (MTS) is a robust predictive analytics technique used for diagnostic, forecasting, nonlinear classification, and feature selection of multivariate systems. MTS concept is applied in fault detection, medical diagnostic, and health hazards. However, most of the work on MTS addressed binary or two-class classification problems. There seems to be a lack of research that illustrates suitability and compares MTS performance for various nonlinear multi-class classification problems. This work's primary objective is to propose an efficient MTS (e-MTS) for nonlinear multi-class classification problems. A new approach to defining threshold Mahalanobis distance (MD) is also suggested to improve the classification performance of MTS. The secondary aim of this research is to illustrate the efficiency of e-MTS and compare its performance with back-propagation artificial neural network (BPNN) and radial basis function (RBF) kernel-based support vector machine (SVM). Analysis of various cases confirms the suitability of e-MTS for multi-class classification.

Kernel-Based Constrained Energy Minimization for Hyperspectral Mixed Pixel Classification

One fundamental task of hyperspectral imaging is spectral unmixing. In this case, the conventional pure pixel-based hyperspectral image classification (HSIC) may not work effectively for mixed pixels. This article presents a kernel-based approach to hyperspectral mixed pixel classification (HMPC) which includes two nonlinear mixed pixel classifiers, kernel constrained energy minimization (KCEM) and kernel linearly constrained minimum variance (KLCMV) to replace the widely used pure pixel-based support vector machine (SVM) classifier. Interestingly, what the binary-class and multiclass SVM classifiers are to pure pixel-based HSIC can be similarly derived for what a single-class KCEM detector and a multiclass KLCMV detector are to HMPC. In particular, the commonly used discrete classification map-based hard classification measures, average accuracy (AA) and overall accuracy (OA) for performance evaluation can be further generalized to real-valued mixed class abundance fractional map-based soft classification measures via 3-D receiver operating characteristic (3-D ROC) analysis-derived detection measures. Extensive experiments are conducted to demonstrate the utility of HMPC where KCEM/KLCMV not only significantly improve the classification performance of CEM/LCMV-based classifiers but also outperform many existing spectral-spatial classification methods.

Band Sampling of Kernel Constrained Energy Minimization Using Training Samples for Hyperspectral Mixed Pixel Classification

Constrained energy minimization (CEM) has been extended to several generalized versions, iterative CEM (ICEM), Nystrom method-based kernel CEM (NKCEM), and iterative training sampling-based NKCEM (ITS-NKCEM) for hyperspectral image classification (HSIC). Since CEM is a subpixel target detector that specifies a target signature to detect its target abundance fractions present in data samples, this article takes the advantage of CEM’s ability in subpixel detection to consider NKCEM as a nonlinear mixed pixel classifier for hyperspectral mixed pixel classification (HMPC). Recently, a new concept of band sampling (BSam) was proposed by utilizing random signal sampling derived from compressive sensing (CS) to show its performance better than band selection (BSel) for HSIC. Thus, incorporating BSam into NKCEM and ITS-NKCEM yields two new versions for HPMC, called band sampling NKCEM (BSam-NKCEM) and band sampling ITS-NKCEM (BSam-ITS-NKCEM). Interestingly, despite that BSam-ITS-NKCEM uses sampled bands as well as training samples, extensive experiments demonstrate that it can perform better than all other CEM and KCEM versions using full bands and ground truth. Specifically, it also shows that BSam-ITS-NKCEM can do better than spectral–spatial HSIC techniques using BSel.

A Hybrid Learning Framework for Multi-Modal Facial Prediction and Recognition Using Improvised Non-Linear SVM Classifier

A Hybrid Learning Framework for Multi-Modal Facial Prediction and Recognition Using Improvised Non-Linear SVM Classifier

An efficient Mahalanobis-Taguchi system for nonlinear multi-class classification problem

An efficient Mahalanobis-Taguchi system for nonlinear multi-class classification problem

Lie analysis, conserved vectors, nonlinear self-adjoint classification and exact solutions of generalized $ \left(N+1\right) $-dimensional nonlinear Boussinesq equation

<abstract><p>In this article, the generalized $ \left(N+1\right) $-dimensional nonlinear Boussinesq equation is analyzed via Lie symmetry method. Lie point symmetries of the considered equation and accompanying invariant groups are computed. After transforming the equation into a nonlinear ordinary differential equation (ODE), analytical solutions of various types are obtained using the $ \left(G^\prime/G, 1/G\right) $ expansion method. The concept of nonlinear self-adjointness is used in order to determine nonlocal conservation laws of the equation in lower dimensions. By selecting the appropriate parameter values, the study provides a graph of the solutions to the equation under study.</p></abstract>

Review on Unmanned Aerial Vehicle Assisted Sensor Node Localization in Wireless Networks: Soft Computing Approaches

Node positioning or localization is a critical requisite for numerous position-based applications of wireless sensor network (WSN). Localization using the unmanned aerial vehicle (UAV) is preferred over localization using fixed terrestrial anchor node (FTAN) because of low implementation complexity and high accuracy. The conventional multilateration technique estimates the position of the unknown node (UN) based on the distance from the anchor node (AN) to UN that is obtained from the received signal strength (RSS) measurement. However, distortions in the propagation medium may yield incorrect distance measurement and as a result, the accuracy of RSS-multilateration is limited. Though the optimization based localization schemes are considered to be a better alternative, the performance of these schemes is not satisfactory if the distortions are non-linear. In such situations, the neural network (NN) architecture such as extreme learning machine (ELM) can be a better choice as it is a highly non-linear classifier. The ELM is even superior over its counterpart NN classifiers like multilayer perceptron (MLP) and radial basis function (RBF) due to its fast and strong learning ability. Thus, this paper provides a comparative review of various soft computing based localization techniques using both FTAN and aerial ANs for better acceptability.

Distributed Healthcare Framework Using MMSM-SVM and P-SVM Classification

With the modernization of machine learning techniques in healthcare, different innovations including support vector machine (SVM) have predominantly played a major role in classifying lung cancer, predicting coronavirus disease 2019, and other diseases. In particular, our algorithm focuses on integrated datasets as compared with other existing works. In this study, parallel-based SVM (P-SVM) and multiclass-based multiple submodels (MMSM-SVM) were used to analyze the optimal classification of lung diseases. This analysis aimed to find the optimal classification of lung diseases with id and stages, such as key-value pairs in MapReduce combined with P-SVM and MMSVM for binary and multiclasses, respectively. For non-linear classification, kernel clustering-based SVM embedded with multiple submodels was developed. Both algorithms were developed using Apache spark environment, and data for the analysis were retrieved from microscope lab, UCI, Kaggle, and General Thoracic surgery database along with some electronic health records related to various lung diseases to increase the dataset size to 5 GB. Performance measures were conducted using a 5 GB dataset with five nodes. Dataset size was finally increased, and task analysis and CPU utilization were measured.

Kernel-based adversarial attacks and defenses on support vector classification

While malicious samples are widely found in many application fields of machine learning, suitable countermeasures have been investigated in the field of adversarial machine learning. Due to the importance and popularity of Support Vector Machines (SVMs), we first describe the evasion attack against SVM classification and then propose a defense strategy in this paper. The evasion attack utilizes the classification surface of SVM to iteratively find the minimal perturbations that mislead the nonlinear classifier. Specially, we propose what is called a vulnerability function to measure the vulnerability of the SVM classifiers. Utilizing this vulnerability function, we put forward an effective defense strategy based on the kernel optimization of SVMs with Gaussian kernel against the evasion attack. Our defense method is verified to be very effective on the benchmark datasets, and the SVM classifier becomes more robust after using our kernel optimization scheme.

Sparse coded spatial pyramid matching and multi-kernel integrated SVM for non-linear scene classification

AbstractSupport vector machine (SVM) techniques and deep learning have been prevalent in object classification for many years. However, deep learning is computation-intensive and can require a long training time. SVM is significantly faster than Convolution Neural Network (CNN). However, the SVM has limited its applications in the mid-size dataset as it requires proper tuning. Recently the parameterization of multiple kernels has shown greater flexibility in the characterization of the dataset. Therefore, this paper proposes a sparse coded multi-scale approach to reduce training complexity and tuning of SVM using a non-linear fusion of kernels for large class natural scene classification. The optimum features are obtained by parameterizing the dictionary, Scale Invariant Feature Transform (SIFT) parameters, and fusion of multiple kernels. Experiments were conducted on a large dataset to examine the multi-kernel space capability to find distinct features for better classification. The proposed approach founds to be promising than the linear multi-kernel SVM approaches achieving 91.12 % maximum accuracy.

Weight-of-evidence through shrinkage and spline binning for interpretable nonlinear classification

In many practical applications, such as fraud detection, credit risk modeling or medical decision making, classification models for assigning instances to a predefined set of classes are required to be both precise and interpretable. Linear modeling methods such as logistic regression are often adopted since they offer an acceptable balance between precision and interpretability. Linear methods, however, are not well equipped to handle categorical predictors with high cardinality or to exploit nonlinear relations in the data. As a solution, data preprocessing methods such as weight of evidence are typically used for transforming the predictors. The binning procedure that underlies the weight-of-evidence approach, however, has been little researched and typically relies on ad hoc or expert-driven procedures. The objective in this paper, therefore, is to propose a formalized, data-driven and powerful method. To this end, we explore the discretization of continuous variables through the binning of spline functions, which allows for capturing nonlinear effects in predictor variables and yields highly interpretable predictors that take only a small number of discrete values. Moreover, we extend the weight-of-evidence approach and propose to estimate the proportions using shrinkage estimators. Together, this method offers an improved ability to exploit both nonlinear and categorical predictors to achieve increased classification precision while maintaining the interpretability of the resulting model and decreasing the risk of overfitting. We present the results of a series of experiments in fraud detection and credit risk settings, which illustrate the effectiveness of the presented approach.

Regularized twin minimax probability machine for pattern classification and regression

As an excellent discriminant classifier based on generating prior knowledge, the minimax probability machine (MPM) has been widely used and deeply researched in many fields. The core idea of minimax probability machine is to directly estimate probability accuracy bound by minimizing the maximum probability of misclassification. However, minimax probability machine does not include a regularization term for the construction of the separating hyperplane, and it needs to solve a large-scale second-order cone programming problem in the solution process, which greatly limits it development and application. In this paper, to improve the performance of minimax probability machine, we propose a novel binary classification method called regularized twin minimax probability machine classification (TMPMC). The TMPMC constructs two non-parallel hyperplanes for final classification by solving two smaller second-order cone programming problems to improve the performance of the MPM. For each hyperplane, our method is theoretically well grounded on the idea of minimizing the worst case (maximum) probability of misclassification of a class of samples while the distance to the other class is as large as possible. Our approach was first derived as linear methods, and subsequently extended as kernel-based strategies for nonlinear classification. Additionally, we extend TMPMC to the regression problem and propose a new regularized twin minimax probability machine regression (TMPMR). Experimental results on several datasets show that our methods are competitive in terms of generalization performance compared to other algorithms.

Semiparametric Averaging of Nonlinear Marginal Logistic Regressions and Forecasting for Time Series Classification

Binary classification is an important issue in many applications but mostly studied for independent data in the literature. A binary time series classification is investigated by proposing a semiparametric procedure named “Model Averaging nonlinear MArginal LOgistic Regressions” (MAMaLoR) for binary time series data based on the time series information of predictor variables. The procedure involves approximating the logistic multivariate conditional regression function by combining low-dimensional non-parametric nonlinear marginal logistic regressions, in the sense of Kullback-Leibler distance. A time series conditional likelihood method is suggested for estimating the optimal averaging weights together with local maximum likelihood estimations of the nonparametric marginal time series logistic (auto)regressions. The asymptotic properties of the procedure are established under mild conditions on the time series observations that are of β-mixing property. The procedure is less computationally demanding and can avoid the “curse of dimensionality” for, and be easily applied to, high dimensional lagged information based nonlinear time series classification forecasting. The performances of the procedure are further confirmed both by Monte-Carlo simulation and an empirical study for market moving direction forecasting of the financial FTSE 100 index data.

Online Interval Type-2 Fuzzy Extreme Learning Machine applied to 3D path following for Remotely Operated Underwater Vehicles

In marine missions that involve 3D path following tasks, the overall goal of Underwater Vehicles (UVs) is the successful completion of a path previously specified by the operator. This implies that the path must be followed by the UV as closely as possible and arrive at a location for collection by a vessel. In this paper, an Online Interval Type-2 Fuzzy Extreme Learning Machine (OIT2-FELM) is suggested to achieve a robust following behaviour along a predefined 3D path using a Remotely Operated Underwater Vehicle (ROV). The proposed machine is a fast sequential learning scheme to the training of a more generalised model of TSK Interval Type-2 Fuzzy Inference Systems (TSK IT2 FISs) equivalent to Single Layer Feedforward Neural Networks (SLFNs). Learning new input data in the OIT2-FELM can be done one-by-one or chunk-by-chunk with a fixed or varying size. The OIT2-FELM is implemented in a hierarchical navigation strategy (HNS) as the main guidance mechanism to infer local control motions and to provide the ROV with the necessary autonomy to complete a predefined 3D path. For local path-planning, the OIT2-FELM performs signal classification for obstacle avoidance and target detection based on data collected by an on-board scan sonar. To evaluate the performance of the proposed OIT2-FELM, two different experiments are suggested. First, a number of benchmark problems in the field of non-linear system identification, regression and classification problems are used. Secondly, a number of experiments to the completion of a predefined 3D path using an ROV is implemented. Compared to other fuzzy strategies, the OIT2-FELM offered two significant capabilities. On the one hand, the OIT2-FELM provides a better treatment of uncertainty and noisy signals in underwater environments while improving the ROV’s performance. Secondly, online learning in OIT2-FELM allows continuous knowledge discovery from survey data to infer the surroundings of the ROV. Experiment results to the completion of 3D paths show the effectiveness of the proposed approach to handle uncertainty and produce reasonable classification predictions (∼90.5% accuracy in testing data).

Predicting speech intelligibility from EEG in a non-linear classification paradigm * *Submitted for review on 7 June 2021. The research conducted in this paper is funded by KU Leuven Special Research Fund C24/18/099 (C2 project to Tom Francart and Hugo Van hamme), by a PhD Grant (1S89620N) of the Research Foundation Flanders (FWO) and from the European Research Council (ERC)

Objective. Currently, only behavioral speech understanding tests are available, which require active participation of the person being tested. As this is infeasible for certain populations, an objective measure of speech intelligibility is required. Recently, brain imaging data has been used to establish a relationship between stimulus and brain response. Linear models have been successfully linked to speech intelligibility but require per-subject training. We present a deep-learning-based model incorporating dilated convolutions that operates in a match/mismatch paradigm. The accuracy of the model’s match/mismatch predictions can be used as a proxy for speech intelligibility without subject-specific (re)training. Approach. We evaluated the performance of the model as a function of input segment length, electroencephalography (EEG) frequency band and receptive field size while comparing it to multiple baseline models. Next, we evaluated performance on held-out data and finetuning. Finally, we established a link between the accuracy of our model and the state-of-the-art behavioral MATRIX test. Main results. The dilated convolutional model significantly outperformed the baseline models for every input segment length, for all EEG frequency bands except the delta and theta band, and receptive field sizes between 250 and 500 ms. Additionally, finetuning significantly increased the accuracy on a held-out dataset. Finally, a significant correlation (r = 0.59, p = 0.0154) was found between the speech reception threshold (SRT) estimated using the behavioral MATRIX test and our objective method. Significance. Our method is the first to predict the SRT from EEG for unseen subjects, contributing to objective measures of speech intelligibility.

Multi-Target In Silico Prediction of Inhibitors for Mitogen-Activated Protein Kinase-Interacting Kinases.

The inhibitors of two isoforms of mitogen-activated protein kinase-interacting kinases (i.e., MNK-1 and MNK-2) are implicated in the treatment of a number of diseases including cancer. This work reports, for the first time, a multi-target (or multi-tasking) in silico modeling approach (mt-QSAR) for probing the inhibitory potential of these isoforms against MNKs. Linear and non-linear mt-QSAR classification models were set up from a large dataset of 1892 chemicals tested under a variety of assay conditions, based on the Box–Jenkins moving average approach, along with a range of feature selection algorithms and machine learning tools, out of which the most predictive one (>90% overall accuracy) was used for mechanistic interpretation of the likely inhibition of MNK-1 and MNK-2. Considering that the latter model is suitable for virtual screening of chemical libraries—i.e., commercial, non-commercial and in-house sets, it was made publicly accessible as a ready-to-use FLASK-based application. Additionally, this work employed a focused kinase library for virtual screening using an mt-QSAR model. The virtual hits identified in this process were further filtered by using a similarity search, in silico prediction of drug-likeness, and ADME profiles as well as synthetic accessibility tools. Finally, molecular dynamic simulations were carried out to identify and select the most promising virtual hits. The information gathered from this work can supply important guidelines for the discovery of novel MNK-1/2 inhibitors as potential therapeutic agents.