Nonlinear Classification Research Articles

Kalman filtering-based decision threshold dynamic optimization in photonics-aided millimeter-wave systems

We propose a novel, to the best of our knowledge, decision threshold dynamic optimization (DTDO) method based on Kalman filtering to mitigate the nonlinear effect impacts of the time-domain jitter and saturation distortion on a bit error rate (BER) performance. This dynamic optimization method was validated in a photonics-aided W-band millimeter-wave system. Compared to nonlinear algorithms and clustering classification methods, DTDO offers lower complexity and superior dynamic tracking capabilities when optimizing the BER performance in photonics-aided systems. The four-level pulse amplitude modulation (PAM4) signal, after being transmitted over a 50-m W-band link, was equalized using a low-complexity filter, maintaining an exceptionally low BER. This approach effectively leverages the inherent low-complexity advantages of single-carrier transmission over the link.

GeM-LR: Discovering predictive biomarkers for small datasets in vaccine studies.

Despite significant progress in vaccine research, the level of protection provided by vaccination can vary significantly across individuals. As a result, understanding immunologic variation across individuals in response to vaccination is important for developing next-generation efficacious vaccines. Accurate outcome prediction and identification of predictive biomarkers would represent a significant step towards this goal. Moreover, in early phase vaccine clinical trials, small datasets are prevalent, raising the need and challenge of building a robust and explainable prediction model that can reveal heterogeneity in small datasets. We propose a new model named Generative Mixture of Logistic Regression (GeM-LR), which combines characteristics of both a generative and a discriminative model. In addition, we propose a set of model selection strategies to enhance the robustness and interpretability of the model. GeM-LR extends a linear classifier to a non-linear classifier without losing interpretability and empowers the notion of predictive clustering for characterizing data heterogeneity in connection with the outcome variable. We demonstrate the strengths and utility of GeM-LR by applying it to data from several studies. GeM-LR achieves better prediction results than other popular methods while providing interpretations at different levels.

Subjective well-being of children with special educational needs: Longitudinal predictors using machine learning.

Children with special educational needs (SEN) are a diverse group facing numerous challenges related to well-being and mental health. Understanding the predictors of well-being in this population requires the incorporation of diverse factors along with approaches that can uncover complexity in how these factors work together to influence well-being. We longitudinally predicted subjective well-being in a group of children with diverse special educational needs (N = 499; M = 8.4 ± 0.9years). Thirty-two variables - ranging from demographics to various categories of life experiences - were used as predictors for both nonlinear machine learning and classical linear classifiers. Nonlinear machine learning classifiers exhibited much performance in predicting subjective well-being (F1 score = 0.72 to 0.84) compared to traditional linear classifiers. Overall, across all children, prior subjective well-being, numeracy, literacy skills, and interpersonal dimensions played important roles. However, clustering further identified four distinct clusters sharing important predictors: a 'socializer' cluster dominated by interpersonal functioning predictors, an 'analyzer' cluster emphasizing academic skills predictors, and two clusters with more diverse sets of important predictors. Our research highlights the multiple pathways toward well-being in children with SEN as uncovered by machine learning, with implications for understanding and supporting their well-being.

A semiparametric accelerated failure time-based mixture cure tree

The mixture cure rate model (MCM) is the most widely used model for the analysis of survival data with a cured subgroup. In this context, the most common strategy to model the cure probability is to assume a generalized linear model with a known link function, such as the logit link function. However, the logit model can only capture simple effects of covariates on the cure probability. In this article, we propose a new MCM where the cure probability is modeled using a decision tree-based classifier and the survival distribution of the uncured is modeled using an accelerated failure time structure. To estimate the model parameters, we develop an expectation maximization algorithm. Our simulation study shows that the proposed model performs better in capturing nonlinear classification boundaries when compared to the logit-based MCM and the spline-based MCM. This results in more accurate and precise estimates of the cured probabilities, which in-turn results in improved predictive accuracy of cure. We further show that capturing nonlinear classification boundary also improves the estimation results corresponding to the survival distribution of the uncured subjects. Finally, we apply our proposed model and the EM algorithm to analyze an existing bone marrow transplant data.

Efficient microplastic identification by hyperspectral imaging: A comparative study of spatial resolutions, spectral ranges and classification models to define an optimal analytical protocol

Microplastics (MPs) pollution is a global and challenging issue, necessitating the development of efficient analytical strategies for their detection to monitor their environmental impact.This study aims to define an optimal analytical protocol for characterizing MPs by hyperspectral imaging (HSI), comparing different setups based on spatial resolution, spectral range and classification models. The investigated MPs include polymers commonly found in the environment, such as polystyrene (PS), polypropylene (PP) and high-density polyethylene (HDPE), subdivided in three size classes (1000–2000 μm, 500–1000 μm, 250–500 μm). Furthermore, MP particles with diameters ranging from 30 to 250 μm were assessed to determine the limit of detection (LOD) in the different configurations. Hyperspectral images were acquired with two spatial resolutions, 150 and 30 μm/pixel, and two spectral ranges, 1000–1700 nm (NIR) and 1000–2500 nm (SWIR). Three classification models, Partial Least Square-Discriminant Analysis (PLS-DA), Error Correction Output Coding-Support Vector Machine (ECOC-SVM) and Neural Network Pattern Recognition (NNPR) were tested on the acquired images. The correctness of these models was evaluated by prediction maps and statistical parameters (Recall, Specificity and Accuracy). The results demonstrated that for MP particles larger than 250 μm, the optimal setup is a spatial resolution of 150 μm/pixel and a spectral range of 1000–1700 nm, utilizing a linear classification model like PLS-DA. This approach offers accurate predictions while being time- and cost-efficient. For MPs smaller than 250 μm, a higher spatial resolution of 30 μm/pixel with a spectral range of 1000–2500 nm and a non-linear classification method like ECOC-SVM is preferable. The LOD is 250 μm for the 150 μm/pixel resolution and ranges from 100 to 200 μm for the 30 μm/pixel resolution. These findings provide a valuable guide for selecting the appropriate HSI acquisition conditions and data processing methods to optimally characterize MPs of different sizes.

Engineering Sequestration-Based Biomolecular Classifiers with Shared Resources.

Constructing molecular classifiers that enable cells to recognize linear and nonlinear input patterns would expand the biocomputational capabilities of engineered cells, thereby unlocking their potential in diagnostics and therapeutic applications. While several biomolecular classifier schemes have been designed, the effects of biological constraints such as resource limitation and competitive binding on the function of those classifiers have been left unexplored. Here, we first demonstrate the design of a sigma factor-based perceptron as a molecular classifier working based on the principles of molecular sequestration between the sigma factor and its antisigma molecule. We then investigate how the output of the biomolecular perceptron, i.e., its response pattern or decision boundary, is affected by the competitive binding of sigma factors to a pool of shared and limited resources of core RNA polymerase. Finally, we reveal the influence of sharing limited resources on multilayer perceptron neural networks and outline design principles that enable the construction of nonlinear classifiers using sigma-based biomolecular neural networks in the presence of competitive resource-sharing effects.

Laplacian nonlinear logistic stepwise and gravitational deep neural classification for facial expression recognition

Laplacian nonlinear logistic stepwise and gravitational deep neural classification for facial expression recognition

Binary ant colony optimization algorithm in learning random satisfiability logic for discrete hopfield neural network

This study introduced a novel ant colony optimization algorithm that implements the population selection strategy of the Estimation of Distribution Algorithm and a new pheromone updating formula. It aimed to optimize the performance of G-type random high-order satisfiability logic structures embedded in Discrete Hopfield Neural Networks, thereby enhancing the efficiency of the Hopfield Neural Network learning algorithm. Through comparative analysis with other metaheuristic algorithms, our model demonstrated superior performance in terms of global convergence, time complexity, and algorithm complexity. Additionally, we evaluated the learning phase, retrieval phase, and similarity analysis using various ratios of literals and clauses. It was shown that our proposed model exhibits stronger search ability compared to other metaheuristic algorithms and Exhaustive Search. Our model enhanced the efficiency of the learning phase, resulting in the number of global solutions accounting for 100 %, and significantly improved the global solution diversity. These advancements contributed to the efficiency of the model in convergence, rendering it applicable to a wide range of nonlinear classification and prediction problems.

Quantum-enhanced learning with a controllable bosonic variational sensor network

Abstract The emergence of quantum sensor networks has presented opportunities for enhancing complex sensing tasks, while simultaneously introducing significant challenges in designing and analyzing quantum sensing protocols due to the intricate nature of entanglement and physical processes. Supervised learning assisted by an entangled sensor network (SLAEN) (Zhuang and Zhang 2019 Phys. Rev. X 9 041023) represents a promising paradigm for automating sensor-network design through variational quantum machine learning. However, the original SLAEN, constrained by the Gaussian nature of quantum circuits, is limited to learning linearly separable data. Leveraging the universal quantum control available in cavity quantum electrodynamics experiments, we propose a generalized SLAEN capable of handling nonlinear data classification tasks. We establish a theoretical framework for physical-layer data classification to underpin our approach. Through training quantum probes and measurements, we uncover a threshold phenomenon in classification error across various tasks—when the energy of probes exceeds a certain threshold, the error drastically diminishes to zero, providing a significant improvement over the Gaussian SLAEN. Despite the non-Gaussian nature of the problem, we offer analytical insights into determining the threshold and residual error in the presence of noise. Our findings carry implications for radio-frequency photonic sensors and microwave dark matter haloscopes.

Unraveling druggable cancer-driving proteins and targeted drugs using artificial intelligence and multi-omics analyses

The druggable proteome refers to proteins that can bind to small molecules with appropriate chemical affinity, inducing a favorable clinical response. Predicting druggable proteins through screening and in silico modeling is imperative for drug design. To contribute to this field, we developed an accurate predictive classifier for druggable cancer-driving proteins using amino acid composition descriptors of protein sequences and 13 machine learning linear and non-linear classifiers. The optimal classifier was achieved with the support vector machine method, utilizing 200 tri-amino acid composition descriptors. The high performance of the model is evident from an area under the receiver operating characteristics (AUROC) of 0.975 ± 0.003 and an accuracy of 0.929 ± 0.006 (threefold cross-validation). The machine learning prediction model was enhanced with multi-omics approaches, including the target-disease evidence score, the shortest pathways to cancer hallmarks, structure-based ligandability assessment, unfavorable prognostic protein analysis, and the oncogenic variome. Additionally, we performed a drug repurposing analysis to identify drugs with the highest affinity capable of targeting the best predicted proteins. As a result, we identified 79 key druggable cancer-driving proteins with the highest ligandability, and 23 of them demonstrated unfavorable prognostic significance across 16 TCGA PanCancer types: CDKN2A, BCL10, ACVR1, CASP8, JAG1, TSC1, NBN, PREX2, PPP2R1A, DNM2, VAV1, ASXL1, TPR, HRAS, BUB1B, ATG7, MARK3, SETD2, CCNE1, MUTYH, CDKN2C, RB1, and SMARCA4. Moreover, we prioritized 11 clinically relevant drugs targeting these proteins. This strategy effectively predicts and prioritizes biomarkers, therapeutic targets, and drugs for in-depth studies in clinical trials. Scripts are available at https://github.com/muntisa/machine-learning-for-druggable-proteins.

Variational tensor neural networks for deep learning

Deep neural networks (NNs) encounter scalability limitations when confronted with a vast array of neurons, thereby constraining their achievable network depth. To address this challenge, we propose an integration of tensor networks (TN) into NN frameworks, combined with a variational DMRG-inspired training technique. This in turn, results in a scalable tensor neural network (TNN) architecture capable of efficient training over a large parameter space. Our variational algorithm utilizes a local gradient-descent technique, enabling manual or automatic computation of tensor gradients, facilitating design of hybrid TNN models with combined dense and tensor layers. Our training algorithm further provides insight on the entanglement structure of the tensorized trainable weights and correlation among the model parameters. We validate the accuracy and efficiency of our method by designing TNN models and providing benchmark results for linear and non-linear regressions, data classification and image recognition on MNIST handwritten digits.

Olfactory analysis of oolong tea sensory quality using composite nano-colorimetric sensor array

Chinese oolong tea is famous for its rich and diverse aromas, which is an important indicator for sensor quality evaluation. To accurately and rapidly evaluate sensory quality, a novel colorimetric sensor array (CSA) was developed to detect volatile organic compounds (VOCs) in oolong tea. We further explored the binding mechanism between colorimetric dyes that trigger changes in charge transfer and visible color changes. Based on this, we modified and optimized the CSA to improve the sensitivity by 17.1–234.9% and the stability by 8.7–33.3%. The study also assessed the effectiveness of this method by comparing two linear and two non-linear classification models, with the support vector machine (SVM) model achieving the highest accuracy, identifying different flavor intensity and grades with rates of 100% and 95.83%, respectively. These findings sufficiently demonstrated that the novel CSA, integrated with the SVM model, has promising potential for predicting the sensory quality of oolong tea.

Geometric nonlinear classification of Hilbert $$C^*$$-modules based on strong Birkhoff–James orthogonality

Geometric nonlinear classification of Hilbert $$C^*$$-modules based on strong Birkhoff–James orthogonality

XAI-TRIS: non-linear image benchmarks to quantify false positive post-hoc attribution of feature importance

The field of ‘explainable’ artificial intelligence (XAI) has produced highly acclaimed methods that seek to make the decisions of complex machine learning (ML) methods ‘understandable’ to humans, for example by attributing ‘importance’ scores to input features. Yet, a lack of formal underpinning leaves it unclear as to what conclusions can safely be drawn from the results of a given XAI method and has also so far hindered the theoretical verification and empirical validation of XAI methods. This means that challenging non-linear problems, typically solved by deep neural networks, presently lack appropriate remedies. Here, we craft benchmark datasets for one linear and three different non-linear classification scenarios, in which the important class-conditional features are known by design, serving as ground truth explanations. Using novel quantitative metrics, we benchmark the explanation performance of a wide set of XAI methods across three deep learning model architectures. We show that popular XAI methods are often unable to significantly outperform random performance baselines and edge detection methods, attributing false-positive importance to features with no statistical relationship to the prediction target rather than truly important features. Moreover, we demonstrate that explanations derived from different model architectures can be vastly different; thus, prone to misinterpretation even under controlled conditions.

Comprehensive assessment of machine learning methods for diagnosing gastrointestinal diseases through whole metagenome sequencing data

ABSTRACT The gut microbiome, linked significantly to host diseases, offers potential for disease diagnosis through machine learning (ML) pipelines. These pipelines, crucial in modeling diseases using high-dimensional microbiome data, involve selecting profile modalities, data preprocessing techniques, and classification algorithms, each impacting the model accuracy and generalizability. Despite whole metagenome shotgun sequencing (WMS) gaining popularity for human gut microbiome profiling, a consensus on the optimal methods for ML pipelines in disease diagnosis using WMS data remains elusive. Addressing this gap, we comprehensively evaluated ML methods for diagnosing Crohn’s disease and colorectal cancer, using 2,553 fecal WMS samples from 21 case-control studies. Our study uncovered crucial insights: gut-specific, species-level taxonomic features proved to be the most effective for profiling; batch correction was not consistently beneficial for model performance; compositional data transformations markedly improved the models; and while nonlinear ensemble classification algorithms typically offered superior performance, linear models with proper regularization were found to be more effective for diseases that are linearly separable based on microbiome data. An optimal ML pipeline, integrating the most effective methods, was validated for generalizability using holdout data. This research offers practical guidelines for constructing reliable disease diagnostic ML models with fecal WMS data.

Distributionally robust chance-constrained kernel-based support vector machine

Support vector machine (SVM) is a powerful model for supervised learning. This article addresses the nonlinear binary classification problem using kernel-based SVM with uncertainty involved in the input data specified by the first- and second-order moments. To achieve a robust classifier with small probabilities of misclassification, we investigate a distributionally robust chance-constrained kernel-based SVM model. Since the moment information in the original problem becomes unclear/unavailable in the feature space via kernel transformation, we develop a data-driven approach utilizing empirical moments to provide a second-order cone programming (SOCP) reformulation for efficient computation. To speed up the required computations for solving large-size problems in higher dimensional space and/or with more sampling points involved in estimating empirical moments, we further design an alternating direction multipliers-based algorithm for fast computations. Extensive computational results support the effectiveness and efficiency of the proposed model and solution method. Results on public benchmark datasets without any moment information indicate that the proposed approach still works and, surprisingly, outperforms some commonly used state-of-the-art kernel-based SVM models.

Predictive modeling of perioperative patient deterioration: combining unanticipated ICU admissions and mortality for improved risk prediction

ObjectiveThis paper presents a comprehensive analysis of perioperative patient deterioration by developing predictive models that evaluate unanticipated ICU admissions and in-hospital mortality both as distinct and combined outcomes.Materials and MethodsWith less than 1% of cases resulting in at least one of these outcomes, we investigated 98 features to identify their role in predicting patient deterioration, using univariate analyses. Additionally, multivariate analyses were performed by employing logistic regression (LR) with LASSO regularization. We also assessed classification models, including non-linear classifiers like Support Vector Machines, Random Forest, and XGBoost.ResultsDuring evaluation, careful attention was paid to the data imbalance therefore multiple evaluation metrics were used, which are less sensitive to imbalance. These metrics included the area under the receiver operating characteristics, precision-recall and kappa curves, and the precision, sensitivity, kappa, and F1-score. Combining unanticipated ICU admissions and mortality into a single outcome improved predictive performance overall. However, this led to reduced accuracy in predicting individual forms of deterioration, with LR showing the best performance for the combined prediction.DiscussionThe study underscores the significance of specific perioperative features in predicting patient deterioration, especially revealed by univariate analysis. Importantly, interpretable models like logistic regression outperformed complex classifiers, suggesting their practicality. Especially, when combined in an ensemble model for predicting multiple forms of deterioration. These findings were mostly limited by the large imbalance in data as post-operative deterioration is a rare occurrence. Future research should therefore focus on capturing more deterioration events and possibly extending validation to multi-center studies.ConclusionsThis work demonstrates the potential for accurate prediction of perioperative patient deterioration, highlighting the importance of several perioperative features and the practicality of interpretable models like logistic regression, and ensemble models for the prediction of several outcome types. In future clinical practice these data-driven prediction models might form the basis for post-operative risk stratification by providing an evidence-based assessment of risk.

Methods and algorithms of swarm intelligence for the problems of nonlinear regression analysis and optimization of complex processes, objects, and systems: review and modification of methods and algorithms

The development of high-speed methods and algorithms for global multidimensional optimization and their modifications in various fields of science, technology, and economics is an urgent problem that involves reducing computing costs, accelerating, and effectively searching for solutions to such problems. Since most serious problems involve the search for tens, hundreds, or thousands of optimal parameters of mathematical models, the search space for these parameters grows non-linearly. Currently, there are many modern methods and algorithms of swarm intelligence that solve today's scientific and applied problems, but they require modifications due to the large spaces of searching for optimal model parameters. Modern swarm intelligence has significant potential for application in the energy industry due to its ability to optimize and solve complex problems. It can be used to solve scientific and applied problems of optimizing energy consumption in buildings, industrial complexes, and urban systems, reducing energy losses, and increasing the efficiency of resource use, as well as for the construction of various elements of energy systems in general. Well-known methods and algorithms of swarm intelligence are also actively applied to forecast energy production from renewable sources, such as solar and wind energy. This allows better management of energy sources and planning of their use. The relevance of modifications of methods and algorithms is due to the issues of speeding up their work when solving machine learning problems, in particular, in nonlinear regression models, classification, and clustering problems, where the number of observed data can reach tens and hundreds of thousands or more. The work considers and modifies well-known effective methods and algorithms of swarm intelligence (particle swarm optimization algorithm, bee optimization algorithm, differential evolution method) for finding solutions to multidimensional extremal problems with and without restrictions, as well as problems of nonlinear regression analysis. The obtained modifications of the well-known classic effective methods and algorithms of swarm intelligence, which are present in the work, effectively solve complex scientific and applied tasks of designing complex objects and systems. A comparative analysis of methods and algorithms will be conducted in the next study on this topic. Keywords: optimization, swarm intelligence, mathematical modelling, nonlinear regression, complex objects and systems.

Ensemble One-Class Support Vector Machine for Sea Surface Target Detection Based on k-Means Clustering

Sea surface target detection is a key stage in a typical target detection system and directly influences the performance of the whole system. As an effective discriminator, the one-class support vector machine (OCSVM) has been widely used in target detection. In OCSCM, training samples are first mapped to the hypersphere in the kernel space with the Gaussian kernel function, and then, a linear classification hyperplane is constructed in each cluster to separate target samples from other classes of samples. However, when the distribution of the original data is complex, the transformed data in the kernel space may be nonlinearly separable. In this situation, OCSVM cannot classify the data correctly, because only a linear hyperplane is constructed in the kernel space. To solve this problem, a novel one-class classification algorithm, referred to as ensemble one-class support vector machine (En-OCSVM), is proposed in this paper. En-OCSVM is a hybrid model based on k-means clustering and OCSVM. In En-OCSVM, training samples are clustered in the kernel space with the k-means clustering algorithm, while a linear decision hyperplane is constructed in each cluster. With the combination of multiple linear classification hyperplanes, a complex nonlinear classification boundary can be achieved in the kernel space. Moreover, the joint optimization of the k-means clustering model and OCSVM model is realized in the proposed method, which ensures the linear separability of each cluster. The experimental results based on the synthetic dataset, benchmark datasets, IPIX datasets, and SAR real data demonstrate the better performance of our method over other related methods.

Photonic Neural Network Fabricated on Thin Film Lithium Niobate for High‐Fidelity and Power‐Efficient Matrix Computation

AbstractPhotonic neural networks (PNNs) have emerged as a promising platform to address the energy consumption issue that comes with the advancement of artificial intelligence technology, and thin film lithium niobate (TFLN) offers an attractive solution as a material platform mainly for its combined characteristics of low optical loss and large electro‐optic (EO) coefficients. Here, the first implementation of an EO tunable Mach‐Zehnder interferometer (MZI) mesh‐based TFLN PNN is presented. The device features ultra‐high fidelity, high computation speed, and exceptional power efficiency. The performance of the device is benchmarked with several deep learning missions including in situ training of Circle and Moons nonlinear datasets classification, Iris flower species recognition, and handwriting digits recognition. The work paves the way for sustainable up‐scaling of high‐speed, energy‐efficient PNNs.