Tikhonov Regularization Research Articles

A new general biased estimator in linear measurement error model

Numerous biased estimators are known to circumvent the multicollinearity problem in linear measurement error models. This article proposes a general biased estimator with the ridge regression and the Liu estimators as special cases. The efficiency of the suggested estimator is compared with ridge regression and Liu estimators under the mean squared error matrix criterion. In addition, a Monte Carlo simulation study and a numerical evaluation have been conducted to elucidate the superiority of the new general biased estimator over other estimators.

Exploring the dose-response of landscape preference: A case study in Singapore

Despite pursuing greening initiatives, cities might experience excessive greenery at the neighborhood level. The intension of this research is to promote ecosystem services and avoid the mantle of so-called “green urbanism”. Hence, examining the dose-response relationship is essential to understanding the intricate interplay between residential greenery quantity and landscape preference. Drawing upon preference assessment data from multiple neighborhoods with 30 typical scenes (n = 147), we employ Kaplan's landscape preference matrix (KLPM) to quantify the spatial distribution of greenery and individuals' preference assessment from the perspectives of Mystery, Coherence, Legibility, and Complexity. Results from multiple and ridge regressions indicate that the dose-response curve follows a polynomial shape, with landscape preferences reaching a threshold as the NDVI value increased from 0.35 to 0.42. Beyond the threshold, the trend of landscape preference levels tapers off and eventually plateaus. Therefore, the conclusion of our cross-sectional experiment adequately reflects the threshold effect of landscape preference toward greenery quantity. Furthermore, the weight of Mystery, Coherence, and Legibility in KLPM all showed a positive strong correlation, but the dose-response relationship cannot be fully explained by Complexity. These results inform how designers can more effectively promote optimal doses of nearby nature to circumvent worldwide overheated "Garden City” initiatives.

Non‐linear phylogenetic regression using regularised kernels

Abstract Phylogenetic regression is a type of generalised least squares (GLS) method that incorporates a modelled covariance matrix based on the evolutionary relationships between species (i.e. phylogenetic relationships). While this method has found widespread use in hypothesis testing via phylogenetic comparative methods, such as phylogenetic ANOVA, its ability to account for non‐linear relationships has received little attention. To address this, here we implement a phylogenetic Kernel Ridge Regression (phyloKRR) method that utilises GLS in a high‐dimensional feature space, employing linear combinations of phylogenetically weighted data to account for non‐linearity. We analysed two biological datasets using the Radial Basis Function and linear kernel function. The first dataset contained morphometric data, while the second dataset comprised discrete trait data and diversification rates as response variable. Hyperparameter tuning of the model was achieved through cross‐validation rounds in the training set. In the tested biological datasets, phyloKRR reduced the error rate (as measured by RMSE) by around 20% compared to linear‐based regression when data did not exhibit linear relationships. In simulated datasets, the error rate decreased almost exponentially with the level of non‐linearity. These results show that introducing kernels into phylogenetic regression analysis presents a novel and promising tool for complementing phylogenetic comparative methods. We have integrated this method into Python package named phyloKRR, which is freely available at: https://github.com/ulises‐rosas/phylokrr.

Kernel-based framework for improved prediction of discharge coefficient in vertically supported cylindrical weirs

ABSTRACT The present study represents the first use of kernel-based models to predict discharge coefficient (Cd) for two distinct types of cylindrical weirs, featuring vertical support and a 30-degree upstream ramp. For this purpose, kernel-based methods, including support vector machine, Gaussian process regression (GPR), Kernel extreme learning machine, and Kernel ridge regression, were used, as they offer notable advantages compared to other machine learning models, such as flexibility in handling various data patterns, robustness against overfitting, and effectiveness in high-dimensional data scenarios. The results indicated that the GPR model, with statistical metrics of R = 0.967, Nash–Sutcliffe efficiency (NSE) = 0.935, and root-mean-square error (RMSE) = 0.027, demonstrates superior accuracy in modeling the overall dataset collected from two distinct types of weirs. Through a conducted sensitivity analysis, it was identified that the upstream Froude number is pivotal in accurately predicting the Cd of a cylindrical weir. The modeling conducted for two distinct weir types revealed that a cylindrical weir with vertical support exhibits enhanced predictive capabilities (R = 0.997, NSE = 0.994, and RMSE = 0.007) for Cd. The findings indicate that the introduction of the upstream ramp alters hydraulic conditions, resulting in reduced modeling accuracy (R = 0.760, NSE = 0.529, and RMSE = 0.060).

Distributed Wearable Ultrasound Sensors Predict Isometric Ground Reaction Force.

Rehabilitation from musculoskeletal injuries focuses on reestablishing and monitoring muscle activation patterns to accurately produce force. The aim of this study is to explore the use of a novel low-powered wearable distributed Simultaneous Musculoskeletal Assessment with Real-Time Ultrasound (SMART-US) device to predict force during an isometric squat task. Participants (N = 5) performed maximum isometric squats under two medical imaging techniques; clinical musculoskeletal motion mode (m-mode) ultrasound on the dominant vastus lateralis and SMART-US sensors placed on the rectus femoris, vastus lateralis, medial hamstring, and vastus medialis. Ultrasound features were extracted, and a linear ridge regression model was used to predict ground reaction force. The performance of ultrasound features to predict measured force was tested using either the Clinical M-mode, SMART-US sensors on the vastus lateralis (SMART-US: VL), rectus femoris (SMART-US: RF), medial hamstring (SMART-US: MH), and vastus medialis (SMART-US: VMO) or utilized all four SMART-US sensors (Distributed SMART-US). Model training showed that the Clinical M-mode and the Distributed SMART-US model were both significantly different from the SMART-US: VL, SMART-US: MH, SMART-US: RF, and SMART-US: VMO models (p < 0.05). Model validation showed that the Distributed SMART-US model had an R2 of 0.80 ± 0.04 and was significantly different from SMART-US: VL but not from the Clinical M-mode model. In conclusion, a novel wearable distributed SMART-US system can predict ground reaction force using machine learning, demonstrating the feasibility of wearable ultrasound imaging for ground reaction force estimation.

Combination of gas chromatography-mass spectrometry and hyperspectral imaging for identification of adulterated Safflower seed oil

Safflower seed oil (SSO), a crucial edible oil, occupies a significant position in human diet due to its unique nutritional components. However, because of its high economic value, SSO has become a primary target for food adulteration. To rapidly and effectively detect the concentration of adulterants in SSO, a research approach using hyperspectral imaging and gas chromatography-mass spectrometry (GC-MS) was proposed as technical means, employing machine learning as the methodology. Specifically, different preprocessing methods were compared, with median filtering (MF) selected for spectral data to significantly reduce noise and improve the robustness and generalization ability of the model. Regarding the selection of feature bands, bands around 440 nm, 530 nm, and 880 nm – 950 nm were identified as more favorable for establishing a predictive model for the concentration of adulterants in SSO, while also shortening the modeling time. By constructing an ensemble learning model with ridge regression (Ridge) and partial least squares regression (PLSR) as base models and LightGBM as the meta-model, achieving high-precision prediction of the concentration of adulterants in SSO. Furthermore, by jointly modeling the linoleic acid, oleic acid and palmitic acid measured by GC-MS with hyperspectral data, the model's R2 was improved to 0.976, highlighting its outstanding performance. Therefore, this study identifies MF-Ridge-Stacking as the optimal model for predicting the concentration of adulterants in SSO. The research not only provides theoretical and technical support for adulteration identification but also presents a new method for predicting the concentration of adulterants in SSO, with potential practical application value.

An RFE/Ridge-ML/DL based anomaly intrusion detection approach for securing IoMT system

Smart healthcare is one of the promising areas of the Internet of Things (IoT), particularly in the case of the Covid-19 pandemic. Real-time patient monitoring and remote diagnostics facilitate better medical services to preserve human lives using Internet of Medical Things (IoMT) technology. Regardless of the numerous benefits, IoMT devices are susceptible to sophisticated cyber-attacks at a breakneck pace, which lead to tampering with healthcare data and threaten patients' lives. In a similar context, the 2022 ransomware cyber-attack on Versailles André-Mignot Hospital compromised the healthcare system and disclosed tremendous amounts of patient information. Towards this direction, most researchers have solely developed either machine learning or Deep learning algorithms to identify network traffic anomalies. Motivated by the above challenges, an effort has been made in this paper to design a Recursive Feature Elimination (RFE) integrated with machine learning paradigms and a Ridge regression merged into deep learning models for implementing accurate anomaly intrusion detection based on the real-time dataset WUSTL-EHMS. Among the paradigms used, the proposed approach confirms that the RFE-based Decision Tree (DT) outperforms state-of-the-art techniques with a training accuracy of 99 % and a testing accuracy of 97.85 % while maintaining a reduction of FAR to 0.03. In a nutshell, it has been proven that the suggested framework can be deployed to build anomaly intrusion detection, reinforcing IoMT against widespread cyber-attacks and safeguarding the integrity of advanced healthcare systems.

Predicting the financial performance of microfinance institutions with machine learning techniques

Purpose Given the growing emphasis among scholars, practitioners and policymakers on financial sustainability, this study aims to explore the applicability of machine learning techniques in predicting the financial performance of microfinance institutions (MFIs). Design/methodology/approach This study gathered 9,059 firm-year observations spanning from 2003 to 2018 from the World Bank's Mix Market database. To predict the financial performance of MFIs, the authors applied a range of machine learning regression approaches to both training and testing data sets. These included linear regression, partial least squares, linear regression with stepwise selection, elastic net, random forest, quantile random forest, Bayesian ridge regression, K-Nearest Neighbors and support vector regression. All models were implemented using Python. Findings The findings revealed the random forest model as the most suitable choice, outperforming the other models considered. The effectiveness of the random forest model varied depending on specific scenarios, particularly the balance between training and testing data set proportions. More importantly, the results identified operational self-sufficiency as the most critical factor influencing the financial performance of MFIs. Research limitations/implications This study leveraged machine learning on a well-defined data set to identify the factors predicting the financial performance of MFIs. These insights offer valuable guidance for MFIs aiming to predict their long-term financial sustainability. Investors and donors can also use these findings to make informed decisions when selecting their potential recipients. Furthermore, practitioners and policymakers can use these findings to identify potential financial performance vulnerabilities. Originality/value This study stands out by using a global data set to investigate the best model for predicting the financial performance of MFIs, a relatively scarce subject in the existing microfinance literature. Moreover, it uses advanced machine learning techniques to gain a deeper understanding of the factors affecting the financial performance of MFIs.

Comparative Study in Controlling Outliers and Multicollinearity Using Robust Performance Jackknife Ridge Regression Estimator Based on Generalized-M and Least Trimmed Square Estimator

Regression analysis is one of the statistical methods used to determine the causal relationship between one or more explanatory variables to the affected variable. The problem that often occurs in regression analysis is that there are multicollonity and outliers. To deal with such problems can be solved using ridge regression analysis and robust regression. Ridge regression can solve the problem of multicollinearas by assigning a constant k to the matrix Z′Z. But in this method the resulting bias value is still high, so to overcome this problem, the jackknife ridge regression method is used. Meanwhile, to overcome outliers in the data using robust regression methods which have several estimation methods, two of which are the Generalized-M (GM) estimator and the Least Trimmed Square (LTS) estimator. The aim of the study is to solve the problem of multicollinearity and outliers simultaneously using robust jackknife ridge regression method with GM estimators and LTS estimators. The results showed that the robust ridge jackknife regression method with LTS estimator can control multicollinearity and outliers simultaneously better based on MSE, AIC and BIC values compared to the robust ridge jackknife regression method with GM estimators. This is indicated by the value MSE = -6.60371, AIC = 75.823 and BIC = 81.642 on LTS estimators that are of lower value than GM estimators.

Dynamical system identification, model selection, and model uncertainty quantification by Bayesian inference.

This study presents a Bayesian maximum a posteriori (MAP) framework for dynamical system identification from time-series data. This is shown to be equivalent to a generalized Tikhonov regularization, providing a rational justification for the choice of the residual and regularization terms, respectively, from the negative logarithms of the likelihood and prior distributions. In addition to the estimation of model coefficients, the Bayesian interpretation gives access to the full apparatus for Bayesian inference, including the ranking of models, the quantification of model uncertainties, and the estimation of unknown (nuisance) hyperparameters. Two Bayesian algorithms, joint MAP and variational Bayesian approximation, are compared to the least absolute shrinkage and selection operator (LASSO), ridge regression, and the sparse identification of nonlinear dynamics (SINDy) algorithms for sparse regression by application to several dynamical systems with added Gaussian or Laplace noise. For multivariate Gaussian likelihood and prior distributions, the Bayesian formulation gives Gaussian posterior and evidence distributions, in which the numerator terms can be expressed in terms of the Mahalanobis distance or "Gaussian norm" ||y-y^||M-12=(y-y^)⊤M-1(y-y^), where y is a vector variable, y^ is its estimator, and M is the covariance matrix. The posterior Gaussian norm is shown to provide a robust metric for quantitative model selection for the different systems and noise models examined.

Effect of dimensionality on convergence rates of kernel ridge regression estimator

Despite the curse of dimensionality, kernel ridge regression often exhibits good performance in practical applications, even when the dimension is moderately large. However, it has been shown that kernel ridge regression cannot be free from the curse of dimensionality. Until now, the literature on kernel ridge regression has suggested that the gap between theory and practice in relation to dimensionality has not narrowed. In this study, we first investigate when the influence of dimensionality does not significantly affect the convergence rate of the kernel ridge regression. Specifically, we study the convergence rate of L2 and L∞ risks for the kernel ridge estimator, with a focus on reproducing kernel Hilbert space (RKHS) generated by a product kernel. We show that the univariate optimal convergence rate up to a logarithmic factor in L2 and L∞ risks can be achieved by controlling the size of the RKHS. The result of a numerical study confirms our theoretical findings.

Alzheimer’s Disease Diagnosis via Intuitionistic Fuzzy Random Vector Functional Link Network

Alzheimer's disease (AD) is a prominent neurodegenerative disorder, which leads to memory loss and cognitive impairment. The progression is irreversible and shows atrophies in cerebral cortex. Multiple studies revealed that the early diagnosis and early treatment can slow the progress of dementia, and hence, further atrophies can be controlled. Brain imaging data, such as magnetic resonance imaging (MRI), have been prominently used for the diagnosis of AD. Multiple approaches have been proposed for the diagnosis of AD. We propose a novel intuitionistic fuzzy random vector functional link network (IFRVFL) for the diagnosis of AD. Unlike standard random vector functional link (RVFL) network, extreme learning machine (ELM), and kernel ridge regression (KRR), which uses a uniform weighting approach for generating the optimal classifier, the proposed IFRVFL uses a fuzzy weighted approach for generating the optimal classifier. A uniform weighting scheme assumes that all the data samples are equally important; however, in real-world scenarios, this assumption may not hold true due to the presence of outliers and noise. Hence, it results in lower generalization. The proposed IFRVFL assigns each sample an intuitionistic fuzzy number (IFN), which is a function of membership and nonmembership score of a sample. The membership score is a function of the sample distance from the centroid of its corresponding class and the nonmembership score is a function of sample distance from the centroid as well as the neighborhood of the given sample. The proposed IFRVFL effectively reduces the influence of outliers. To evaluate the efficiency of the proposed IFRVFL model, we employed it for the diagnosis of AD. Experimental results demonstrate that the proposed IFRVFL model is superior in mild cognitive impairment (MCI) versus AD case. Thus, IFRVFL can be used in the clinical setting for the early diagnosis of AD. Furthermore, to check the robustness of the proposed IFRVFL model, we also evaluated it on benchmark datasets. Experimental results and the statistical tests reveal that the proposed IFRVFL is better in comparison to baseline models. The source code of the proposed IFRVFL formulation is available at https://github.com/mtanveer1/.

Relevance Vector Machines for Index Direction Predictions: An Application on Borsa Istanbul

This study investigates index prediction performance of Relevance Vector Machines (RVM) and frequently applied Ridge Regression and Support Vector Machines (SVM). Daily prices of BIST Banks and BIST Financials indices of Borsa Istanbul are used to obtain one-day-ahead predictions of the algorithms. According to estimated performance measures, RVM yielded mostly the best metrics in both periods of BIST Banks. While SVM obtained the best performance metrics on BIST Financials index, metrics of RVM were not far from the best. Overall, the results indicate the applicability of RVM in predicting index directions and has a potential to be a good rival of SVM.

Predicting tremor improvement after MRgFUS thalamotomy in essential tremor from preoperative spontaneous brain activity: A machine learning approach

Magnetic resonance-guided focused ultrasound surgery (MRgFUS) thalamotomy is an emerging technique for medication-refractory essential tremor (ET), but with variable outcomes. This study used pattern regression analysis to identify brain signatures predictive of tremor improvements. Fifty-four ET patients (mean age = 63.06 years, standard deviation (SD) = 10.55 years, 38 males) underwent unilateral MRgFUS thalamotomy and were scanned for resting-state functional magnetic resonance imaging (rs-fMRI). Seventy-four healthy controls (mean age = 58.09 years, SD = 10.30 years, 38 males) were recruited for comparison. Tremor responses at 12 months posttreatment were evaluated by the Clinical Rating Scale for Tremor. The fractional amplitude of low-frequency fluctuations (fALFF) was calculated from rs-fMRI data. Two-sample t-test was used to generate a disease-specific mask, within which Multivariate Kernel Ridge Regression analyses were conducted. Predicted and actual clinical scores were compared using Pearson’s correlation coefficient (r) and normalized mean squared error (Norm. MSE). Permutation test and leave-one-out strategy were applied for results validation. KRR identified fALFF patterns that significantly predicted the hand tremor improvement (r = 0.23, P = 0.025; Norm. MSE = 0.05, P = 0.026) and the postural tremor improvement (r = 0.28, P = 0.025; Norm. MSE = 0.06, P = 0.023), but not action tremor improvement. Lobule VI of right cerebellum (Cerebelum_6_R), right superior occipital gyrus (Occipital_Sup_R) and lobule X of vermis (Vermis_10) contributed most for hand tremor prediction (normalized weights (NW): 2.77%, 2.40%, 2.34%) while Vermis_10, left supplementary motor area (Supp_Motor_Area_L) and right hippocampus (Hippocampus_R) for postural tremor prediction (NW: 2.69%, 2.12%, 2.05%). The low contributing NW of the individual brain regions suggested that the fALFF pattern as a whole is an overall predicting feature. Preoperative fALFF pattern predicts tremor benefits induced by MRgFUS thalamotomy. ClinicalTrials.gov number: NCT04570046.

Computational modeling of petroleum purification for removal of sulfur compounds: Process analysis for reduction of environmental impacts and material costs

In the course of this investigation, three machine learning models (Gaussian Process Regression, Decision Tree Regression, and Kernel Ridge Regression) were examined for determining the correlation between the input variables (x and y) which are spatial coordinates of model’s geometry, and the content of species in adsorption for sulfur capture. For the process modeling, mass transfer was analyzed, and the concentration distribution of sulfur compound was obtained via numerical solution of mass transfer equations, and then used for machine learning models. The machine learning models were trained using a dataset of 19,000 observations, and their performance was assessed through metrics including R2 score, MAE, and RMSE. Analysis of the results reveals that Decision Tree Regression surpassed the other two models in performance, with an R2 score of 0.9989, MAE of 6.64405E-01, and RMSE of 1.1277E+00. Gaussian Process Regression had an R2 score of 0.97106, MAE of 3.65541E+00, and RMSE of 5.6821E+00, while Kernel Ridge Regression had an R2 score of 0.86347, MAE of 8.26121E+00, and RMSE of 1.1330E+01. The Clonal Selection Algorithm was used for hyper-parameter optimization for all models. These findings demonstrate the potential of machine learning techniques for accurately and reliably predicting the concentration of chemical species and highlight the importance of considering the choice of model and hyper-parameter optimization for optimal performance.

Predicting wear resistance of high-carbon Cr-V alloy steel based on machine learning

The microstructure of Cr-V wear-resistant alloy steel was characterized by its complexity, with the evaluation of its wear resistance exhibiting multidimensional, strongly coupled, and nonlinear attributes. Contrary to the conventional research paradigm that relied on microstructure analysis and performance comparison, accurate prediction of wear resistance based on data-driven approaches was found to significantly expedite the research and development process for materials. In this study, microstructural analysis demonstrated that the grid-like arrangement of M7C3, MC, and M2C eutectic carbides, along with the dispersed granular M23C6 carbides within the matrix, significantly enhanced material hardness. The uniform distribution of carbides played a critical role in resisting abrasive wear. This investigation further introduced the mass score ratios of (Cr+V)/C and V/Cr as evaluation indices for carbides to optimize the characteristic dataset. The findings unequivocally revealed that optimization of the feature set significantly enhanced the performance of the predictive models, with the correlation coefficient (R) value showing an increase of 3.17–25.18 %, while the root mean square error (RMSE) value decreased by 33.26–71.15 %. Compared to the Random Forest (RF), Gradient Boosting Trees (GBT), and Kernel Ridge Regression (KRR) models，the Support Vector Regression(SVR) model demonstrated preeminent performance, evidenced by an R value of 0.979 and an RMSE value of 0.866 mg. The application of the trained SVR model to predict new samples yielded satisfactory results, with the predicted wear resistance rankings being entirely consistent with the experimental outcomes and exhibiting high prediction accuracy. The model possessed robust quantitative predictive capabilities, enabling rapid evaluation of the wear resistance of this category of steel.

Sparse graphs-based dynamic attention networks

In previous research, the prevailing assumption was that Graph Neural Networks (GNNs) precisely depicted the interconnections among nodes within the graph's architecture. Nonetheless, real-world graph datasets are often rife with noise, elements that can disseminate through the network and ultimately affect the outcome of the downstream tasks. Facing the complex fabric of real-world graphs and the myriad potential disturbances, we introduce the Sparse Graph Dynamic Attention Networks (SDGAT) in this research. SDGAT employs the L0 regularization technique to achieve a sparse representation of the graph structure, which eliminates noise and generates a more concise sparse graph. Building upon this foundation, the model integrates a dynamic attention mechanism, allowing it to selectively focus on key nodes and edges, filter out irrelevant data, and simultaneously facilitate effective feature aggregation with important neighbors. To evaluate the performance of SDGAT, we conducted experiments on three citation datasets and compared its performance against commonly employed models. The outcomes indicate that SDGAT excels in node classification tasks, notably on the Cora dataset, with an accuracy rate of 85.29%, marking a roughly 3% enhancement over the majority of baseline models. The experimental findings provide evidence that SDGAT delivers effective performance on all three citation datasets, underscoring the efficacy of the dynamic attention network built upon a sparse graph.

NeRF-Art: Text-Driven Neural Radiance Fields Stylization.

As a powerful representation of 3D scenes, the neural radiance field (NeRF) enables high-quality novel view synthesis from multi-view images. Stylizing NeRF, however, remains challenging, especially in simulating a text-guided style with both the appearance and the geometry altered simultaneously. In this paper, we present NeRF-Art, a text-guided NeRF stylization approach that manipulates the style of a pre-trained NeRF model with a simple text prompt. Unlike previous approaches that either lack sufficient geometry deformations and texture details or require meshes to guide the stylization, our method can shift a 3D scene to the target style characterized by desired geometry and appearance variations without any mesh guidance. This is achieved by introducing a novel global-local contrastive learning strategy, combined with the directional constraint to simultaneously control both the trajectory and the strength of the target style. Moreover, we adopt a weight regularization method to effectively suppress cloudy artifacts and geometry noises which arise easily when the density field is transformed during geometry stylization. Through extensive experiments on various styles, we demonstrate that our method is effective and robust regarding both single-view stylization quality and cross-view consistency.

Non-convex sparse regularization via convex optimization for blade tip timing

Blade Tip Timing (BTT), an emerging technology poised to replace strain gauges, enables contactless measurement of rotor blade vibration. However, the blade vibration signals measured by BTT systems often suffer from significant undersampling. Sparse reconstruction methods are instrumental in addressing the challenge of undersampled signal reconstruction. However, traditional approaches grounded in ℓ1 regularization tend to underestimate the amplitude of the true solution. This underestimation is particularly pronounced in the resonance state of the rotor blade, hindering effective prediction of the blade’s operational state. To overcome this limitation, this paper introduces a novel non-convex regularized BTT model, employing a non-convex penalty term with convex-preserving properties to achieve nearly unbiased reconstruction accuracy. Additionally, we propose a new threshold iteration algorithm designed for the swift solution of this model. The accuracy and robustness of the proposed method in identifying the multimodal vibration parameters of rotor blades are validated through simulations and experiments. Comparatively, the proposed method closely aligns with the Orthogonal Matching Pursuit (OMP) method in recognizing blade multimodal vibration amplitude, showcasing significant improvement over the ℓ1 regularization method. Furthermore, it demonstrates lower sensitivity to changes in BTT probe layout when compared to the OMP and ℓ1 regularization methods.

Bipolar disorder: Construction and analysis of a joint diagnostic model using random forest and feedforward neural networks

BackgroundTo construct a diagnostic model for Bipolar Disorder (BD) depressive phase using peripheral tissue RNA data from patients and combining Random Forest with Feedforward Neural Network methods. MethodsDatasets GSE23848, GSE39653, and GSE69486 were selected, and differential gene expression analysis was conducted using the limma package in R. Key genes from the differentially expressed genes were identified using the Random Forest method. These key genes' expression levels in each sample were used to train a Feedforward Neural Network model. Techniques like L1 regularization, early stopping, and dropout layers were employed to prevent model overfitting. Model performance was then validated, followed by GO, KEGG, and protein-protein interaction network analyses. ResultsThe final model was a Feedforward Neural Network with two hidden layers and two dropout layers, comprising 2345 trainable parameters. Model performance on the validation set, assessed through 1000 bootstrap resampling iterations, demonstrated a specificity of 0.769 (95 % CI 0.571–1.000), sensitivity of 0.818 (95 % CI 0.533–1.000), AUC value of 0.832 (95 % CI 0.642–0.979), and accuracy of 0.792 (95 % CI 0.625–0.958). Enrichment analysis of key genes indicated no significant enrichment in any known pathways. ConclusionKey genes with biological significance were identified based on the decrease in Gini coefficient within the Random Forest model. The combined use of Random Forest and Feedforward Neural Network to establish a diagnostic model showed good classification performance in Bipolar Disorder.