Variable Selection Techniques Research Articles

A data science approach to optimize ADHD assessment with the BRIEF-2 questionnaire.

Attention deficit hyperactivity disorder (ADHD) is a prevalent neurodevelopmental disorder. A key challenge associated with this condition is achieving an early diagnosis. The current study seeks to anticipate and delineate the assessments offered by both parents and teachers concerning a child's behavior and overall functioning with the Behavior Rating Inventory of Executive Function-2 (BRIEF-2). Mothers, fathers, and teachers of 59 children diagnosed or in the process of being assessed for ADHD participated in this study. The responses provided by 59 mothers, 59 fathers, and 57 teachers to the BRIEF-2 questionnaire were collected. The performance of various feature selection techniques, including Lasso, decision trees, random forest, extreme gradient boosting, and forward stepwise regression, was evaluated. The results indicate that Lasso stands out as the optimal method for our dataset, striking an ideal balance between accuracy and interpretability. A repeated validation analysis reveals an average positive correlation exceeding 0.5 between the inattention/hyperactivity scores reported by informants (mother, father, or teacher) and the predictions derived from Lasso. This performance is achieved using only approximately 18% of the BRIEF-2 items. These findings underscore the usefulness of variable selection techniques in accurately characterizing a patient's condition while employing a small subset of assessment items. This efficiency is particularly valuable in time-constrained settings and contributes to improving the comprehension of ADHD.

Post-Estimation Shrinkage in Full and Selected Linear Regression Models in Low-Dimensional Data Revisited.

The fit of a regression model to new data is often worse due to overfitting. Analysts use variable selection techniques to develop parsimonious regression models, which may introduce bias into regression estimates. Shrinkage methods have been proposed to mitigate overfitting and reduce bias in estimates. Post-estimation shrinkage is an alternative to penalized methods. This study evaluates effectiveness of post-estimation shrinkage in improving prediction performance of full and selected models. Through a simulation study, results were compared with ordinary least squares (OLS) and ridge in full models, and best subset selection (BSS) and lasso in selected models. We focused on prediction errors and the number of selected variables. Additionally, we proposed a modified version of the parameter-wise shrinkage (PWS) approach named non-negative PWS (NPWS) to address weaknesses of PWS. Results showed that no method was superior in all scenarios. In full models, NPWS outperformed global shrinkage, whereas PWS was inferior to OLS. In low correlation with moderate-to-high signal-to-noise ratio (SNR), NPWS outperformed ridge, but ridge performed best in small sample sizes, high correlation, and low SNR. In selected models, all post-estimation shrinkage performed similarly, with global shrinkage slightly inferior. Lasso outperformed BSS and post-estimation shrinkage in small sample sizes, low SNR, and high correlation but was inferior when the opposite was true. Our study suggests that, with sufficient information, NPWS is more effective than global shrinkage in improving prediction accuracy of models. However, in high correlation, small sample sizes, and low SNR, penalized methods generally outperform post-estimation shrinkage methods.

Short wave-near infrared spectroscopy for predicting soluble solid content in intact mango with variable selection algorithms and chemometric model

Mango sweetness is one of the most prominent internal quality constituents that attract consumer's attention. The current common methods used to determine the sweetness of mango have significant disadvantages in that labor-intensive, time-consuming, and damaging techniques. This research aims to predict sweetness in mangoes using short wave-near infrared (SW-NIR) spectroscopy in the ranges 900–1650 nm, along with variable selection algorithms and chemometric model. A total of 120 mango samples were used to collect the spectra using a fibre module SW-NIR spectrometer. The partial least squares (PLS) regression with several spectral preprocessing methods was employed to develop the calibration model, and the best preprocessing technique was selected. The Savitzky–Golay second-derivative preprocessing technique performed better among the other preprocessing techniques with a correlation coefficient of prediction (rpred) of 0.74 and standard error of prediction (SEP) is 0.78 %Brix. After that, two variable selection techniques were used to select effective wavelength variables, including regression coefficient and successive projections algorithm (SPA). For SSC prediction in the range 900–1650 nm, the SPA-PLS model obtained a rpred of 0.78 and SEP of 0.67 %Brix. The current study unequivocally shows that the proposed SW-NIR spectroscopy coupled with a suitable chemometrics method can evaluate mango sweetness nondestructively.

Research on model transfer strategies based on the fusion of NIR-MIR spectral data

This study investigated the impact of different data fusion strategies on the performance of soluble solids content (SSC) prediction models based on near-infrared and mid-infrared spectroscopic techniques. In the data-level fusion approach, we applied standard normal variate and multiplicative scatter correction for pre-processing the NIR and MIR data. For the feature-level fusion, we utilized successive projections algorithm and competitive adaptive reweighted sampling to select informative wavelengths, and then applied direct orthogonal projection (DOP) for model transfer. The study employed a dataset of 150 honey samples to evaluate the impact of different data fusion strategies on model performance. To effectively evaluate model performance, we utilized the coefficient of R2 and RMSEP as evaluation metrics. By comparing the results of data-level fusion, feature-level fusion and single-spectrum model transfer, the results showed that spectral data fusion improved the model transfer performance compared to the single-spectrum approach, with feature-level fusion exhibiting the most significant advantages. The effective variable selection techniques in feature-level fusion successfully removed a substantial amount of interfering data and significantly reduced noise influence, thereby improving the model accuracy. Specifically, the use of feature-level fusion improved the predictive model’s R2 from 0.319 to 0.878 and reduced the RMSEP from 1.974 to 0.613°Brix, demonstrating the significant advantages of this approach in enhancing model transfer performance. The research findings provide important reference and theoretical support for future studies in the field of food quality assessment and other near-infrared spectroscopic data applications. This not only validates the effectiveness of the feature-level fusion approach, but also lays the foundation for establishing efficient and reliable predictive models.

An interpretable machine learning approach for predicting the capacity and failure mode of reinforced concrete columns

During seismic events, reinforced concrete (RC) columns play a crucial role in maintaining buildings’ structural integrity. This motivated engineers and practitioners to search for key parameters that influence the load-carrying capacity and failure mechanisms of such columns. However, the complexity and nonlinearity of seismic effects along with the intricate nature of RC columns as a composite system challenge the capabilities of analytical and empirical approaches to accurately capture the response of RC columns. Subsequently, the present study utilizes Machine Learning (ML) techniques to identify the failure modes and predict the corresponding capacities of RC columns based on both their geometrical and material properties. Decision trees and different ensemble methods were employed to predict both the columns’ failure mode and ultimate capacity. A multivariate dataset consisting of 486 cyclically loaded rectangular and circular columns was used to develop and validate the models. In addition, different embedded variable selection techniques were employed to evaluate the significance of input parameters in predicting the performance of columns. Moreover, partial dependence plots and accumulated local effects were employed to uncover the interrelationships between the input features and the modelled outputs. The developed models yielded an average accuracy of 90% and 95% for predicting the failure mode and ultimate capacity of RC columns, respectively. Given such high accuracy, it can be inferred that, ML techniques have the potential to provide efficient and reliable prediction tools to support seismic design and assessment decisions - mitigating seismic risks and empowering resilience planning in the face of extreme events.

Bootstrap variable selection for corporate distress prediction: evidence from the Chinese manufacturing industry

ABSTRACT This study explores the corporate distress prediction of Chinese manufacturing firms over 2013–2018 by using a large number of financial predictors. We extend the bootstrap resampling procedure into corporate distress prediction, combining with the existing selection techniques such as best subset selection, forward stepwise selection, backward stepwise selection, and least absolute shrinkage and selection operator (LASSO). We provide empirical evidence of the advantage of LASSO in selecting key predictors. The models constructed by these four techniques all outperform the prominent two models in the literature. This study proves the effectiveness of combining the bootstrap resampling method with variable selection techniques to predict corporate distress.

Construction of uniform designs for high-dimensional screening via an adjusted multiple quadrupling algorithm

One of the most common strategies to achieve optimum designs for physical and computer high-dimensional experiments with no model pre-specification and limited budget is to uniformly spread the design points over the experimental domain, which are called uniform designs. A uniform design provides good design space coverage (space-filling), resulting in more accurate estimates using fewer points. Algorithmic search is commonly used for finding such designs, but this approach becomes ineffective for large problems. Though the construction of uniform designs theoretically is challenging, intriguing and encouraging findings have recently been accomplished for highly specialized cases. This paper gives a new theoretical construction method for high-dimensional experiments with a mixture of four-level and sixteen-level factors via an adjusted version of the multiple quadrupling algorithm (Elsawah, Communication in Mathematics and Statistics 10:623-652, 2022). The new technique (called AMQA) is simple to use since it only needs small suitable four-level designs, which may be easily generated theoretically or by algorithmic search. The quality of the small designs ensures the quality of the large designs. The effectiveness of the AMQA is investigated both theoretically and computationally. The main findings demonstrate that, compared with other widely used techniques, the AMQA provides more accurate results for high-dimensional screening using a variety of models, including polynomial, spline, and kriging models, as well as variable selection techniques, including AIC, BIC, RIC, SCAD, and LASSO. To further improve performance, a hybrid algorithm that combines the AMQA with the iterative threshold accepting algorithm is proposed.

Forest above-ground biomass estimation based on strongly collinear variables derived from airborne laser scanning data

Airborne laser scanning technique (ALS) is the most appealing remote sensing technique for the precise estimation of forest above-ground biomass (AGB). Significantly strong correlations (collinearity) among the independent variables derived from ALS data decrease the accuracy of developed AGB models. To address this issue, we propose a novel variable selection algorithm-an improved sure independence screening (SPV), which integrates the Pearson correlation coefficient, threshold μ, and variance inflation factor. We further compared the performance of SPV-based, stepwise feature selection (SFS)-based, and least absolute shrinkage and selection operator (LASSO)-based AGB models developed with different regression approaches that were sensitive to strong collinearity. Field-measured data and corresponding ALS data, acquired from 1002 sample plots distributed across four distinct forest types within the Guangxi Zhuang Autonomous Region in Southern China, were used to evaluate variable selection techniques and develop AGB models. Results indicated that ALS variables selected by SPV exhibited weaker collinearity compared to those selected by SFS and LASSO. SPV-based AGB models outperformed SFS-based AGB models with higher leave-one-out cross-validation adjusted R2 (LOOCV Radj2) by 0.1% − 27.8%. SPV-based AGB models outperformed LASSO-based AGB models with higher LOOCV Radj2 by 0.4% − 16.3%. Hence, for variable selection in constructing AGB models (linear regression model, log–log regression model, and generalized additive model (GAM)) based on strongly collinear ALS variables, SPV is most preferred, followed by SFS and LASSO. The smooth curves from our GAMs developed using SPV-selected variables revealed that five canopy height variables (whp40, hmean, hp25, hp70, hskew), one canopy density variable (whp85), three density-related variables (nVegPts, bcc, and mcc), and one vertical structural variable (LADmean) were positively correlated with AGB. The canopy height variables (whp40, hmean, hp25, and hp70) were identified as the most important variables in estimating AGB for four forest types. The canopy density variable wdp85 showed a strong effect on AGB of the coniferous forests, whereas it had almost no effect on the AGB of broadleaved forests. Overall, this manuscript proposes a novel variable selection algorithm named SPV, aimed at addressing collinearity of variables derived from ALS data, which has significant implications for the application of ALS in forest inventory and forest modeling.

Determination of total protein and sugar content in soy-based beverages using variable selection methods applied to ATR-FTIR spectroscopy

This study explores the potential of using ATR-FTIR coupled with partial least squares (PLS) and random forest (RF) for predicting total protein (TP), total sugar (TS), reducing sugars (RS), and non-reducing sugars (NRS) in soy-based beverages (SBBs). Employing variable selection techniques such as interval PLS (iPLS), synergy interval PLS (siPLS), uninformative variable elimination (UVE), ordered predictors selection (OPS), bat algorithm (BA), genetic algorithm (GA), and particle swarm optimization (PSO). The OPS-PLS emerges as the optimal model for TS prediction, yielding an RMSEP of 0.114 wt% and R2p of 0.934. GA-PLS excels in TP, RS, and NRS prediction, achieving RMSEP values of 0.024 wt%, 0.273 wt%, and 0.339 wt%, with corresponding R2p values of 0.880, 0.847, and 0.450. In RF modeling, GA-RF is identified as the superior model for all properties. These findings underscore the potential of regression models and ATR-FTIR as effective and environmentally friendly tools for analyzing SBB composition.

Neural interval‐censored survival regression with feature selection

AbstractSurvival analysis is a fundamental area of focus in biomedical research, particularly in the context of personalized medicine. This prominence is due to the increasing prevalence of large and high‐dimensional datasets, such as omics and medical image data. However, the literature on nonlinear regression algorithms and variable selection techniques for interval‐censoring is either limited or nonexistent, particularly in the context of neural networks. Our objective is to introduce a novel predictive framework tailored for interval‐censored regression tasks, rooted in Accelerated Failure Time (AFT) models. Our strategy comprises two key components: (i) a variable selection phase leveraging recent advances on sparse neural network architectures; (ii) a regression model targeting prediction of the interval‐censored response. To assess the performance of our novel algorithm, we conducted a comprehensive evaluation through both numerical experiments and real‐world applications that encompass scenarios related to diabetes and physical activity. Our results outperform traditional AFT algorithms, particularly in scenarios featuring nonlinear relationships.

Analysis of COVID-19 Virus Spread in Jakarta Using Multiple Linear Regression

COVID-19, first identified in Wuhan, China in December 2019, quickly spread worldwide and was declared a pandemic by WHO in March 2020. Indonesia reported its first case on March 2, 2020, and the pandemic has had a significant impact on the country's economic, social, and health sectors. This study aims to predict the death rate due to COVID-19 in Jakarta using multiple linear regression method. The dataset collected from Andra Farm - Go Green website includes COVID-19 cases recorded in all sub-districts in Jakarta on November 1, 2023. Pre-processing was performed to improve the quality and accuracy of the model. The method used was multiple linear regression. The analysis results show that variables such as total travel and discarded trip have a significant influence in predicting the number of positive cases. The study found that lowering the correlation threshold for selecting independent variables reduced the mean squared error (MSE) and improved model performance, highlighting the importance of variable selection in developing accurate predictive models. These findings provide important insights for the government in making informed decisions regarding post-pandemic healthcare. This research underscores the value of robust data processing and variable selection techniques in enhancing predictive accuracy for public health planning.

A Branch-and-Cut-and-Price Algorithm for Cutting Stock and Related Problems

In this project, we introduce a branch-and-cut-and-price framework to solve the Cutting Stock Problems with strong relaxations using the Set Covering (Packing) Formulations, which are solved through column generation. We propose an extended Ryan-Foster branching scheme tailored to non-binary models, a pricing algorithm that produces convergence in a few iterations, and a variable selection technique based on branching history. These strategies are combined with subset-row cuts and custom primal heuristics to create a framework that overcomes the current state-of-the-art of Cutting Stock Problem, Skiving Stock Problem, and other related problems, being at least twice faster in the first problem and at least 60% faster in the second one.

REDOMA: Bayesian random-effects dose-optimization meta-analysis using spike-and-slab priors.

The rise of cutting-edge precision cancer treatments has led to a growing significance of the optimal biological dose (OBD) in modern oncology trials. These trials now prioritize the consideration of both toxicity and efficacy simultaneously when determining the most desirable dosage for treatment. Traditional approaches in early-phase oncology trials have conventionally relied on the assumption of a monotone relationship between treatment efficacy and dosage. However, this assumption may not hold valid for novel oncology therapies. In reality, the dose-efficacy curve of such treatments may reach a plateau at a specific dose, posing challenges for conventional methods in accurately identifying the OBD. Furthermore, achieving reliable identification of the OBD is typically not possible based on a single small-sample trial. With data from multiple phase I and phase I/II trials, we propose a novel Bayesian random-effects dose-optimization meta-analysis (REDOMA) approach to identify the OBD by synthesizing toxicity and efficacy data from each trial. The REDOMA method can address trials with heterogeneous characteristics. We adopt a curve-free approach based on a Gamma process prior to model the average dose-toxicity relationship. In addition, we utilize a Bayesian model selection framework that uses the spike-and-slab prior as an automatic variable selection technique to eliminate monotonic constraints on the dose-efficacy curve. The good performance of the REDOMA method is confirmed by extensive simulation studies.

Variable selection for semivarying coefficient models via local averaging

AbstractThis study aims to provide novel insights into variable selection in the semivarying coefficient model. We focus on the problem of variable selection and screening for the constant coefficient part. A common approach in the existing literature is to infer the constant coefficients by transforming the problem into a linear model scenario, utilizing a fine estimator of the varying coefficients. In this paper, we propose an approximation method for the varying coefficient functions using local averaging, which is characterized by its simplicity, rough and computational efficiency. Additionally, we introduce an adaptive lasso estimator and a forward regression algorithm specifically designed for semivarying coefficient models. Theoretical and experimental results highlight the effectiveness of the local averaging method in extending variable selection techniques from the linear model to the semivarying coefficient model. Our proposed approaches demonstrate a significant improvement in inference speed compared with baseline methods, with little loss of asymptotic efficiency.

Machine learning regression algorithms for predicting muscle, bone, carcass fat and commercial cuts in hairless lambs

The growth in demand and demand for quality in the sheep chain has generated the need for automation techniques in the meat industry and the need to obtain responses with greater speed and standardization. The research aimed to predict tissue characteristics of the carcass and commercial cuts based on measurements obtained by VIA – see oimage analysis, carried out on cold carcasses of hairless lambs, using machine learning employing regressive techniques for variable selection. Information from 72 carcasses of castrated male lambs, aged between 8 and 11 months, with an average cold carcass weight of 16.13 ± 3.98 kg, was used. Images of the right side of the carcasses were captured from the dorsal and lateral views using a digital camera. From the ImageJ2 software, VIA data, measurements and shape descriptors (areas, perimeters, widths, lengths, convexities, solidities) were obtained, combined with cold carcass weight and used to generate four sets of data, called descriptor sets (DSs). Obtaining DS1, DS1’, DS2, DS2’, DS3, DS3’, DS4 AND DS4’. To generate these sets, a database was formed and divided into a training bank (with 70% of the observations) and a test bank (30% of the observations). Multiple linear regression models were developed using Stepwise, LASSO, and Elastic Net regression methods, combined with k-fold cross-validation, to evaluate the performance of the models. The accuracy of the estimates was based on RMSE, R2, Pearson correlation and bias metrics. For the variables tested in this study, the proposed shape descriptors were mostly efficient in predicting tissue and weight variables. DS1' with the LASSO technique presented the best adjustments for variables total muscle and fat followed by shoulder, loin and rib cuts. The descriptors tested by this study were able to predict with quality the vast majority of the characteristics tested, the variable cold carcass weight (CCW), introduced as additional predictor, promoted a consistent improvement in the fits of all models. DS1 presented greater constancy for the twenty-three predicted characteristics and Stepwise presented the worst predictive performance, in relation to LASSO and Elastic Net. Despite close adjustments between the generated models, in general, Elastic Net presented lower performance than LASSO.

Longitudinal artificial intelligence-based deep learning models for diagnosis and prediction of the future occurrence of polyneuropathy in diabetes and prediabetes

ObjectiveThe objective of this study was to develop artificial intelligence-based deep learning models and assess their potential utility and accuracy in diagnosing and predicting the future occurrence of diabetic distal sensorimotor polyneuropathy (DSPN) among individuals with type 2 diabetes mellitus (T2DM) and prediabetes. MethodsIn 394 patients (T2DM=300, Prediabetes=94), we developed a DSPN diagnostic and predictive model using Random Forest (RF)-based variable selection techniques, specifically incorporating the combined capabilities of the Clinical Toronto Neuropathy Score (TCNS) and nerve conduction study (NCS) to identify relevant variables. These important variables were then integrated into a deep learning framework comprising Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks. To evaluate temporal predictive efficacy, patients were assessed at enrollment and one-year follow-up. ResultsRF-based variable selection identified key factors for diagnosing DSPN. Numbness scores, sensory test results (vibration), reflexes (knee, ankle), sural nerve attributes (sensory nerve action potential [SNAP] amplitude, nerve conduction velocity [NCV], latency), and peroneal/tibial motor NCV were candidate variables at baseline and over one year. Tibial compound motor action potential amplitudes were used for initial diagnosis, and ulnar SNAP amplitude for subsequent diagnoses. CNNs and LSTMs achieved impressive AUC values of 0.98 for DSPN diagnosis prediction, and 0.93 and 0.89 respectively for predicting the future occurrence of DSPN. RF techniques combined with two deep learning algorithms exhibited outstanding performance in diagnosing and predicting the future occurrence of DSPN. These algorithms have the potential to serve as surrogate measures, aiding clinicians in accurate diagnosis and future prediction of DSPN.

Improved nonparametric survival prediction using CoxPH, Random Survival Forest & DeepHit Neural Network

In recent times, time-to-event data such as time to failure or death is routinely collected alongside high-throughput covariates. These high-dimensional bioinformatics data often challenge classical survival models, which are either infeasible to fit or produce low prediction accuracy due to overfitting. To address this issue, the focus has shifted towards introducing a novel approaches for feature selection and survival prediction. In this article, we propose a new hybrid feature selection approach that handles high-dimensional bioinformatics datasets for improved survival prediction. This study explores the efficacy of four distinct variable selection techniques: LASSO, RSF-vs, SCAD, and CoxBoost, in the context of non-parametric biomedical survival prediction. Leveraging these methods, we conducted comprehensive variable selection processes. Subsequently, survival analysis models—specifically CoxPH, RSF, and DeepHit NN—were employed to construct predictive models based on the selected variables. Furthermore, we introduce a novel approach wherein only variables consistently selected by a majority of the aforementioned feature selection techniques are considered. This innovative strategy, referred to as the proposed method, aims to enhance the reliability and robustness of variable selection, subsequently improving the predictive performance of the survival analysis models. To evaluate the effectiveness of the proposed method, we compare the performance of the proposed approach with the existing LASSO, RSF-vs, SCAD, and CoxBoost techniques using various performance metrics including integrated brier score (IBS), concordance index (C-Index) and integrated absolute error (IAE) for numerous high-dimensional survival datasets. The real data applications reveal that the proposed method outperforms the competing methods in terms of survival prediction accuracy.

Fecal calprotectin and platelet count predict histologic disease activity in pediatric ulcerative colitis: results from a projection-predictive feature selection

Especially for pediatric patients, proxies of mucosal inflammation are needed. The Pediatric Ulcerative Colitis Activity Index (PUCAI) has been established to predict clinical and endoscopic disease activity. However, histologic inflammation might persist. We applied a special variable selection technique to predict histologic healing in pediatric ulcerative colitis (UC) as parsimoniously (but still as precisely) as possible. The retrospective analysis included data from two study cohorts, comprising 91 visits from 59 pediatric patients with UC. A Bayesian ordinal regression model was used in combination with a projection-predictive feature selection (PPFS) to identify a minimal subset of clinical and laboratory parameters sufficient for the prediction of histologic disease activity. Following the PPFS, CEDATA-GPGE patient registry data were analyzed to investigate the relevance of the selected predictors in relation to PUCAI and Physician Global Assessment (PGA) in up to 6697 patient visits. Fecal calprotectin (FC) and platelet count were identified as the minimal subset of predictors sufficient for prediction of histologic disease activity in pediatric UC. FC and platelet count also appeared to be associated with increasing disease activity as measured by PUCAI and PGA in the CEDATA-GPGE registry. Based on the selected model, predictions can be performed with a Shiny web app. Conclusion: Our statistical approach constitutes a reproducible and objective tool to select a minimal subset of the most informative parameters to predict histologic inflammation in pediatric UC. A Shiny app shows how physicians may predict the histologic activity in a user-friendly way using FC and platelet count. To generalize the findings, further prospective studies will be needed.What is Known:• Histologic healing is a major endpoint in the therapy of ulcerative colitis (UC).• The PUCAI score has been established to predict disease activity in pediatric UC but is not suitable for the prediction of histologic healing.What is New:• Our Bayesian ordinal regression model in combination with a projection-predictive feature selection is a reproducible and objective tool to select the minimal subset of clinical and laboratory parameters to predict histologic inflammation in pediatric UC.• Histologic inflammation in pediatric UC can be non-invasively predicted based on the combination of fecal calprotectin levels and platelet count.

Variable selection using data splitting and projection for principal fitted component models in high dimension

Sufficient dimension reduction (SDR) is such an effective way to detect nonlinear relationship between response variable and covariates by reducing the dimensionality of covariates without information loss. The principal fitted component (PFC) model is a way to implement SDR using some class of basis functions, however the PFC model is not efficient when there are many irrelevant or noisy covariates. There have been a few studies on the selection of variables in the PFC model via penalized regression or sequential likelihood ratio test. A novel variable selection technique in the PFC model has been proposed by incorporating a recent development in multiple testing such as mirror statistics and random data splitting. It is highlighted how we construct a mirror statistic in the PFC model using the idea of projection of coefficients to the other space generated from data splitting. The proposed method is superior to some existing methods in terms of false discovery rate (FDR) control and applicability to high-dimensional cases. In particular, the proposed method outperforms other methods as the number of covariates tends to be getting larger, which would be appealing in high dimensional data analysis. Simulation studies and analyses of real data sets have been conducted to show the finite sample performance and the gain that it yields compared to existing methods.

Enhancing corn quality prediction: Variable selection and explainable AI in spectroscopic analysis

This study addresses the challenge of effectively selecting relevant variables and providing interpretable insights in spectroscopic analysis (1100–2498 nm) of corn quality (moisture, fat, protein, and starch), incorporating variable selection techniques and explainable artificial intelligence (AI). Three variable selection algorithms were used to select 36 important variables, and through combinatorial optimization, only 11 common wavelengths were chosen. Partial least squares regression (PLSR) models were developed using the full spectral range, individual variables, and common variables. Based on the root mean square errors of prediction (RMSEP), the PLSR models with common variables outperformed individual feature variables for starch (0.22 % vs. 0.24 %) and were comparable for oil (0.05 % vs. 0.05 %), water (0.02 % vs. 0.01 %) and protein (0.12 % vs. 0.09 %). The SHapley Additive exPlanation (SHAP) method was employed to explain the PLSR model and assess the contribution of common variables. This combined approach significantly improved the accuracy and interpretability of corn prediction models, providing trust in the results and facilitating the understanding of the relationship between spectral features and corn quality attributes.