Final Estimator Research Articles

Complexities of feature-based learning systems, with application to reservoir computing

This paper studies complexity measures of reservoir systems. For this purpose, a more general model that we call a feature-based learning system, which is the composition of a feature map and of a final estimator, is studied. We study complexity measures such as growth function, VC-dimension, pseudo-dimension and Rademacher complexity. On the basis of the results, we discuss how the unadjustability of reservoirs and the linearity of readouts can affect complexity measures of the reservoir systems. Furthermore, some of the results generalize or improve the existing results.

Robust causal inference for point exposures with missing confounders

AbstractLarge observational databases are often subject to missing data. As such, methods for causal inference must simultaneously handle confounding and missingness; surprisingly little work has been done at this intersection. Motivated by this, we propose an efficient and robust estimator of the causal average treatment effect from cohort studies when confounders are missing at random. The approach is based on a novel factorization of the likelihood that, unlike alternative methods, facilitates flexible modelling of nuisance functions (e.g., with state‐of‐the‐art machine learning methods) while maintaining nominal convergence rates of the final estimators. Simulated data, derived from an electronic health record‐based study of the long‐term effects of bariatric surgery on weight outcomes, verify the robustness properties of the proposed estimators in finite samples. Our approach may serve as a theoretical benchmark against which ad hoc methods may be assessed.

Explainable Ensemble Learning and Multilayer Perceptron Modeling for Compressive Strength Prediction of Ultra-High-Performance Concrete.

The performance of ultra-high-performance concrete (UHPC) allows for the design and creation of thinner elements with superior overall durability. The compressive strength of UHPC is a value that can be reached after a certain period of time through a series of tests and cures. However, this value can be estimated by machine-learning methods. In this study, multilayer perceptron (MLP) and Stacking Regressor, an ensemble machine-learning models, is used to predict the compressive strength of high-performance concrete. Then, the ML model's performance is explained with a feature importance analysis and Shapley additive explanations (SHAPs), and the developed models are interpreted. The effect of using different random splits for the training and test sets has been investigated. It was observed that the stacking regressor, which combined the outputs of Extreme Gradient Boosting (XGBoost), Category Boosting (CatBoost), Light Gradient Boosting Machine (LightGBM), and Extra Trees regressors using random forest as the final estimator, performed significantly better than the MLP regressor. It was shown that the compressive strength was predicted by the stacking regressor with an average R2 score of 0.971 on the test set. On the other hand, the average R2 score of the MLP model was 0.909. The results of the SHAP analysis showed that the age of concrete and the amounts of silica fume, fiber, superplasticizer, cement, aggregate, and water have the greatest impact on the model predictions.

Postfilter for Dual Channel Speech Enhancement Using Coherence and Statistical Model-Based Noise Estimation.

A multichannel speech enhancement system usually consists of spatial filters such as adaptive beamformers followed by postfilters, which suppress remaining noise. Accurate estimation of the power spectral density (PSD) of the residual noise is crucial for successful noise reduction in the postfilters. In this paper, we propose a postfilter utilizing proposed a posteriori speech presence probability (SPP) and noise PSD estimators, which are based on both the coherence and the statistical models. We model the coherence-based a posteriori SPP as a simple function of the magnitude of coherence between two microphone signals and combine it with a single-channel SPP based on statistical models. The coherence-based estimator for the PSD of the noise remaining in the beamformer output in the presence of speech is derived using the pseudo-coherence considering the effect of the beamformers, which is used to construct the coherence-based noise PSD estimator. Then, the final noise PSD estimator is obtained by combining the coherence-based and statistical model-based noise PSD estimators with the proposed SPP. The spectral gain function is also modified, incorporating the proposed SPP. Experimental results demonstrate that the proposed method led to more accurate noise PSD estimation and perceptual evaluation of speech quality scores in various diffuse noise environments, and did not degrade the speech quality under the presence of directional interference, although the proposed method utilizes the coherence information.

Adaptive Huber trace regression with low-rank matrix parameter via nonconvex regularization

In this paper, we consider the adaptive Huber trace regression model with matrix covariates. A non-convex penalty function is employed to account for the low-rank structure of the unknown parameter. Under some mild conditions, we establish an upper bound for the statistical rate of convergence of the regularized matrix estimator. Theoretically, we can deal with heavy-tailed distributions with bounded (1+δ)-th moment for any δ>0. Furthermore, we derive the effect of the adaptive parameter on the final estimator. Some simulations, as well as a real data example, are designed to show the finite sample performance of the proposed method.

A new robust covariance matrix estimation for high‐dimensional microbiome data

SummaryMicrobiome data typically lie in a high‐dimensional simplex. One of the key questions in metagenomic analysis is to exploit the covariance structure for this kind of data. In this paper, a framework called approximate‐estimate‐threshold (AET) is developed for the robust basis covariance estimation for high‐dimensional microbiome data. To be specific, we first construct a proxy matrix , which is almost indistinguishable from the real basis covariance matrix . Then, any estimator satisfying some conditions can be used to estimate . Finally, we impose a thresholding step on to obtain the final estimator . In particular, this paper applies a Huber‐type estimator , and achieves robustness by only requiring the boundedness of 2+ moments for some . We derive the convergence rate of under the spectral norm, and provide theoretical guarantees on support recovery. Extensive simulations and a real example are used to illustrate the empirical performance of our method.

Enhancing machining accuracy of banana fiber-reinforced composites with ensemble machine learning

Through innovative predictive modeling, this study advances Abrasive Water Jet Machining (AWJM) for banana fiber-reinforced biocomposites. Utilizing Artificial Neural Network (ANN) and Long Short-Term Memory (LSTM) models, hyperparameters are meticulously tuned for predicting Surface Roughness (Ra), Material Removal Rate (MRR), and Kerf Angle (Ka). Optimal configurations are identified, such as 3–6-6–4-1, 3–5-5–4-1, and 3–5–4–4-1 for ANN models, and 250, 100, and 50 units for LSTM models. Beyond individual models, the study explores stacking ensemble models, merging ANN and LSTM strengths with a Linear Regression final estimator, showcasing robust performance validated with high R-squared values. Regression models from Design Expert 13 contribute to understanding process parameter-outcome relationships. Optimized input parameters offer insights into minimizing surface roughness and kerf angle while maximizing MRR. This holistic approach, integrating advanced machine learning and ensemble learning, enhances predictive accuracy for banana-reinforced biocomposites, providing a versatile framework for diverse materials processing applications.

Influencing factors for the human development index in West Java using geographically and temporally weighted regression kernel functions

Human Development Index (HDI) is a competitive index that serves as one of the crucial metrics for evaluating the effectiveness of enhancing the quality of human resources. HDI values from different areas can be compared. This study aims to spatially and temporally explore the HDI data from districts or cities in West Java and examine the factors that influence HDI in each of these districts or cities using the GTWR Great Circle Distance Fixed Kernels model. In this study, we used a combination of cross-sectional data from districts or cities in West Java and time series data with seven annual periods from 2015-2021. The GTWR Great Circle Distance Fixed Kernels model was expected to display coefficient values at each location and time simultaneously, providing more in-depth information and analysis results at each location and time. The analysis results using the GTWR Great Circle Distance Fixed Kernels model show that HDI in West Java carries a positive influence on the location and time. This finding should be of particular concern to the relevant government, particularly the factors presenting a natural effect on HDI based on location and time. The positive influence obtained by an area at a particular time will also have a positive impact on other regions, and if there is a negative influence, it will undoubtedly affect other regions as well. Analysis of the HDI model in West Java using the GTWR Great Circle Distance Fixed Exponential Kernel model also presents better results in comparison to the Global OLS model and the GTWR model without the Great Circle Fixed Exponential Kernel. The final parameter estimator results are displayed in the form of a geographic map to facilitate ease of understanding.

Fitting the Cox proportional hazards model to big data.

The semiparametric Cox proportional hazards model, together with the partial likelihood principle, has been widely used to study the effects of potentially time-dependent covariates on a possibly censored event time. We propose a computationally efficient method for fitting the Cox model to big data involving millions of study subjects. Specifically, we perform maximum partial likelihood estimation on a small subset of the whole data and improve the initial estimator by incorporating the remaining data through one-step estimation with estimated efficient score functions. We show that the final estimator has the same asymptotic distribution as the conventional maximum partial likelihood estimator using the whole dataset but requires only a small fraction of computation time. We demonstrate the usefulness of the proposed method through extensive simulation studies and an application to the UK Biobank data.

Oracle Efficient Estimation of Heterogeneous Dynamic Panel Data Models with Interactive Fixed Effects

We propose a two-step procedure to estimate a heterogeneous dynamic panel data model with interactive fixed effects. We establish the asymptotic properties of the estimators and show that the final estimator is oracle efficient. We also propose a specification test for the null hypothesis of homogeneous slopes and study the asymptotic properties of the test statistic under both local and global alternatives. Simulations demonstrate the fine performance of the estimator and test statistic. The new estimation and inference methods are applied to study the heterogeneous effects of minimum wage on employment across different counties in the United States. Our dynamic model suggests that the changes of employment range from about–1% to 1% when the minimum wage increases by 1%.

Robust Optimization and Inference on Manifolds

We propose a robust and scalable procedure for general optimization and inference problems on manifolds leveraging the classical idea of `median-of-means' estimation. This is motivated by ubiquitous examples and applications in modern data science in which a statistical learning problem can be cast as an optimization problem over manifolds. Being able to incorporate the underlying geometry for inference while addressing the need for robustness and scalability presents great challenges. We address these challenges by first proving a key lemma that characterizes some crucial properties of geometric medians on manifolds. In turn, this allows us to prove robustness and tighter concentration of our proposed final estimator in a subsequent theorem. This estimator aggregates a collection of subset estimators by taking their geometric median over the manifold. We illustrate bounds on this estimator via calculations in explicit examples. The robustness and scalability of the procedure is illustrated in numerical examples on both simulated and real data sets.

Machine Learning Algorithm: Texture Analysis in CNO and Application in Distinguishing CNO and Bone Marrow Growth-Related Changes on Whole-Body MRI.

The purpose of this study is to analyze the texture characteristics of chronic non-bacterial osteomyelitis (CNO) bone lesions, identified as areas of altered signal intensity on short tau inversion recovery (STIR) sequences, and to distinguish them from bone marrow growth-related changes through Machine Learning (ML) and Deep Learning (DL) analysis. We included a group of 66 patients with confirmed diagnosis of CNO and a group of 28 patients with suspected extra-skeletal systemic disease. All examinations were performed on a 1.5 T MRI scanner. Using the opensource 3D Slicer software version 4.10.2, the ROIs on CNO lesions and on the red bone marrow were sampled. Texture analysis (TA) was carried out using Pyradiomics. We applied an optimization search grid algorithm on nine classic ML classifiers and a Deep Learning (DL) Neural Network (NN). The model's performance was evaluated using Accuracy (ACC), AUC-ROC curves, F1-score, Positive Predictive Value (PPV), Mean Absolute Error (MAE) and Root-Mean-Square Error (RMSE). Furthermore, we used Shapley additive explanations to gain insight into the behavior of the prediction model. Most predictive characteristics were selected by Boruta algorithm for each combination of ROI sequences for the characterization and classification of the two types of signal hyperintensity. The overall best classification result was obtained by the NN with ACC = 0.91, AUC = 0.93 with 95% CI 0.91-0.94, F1-score = 0.94 and PPV = 93.8%. Between classic ML methods, ensemble learners showed high model performance; specifically, the best-performing classifier was the Stack (ST) with ACC = 0.85, AUC = 0.81 with 95% CI 0.8-0.84, F1-score = 0.9, PPV = 90%. Our results show the potential of ML methods in discerning edema-like lesions, in particular by distinguishing CNO lesions from hematopoietic bone marrow changes in a pediatric population. The Neural Network showed the overall best results, while a Stacking classifier, based on Gradient Boosting and Random Forest as principal estimators and Logistic Regressor as final estimator, achieved the best results between the other ML methods.

Generalized Partially Functional Linear Model with Unknown Link Function

In existing models with an unknown link function, the issue of predictors containing both multiple functional data and multiple scalar data has not been studied. To fill this gap, we propose a generalized partially functional linear model, which not only models the relationship between multiple scalar and functional predictors and responses, but also automatically estimates the link function. Specifically, we use the functional principal component analysis method to reduce the dimensionality of functional predictors, estimate the regression coefficients using the maximum likelihood estimation method, estimate the link function using the method of local linear regression, iteratively obtain the final estimator, and establish the asymptotic normality of the estimator. The asymptotic normality is illustrated through simulation experiments. Finally, the proposed model is applied to study the influence of environmental, economic, and medical levels on life expectancy in China. In the study, functional predictors are the daily air quality index, temperature, and humidity of 58 cities in 2020, and scalar predictors are GDP and the number of beds in hospitals. The experimental results indicate that the unknown link function model has a smaller prediction error and better performance than both the model with the known link function and the model without a link function.

Intelligent monitoring and diagnosis of telescope image quality

ABSTRACT The imaging quality of a telescope directly affects the reliability of astronomical research. Through the monitoring and diagnosis of imaging quality, the cause of the deterioration of imaging quality can be found in time, which is essential for ensuring the peaking performance of the telescope and high-quality imaging. Moreover, these operations are complex and crucial for achieving high-quality imaging of future giant telescope systems involving active optics, adaptive optics, and other advanced techniques. We propose a three-component method based on cutting-edge artificial intelligence technology to real-time monitor and efficiently diagnose the telescope image quality. The first component, an image quality monitoring system, monitors and outputs the telescope’s image quality. The second component is a query system with a knowledge graph, which outputs the node chains as the possible cause of poor image quality based on the input. The third component, a final estimator, uses the node parameter, which contains historical fault data and real-time updated data from sensors, to give the probability of each node chain. We construct and test the system in the Large Sky Area Multi-Object Fiber Spectroscopy Telescope.

Cardiac Failure Forecasting Based on Clinical Data Using a Lightweight Machine Learning Metamodel.

Accurate prediction of heart failure can help prevent life-threatening situations. Several factors contribute to the risk of heart failure, including underlying heart diseases such as coronary artery disease or heart attack, diabetes, hypertension, obesity, certain medications, and lifestyle habits such as smoking and excessive alcohol intake. Machine learning approaches to predict and detect heart disease hold significant potential for clinical utility but face several challenges in their development and implementation. This research proposes a machine learning metamodel for predicting a patient's heart failure based on clinical test data. The proposed metamodel was developed based on Random Forest Classifier, Gaussian Naive Bayes, Decision Tree models, and k-Nearest Neighbor as the final estimator. The metamodel is trained and tested utilizing a combined dataset comprising five well-known heart datasets (Statlog Heart, Cleveland, Hungarian, Switzerland, and Long Beach), all sharing 11 standard features. The study shows that the proposed metamodel can predict heart failure more accurately than other machine learning models, with an accuracy of 87%.

CT Angiography Radiomics Combining Traditional Risk Factors to Predict Brain Arteriovenous Malformation Rupture: a Machine Learning, Multicenter Study.

This study aimed to develop a machine learning model for predicting brain arteriovenous malformation (bAVM) rupture using a combination of traditional risk factors and radiomics features. This multicenter retrospective study enrolled 586 patients with unruptured bAVMs from 2010 to 2020. All patients were grouped into the hemorrhage (n = 368) and non-hemorrhage (n = 218) groups. The bAVM nidus were segmented on CT angiography images using Slicer software, and radiomic features were extracted using Pyradiomics. The dataset included a training set and an independent testing set. The machine learning model was developed on the training set and validated on the testing set by merging numerous base estimators and a final estimator based on the stacking method. The area under the receiver operating characteristic (ROC) curve, precision, and the f1 score were evaluated to determine the performance of the model. A total of 1790 radiomics features and 8 traditional risk factors were contained in the original dataset, and 241 features remained for model training after L1 regularization filtering. The base estimator of the ensemble model was Logistic Regression, whereas the final estimator was Random Forest. In the training set, the area under the ROC curve of the model was 0.982 (0.967-0.996) and 0.893 (0.826-0.960) in the testing set. This study indicated that radiomics features are a valuable addition to traditional risk factors for predicting bAVM rupture. In the meantime, ensemble learning can effectively improve the performance of a prediction model.

Combining estimators in interlaboratory studies and meta-analyses.

Many statistical methods (estimators) are available for estimating the consensus value (or average effect) and heterogeneity variance in interlaboratory studies or meta-analyses. These estimators are all valid because they are developed from or supported by certain statistical principles. However, no estimator can be perfect and must have error or uncertainty (known as estimator uncertainty). For a given dataset, the consensus value and heterogeneity variance given by different estimators can often differ significantly. Consequently, the choice of different estimators can affect the conclusion of an interlaboratory study or meta-analysis. However, there is no universally accepted metric for determining which estimator is optimal among a set of candidate estimators. Instead of selecting and using a single estimator, this paper proposes an estimator-averaging approach to combine a set of individual estimators. The final averaged estimator is a linear combination of individual estimators, which accounts for three sources of uncertainties including the estimator uncertainty. Monte Carlo simulations were performed to examine the long-run performance of four individual estimators and the proposed averaged estimators. A case study: the determination of the Newtonian constant of gravitation is presented, where ten individual estimators (eight frequentist weighted average methods and two Bayesian methods) are combined using the proposed estimator-averaging approach. This article is protected by copyright. All rights reserved.

A Statistical Relation between Mass, Age, and Velocity Dispersion in the Solar Neighborhood* *Based on the third data release of the Gaia space telescope.

The stellar kinematics of the Galactic disk are the main factors constraining disk formation and evolution processes in the Milky Way. In this paper, we investigate a statistical relation between stellar mass, age, and velocity dispersion for stars in the solar neighborhood. Age–velocity dispersion relations, with their applications, have been studied in detail before, but their correlation with mass was mostly neglected. To investigate this relation, we use proper motion data of more than 113,035 stars in the Galactic disk (with solar distances less than 150 pc) provided by the third data release of the Gaia mission, and for stellar mass and age, Gaia's Final Luminosity Age Mass Estimator is implemented. We analyze this data and the correlations between the parameters with random forest regression, which is an ensemble statistical-learning technique. Finally, we show that, by considering stellar mass alongside age, we can determine velocity dispersions with an average relative error and a mean absolute error of about 9% and 2.68 km s−1, respectively. We also find that the correlation of stellar age with velocity dispersion is 3–8 times more than mass, which varies due to the different stellar types and masses.

Non-Iterative Multiscale Estimation for Spatial Autoregressive Geographically Weighted Regression Models

Multiscale estimation for geographically weighted regression (GWR) and the related models has attracted much attention due to their superiority. This kind of estimation method will not only improve the accuracy of the coefficient estimators but also reveal the underlying spatial scale of each explanatory variable. However, most of the existing multiscale estimation approaches are backfitting-based iterative procedures that are very time-consuming. To alleviate the computation complexity, we propose in this paper a non-iterative multiscale estimation method and its simplified scenario for spatial autoregressive geographically weighted regression (SARGWR) models, a kind of important GWR-related model that simultaneously takes into account spatial autocorrelation in the response variable and spatial heterogeneity in the regression relationship. In the proposed multiscale estimation methods, the two-stage least-squares (2SLS) based GWR and the local-linear GWR estimators of the regression coefficients with a shrunk bandwidth size are respectively taken to be the initial estimators to obtain the final multiscale estimators of the coefficients without iteration. A simulation study is conducted to assess the performance of the proposed multiscale estimation methods, and the results show that the proposed methods are much more efficient than the backfitting-based estimation procedure. In addition, the proposed methods can also yield accurate coefficient estimators and such variable-specific optimal bandwidth sizes that correctly reflect the underlying spatial scales of the explanatory variables. A real-life example is further provided to demonstrate the applicability of the proposed multiscale estimation methods.

Distributed Adaptive Tracking Control for High-Order Nonlinear Multiagent Systems Over Event-Triggered Communication

In this article, we investigate the consensus problem for high-order nonlinear multiagent systems (MASs) with an uncertain leader under event-triggered communication. Compared with the existing consensus results for nonlinear MASs under event-triggered communication, the class of systems considered is more general, while achieving better performance in term of asymptotic tracking. To estimate the unknown parameters of the uncertain leader, distributed intermediate parameter estimators based on event-triggered communication mechanism are first introduced. To guarantee that high-order derivatives exist when applying the backstepping method, such estimators are further modified to obtain final estimators having a polynomial form by using the Hermite interpolation method. Moreover, novel high-order filters are proposed to generate local reference signals and to ensure the existence of high-order derivatives of the filter states. Based on the developed filters, a backstepping-based decentralized adaptive controller is developed. It is proved that consensus errors are asymptotically convergent with the developed method. Finally, simulation examples are provided to show the effectiveness of the proposed method.