Huber Loss Research Articles

Joint modelling location and scale with the LTS estimation method

The joint modelling of location and scale presents significant challenges in regression, particularly in handling outliers and variance heterogeneity. Robust estimation methods are crucial in this context to ensure reliable analyses. This paper introduces a novel joint location and scale model (JLSM-LTS) based on the least trimmed squares (LTS) estimation method, specifically designed to address these issues. We present a detailed estimation procedure for accurately determining JLSMLTS parameters. To evaluate its effectiveness, we perform extensive simulations and apply the model to a real dataset. For comparison, we include alternative models: the joint location and scale model with normal distribution assumptions (JLSMN) and the M-regression-based models (JLSM-M) using Huber and Tukey functions. The results demonstrate the robustness of JLSM-LTS in managing outliers and heteroscedasticity, highlighting its advantages in joint modelling scenarios and its potential to enhance regression analysis under challenging conditions.

CONVOLUTIONAL LSTM FOR EGG PRICES FORECASTING IN INDONESIA’S HIGH STUNTING PREVALENCE PROVINCE

The Sustainable Development Goals (SDGs) are a series of 17 goals fixed by the United Nations and adopted by 193 countries in 2015, including Indonesia. By 2030, to end all forms of malnutrition, targeting on stunting and wasting in children under 5 years of age is one of the targets from Goals Number 2. One affordable source of protein and nutrition used as a solution to overcome malnutrition problems such as stunting is eggs. Egg price modeling was carried out to see the affordability of prices for the community. Weekly dataset of egg price in NTT Province from 2018 to 2023 used to modeling with Convolutional LSTM. The Convolutional LSTM components are Adam optimizer, ReLU activation function, Huber loss function, with batch size and neurons of 32. The MAPE value obtained from the model is relatively small, with MAPE for training, validation, and testing of 1.97%, 1%, and 1.19% respectively. The results of egg price forecasting for December 11, 2023, to January 8, 2024, show that egg prices tend to continue to decline per week. Thus, a decrease in egg prices can be a good thing in providing more affordable nutrition for the community.

ConCave-Convex procedure for support vector machines with Huber loss for text classification

The classical support vector machine (SVM) adopts the linear Hinge loss whereas the least squares SVM (LS-SVM) employs the quadratically growing least squares loss function. The robust Ramp loss function is employed in Ramp loss SVM (RSVM) that truncates the Hinge loss function and becomes flat a specified point afterwards, thus, increases robustness to outliers. Recently proposed SVM with pinball loss (pin-SVM) utilizes pinball loss function that maximizes the margin between the class hyperplanes based on quantile distance. Huber loss function is the generalization of linear Hinge loss and quadratic loss. Huber loss solves sensitivity issues of least squares loss to noise and outlier. In this work, we employ the robust Huber loss function for SVM classification for improved generalization performance. The cost function of the proposed approach consists of one convex and one non-convex part, which might sometimes provide local optimum solution instead of a global optimum. We suggest a ConCave-Convex Procedure (CCCP) to resolve this issue. Additionally, the proximal cost is scaled for each class sample based on their class size to reduce the effect of the class imbalance problem. Thus, it can be claimed that the proposed approach incorporates class imbalance learning as well. Extensive experimental analysis establishes efficacy of the proposed method. Furthermore, a sequential minimal optimization (SMO) procedure for high dimensional HSVM is proposed and its performance is tested on two text classification datasets.

An attention-based weight adaptive multi-task learning framework for slab head shape prediction and optimization during the rough rolling process

During the hot rolling process, predicting and optimizing the slab head unstable deformation is crucial for improving product yield. However, the multivariate, strongly coupled, nonlinear characteristics of the hot rolling process pose a significant challenge to information extraction. In this paper, an attention-based weight adaptive multi-task learning framework (AWAMTL) is proposed for slab head unstable deformation prediction and optimization. Initially, the unstable deformation zone of the slab is represented using the defect width deviation, defect length, and defect area, and the process variables are converted into image data to construct a slab head shape dataset. Subsequently, a convolutional neural network (CNN) architecture that incorporates inter-block connections and attention mechanisms is designed for information extraction. A weight adaptive method multi-task learning method is introduced to balance the training process of three targets, in addition, the Huber loss function and branching network are incorporated into the model training. The experiment results demonstrate that the proposed data conversion method can improve information aggregation, and the AWAMTL framework enhances the model performance, which outperforms the other compared models. Furthermore, based on the proposed prediction model and optimization algorithm, the short stroke control process is optimized. For the defect width deviation, defect length and defect area, the mean improvement rate for all slabs are 10.30 %, 21.89 % and 23.50 %, respectively.

High-dimensional robust regression under heavy-tailed data: asymptotics and universality

Abstract We investigate the high-dimensional properties of robust regression estimators in the presence of heavy-tailed contamination of both the covariates and response functions. In particular, we provide a sharp asymptotic characterisation of M-estimators trained on a family of elliptical covariate and noise data distributions including cases where second and higher moments do not exist. We show that, despite being consistent, the Huber loss with optimally tuned location parameter δ is suboptimal in the high-dimensional regime in the presence of heavy-tailed noise, highlighting the necessity of further regularisation to achieve optimal performance. This result also uncovers the existence of a transition in δ as a function of the sample complexity and contamination. Moreover, we derive the decay rates for the excess risk of ridge regression. We show that, while it is both optimal and universal for covariate distributions with finite second moment, its decay rate can be considerably faster when the covariates’ second moment does not exist. Finally, we show that our formulas readily generalise to a richer family of models and data distributions, such as generalised linear estimation with arbitrary convex regularisation trained on mixture models.

A new outlier rejection approach for non-Lambertian photometric stereo

Photometric stereo (PS) has garnered increasing attention due to its adeptness in restoring local fine textures. However, non-Lambertian reflections present in almost all real-world objects limit the effectiveness of the Lambertian model for surface normal vector estimation. Although BRDF-based and deep learning-based methods have become mainstream, recent research shows that comparable accuracy can also be achieved through simple filtering of observed intensity values. Nevertheless, these methods only consider the relative bias of pixel values and require manual specification of the number of pixels to be culled and the number of iterations. To address these issues, this paper proposes corresponding improvement methods. Firstly, a weighted inter-relationship function (IRF) fused with Huber loss is introduced to robustly and effectively evaluate the abnormal degree of pixel value. Secondly, based on the IRF curve and histogram statistical analysis, the number of excluded pixels is adaptively calculated. Thirdly, linear equations are then constructed based on the photometric equations, and the maximum between-class variance method is employed to achieve a high degree of sparsity, enabling fast and effective normal vector estimation. Finally, to verify the effectiveness of the proposed algorithm for non-Lambertian PS vision, quantitative verification tests on the synthetic dataset and the open-source datasets “DiLiGenT” and “DiLiGenT-PI” in real-world scenarios, and qualitative assessment experiments on real metal roughness samples are conducted. The experimental results demonstrate that, compared with the position threshold and IRF methods, our algorithms not only significantly enhance the accuracy of normal vector solutions but also markedly improve the operational efficiency of the algorithm, laying a solid foundation for practical online applications. These results fully validate the correctness and effectiveness of the proposed algorithm and provide a reference for the further development of outlier removal algorithms.

A Squared Smoothing Newton Method for Semidefinite Programming

This paper proposes a squared smoothing Newton method via the Huber smoothing function for solving semidefinite programming problems (SDPs). We first study the fundamental properties of the matrix-valued mapping defined upon the Huber function. Using these results and existing ones in the literature, we then conduct rigorous convergence analysis and establish convergence properties for the proposed algorithm. In particular, we show that the proposed method is well-defined and admits global convergence. Moreover, under suitable regularity conditions, that is, the primal and dual constraint nondegenerate conditions, the proposed method is shown to have a super-linear convergence rate. To evaluate the practical performance of the algorithm, we conduct extensive numerical experiments for solving various classes of SDPs. Comparison with the state-of-the-art SDP solvers demonstrates that our method is also efficient for computing accurate solutions of SDPs. Funding: The research of D. Sun was supported in part by the Hong Kong Research Grants Council under Grant 15307523, and the research of K.-C. Toh was supported by the Ministry of Education, Singapore, under its Academic Research Fund Tier 3 grant call [MOE-2019-T3-1-010].

Layout Congestion Prediction Based on Regression-ViT

To accelerate the back-end design flow of integrated circuit (IC), numerous studies have made exploratory advancements in machine learning (ML) for electronic design automation (EDA). However, most research works are limited to deep learning (DL) models predominantly based on convolutional neural networks, and the models often suffer from poor generalization due to the scarcity of data. In this study, we propose the Double generative adversarial networks (D-GAN) model to enrich the dataset and propose the Regression Vision Transformer (R-ViT) model to predict layout congestion information. Compared with the baseline model, experimental results show improvements of 3.03% and 2.64% in Receiver Operating Characteristic-Area under Curve (ROC-AUC) and Precision-Recall Curve-Area under Curve (PRC-AUC) respectively. To further enhance the prediction accuracy of the model, an adaptive Huber loss function is designed to optimize the training process, resulting in an improvement of up to 11.03% in ROC-AUC compared with the baseline model. Lastly, extended experiments are conducted to study the effects of parameters and convolutional kernel size on performance, which find a better configuration.

Adaptive Weighted Bregman Huber Total Variation Method for Terahertz Computed Tomography

ABSTRACTTo improve the image quality of terahertz (THz) continuous wave (CW) computed tomography (CT) under sparse‐view projections, a sparse‐view adaptive‐weighted Bregman Huber total variation (AW‐BHTV) method is proposed and experimentally evaluated at 0.11 THz. The Huber function is served as the fidelity term and TV function is worked for the regularization term, optimized by the adaptive weights, incorporating with iterative algorithms containing the modified simultaneous algebraic reconstruction technique (MSART) and the iterative filtered back‐projection (FBP). Sparse‐view reconstruction experiments are carefully implemented via polystyrene (PS) foam samples: Sample 1 and Sample 2, respectively. Under 20 projection angles by MSART based AW‐BHTV, the values of rooted mean‐square‐error (RMSE), peak signal‐to‐noise ratio (PSNR), structure similarity (SSIM) and feature similarity (FSIM) are 0.0061, 22.1427, 0.8738 and 0.9902 for Sample 1 together with 0.0051, 22.9101, 0.8376 and 0.9885 for Sample 2 separately. All of the results above suggest that the proposed AW‐BHTV method can effectively protect image details and strengthen image quality in sparse‐view THz CW CT.

Remote physiological signal recovery with efficient spatio-temporal modeling.

Contactless physiological signal measurement has great applications in various fields, such as affective computing and health monitoring. Physiological measurements based on remote photoplethysmography (rPPG) are realized by capturing the weak periodic color changes. The changes are caused by the variation in the light absorption of skin surface during systole and diastole stages of a functioning heart. This measurement mode has advantages of contactless measurement, simple operation, low cost, etc. In recent years, several deep learning-based rPPG measurement methods have been proposed. However, the features learned by deep learning models are vulnerable to motion and illumination artefacts, and are unable to fully exploit the intrinsic temporal characteristics of the rPPG. This paper presents an efficient spatiotemporal modeling-based rPPG recovery method for physiological signal measurements. First, two modules are utilized in the rPPG task: 1) 3D central difference convolution for temporal context modeling with enhanced representation and generalization capacity, and 2) Huber loss for robust intensity-level rPPG recovery. Second, a dual branch structure for both motion and appearance modeling and a soft attention mask are adapted to take full advantage of the central difference convolution. Third, a multi-task setting for joint cardiac and respiratory signals measurements is introduced to benefit from the internal relevance between two physiological signals. Last, extensive experiments performed on three public databases show that the proposed method outperforms prior state-of-the-art methods with the Pearson's correlation coefficient higher than 0.96 on all three datasets. The generalization ability of the proposed method is also evaluated by cross-database and video compression experiments. The effectiveness and necessity of each module are confirmed by ablation studies.

Predicting actual crack size through crack signal obtained by advanced Flexible Eddy Current Sensor using ResNet integrated with CBAM and Huber loss function

This study presents an advanced FEC sensor, engineered by arranging coils in a co-directional current configuration. Moreover, boasting a compact design, the FEC sensor showcases significantly enhanced spatial resolution, enabling robust detection of small cracks even at low excitation frequencies and mitigating issues of overlapping in adjacent crack detection. Results indicate successful crack detection through voltage and phase measurements, albeit with phase signals demonstrating variation at specific excitation frequencies, complicating the determination of actual crack sizes. Consequently, a novel model is proposed to forecast actual crack sizes, leveraging experimental data from the FEC sensor system. This model integrates a Residual Neural Network (ResNet) architecture with a Convolutional Block Attention Module (CBAM) and utilizes the Huber loss function to minimize errors during model training. Comparative analysis underscores the superior performance of the proposed model in predicting crack length and depth compared to the standalone ResNet, particularly when utilizing the Huber loss function with a δ value of 1.0. Evaluation metrics, encompassing Mean Squared Error (MSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE), illustrate an average accuracy surpassing 95 % for crack size predictions. Consequently, the proposed model demonstrates remarkable performance, significantly reducing the time required to ascertain actual crack sizes by leveraging voltage and phase measurements.

Robust and stochastic sparse subspace clustering

Sparse subspace clustering (SSC) has been widely employed in machine learning and pattern recognition, but it still faces scalability challenges when dealing with large-scale datasets. Recently, stochastic SSC (SSSC) has emerged as an effective solution by leveraging the dropout technique. However, SSSC cannot robustly handle noise, especially non-Gaussian noise, leading to unsatisfactory clustering performance. To address the above issues, we propose a novel robust and stochastic method called stochastic sparse subspace clustering with the Huber function (S3CH). The key idea is to introduce the Huber surrogate to measure the loss of the stochastic self-expression framework, thus S3CH inherits the advantage of the stochastic framework for large-scale problems while mitigating sensitivity to non-Gaussian noise. In algorithms, an efficient proximal alternating minimization (PAM)-based optimization scheme is developed. In theory, the convergence of the generated sequence is rigorously proved. Extensive numerical experiments on synthetic and six real datasets validate the advantages of the proposed method in clustering accuracy, noise robustness, parameter sensitivity, post-hoc analysis, and model stability.

Multi-step prediction of offshore wind power based on Transformer network and Huber loss

In the context of the burgeoning expansion of renewable energy sources, the precise prediction of offshore wind power assumes a pivotal role in safeguarding the reliability, economic viability, and sustainable progression of offshore wind farms. The present study introduces a novel methodology for offshore wind power prediction, predicated upon the synergy of the Transformer network and Huber loss function. Empirical validation is conducted utilizing authentic data from a European offshore wind farm. The resulting analyses delineate a discernible superiority of the Transformer network over classical LSTM and GRU models in capturing the intricate long-term dependencies intrinsic to the time series. Furthermore, the inclusion of the Huber loss function effectively mitigates the challenges posed by the high volatility often characteristic of offshore wind power data. The study also demonstrates the beneficial integration of autoencoder reconstruction for denoising and slime mould optimization algorithm to augment prediction performance. Distinctively diverging from traditional single-step prediction paradigms, the multi-step prediction model constructed within this research offers a more comprehensive and precise prediction of wind power. Such an innovative approach represents a valuable contribution to the field, with tangible implications for the dependable operation and future advancement of wind power.

Multi-step prediction of ship heave motion using transformer-enhanced multi-scale CNN

To achieve precise multi-step heave motion prediction for active compensation control in marine equipment, an innovative approach that integrates an attention-fused multi-scale CNN with a Transformer encoder is introduced in this study. The dual-path CNN and point-wise convolution-enhanced Transformer encoder are designed to capture local features across various scales and the global characteristics of heave motion signals, respectively. Furthermore, a novel optimization objective which combines a slope factor-optimized Huber loss function with a maximum square distance loss function is proposed to better approximate the original signals at extreme points. The proposed model is trained and tested on simulated data under multiple sea conditions. In typical active compensation scenarios (Hs=3.5m), the proposed model demonstrates superior performance compared to baseline methods, achieving an RMSE of 0.0083 m, an MAE of 0.0066 m, and the upper whisker of the box plot for the prediction error is less than 0.02 m. These outcomes effectively satisfy the accuracy requirements for active compensation systems. The generalization test on real-ship collected data demonstrates excellent performance after fine-tuning. Additionally, the model is considered suitable for deployment in real-world applications due to its memory efficiency and fast inference speed.

Robust Matrix Completion with Heavy-Tailed Noise

This article studies noisy low-rank matrix completion in the presence of heavy-tailed and possibly asymmetric noise, where we aim to estimate an underlying low-rank matrix given a set of highly incomplete noisy entries. Though the matrix completion problem has attracted much attention in the past decade, there is still lack of theoretical understanding when the observations are contaminated by heavy-tailed noises. Prior theory falls short of explaining the empirical results and is unable to capture the optimal dependence of the estimation error on the noise level. In this article, we adopt an adaptive Huber loss to accommodate heavy-tailed noise, which is robust against large and possibly asymmetric errors when the parameter in the Huber loss function is carefully designed to balance the Huberization biases and robustness to outliers. Then, we propose an efficient nonconvex algorithm via a balanced low-rank Burer-Monteiro matrix factorization and gradient descent with robust spectral initialization. We prove that under merely a bounded second-moment condition on the error distributions, rather than the sub-Gaussian assumption, the Euclidean errors of the iterates generated by the proposed algorithm decrease geometrically fast until achieving a minimax-optimal statistical estimation error, which has the same order as that in the sub-Gaussian case. The key technique behind this significant advancement is a powerful leave-one-out analysis framework. The theoretical results are corroborated by our numerical studies. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Lightweight omni-dimensional dynamic convolution with spatial-spectral self-attention network for hyperspectral image classification

ABSTRACT Omni-dimensional dynamic convolution can adjust spatial sizes, the amount of input/output channels, and kernel size based on the characteristics of the input, thus improving the capacity to extract characteristics through convolution, resulting in increased computation and model complexity. As a result, it is challenging to deploy these models on resource-limited airborne high-spectral testing systems. This paper proposes a lightweight omni-dimensional dynamic convolution network as a possible solution to this issue. In this network, the LS2CM module is used for spatial-spectral information extraction, and the pyramidal residual network is the basic architecture for extracting features from shallow to deep layers, and lightweight omni-dimensional dynamic convolution is adopted in the convolution module. Overcoming the pyramidal structure can only extract local information, we proposed a new simple spatial-spectral self-attention mechanism to extract global information from hyperspectral. Besides, the Huber function as an activation function. The IP, WHU-Hi-LongKou, SA, and PU datasets were used to verify the accuracy of the suggested network model. The results show that the proposed lightweight network maintains an appropriate equilibrium among model complexity as well as accuracy.

Robust estimate for count time series using GLARMA models: An application to environmental and epidemiological data

The Generalized Linear Autoregressive Moving Average (GLARMA) model has been used in epidemiological studies to evaluate the impact of air pollutants on health. Due to the nature of the data, a robust approach for the GLARMA model is proposed here based on the robustification of the quasi-likelihood function. Outlying observations are bounded separately by weight functions on covariates and the Huber loss function on the response variable. Some technical issues related to the robust approach are discussed and a Monte Carlo study revealed that the robust approach is more reliable than the classic one for contaminated data with additive outliers. The real data analysis investigates the impact of PM10 in the number of deaths by respiratory diseases in Vitória, Brazil.

6DoF Object Pose and Focal Length Estimation from Single RGB Images in Uncontrolled Environments.

Accurate 6DoF (degrees of freedom) pose and focal length estimation are important in extended reality (XR) applications, enabling precise object alignment and projection scaling, thereby enhancing user experiences. This study focuses on improving 6DoF pose estimation using single RGB images of unknown camera metadata. Estimating the 6DoF pose and focal length from an uncontrolled RGB image, obtained from the internet, is challenging because it often lacks crucial metadata. Existing methods such as FocalPose and Focalpose++ have made progress in this domain but still face challenges due to the projection scale ambiguity between the translation of an object along the z-axis (tz) and the camera's focal length. To overcome this, we propose a two-stage strategy that decouples the projection scaling ambiguity in the estimation of z-axis translation and focal length. In the first stage, tz is set arbitrarily, and we predict all the other pose parameters and focal length relative to the fixed tz. In the second stage, we predict the true value of tz while scaling the focal length based on the tz update. The proposed two-stage method reduces projection scale ambiguity in RGB images and improves pose estimation accuracy. The iterative update rules constrained to the first stage and tailored loss functions including Huber loss in the second stage enhance the accuracy in both 6DoF pose and focal length estimation. Experimental results using benchmark datasets show significant improvements in terms of median rotation and translation errors, as well as better projection accuracy compared to the existing state-of-the-art methods. In an evaluation across the Pix3D datasets (chair, sofa, table, and bed), the proposed two-stage method improves projection accuracy by approximately 7.19%. Additionally, the incorporation of Huber loss resulted in a significant reduction in translation and focal length errors by 20.27% and 6.65%, respectively, in comparison to the Focalpose++ method.

Multi-task Support Vector Machine Classifier with Generalized Huber Loss

Multi-task Support Vector Machine Classifier with Generalized Huber Loss

Attention-Enhanced Bi-LSTM with Gated CNN for Ship Heave Multi-Step Forecasting

This study addresses the challenges of predicting ship heave motion in real time, which is essential for mitigating sensor–actuator delays in high-performance active compensation control. Traditional methods often fall short due to training on specific sea conditions, and they lack real-time prediction capabilities. To overcome these limitations, this study introduces a multi-step prediction model based on a Seq2Seq framework, training with heave data taken from various sea conditions. The model features a long-term encoder with attention-enhanced Bi-LSTM, a short-term encoder with Gated CNN, and a decoder composed of multiple fully connected layers. The long-term encoder and short-term encoder are designed to maximize the extraction of global characteristics and multi-scale short-term features of heave data, respectively. An optimized Huber loss function is used to improve the fitting performance in peak and valley regions. The experimental results demonstrate that this model outperforms baseline methods across all metrics, providing precise predictions for high-sampling-rate real-time applications. Trained on simulated sea conditions and fine-tuned through transfer learning on actual ship data, the proposed model shows strong generalization with prediction errors smaller than 0.02 m. Based on both results from the regular test and the generalization test, the model’s predictive performance is shown to meet the necessary criteria for active heave compensation control.