Huber Loss Research Articles

A phase model using the Huber norm for estimating point spread function under frozen flow hypothesis

In astronomical observation and space exploration, the light of an object in the outer space is inevitably distorted by the atmospheric turbulence around the Earth before it reaches the ground-based telescope aperture. The target object so observed is degraded by unknown point spread functions (PSFs). Estimating the PSFs is thus crucial for ground-based astronomy. In order to know the PSFs, we first need to estimate the phase, i.e., the wavefront aberration of incoming light at telescope aperture. In this paper, under the frozen flow hypothesis, we propose a new model for estimating the phase in the case of single- and multi-layered atmospheric turbulence. The new model, which deploys the Huber norm as the regularizer, can be cast as a saddle-point problem and solved efficiently by existing well-developed optimization algorithms. We test a spatially-varying image deblurring problem by using the estimated PSFs, which demonstrates the compelling numerical performance of the new phase model.

Forecasting stock return volatility using a robust regression model

AbstractThis paper aims to accurately forecast stock return volatility based on a robust regression model. The robust regression model is developed by replacing the mean squared error (MSE) in the autoregressive (AR) model with the Huber loss function, and the resulting model is called the ARH model. The empirical results show that the ARH model displays significantly stronger predictive power than the AR benchmark model for different evaluation periods and forecasting horizons. From an asset allocation perspective, a mean–variance investor can obtain sizeable utility gains based on the volatility forecasts produced by the ARH model. Furthermore, we find that the superior performance of the ARH model comes from assigning small weights for the extreme values, which are mainly found during recessions and periods of high volatility. Finally, our results are robust to various settings.

Sparse and robust estimation with ridge minimax concave penalty

• Derived a novel penalty function with ridge and minimax concave penalty. • A simple algorithm was designed based on the local quadratic approximation . • Derived the forecasting and estimation error bounds of the proposed estimator. • Experiment reveals that our developed models get better performance. Feature selection is an important procedure that is used in data mining to extract valuable information from large quantities of data. Existing penalization methods use a single penalty function to select important features. However, these methods do not yield sufficiently accurate predictions and selection outcomes. Therefore, construction of a concise and efficient prediction model would be beneficial. In this study, we propose a novel penalty function using a ridge and minimax concave penalty to overcome the limitations of individual penalty functions. Furthermore, we introduce a robust penalized feature selection method with Huber loss function, which is implemented by a local approximation algorithm. The theoretical properties of the algorithm have been described. Simulated and real-world data analyses are used to demonstrate the efficacy of the proposed method.

Robust pairwise learning with Huber loss

Pairwise learning naturally arises from machine learning tasks such as AUC maximization, ranking, and metric learning. In this paper we propose a new pairwise learning algorithm based on the additive noise regression model, which adopts the pairwise Huber loss and applies effectively even to the situation where the noise only satisfies a weak moment condition. Owing to the robustness of Huber loss function, this new method is resistant to heavy-tailed errors or outliers in the response variable. We establish a comparison theorem to characterize the gap between the excess generalization error and the prediction error. We derive the error bounds and convergence rates under appropriate conditions. It is worth mentioning that all the results are established under the (1+ϵ)-th moment condition of the noise variable. It is rather weak particularly in the case of ϵ<1, which means the noise variable does not even admit a finite variance.

Regularization Parameter Selection for the Low Rank Matrix Recovery

A popular approach to recover low rank matrices is the nuclear norm regularized minimization (NRM) for which the selection of the regularization parameter is inevitable. In this paper, we build up a novel rule to choose the regularization parameter for NRM, with the help of the duality theory. Our result provides a safe set for the regularization parameter when the rank of the solution has an upper bound. Furthermore, we apply this idea to NRM with quadratic and Huber functions, and establish simple formulae for the regularization parameters. Finally, we report numerical results on some signal shapes by embedding our rule into the cross validation, which state that our rule can reduce the computational time for the selection of the regularization parameter. To the best of our knowledge, this is the first attempt to select the regularization parameter for the low rank matrix recovery.

The Huber Functions and Their Application to a Classification Problem

In the following paper a classification problem with two multivariate normally distributed classes is considered. The problem is solved in a case of an empirical real situation (a motors data) using the Karhunen-Loeve Transform and classifying functions based on estimators for unknown parameters of a multivariate normal distribution. We consider the maximum likelihood estimator and a robust one. The robust estimator bases on the Huber functions. The corresponding classifying functions (classifiers) are compared using the Leave-One-Out method.

Radar reflectivity assimilation using hourly cycling4D‐Varin the Met Office Unified Model

AbstractA new method has been developed to directly assimilate volume scans of radar reflectivity data with 4D‐Var in the Met Office Unified Model. The method has been demonstrated in the convective‐scale hourly‐cycling UKV forecast model. Reflectivity observations from 18 C‐band radars in the British Isles are assimilated. This article describes the method of observation processing and quality control, the observation operator, and assimilation method. The assimilation method uses a minimum threshold rainwater content in the forward operator to give sensitivity to reflectivity observations where there is no rain in the background. Furthermore, the use of the Huber norm in the observation penalty function allows the use of observations with large innovations in the assimilation. A change was made to the precipitation efficiency in the microphysics scheme of the linear perturbation forecast model to ensure stability of the scheme. A case‐study is presented which demonstrates how the inclusion of reflectivity observations enhances convergence through analysis increments to the wind field, leading to improvements to the location of convective precipitation features in the forecast. Two‐month trials for summer and winter seasons demonstrate significant improvements to rain forecasts in the nowcasting range.

Residual encoder-decoder up-sampling for structural preservation in noise removal

While denoising 3D MRI, structural preservation is a very critical process in the medical region. However, Rician noise in MRI has affected the image quality which is complicated for diagnosing the disease like a brain tumor. So the Rician noise removal process is introduced in this article using filters based method named Residual Encoder-Decoder Up-sampling Non-Similar Wassertein Generative Adversarial Network (REDUPNSWGAN) which preserves the structural similarity between the neighbor-hood slices in 3D configuration is utilized as GPU. Residual auto-encoders are connected with De-convolution processes that are sent to the Generator Net (GNET). Furthermore, to reduce the level of Huber loss, the perceptual loss for similarity is implemented by the extraction of the feature space using Average-pool Net (AVGNET) which is incorporated with Huber loss, signed structural non-similar loss and Wassertein Adversarial loss to form the REDUPNSWGAN loss. Experimentally, this proposed method shows better performance quality based on mean PSNR than REDWGANVGG19 and also time complexity based on training and testing period which shows better than REDWGANVGG19. Particularly, the proposed network Residual Encoder-Decoder Upsampling Non-Similar REDUPNSWGAN reduces the noise level and preserves the structural similarity.

The Mag.num core field model as a parent for IGRF-13, and the recent evolution of the South Atlantic Anomaly

We present the GFZ candidate field models for the 13{mathrm{th}} Generation International Geomagnetic Reference Field (IGRF-13). These candidates were derived from the Mag.num.IGRF13 geomagnetic core field model, which is constrained by Swarm satellite and ground observatory data from November 2013 to August 2019. Data were selected from magnetically quiet periods, and the model parameters have been obtained using an iteratively reweighted inversion scheme approximating a robust modified Huber norm as a measure of misfit. The root mean square misfit of the Mag.num.IGRF13 model to Swarm and observatory data is in the order of 3–5 nT for mid and low latitudes, with a maximum of 44 nT for the satellite east component data at high latitudes. The time-varying core field is described by order 6 splines and spherical harmonic coefficients up to degree and order 20. We note that the temporal variation of the core field component of the Mag.num.IGRF13 model is strongly damped and shows a smooth secular variation that suits well for the IGRF, where secular variation is represented as constant over 5-year intervals. Further, the external field is parameterised by a slowly varying part and a more rapidly varying part controlled by magnetic activity and interplanetary magnetic field proxies. Additionally, the Euler angles of the magnetic field sensor orientation are co-estimated. A widely discussed feature of the geomagnetic field is the South Atlantic Anomaly, a zone of weak and decreasing field strength stretching from southern Africa over to South America. The IGRF and Mag.num.IGRF13 indicate that the anomaly has developed a second, less pronounced eastern minimum at Earth’s surface since 2007. We observe that while the strong western minimum continues to drift westwards, the less pronounced eastern minimum currently drifts eastward at Earth’s surface. This does not seem to be linked to any eastward motion at the core–mantle boundary, but rather to intensity changes of westward drifting flux patches contributing to the observed surface field. Also, we report a sudden change in the secular variation measured at two South Atlantic observatories around 2015.0, which occurred shortly after the well-known jerk of 2014.0.

Minibatch least-squares reverse time migration in a deep-learning framework

Migration techniques are an integral part of seismic imaging workflows. Least-squares reverse time migration (LSRTM) overcomes some of the shortcomings of conventional migration algorithms by compensating for illumination and removing sampling artifacts to increase spatial resolution. However, the computational cost associated with iterative LSRTM is high and convergence can be slow in complex media. We implement prestack LSRTM in a deep-learning framework and adopt strategies from the data science domain to accelerate convergence. Our hybrid framework leverages the existing physics-based models and machine-learning optimizers to achieve better and cheaper solutions. Using a time-domain formulation, we find that minibatch gradients can reduce the computation cost by using a subset of total shots for each iteration. The minibatch approach not only reduces source crosstalk, but it is also less memory-intensive. Combining minibatch gradients with deep-learning optimizers and loss functions can improve the efficiency of LSRTM. Deep-learning optimizers such as adaptive moment estimation are generally well-suited for noisy and sparse data. We compare different optimizers and determine their efficacy in mitigating migration artifacts. To accelerate the inversion, we adopt the regularized Huber loss function in conjunction. We apply these techniques to 2D Marmousi and 3D SEG/EAGE salt models and find improvements over conventional LSRTM baselines. Our approach achieves higher spatial resolution in less computation time measured by various qualitative and quantitative evaluation metrics.

Efficient and robust state estimation: Application to a copolymerization process

AbstractPolymerization processes are highly non‐linear systems that require strict control of their dynamic operation to be competitive. The unscented Kalman filter is a filtering strategy that has shown a rewarding performance for non‐linear state estimation. Besides, filters based on robust statistics have been proposed to deal with the presence of outliers. However, reported robust filters have employed only the Huber M‐estimator as the loss function of the estimation problem. This work presents a new state‐estimation procedure based on the unscented transformation and robust statistics concepts. When outliers are present, estimates are more accurate than when using the conventional filter. In contrast to previous research, our methodology is also efficient when there are no outliers. The performances of different loss functions for solving the estimation problem are presented. The results show that redescending M‐estimators outperform the Huber function. The behaviour of the technique is analyzed for a copolymerization process.

Saliency aware image cropping with latent region pair

As one of the fundamental problems for image enhancing editing, image cropping seeks the best crop with high aesthetic quality and removes extraneous areas. Many recent deep learning methods have been proposed to address the problem, but they do not reveal the intrinsic mechanism of image cropping. In this paper, we explore the latent region pair and then fulfill its potential in our proposed deep learning methodology (SAIC-Net) with saliency map for automatic image cropping. For each image, a lightweight multi-scale feature extraction network is first adopted to produce deep and informative features. Then, the features of latent region pair (ROI and ROD) are aligned and refined by the proposed saliency-aware align operators and context channel attentions. Finally, hybrid loss composed by ranking loss and Huber loss is minimized when training our model. In our experiments, to reduce the searching space for candidate crops, we conduct a saliency-aware grid cropping candidates generation method to eliminate irrelevant crops. Afterwards, we provide a thorough ablation study to better figure out the superiority of each part in our method, and conduct user study against the state-of-the-art methods on a fraction of performance metrics. The quantitative and qualitative results on three benchmark datasets demonstrate the superiority of our SAIC-Net in the task of automatic cropping.

On Regularization Based Twin Support Vector Regression with Huber Loss.

Twin support vector regression (TSVR) is generally employed with varepsilon -insensitive loss function which is not well capable to handle the noises and outliers. According to the definition, Huber loss function performs as quadratic for small errors and linear for others and shows better performance in comparison to Gaussian loss hence it restrains easily for a different type of noises and outliers. Recently, TSVR with Huber loss (HN-TSVR) has been suggested to handle the noise and outliers. Like TSVR, it is also having the singularity problem which degrades the performance of the model. In this paper, regularized version of HN-TSVR is proposed as regularization based twin support vector regression (RHN-TSVR) to avoid the singularity problem of HN-TSVR by applying the structured risk minimization principle that leads to our model convex and well-posed. This proposed RHN-TSVR model is well capable to handle the noise as well as outliers and avoids the singularity issue. To show the validity and applicability of proposed RHN-TSVR, various experiments perform on several artificial generated datasets having uniform, Gaussian and Laplacian noise as well as on benchmark different real-world datasets and compare with support vector regression, TSVR, varepsilon -asymmetric Huber SVR, varepsilon -support vector quantile regression and HN-TSVR. Here, all benchmark real-world datasets are embedded with a different significant level of noise 0%, 5% and 10% on different reported algorithms with the proposed approach. The proposed algorithm RHN-TSVR is showing better prediction ability on artificial datasets as well as real-world datasets with a different significant level of noise compared to other reported models.

RCKF Cooperative Navigation Algorithm for Tightly Coupled Vehicle Ad Hoc Networks Based on Huber M Estimation

As a core technology of cooperative navigation, relative position sensing is crucial for vehicle intelligent driving. It plays a key role in the cooperative positioning algorithm of vehicle ad hoc networks (VANETs). However, because of system nonlinearity and colored noise, the acquisition of a posteriori information of cooperative positioning under the same hardware platform is usually limited to a certain accuracy. To address these problems, a vehicle cooperative positioning state estimation algorithm based on a robust cubature Kalman filter (RCKF) of Huber M estimation is herein proposed. The algorithm combines the cubature rule for nonlinear updating, converts the measurement equation into a linear regression problem, and uses M estimation to solve it. The colored noise is suppressed by reducing the weight of disturbed measurements by the Huber loss function. The experimental results of tightly coupled vehicle cooperative navigation show that compared with the extended Kalman filter (EKF) and cubature Kalman filter (CKF), the positioning accuracy of the RCKF proposed in this paper is improved by 26.04% and 27.10% respectively, which effectively improves the accuracy and robustness of relative position estimation. It provides a reference system quality control strategy for vehicle cooperative positioning solutions.

Enhancing Market Price Decision-Making in Fintech through A Busines¬s Intelligence Technique

The surge of Fintech data and its implications on informed decision-making within the transportation sector have spurred the need for advanced analytical frameworks. This study addresses the challenge of leveraging Fintech data's temporal dynamics to enhance predictive capabilities and decision-making. The methodologies encompass an AutoEncoder (AE) for spatial feature extraction and an Improved Gated Recurrent Unit (IGRU) to capture temporal dependencies. Additionally, the Huber loss function optimizes model parameters, particularly in handling outliers. Integrating these techniques, our study explores Fintech data's spatial and temporal patterns, contributing insights for transportation planners and Fintech industries. Results demonstrate the efficacy of AE in learning spatial features, while IGRU effectively captures temporal dependencies, enabling the prediction of Fintech data with enhanced accuracy. The application of Huber loss ensures robustness by mitigating outlier influence. By the study's end, the model's predictive capabilities foster informed decision-making, offering opportunities to enhance Fintech data quality, reduce congestion, and bolster road safety. Overall, this research underscores the significance of advanced machine learning methodologies in decoding Fintech data's intricacies, laying a foundation for data-driven decision-making in the transportation and Fintech sectors.

Modeling of daily confirmed Saudi COVID-19 cases using inverted exponential regression.

The coronavirus disease 2019 (COVID-19) pandemic caused by the coronavirus strain has had massive global impact, and has interrupted economic and social activity. The daily confirmed COVID-19 cases in Saudi Arabia are shown to be affected by some explanatory variables that are recorded daily: recovered COVID-19 cases, critical cases, daily active cases, tests per million, curfew hours, maximal temperatures, maximal relative humidity, maximal wind speed, and maximal pressure. Restrictions applied by the Saudi Arabia government due to the COVID-19 outbreak, from the suspension of Umrah and flights, and the lockdown of some cities with a curfew are based on information about COVID-15. The aim of the paper is to propose some predictive regression models similar to generalized linear models (GLMs) for fitting COVID-19 data in Saudi Arabia to analyze, forecast, and extract meaningful information that helps decision makers. In this direction, we propose some regression models on the basis of inverted exponential distribution (IE-Reg), Bayesian (BReg) and empirical Bayesian regression (EBReg) models for use in conjunction with inverted exponential distribution (IE-BReg and IE-EBReg). In all approaches, we use the logarithm (log) link function, gamma prior and two loss functions in the Bayesian approach, namely, the zero-one and LINEX loss functions. To deal with the outliers in the proposed models, we apply Huber and Tukey's bisquare (biweight) functions. In addition, we use the iteratively reweighted least squares (IRLS) algorithm to estimate Bayesian regression coefficients. Further, we compare IE-Reg, IE-BReg, and IE-EBReg using some criteria, such as Akaike's information criterion (AIC), Bayesian information criterion (BIC), deviance (D), and mean squared error (MSE). Finally, we apply the collected data of the daily confirmed from March 23 - June 21, 2020 with the corresponding explanatory variables to the theoretical findings. IE-EBReg shows good model for the COVID-19 cases in Saudi Arabia compared with the other models.

HB-PLS: A statistical method for identifying biological process or pathway regulators by integrating Huber loss and Berhu penalty with partial least squares regression

Gene expression data features high dimensionality, multicollinearity, and non-Gaussian distribution noise, posing hurdles for identification of true regulatory genes controlling a biological process or pathway. In this study, we integrated the Huber loss function and the Berhu penalty (HB) into partial least squares (PLS) framework to deal with the high dimension and multicollinearity property of gene expression data, and developed a new method called HB-PLS regression to model the relationships between regulatory genes and pathway genes. To solve the Huber-Berhu optimization problem, an accelerated proximal gradient descent algorithm with at least 10 times faster than the general convex optimization solver (CVX), was developed. Application of HB-PLS to recognize pathway regulators of lignin biosynthesis and photosynthesis in <i>Arabidopsis thaliana</i> led to the identification of many known positive pathway regulators that had previously been experimentally validated. As compared to sparse partial least squares (SPLS) regression, an efficient method for variable selection and dimension reduction in handling multicollinearity, HB-PLS has higher efficacy in identifying more positive known regulators, a much higher but slightly less sensitivity/(1-specificity) in ranking the true positive known regulators to the top of the output regulatory gene lists for the two aforementioned pathways. In addition, each method could identify some unique regulators that cannot be identified by the other methods. Our results showed that the overall performance of HB-PLS slightly exceeds that of SPLS but both methods are instrumental for identifying real pathway regulators from high-throughput gene expression data, suggesting that integration of statistics, machine leaning and convex optimization can result in a method with high efficacy and is worth further exploration.

Настройка робастных нейронных сетей для решения задачи классификации

The paper highlights the problem of building and adjusting robust neural networks applying different loss functions for solving the classification problem. The considered functions are those of Cauchy, Meshalkin, Geman-McCluer, Charbonnier and Tukey’s Biweight losses. The accuracy of classification is examined for the different values of outliers’ fraction, for several values of learning epochs count and for datasets with various sizes. For all obtained networks the parameter values that maximize the accuracy, are defined. The best practices for choosing the parameter values depending on epoch count are also defined for all the loss functions. The ordinary neural network (with quadratic loss) and the robust neural network applying the Huber loss are also considered. The analysis of the results shows that the use of robust approach can significantly increase the learning rate and the classification accuracy, however, choosing the incorrect parameter value can decrease the accuracy of classification.

Scale calibration for high-dimensional robust regression

We present a new method for robust high-dimensional linear regression when the scale parameter of the additive errors is unknown. The proposed estimator is based on a penalized Huber M-estimator, for which theoretical bounds on the estimation error have recently been proposed in high-dimensional statistics literature. However, the variance of the error term in the linear model is intricately connected to the optimal parameter used to define the shape of the Huber loss. Our main idea is to use an adaptive technique, based on Lepski’s method, to overcome the difficulties of solving a joint nonconvex optimization problem with respect to the location and scale parameters. Furthermore, by including a weight term in the definition of the M-estimator, our consistency results hold even when the covariates are heavy-tailed. We then derive asymptotic normality of a one-step estimator constructed from the penalized Huber estimator, which can be used to construct confidence regions for subsets of coordinates. The one-step estimator is shown to be semiparametrically efficient when the covariates are sub-exponential. Our results substantially generalize previous work on high-dimensional inference, derived under sub-Gaussian assumptions on both the covariate and error distributions.

Robust low-rank multiple kernel learning with compound regularization

Kernel learning is widely used in nonlinear models during the implementation of forecasting tasks in analytics. However, existing forecasting models lack robustness and accuracy. Therefore, in this study, a novel supervised forecasting approach based on robust low-rank multiple kernel learning with compound regularization is investigated. The proposed method extracts the benefits from robust regression, multiple kernel learning with low-rank approximation, and sparse learning systems. Unlike existing hybrid forecasting methods, which frequently combine different models in parallel, we embed a Huber or quantile loss function and a compound regularization composed of smoothly clipped absolute deviation and ridge regularizations in a support vector machine with predefined number of kernels. To select the optimal kernels, L1 penalization with positive constraint is also considered. The proposed model exhibits robustness, forecasting accuracy, and sparsity in the reproducing kernel Hilbert space. For computation, a simple algorithm is designed based on local quadratic approximation to implement the proposed method. Theoretically, the forecasting and estimation error bounds of the proposed estimators are derived under a null consistency assumption. Real data experiments using datasets from various scientific research fields demonstrate the superior performances of the proposed approach compared with other state-of-the-art competitors.