Huber Loss Research Articles

ROBOUT: a conditional outlier detection methodology for high-dimensional data

This paper presents a methodology, called ROBOUT, to identify outliers conditional on a high-dimensional noisy information set. In particular, ROBOUT is able to identify observations with outlying conditional mean or variance when the dataset contains multivariate outliers in or besides the predictors, multi-collinearity, and a large variable dimension compared to the sample size. ROBOUT entails a pre-processing step, a preliminary robust imputation procedure that prevents anomalous instances from corrupting predictor recovery, a selection stage of the statistically relevant predictors (through cross-validated LASSO-penalized Huber loss regression), the estimation of a robust regression model based on the selected predictors (via MM regression), and a criterion to identify conditional outliers. We conduct a comprehensive simulation study in which the proposed algorithm is tested under a wide range of perturbation scenarios. The combination formed by LASSO-penalized Huber loss and MM regression turns out to be the best in terms of conditional outlier detection under the above described perturbed conditions, also compared to existing integrated methodologies like Sparse Least Trimmed Squares and Robust Least Angle Regression. Furthermore, the proposed methodology is applied to a granular supervisory banking dataset collected by the European Central Bank, in order to model the total assets of euro area banks.

Enhancing accuracy of physically informed neural networks for nonlinear Schrödinger equations through multi-view transfer learning

In recent years, significant research efforts have been dedicated to developing solutions for nonlinear partial differential equations (PDEs) with applications in physics. Among these equations, the NonLinear Schrödinger (NLS) equation plays a pivotal role in modern nonlinear science, governing the behavior of optical solitons and serving as a crucial model. However, the practical application of these elegant PDE models is hindered by the challenge of high dimensionality and computational cost. To address these challenges, we introduce a novel approach to enhance the training accuracy of physically informed neural networks (PINNs) for the NLS equation. Leveraging the concept of multi-view transfer learning, we utilize a pre-trained NLS model that considers different initial–boundary conditions as multiple views, enabling exploration of their contributions to the overall understanding and solution of the NLS equation. Additionally, we investigate the influence of different loss functions on PINN training results and find that employing the Huber Loss function yields superior fitting outcomes compared to traditional PINNs that use mean squared error (MSE) loss. We demonstrate significant improvements in precision and accuracy over traditional PINNs. Extensive experiments validate the effectiveness of our proposed optimization techniques. This work paves a novel way for solving complex physical systems using data-driven approaches.

Out-of-sample error estimation for M-estimators with convex penalty

Abstract A generic out-of-sample error estimate is proposed for $M$-estimators regularized with a convex penalty in high-dimensional linear regression where $(\boldsymbol{X},\boldsymbol{y})$ is observed and the dimension $p$ and sample size $n$ are of the same order. The out-of-sample error estimate enjoys a relative error of order $n^{-1/2}$ in a linear model with Gaussian covariates and independent noise, either non-asymptotically when $p/n\le \gamma $ or asymptotically in the high-dimensional asymptotic regime $p/n\to \gamma ^{\prime}\in (0,\infty )$. General differentiable loss functions $\rho $ are allowed provided that the derivative of the loss is 1-Lipschitz; this includes the least-squares loss as well as robust losses such as the Huber loss and its smoothed versions. The validity of the out-of-sample error estimate holds either under a strong convexity assumption, or for the L1-penalized Huber M-estimator and the Lasso under a sparsity assumption and a bound on the number of contaminated observations. For the square loss and in the absence of corruption in the response, the results additionally yield $n^{-1/2}$-consistent estimates of the noise variance and of the generalization error. This generalizes, to arbitrary convex penalty and arbitrary covariance, estimates that were previously known for the Lasso.

Multi-layer double deep Q network for active distribution network equivalent modeling with internal identification for EV loads

With more and more distributed renewable energy generations and diversified electric vehicle (EV) loads connected to active distribution networks (ADNs), the power interaction between the ADNs and backbone network becomes more complicated. Furthermore, it is difficult to exactly simulate the power flows in the ADN due to large amounts of branches without power measurement units. Consequently, this paper proposes an equivalent modeling for the ADN, consisting of a motor and a synthetic constant impedance (Z), constant current (I), constant power (P) load (ZIP) with an internal identification for EV loads. The equivalent modeling utilizes a multi-layer double deep Q network (MLDDQN) to track the dynamics of the original ADN using few power measurements in the boundary with high precision. In the MLDDQN, the types of the ZIP load and motor are selected from the load pool to obtain the dynamic equivalent modeling of the ADN in the first layer, and the weight of the model is determined to obtain the active power (P) and reactive power (Q) (PQ component) of motor and ZIP load in the second layer through the powerful feature extraction ability of deep learning and the optimization decision-making ability of reinforcement learning. Another layer is adopted to optimize the parameters of the equivalent EV to independently trace the characteristics of the original EV loads. The simulation results show that the proposed MLDDQN improves the sample selection and behavior value function evaluation in deep reinforcement learning through prior experience playback, huber loss function strategy and dueling network. Additionally, the precision and applicability of the equivalent modeling is tested with comparisons with traditional DDQN and DQN.

Robust estimators of functional single index models for longitudinal data

Functional single index model is a powerful model for longitudinal data, which effectively reduces the dimension of covariates. Inspired by robust calibration of computer models, we propose a robust estimator based on Huber loss for functional single index model with fixed dimension parameter. Furthermore, we also consider the case of high dimension parameter, and proposed a robust L 1 estimator of the unknown parameter based on adaptive lasso penalty. Theoretical properties including the asymptotic and non asymptotic results have also been investigated. Numerical studies including simulated experiments and an application to AIDS data verify the validity of the proposed estimators.

Robust Data-Driven Sparse Estimation of Distribution Factors Considering PMU Data Quality and Renewable Energy Uncertainty - Part I: Theory

Data-driven sparse estimation of distribution factors (DFs) facilitates online power flow sensitivity analysis for secure system operation. However, existing methods are vulnerable to time-varying non-Gaussian PMU measurement noise, bad data, and uncertain renewable energy sources (RESs). Moreover, they lack scalability to large-scale systems. This two-part paper proposes a robust and scalable sparse DF estimation framework considering PMU data quality and RES uncertainty. In this Part I, a novel Adaptive <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</i> -Lasso estimator with theoretically guaranteed robustness is proposed. It mitigates the impacts of measurement and RES uncertainties to yield accurate dominant DF estimates while promoting sparsity. The key idea is to integrate the robust statistics theory with sparse representation techniques, in particular the Huber loss function, adaptively-weighted <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> regularization, concomitant scale estimate, and pseudo-residuals. Two important robustness properties of this estimator are theoretically proven, i.e., the bounded influence function and the asymptotic consistency of dominant DF estimates given limited samples. The breakdown points of this estimator to measurement and RES uncertainties are derived. Test results validate that the proposed estimator allows accurate estimation without relying on power flow models or massive operating data. It achieves significantly superior robustness over existing methods in multiple scenarios.

Comprehensive Evaluation of Deep Neural Network Architectures for Parawood Pith Estimation

Accurate pith estimation is crucial for maintaining the quality of wood products. This study delves into deep learning techniques for precise Parawood pith estimation, employing popular convolutional neural networks (ResNet50, MobileNet, and Xception) with adapted regression heads. Through variations in regression functions, optimizers, and training epochs, the most effective models were pinpointed. Xception, coupled with Huber Loss regression, Nadam optimizer, and 200 epochs, showcased superior performance, achieving a 4.48 mm mean error (with a standard deviation of 3.69 mm) in Parawood. Notably, benchmarking on the Douglas Fir dataset yielded similar results (2.81 mm mean error, standard deviation: 1.57 mm). These findings underscore deep learning's potential for Parawood and Douglas Fir pith estimation, offering substantial benefits to wood industry quality control and production efficiency. By harnessing advanced machine learning techniques, this study advances wood industry processes, promoting the adoption of state-of-the-art technology in forestry and wood science. Doi: 10.28991/HIJ-2023-04-03-06 Full Text: PDF

Using Histogram Extracted from Satelite Imagery and Convolutional Network to Predict GRDP in Java Region

Inequality is one of the problems faced by all countries in the world, including Indonesia. The data used to measure development inequality between regions mostly use GRDP data. However, the GRDP data issued by BPS has a deficiency, it was released after the current year, and this figure is provisional. Therefore, a new data source is needed that can be used to estimate the value of economic activity so that it can be used to measure the level of inequality in development in a region. Nighttime Light (NTL) satellite imagery data can be an alternative to see socioeconomic activity in an area and have been shown to have a strong correlation with socioeconomic activity. In this study, we used VIIRS NTL satellite imagery data and Dynamic World land cover data to estimate GRDP. Rather than using statistical features for each area of interest, we use features in the form of histograms extracted from NTL images and land cover images for each area of interest. Using a histogram, we do not lose spatial information from satellite imagery. Then we proposed a deep learning method in the form of a one-dimensional convolutional neural network using the Huber loss function. This model obtained good precision with an R-square value of 0.8549, beating the baseline method with two-dimensional convolutional networks. The use of the Huber loss function can improve the performance of the model, which has a smaller total loss and has a smoother gradient.

Robust multitask diffusion normalized M-estimate subband adaptive filter algorithm over adaptive networks

In recent years, the multitask diffusion least mean square (MD-LMS) algorithm has been extensively applied in distributed parameter estimation and target tracking of multitask network. However, its performance degrades under colored input signals or impulsive noises. To overcome these two drawbacks, this paper introduces the subband adaptive filter (SAF) into a multitask network for the first time, and a robust multitask diffusion normalized M-estimate subband adaptive filtering (MD-NMSAF) algorithm is proposed, which solves the global network optimization problem based on the modified Huber (MH) function in a distributed manner, achieves robustness to impulsive noise, and significantly improves the convergence performance of the MD-LMS algorithm. Compared with existing robust multitask diffusion affine projection algorithms (MD-APAs), the computational complexity of the proposed MD-NMSAF algorithm is greatly reduced. In addition, we analyze the mean and mean-square stability conditions of MD-NMSAF, provide theoretical models characterizing the network mean square deviation (MSD) behaviors of transient and steady-state, and further verify their correctness by computer simulations. Simulation results under different network topologies, input signals, filter lengths and impulsive noise types fully demonstrate the performance advantages of the proposed MD-NMSAF algorithm over its competitors in terms of steady-state estimation accuracy and convergence speed.

WITHDRAWAL

AbstractIn order to reduce the noise of TEM signal, a denoising method, in which based on the Variational Mode Decomposition of parameters optimized in sections and Huber Loss Function Curve fitting (SOP‐GWOVMD‐CFH), is proposed in this paper. According to the characteristics of TEM signal changing rate and signal‐to‐noise ratio varying with time, a segmented signal processing method is proposed and the segmented criterion is given. On the basic of different segmentation signals, the Grey Wolf Optimization algorithm is used to optimize the parameters of VMD, and the IMF1 after the optimization decomposition is took as the filtered signal. To further improve the SNR of late‐time attenuation signal, Huber loss function was used to perform exponential curve fitting of Intrinsic Mode Function (IMF1) late data to improve the filtering robustness. Then the data are spliced to obtain the filtered transient electromagnetic curve by the endpoint continuation method. The Simulation and experimental results show that the proposed method can effectively remove the noise in the signal and restore TEM signal with the absolute error less than 0.1μv, which proves the effectiveness of the proposed method, and the denoising effect of the proposed method has better performance than that of other common methods.This article is protected by copyright. All rights reserved

Identifying Anomalous Behavior in Aircraft Landing Trajectory Using a Bayesian Autoencoder

Anomalous behavior during the aircraft landing phase can significantly increase the probability of adverse events. Automated anomaly detection during the landing phase can help aviation safety-related organizations to efficiently detect anomalous behavior and consider mitigation strategies. This paper develops a Bayesian autoencoder neural network model to identify anomalous behavior in landing trajectories by reconstructing the flight data because the reconstruction error is larger for anomalous flights. Different loss functions, such as Huber loss, mean squared error loss, and least trimmed squares are investigated to construct the Bayesian autoencoder model; and their performances are compared using different measures: the mean of the reconstruction error, the standard deviation of the reconstruction error, and both the mean and standard deviation of the reconstruction error. Different loss function-based models show differences in performance, depending on which measure is used for anomaly detection; among all the options considered, one of the Huber loss options appears to give the best performance, as indicated by the F1 score. Furthermore, the mean and standard deviation of the reconstruction error for a single flight are used to identify the time of occurrence and the flight parameters related to anomalous behavior.

Bi‐directional LSTM based channel estimation in 5G massive MIMO OFDM systems over TDL‐C model with Rayleigh fading distribution

SummaryIn this work, a deep learning (DL)‐based massive multiple‐input multiple‐output (mMIMO) orthogonal frequency division multiplexing (OFDM) system is investigated over the tapped delay line type C (TDL‐C) model with a Rayleigh fading distribution at frequencies ranging from 0.5 to 100 GHz. The proposed bi‐directional long short‐term memory (Bi‐LSTM) channel state information (CSI) estimator uses online learning during training and offline learning during the practical implementation phase. The design of the estimator takes into account situations in which prior knowledge of channel statistics is limited and targets excellent performance, even with limited pilot symbols (PS). Three separate loss functions (mean square logarithmic error [MSLE], Huber, and Kullback–Leibler Distance [KLD]) are assessed in three classification layers. The symbol error rate (SER) and outage probability performance of the proposed estimator are evaluated using a number of optimization techniques, such as stochastic gradient descent (SGD), momentum, and the adaptive gradient (AdaGrad) algorithm. The Bi‐LSTM‐based CSI estimator is trained considering a specific number of PS. It can be readily seen that by incorporating a cyclic prefix (CP), the system becomes more resilient to channel impairments, resulting in a lower SER. Simulations show that the SGD optimization approach and Huber loss function‐trained Bi‐LSTM‐based CSI estimator have the lowest SER and very high estimation accuracy. By using deep neural networks (DNNs), the Bi‐LSTM method for CSI estimation achieves a superior channel capacity (in bps/Hz) at 10 dB than long short‐term memory (LSTM) and other conventional CSI estimators, such as minimum mean square error (MMSE) and least squares (LS). The simulation results validate the analytical results in the study.

Convex Relaxation Approaches to Robust RSS-TOA Based Source Localization in NLOS Environments

For handling unreliable datasets contaminated by the non-line-of-sight (NLOS) bias errors, this correspondence statistically robustifies the traditional least squares type hybrid received signal strength and time-of-arrival location estimator using the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> and Huber loss functions. The two robust formulations are then tackled via different convex relaxation approaches. Numerical results are presented to make fair comparisons with the commonly adopted balancing parameter based scheme, demonstrating the positioning accuracy superiority of the proposed methods in various NLOS environments.

Robust Calibration of Computer Models Based on Huber Loss

Robust Calibration of Computer Models Based on Huber Loss

Navigation Line Extraction Method for Broad-Leaved Plants in the Multi-Period Environments of the High-Ridge Cultivation Mode

Navigation line extraction is critical for precision agriculture and automatic navigation. A novel method for extracting navigation lines based on machine vision is proposed herein using a straight line detected based on a high-ridge crop row. Aiming at the low-level automation of machines in field environments of a high-ridge cultivation mode for broad-leaved plants, a navigation line extraction method suitable for multiple periods and with high timeliness is designed. The method comprises four sequentially linked phases: image segmentation, feature point extraction, navigation line calculation, and dynamic segmentation horizontal strip number feedback. The a* component of the CIE-Lab colour space is extracted to preliminarily extract the crop row features. The OTSU algorithm is combined with morphological processing to completely separate the crop rows and backgrounds. The crop row feature points are extracted using an improved isometric segmented vertical projection method. While calculating the navigation lines, an adaptive clustering method is used to cluster the adjacent feature points. A dynamic segmentation point clustering method is used to determine the final clustering feature point sets, and the feature point sets are optimised using lateral distance and point line distance methods. In the optimisation process, a linear regression method based on the Huber loss function is used to fit the optimised feature point set to obtain the crop row centreline, and the navigation line is calculated according to the two crop lines. Finally, before entering the next frame processing process, a feedback mechanism to calculate a number of horizontal strips for the next frame is introduced to improve the ability of the algorithm to adapt to multiple periods. The experimental results show that the proposed method can meet the efficiency requirements for visual navigation. The average time for the image processing of four samples is 38.53 ms. Compared with the least squares method, the proposed method can adapt to a longer growth period of crops.

A spatio-temporal forecasting model using optimally weighted graph convolutional network and gated recurrent unit for wind speed of different sites distributed in an offshore wind farm

Accurate wind speed forecasting plays an essential role in scheduling wind power generation. Currently, most existing models predict wind speed just based on temporal features and geographical information of wind turbine sites is usually neglected or not taken full use of, leading to poor prediction performance. To solve this issue, a novel spatio-temporal prediction model based on optimally weighted graph convolutional network (GCN) and gated recurrent unit (GRU) is proposed. First, data preprocessing based on tensor decomposition is applied to recover missing speed values. Second, geographic and dynamic time warping (DTW) distance are both calculated to construct an optimally weighted graph for different turbine sites, which not only takes fully into account the geographic information but also the temporal similarity of speed series. Third, based on the weighted graph, GCN is used to extract spatial features from speed series adequately. Afterwards, by inputting spatial features of each site into GRU, respectively, the spatio-temporal features can be effectively extracted. Finally, model parameters are solved by iteration with a Huber loss. To demonstrate the proposed model, performance indexes including RMSE, MAE, MAPE, R2 and TIC are calculated. Results show that the proposed model has the smallest prediction error of all sites for multi-step short-term forecasting compared with prevalent models. Therefore, the proposed model effectively enhances prediction accuracy by making full use of spatio-temporal features.

TGPred: efficient methods for predicting target genes of a transcription factor by integrating statistics, machine learning and optimization.

Four statistical selection methods for inferring transcription factor (TF)-target gene (TG) pairs were developed by coupling mean squared error (MSE) or Huber loss function, with elastic net (ENET) or least absolute shrinkage and selection operator (Lasso) penalty. Two methods were also developed for inferring pathway gene regulatory networks (GRNs) by combining Huber or MSE loss function with a network (Net)-based penalty. To solve these regressions, we ameliorated an accelerated proximal gradient descent (APGD) algorithm to optimize parameter selection processes, resulting in an equally effective but much faster algorithm than the commonly used convex optimization solver. The synthetic data generated in a general setting was used to test four TF-TG identification methods, ENET-based methods performed better than Lasso-based methods. Synthetic data generated from two network settings was used to test Huber-Net and MSE-Net, which outperformed all other methods. The TF-TG identification methods were also tested with SND1 and gl3 overexpression transcriptomic data, Huber-ENET and MSE-ENET outperformed all other methods when genome-wide predictions were performed. The TF-TG identification methods fill the gap of lacking a method for genome-wide TG prediction of a TF, and potential for validating ChIP/DAP-seq results, while the two Net-based methods are instrumental for predicting pathway GRNs.

Attention-Guided Huber Loss for Head Pose Estimation Based on Improved Capsule Network.

Head pose estimation is an important technology for analyzing human behavior and has been widely researched and applied in areas such as human-computer interaction and fatigue detection. However, traditional head pose estimation networks suffer from the problem of easily losing spatial structure information, particularly in complex scenarios where occlusions and multiple object detections are common, resulting in low accuracy. To address the above issues, we propose a head pose estimation model based on the residual network and capsule network. Firstly, a deep residual network is used to extract features from three stages, capturing spatial structure information at different levels, and a global attention block is employed to enhance the spatial weight of feature extraction. To effectively avoid the loss of spatial structure information, the features are encoded and transmitted to the output using an improved capsule network, which is enhanced in its generalization ability through self-attention routing mechanisms. To enhance the robustness of the model, we optimize Huber loss, which is first used in head pose estimation. Finally, experiments are conducted on three popular public datasets, 300W-LP, AFLW2000, and BIWI. The results demonstrate that the proposed method achieves state-of-the-art results, particularly in scenarios with occlusions.

RETRACTED: Uncertain regression model based on Huber loss function

This article has been retracted. A retraction notice can be found at https://doi.org/10.3233/JIFS-219433.

Robust Z-Estimators for Semiparametric Moment Condition Models.

In the present paper, we introduce a class of robust Z-estimators for moment condition models. These new estimators can be seen as robust alternatives for the minimum empirical divergence estimators. By using the multidimensional Huber function, we first define robust estimators of the element that realizes the supremum in the dual form of the divergence. A linear relationship between the influence function of a minimum empirical divergence estimator and the influence function of the estimator of the element that realizes the supremum in the dual form of the divergence led to the idea of defining new Z-estimators for the parameter of the model, by using robust estimators in the dual form of the divergence. The asymptotic properties of the proposed estimators were proven, including here the consistency and their asymptotic normality. Then, the influence functions of the estimators were derived, and their robustness is demonstrated.