Outlier Suppression Research Articles

Attention Similarity Outlier Suppression with Hierarchical Feature Integration for Defect Segmentation

Defect segmentation holds significant importance in industrial applications. However, most existing algorithms are tailored to specific scenarios, lacking a unified defect segmentation algorithm for defect segmentation. Based on the analysis of the characteristics of defects, we aim to propose a unified transformer‐based defect segmentation algorithm. On the one hand, the defect foreground exhibits significant differences from the normal background. Nevertheless, the attention mechanism in the transformer involves all feature nodes in the feature map to the similarity score of one patch with all other patches. This approach may compromise the features of potential defect areas where the defect foreground significantly contrasts with the background. We introduce the Attention Similarity Outlier Suppression (ASOS) strategy to address this challenge. Our method is designed to select valid attention scores and filter out outlier scores for appropriate attention fusion, in which the feature of potential defect areas is not easily affected by background. On the other hand, the defect regions often have relatively small areas. We incorporate a Hierarchical Feature Integration (HFI) module in our framework to facilitate defect recognition at different scales. By progressively up‐sampling, features can achieve better integration to generate the final segmentation result. Extensive experiments have verified the effectiveness of our proposed method in defect segmentation tasks. Moreover, thorough ablation studies have demonstrated the efficacy of our algorithm, reinforcing its robust performance. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

A novel robust moving horizon estimator for discrete-time linear systems subject to measurement outliers

State estimation is a crucial problem in modern industries and has been widely applied across various fields. The performance of the estimator depends on the quality of the measurement data. Measurements being corrupted by outliers is becoming an unavoidable phenomenon that leads to degradation of estimator performance. It is critical to develop estimators with outlier suppression capabilities to mitigate the adverse impact of measurement outliers. In this paper, we propose an effective outlier suppression technique for discrete-time linear systems within the framework of moving horizon estimation (MHE). The proposed estimator solves the issues of poor estimation accuracy and low computational efficiency among the existing MHE-based outlier-robust estimators. Moreover, the proposed method allows us to not only achieve robust state estimation but also detect outliers. Specifically, we propose a set of least-squares cost functions and an outlier identification mechanism to implement the estimation process. Subsequently, the stability of the estimation error of the proposed estimator is demonstrated. The estimation error can achieve exponential convergence by choosing appropriate design parameters. Lastly, the proposed estimator is applied to target tracking simulations and compared with state-of-the-art outlier-robust estimation methods, confirming the effectiveness and superiority of the proposed estimator.

A crowdsourcing-based incremental learning framework for automated essays scoring

Automated Essay Scoring (AES) is a challenging topic in Natural Language Processing. Recently, deep learning models have achieved remarkable performance for the AES task. However, applying deep learning models to the AES system in practice is expensive when both data collection and model training are taken into consideration. This paper aims to tackle this problem by proposing the Crowdsourcing-based Automated Essay Scoring (CAES) framework. The proposed framework gradually collects data through crowdsourcing and incrementally trains the AES models. In particular, we propose the Incremental Learning with Dynamic Exemplar Herding (ILDEH) approach to simultaneously tackle catastrophic forgetting and concept drift. The proposed approach dynamically updates the exemplar set by the Dynamic Exemplar Herding algorithm to obtain the best approximation of the overall data distribution and selectively apply knowledge distillation on the model outputs by Linear Outlier Suppression loss to retain the learned knowledge. Moreover, we use a lightweight AES model for effective and efficient essay scoring. The experimental results show that our proposed ILDEH approach outperforms other strong baseline approaches for the AES task. Moreover, the CAES framework is able to steadily improve the AES performance in the crowdsourcing environment with only 10.6% training time of the conventional approach. Further analysis shows that one single CPU server can support daily updates of more than 300 AES models, which is sufficient for most practical AES systems.

Synthetic Aperture Anomaly Imaging for Through-Foliage Target Detection

The presence of foliage is a serious problem for target detection with drones in application fields such as search and rescue, surveillance, early wildfire detection, or wildlife observation. Visual as well as automatic computational methods, such as classification and anomaly detection, fail in the presence of strong occlusion. Previous research has shown that both benefit from integrating multi-perspective images recorded over a wide synthetic aperture to suppress occlusion. In particular, commonly applied anomaly detection methods can be improved by the more uniform background statistics of integral images. In this article, we demonstrate that integrating the results of anomaly detection applied to single aerial images instead of applying anomaly detection to integral images is significantly more effective and increases target visibility as well as precision by an additional 20% on average in our experiments. This results in enhanced occlusion removal and outlier suppression, and consequently, in higher chances of detecting targets that remain otherwise occluded. We present results from simulations and field experiments, as well as a real-time application that makes our findings available to blue-light organizations and others using commercial drone platforms. Furthermore, we outline that our method is applicable for 2D images as well as for 3D volumes.

A robust fusion bus frequency estimation method to improve frequency oscillation damping in power systems

AbstractThis paper proposes a new robust method for accurately and reliably estimating remote bus frequencies in a power system. Two different measurement sources for the remote bus frequency are considered, that is, the ideal Frequency Divider (FD) and the Synchronous Reference Frame Phase Locked Loops (SRF‐PLLs). Each measurement signal encounters different uncertainties and data quality issues. In this paper, both data sources are employed and fused together to better estimate the remote bus frequencies. To this end, the model structure of the power system is selected and an Unscented Kalman Filter (UKF) is utilized together with a fusion covariance intersection method to enhance the accuracy of the estimated bus frequency. Since the fusion estimation method fails in case of quality issues on both channels, a robust Generalized Maximum‐likelihood UKF (GM‐UKF) using a novel outlier detection criteria is developed. The impact of the resulting robust fusion filter on the estimation of the remote bus frequencies and on the performance of WAPSS, which makes use of the estimated frequencies as the feedback signal, is examined via simulations. The results demonstrate the excellent performance and reliability of the proposed method in dealing with noise filtering and outlier suppression while ensuring a high statistical efficiency.

A sea clutter suppression algorithm for over-the-horizon radar based on dictionary learning and subspace estimation

Detecting a ship target by over-the-horizon radar (OTHR) is seriously affected by sea clutter. This paper proposes a clutter suppression algorithm based on dictionary learning and subspace estimation (DLSE). First, the K-singular value decomposition (K-SVD) is used for dictionary learning and sparse representation of the sea clutter. Then, the orthogonal matching pursuit (OMP) is used to reconstruct the sea clutter component to suppress the outliers. Finally, the clutter covariance matrix is calculated using the reconstructed clutter spectrum, and the clutter subspace of the covariance matrix is estimated to achieve clutter suppression. In this paper, the K-SVD is improved to preserve the phase information of the complex signal. The sea clutter spectrum, after sparse representation and reconstruction, is not impacted by outliers in neighboring cells. Therefore, the clutter covariance matrix has a better estimate accuracy than existing subspace decomposition algorithm. The adaptive parameter estimate method also enhances the adaptability of the DLSE. The experimental results of real data demonstrate that the DLSE is as sophisticated as existing subspace decomposition algorithms while having a higher output signal-to-clutter-plus-noise ratio (SCNR) and peak-to-side lobe ratio (PSLR), improving about 2-5 dB. Results of outlier suppression attest to the robustness of the DLSE. Under various coherent integration times (CIT) and frequencies, the DLSE has the best clutter suppression performance among the comparison algorithms.

Centered Error Entropy-Based Variational Bayesian Adaptive and Robust Kalman Filter

In this brief, a centered error entropy based variational Bayesian adaptive and robust Kalman filter (CEEVBKF) is proposed to suppress outlier noise and estimate the unknown noise covariance adaptively. The derived CEEVBKF contains three steps: one-step prediction, centered error entropy (CEE) based outlier suppression, and variational Bayesian (VB) inference. The CEE criterion is first used to suppress outlier noise and obtain rough state estimation value, then they are set as a priori value in VB inference step for accurate a posteriori state estimation. The joint estimation of CEE and VB improves the iterative efficiency and reduces the parameter sensitivity. The simulation results show the effectiveness of CEEVBKF.

An innovative vigorous outlier recognition placed on LROAD for fix-amplitude impulsive noise

Due to large current applications on digital images in these recent years, outlier suppression is one of the primary stages for modern computer vision implementations thereupon there are tremendously invented for creating an efficient and practical outlier suppression, which ordinarily are composed of outlier recognition stage and outlier rebuilt stage. The localised rank ordered difference (LROAD) approach, which is progressed from rank-ordered absolute differences (ROAD), has been invented since 2016. Later, the LROAD approach evolved to be one of the efficient outlier recognition stages from its eminent effectiveness. The paper focus to propose the innovative vigorous outlier recognition placed on localised rank-ordered logarithmic differences (LROLD) approach, which is progressed from LROAD and rank-ordered logarithmic differences (ROLD), which is higher effectiveness than the ordinary LROAD, for applying on FAIN. From the computer experiments, which are examined on many depictions such as Girl, Pepper F16 and Lena, the innovative vigorous outlier recognition placed on LROLD approach has higher eminent effectiveness then the stage-of-art approach such as LROAD and ROAD approaches at numerous consistencies of FAIN.

A Robust Unscented M-Estimation-Based Filter for Vehicle State Estimation With Unknown Input

Longitudinal velocity, lateral velocity, and front wheel steering angle are crucial states for vehicle active safety features. However, direct measurement of these variables requires expensive measurement instruments and can be easily affected by vehicle nonlinear dynamics, outliers and noise pollution. Moreover, unknown inputs bring great challenges to their accurate estimation. Therefore, this paper develops a novel robust unscented M-estimation-based filter (RUMF) for vehicle state estimation with unknown driver steering torque. The nonlinear system model is constructed based on the vehicle dynamics model. Unscented transformation (UT) and statistical linearization are implemented to transform the nonlinear process and measurement function into a linear-like regression form with data redundancy to achieve outlier suppression. Then, an M-estimation-based iterated algorithm is designed to address process uncertainty and innovation and observation outliers for robust vehicle state estimation. The iteratively reweighted least-squares (IRLS) method based on the M-estimation methodology is utilized for unknown input estimation. Simulations and experimental results compared with extended Kalman filter (EKF) and particle filter (PF) have verified the effectiveness and robustness of the proposed algorithm.

Material classification from sparse spectral X-ray CT using vectorial total variation based on L infinity norm

The development of energy resolving photon counting detectors (PCD) has paved the way to spectral X-ray Computed Tomography (CT), with which one can simultaneously extract the energy dependence of a material's linear attenuation coefficient (LAC). Spectral CT has proved to be an advanced technique to classify materials based on their physical properties such as electron density ( ρ e ) and effective atomic number ( Z eff ). However, the application of spectral CT may be hindered by the poor image reconstruction quality when used for material identification in security screening where rapid scanning is required. Image reconstruction from few projections or low radiation exposure time would enable rapid scanning. The reconstruction quality for each energy bin may also be degraded since the division of photon counts into multiple energy bins naturally leads to higher noise levels. In this work, we explore how to perform accurate spectral CT reconstructions from such data, and propose L ∞ norm-based vectorial total variation (L ∞ -VTV) regularization that uses correlations between multiple energy bins. Using experimental data acquired with a custom laboratory instrument for spectral CT, the L ∞ -VTV is tested on “real life” phantoms consisting of materials in the range of 6 ≤ Z eff ≤ 15. For material classification from only 7 projections, the L ∞ -VTV gives the relative deviations of 3.5% for ρ e and 2.4% for Z eff , whereas the total nuclear variation (TNV) of another state-of-the-art joint reconstruction and the total variation (TV) yield those of 3.4% and 3.1%, and 3.8% and 4.2%, respectively. The L ∞ -VTV is now ready for use in security screening. • A joint reconstruction algorithm with L infinity norm improves image reconstruction. • The algorithm is robust to few projections and/or low photon counts. • Using correlations between multiple energy bins enhances material identification. • L infinity norm in the algorithm leads to suppression of outliers in gradient magnitudes. • High noise levels in specific energy bins can be alleviated using unaffected bins.

Robust moving total least squares: A technique for the reconstruction of measurement data in the presence of multiple outliers

This article is concerned with the reconstruction of contaminated measurement data based on the moving total least squares (MTLS) method, which is extensively applied to many engineering and scientific fields. Traditional MTLS method is lack of robustness and sensitive to the outliers in measurement data. Based on the framework of MTLS method, we proposed a robust MTLS method called RMTLS method by introducing a two-step pre-process to detect and remove the anomalous nodes in the support domain. The first step is an iterative regression procedure that combines with k-medoids clustering to automatically reduce the weight of anomalous node for a regression-based reference (curve or surface). Based on the distances between reference and discrete points, the second step adopts a density function defined by a sorted distance sequence to select the normal points without setting a threshold artificially. After the two-step pre-process, weighted total least square is performed on the selected point set to obtain the estimation value. By disposing of the anomalous nodes in each independent support domain, multiple outliers can be suppressed within the whole domain. Furthermore, the suppression of multiple continual outliers is possible by adopting asymmetric support domain and introducing previous estimation points. The proposed method shows great robustness and accuracy in reconstructing the simulation and experiment data.

Triplet Loss With Multistage Outlier Suppression and Class-Pair Margins for Facial Expression Recognition

Deep metric based triplet loss has been widely used to enhance inter-class separability and intra-class compactness of network features. However, the margin parameters in the triplet loss for current approaches are usually fixed and not adaptive to the variations among different expression pairs. Meanwhile, outlier samples like faces with confusing expressions, occlusion and large head poses may be introduced during the selection of the hard triplets, which may deteriorate the generalization performance of the learned features for normal testing samples. In this work, a new triplet loss based on class-pair margins and multistage outlier suppression is proposed for facial expression recognition (FER). In this approach, each expression pair is assigned with an order-insensitive or two order-aware adaptive margin parameters. While expression samples with large head poses or occlusion are firstly detected and excluded, abnormal hard triplets are discarded if their feature distances do not fit the model of normal feature distance distribution. Extensive experiments on seven public benchmark expression databases show that the network using the proposed loss achieves much better accuracy than that using the original triplet loss and the network without using the proposed strategies, and the most balanced performances among state-of-the-art algorithms in the literature.

A new robust delayless subband adaptive filtering algorithm with variable step sizes for active control of broadband noise

Conventional delayless subband active noise control (ANC) systems can be degraded by the unexpected impulsive disturbances in the reference signal and/or the residual error signals despite the fact that the step sizes have been normalized by the variances of subband signals. Another inherent drawback is that an undesirable delay is introduced into the weight update path by using analysis filter banks, which reduces the upper bound of the step sizes thus limits the convergence rate. This paper proposes a novel robust and effective ANC algorithm configured in the delayless subband architecture for broadband noise cancellation. Instead of real-time providing threshold(s) on the reference and/or error signal as applied in most of the existing ANC algorithms for robustness, an online tuning scheme of bounded step sizes for the adaptive learning is developed for better convergence and outlier suppression. Box-constraint and time-averaging scheme deployed in the step size tuning guarantee robustness without requiring very accurate a priori information of the noises which might be corrupted by the impulses in real-world applications. In particular, this paper presents the stability and convergence analysis of the proposed closed-loop ANC system. Moreover, the computational complexity advantage is justified when compared to other state-of-the-art robust ANC algorithms. Numerical simulations are performed to validate the enhanced performance of the proposed algorithm for various colored noises with or without impulsive interferences.

Robust Few-Shot Learning for User-Provided Data.

Few-shot learning (FSL) focuses on distilling transferrable knowledge from existing experience to cope with novel concepts for which the labeled data are scarce. A typical assumption in FSL is that the training examples of novel classes are all clean with no outlier interference. In many realistic applications where examples are provided by users, however, data are potentially noisy or unreadable. In this context, we introduce a novel research topic, robust FSL (RFSL), where we aim to address two types of outliers within user-provided data: the representation outlier (RO) and the label outlier (LO). Moreover, we introduce a metric for estimating robustness and use it to investigate the performance of several advanced methods to FSL when faced with user-provided outliers. In addition, we propose robust attentive profile networks (RapNets) to achieve outlier suppression. The results of a comprehensive evaluation of benchmark data sets demonstrate the shortcomings of current FSL methods and the superiority of the proposed RapNets when dealing with RFSL problems, establishing a benchmark for follow-up studies.

Application of Clustering Filter for Noise and Outlier Suppression in Optical Measurement of Structured Surfaces

In comparison to tactile sensors, optical techniques can provide a fast, nondestructive profile/areal surface measurement solution. Nonetheless, high measurement noise, unmeasured points, and outliers are often observed in optical measurement, particularly for structured surfaces. To alleviate their detrimental impacts on the characterization of surface topography as well as the examination of micro/nanoscale geometries, a post processing filtering technique, i.e., the clustering filter, which is essentially an iterative process to find the aggregation center of a cluster of points, is implemented. The clustering filter is particularly useful for noises and outlier suppression for optical measurement of structured surfaces due to its edge-preserving capability. Five surface samples with structured features are measured by an in-house developed dispersive interferometer and a commercial white light interferometer, thereafter the measured surface data are filtered by the clustering filter. Both noise and outliers are suppressed, which not only facilitates the visualization and characterization of surface topography, but also enables the accurate evaluation of local functional geometries.

A calibrated imputation method for secondary data analysis of survey data

AbstractIn practical survey sampling, missing data are unavoidable due to nonresponse, rejected observations by editing, disclosure control, or outlier suppression. We propose a calibrated imputation approach so that valid point and variance estimates of the population (or domain) totals can be computed by the secondary users using simple complete‐sample formulae. This is especially helpful for variance estimation, which generally require additional information and tools that are unavailable to the secondary users. Our approach is natural for continuous variables, where the estimation may be either based on reweighting or imputation, including possibly their outlier‐robust extensions. We also propose a multivariate procedure to accommodate the estimation of the covariance matrix between estimated population totals, which facilitates variance estimation of the ratios or differences among the estimated totals. We illustrate the proposed approach using simulation data in supplementary materials that are available online.

Ramp-loss nonparallel support vector regression: Robust, sparse and scalable approximation

Although the twin support vector regression (TSVR) has been extensively studied and diverse variants are successfully developed, when it comes to outlier-involved training set, the regression model can be wrongly driven towards the outlier points, yielding extremely poor generalization performance. To overcome such shortcoming, a Ramp-loss nonparallel support vector regression (RL-NPSVR) is proposed in this work. By adopting Ramp ε-insensitive loss function and another Ramp-type linear loss function, RL-NPSVR can not only explicitly filter noise and outlier suppression but also have an excellent sparseness. The non- convexity of RL-NPSVR is solved by concave–convex programming (CCCP). Because a regularized term is added into each primal problem by rigidly following the structural risk minimization (SRM) principle, CCCP actually solves a series of reconstructed convex optimizations which have the same formulation of dual problem as the standard SVR, so that computing inverse matrix is avoided and SMO-type fast algorithm can be used to accelerate the training process. Numerical experiments on various datasets have verified the effectiveness of our proposed RL-NPSVR in terms of outlier sensitivity, generalization ability, sparseness and scalability.

Outlier Suppression via Non-Convex Robust PCA for Efficient Localization in Wireless Sensor Networks

Due to the fact that outliers can degrade localization accuracy significantly in wireless sensor networks, we propose an outlier suppression approach via non-convex robust principal component analysis (Robust PCA). By introducing several non-convex penalty functions to approximate both the rank function and the sparse penalty function in the original Robust PCA problem, we establish a non-convex objective function and solve it efficiently by the augmented Lagrangian multiplier method and difference of convex programming technique. This framework creates the opportunity to safely perform localization with dimension reduction by standard PCA, which has well-known fragile characteristics in the presence of outliers. Thus, the localization algorithms obtained by sparse coding provide the benefits of computational efficiency in low-dimension. The device-free localization (DFL) experiment on a real-world data set shows the improvement in localization accuracy after standard PCA in the presence of outliers. A statistical distance, called the Wasserstein distance, is introduced to evaluate the processed result of outlier suppression, illustrating that the proposed approach can identify and eliminate outliers in DFL problems.

Hybrid scheme for accurate stereo matching

Stereo matching is a key technique in the field of 3D image and video processing and it plays an important role in a practical stereo vision system. Though stereo matching has been extensively studied and many techniques have been proposed, accurate and efficient stereo matching is still challenging and this issue is worthy of being further studied. In this paper, a novel stereo matching scheme is proposed by jointly exploiting segmentation information, disparity continuity constraints as well as the adaptive support-weight (ASW) technique aiming to acquire efficient and accurate matching results. The proposed scheme consists of two major steps, initial matching and disparity refinement. In the initial matching step, both the reference image and the target image are segmented followed by WTA-based matching, where a new matching cost is developed by jointly exploiting segmentation information, disparity continuity constraints as well as the ASW techniques. Compared with the original one in ASW, ours can reduce the errors in depth discontinuities, repetitive patterns and low texture regions effectively and improve the matching accuracy significantly. In the disparity refinement step, an effective comprehensive refinement scheme is designed by performing measures of occlusion detection, boundary disparity initialization, disparity plane fitting, outlier suppression and discontinuity amendment, which can further correct matching errors and improve the matching accuracy effectively. Experiments are conducted and the results show that the proposed scheme can acquire satisfactory results and outperforms most of the existing methods.

Accurate Dense Stereo Matching Based on Image Segmentation Using an Adaptive Multi-Cost Approach

This paper presents a segmentation-based stereo matching algorithm using an adaptive multi-cost approach, which is exploited for obtaining accuracy disparity maps. The main contribution is to integrate the appealing properties of multi-cost approach into the segmentation-based framework. Firstly, the reference image is segmented by using the mean-shift algorithm. Secondly, the initial disparity of each segment is estimated by an adaptive multi-cost method, which consists of a novel multi-cost function and an adaptive support window cost aggregation strategy. The multi-cost function increases the robustness of the initial raw matching costs calculation and the adaptive window reduces the matching ambiguity effectively. Thirdly, an iterative outlier suppression and disparity plane parameters fitting algorithm is designed to estimate the disparity plane parameters. Lastly, an energy function is formulated in segment domain, and the optimal plane label is approximated by belief propagation. The experimental results with the Middlebury stereo datasets, along with synthesized and real-world stereo images, demonstrate the effectiveness of the proposed approach.