Occurrence Of Outliers Research Articles

Adaptive dual control with online outlier detection for uncertain systems

This paper proposes an adaptive dual control with outlier detection that is robust to the occurrence of outliers in uncertain systems. Outliers occasionally exist in system process noise and observation noise, which could cause poor parameter estimation and degraded control performance of uncertain systems. For this reason, we devise an online outlier detection mechanism to filter the outliers so as to enhance the parameter estimation of uncertain systems. The devised mechanism makes decisions on outlier detection via the generated predicted regions where the newly arriving data is expected to locate, and the predicted regions are updated in real-time according to the historical data. The detection mechanism is integrated into the design of adaptive dual control, which is derived based on the bicriterial method. Compared with classical dual control merely considering uncertainty in input and output data stream, we are the first to include the uncontrollable excitations into the structure of dual control to fit practical scenarios, and this inclusion also provides an extensive cover on outliers to be detected. The improved performance of the proposed approach is verified using a mathematical model through one-time simulation and Monte Carlo simulations under different conditions, and we also evaluate our method in the control of fermentation sterilization process for more convincing results.

Robust Pooling Through the Data Mode

The task of learning from point cloud data is always challenging due to the often occurrence of noise and outliers in the data. Such data inaccuracies can significantly influence the performance of state-of-the-art deep learning networks and their ability to classify or segment objects. While there are some robust deep learning approaches, they are computationally too expensive for real-time applications. This paper proposes a deep learning solution that includes a novel robust pooling layer which greatly enhances network robustness and performs significantly faster than state-of-the-art approaches. The proposed pooling layer looks for data a mode/cluster using two methods, RANSAC, and histogram, as clusters are indicative of models. We tested the pooling layer into frameworks such as Point-based and graph-based neural networks, and the tests showed enhanced robustness as compared to robust state-of-the-art methods.

Outlier detection for PIV statistics based on turbulence transport

The occurrence of data outliers in PIV measurements remains nowadays a problematic issue; their effective detection is relevant to the reliability of PIV experiments. This study proposes a novel approach to outliers detection from time-averaged three-dimensional PIV data. The principle is based on the agreement of the measured data to the turbulent kinetic energy (TKE) transport equation. The ratio between the local advection and production terms of the TKE along the streamline determines the admissibility of the inquired datapoint. Planar and 3D PIV experimental datasets are used to demonstrate that in the presence of outliers, the turbulent transport (TT) criterion yields a large separation between correct and erroneous vectors. The comparison between the TT criterion and the state-of-the-art universal outlier detection from Westerweel and Scarano (Exp Fluids 39:1096–1100, 2005) shows that the proposed criterion yields a larger percentage of detected outliers along with a lower fraction of false positives for a wider range of possible values chosen for the threshold.Graphical abstract

Performance Estimation in V2X Networks Using Deep Learning-Based M-Estimator Loss Functions in the Presence of Outliers

Recently, 5G networks have emerged as a new technology that can control the advancement of telecommunication networks and transportation systems. Furthermore, 5G networks provide better network performance while reducing network traffic and complexity compared to current networks. Machine-learning techniques (ML) will help symmetric IoT applications become a significant new data source in the future. Symmetry is a widely studied pattern in various research areas, especially in wireless network traffic. The study of symmetric and asymmetric faults and outliers (anomalies) in network traffic is an important topic. Nowadays, deep learning (DL) is an advanced approach in challenging wireless networks such as network management and optimization, anomaly detection, predictive analysis, lifetime value prediction, etc. However, its performance depends on the efficiency of training samples. DL is designed to work with large datasets and uses complex algorithms to train the model. The occurrence of outliers in the raw data reduces the reliability of the training models. In this paper, the performance of Vehicle-to-Everything (V2X) traffic was estimated using the DL algorithm. A set of robust statistical estimators, called M-estimators, have been proposed as robust loss functions as an alternative to the traditional MSE loss function, to improve the training process and robustize DL in the presence of outliers. We demonstrate their robustness in the presence of outliers on V2X traffic datasets.

Fast local adaptive multiscale image matching algorithm for remote sensing image correlation

Various studies have shown that image correlation calculated in the space domain outperforms frequency-based methods. However, such an approach usually requires great computational efforts, making it challenging to adopt for surveying fast moving processes like glaciers, particularly over wide areas. We present a local adaptive multiscale image matching algorithm (LAMMA), which repeatedly applies image correlation on grids of increasing spatial resolution and adapts the size of the interrogation area according to the local range of displacements. LAMMA allows reducing the number of calculi of several orders of magnitude and limits the occurrence of displacement outliers. We show an example of LAMMA application on Sentinel-2 images to measure glaciers flow of the Southern Patagonian Icefield, where LAMMA's runtime was comparable to that of frequency-based correlation. LAMMA's Matlab code is freely available on GitHub.

Fully Automated Gross Tumor Volume Delineation From PET in Head and Neck Cancer Using Deep Learning Algorithms.

The availability of automated, accurate, and robust gross tumor volume (GTV) segmentation algorithms is critical for the management of head and neck cancer (HNC) patients. In this work, we evaluated 3 state-of-the-art deep learning algorithms combined with 8 different loss functions for PET image segmentation using a comprehensive training set and evaluated its performance on an external validation set of HNC patients. 18F-FDG PET/CT images of 470 patients presenting with HNC on which manually defined GTVs serving as standard of reference were used for training (340 patients), evaluation (30 patients), and testing (100 patients from different centers) of these algorithms. PET image intensity was converted to SUVs and normalized in the range (0-1) using the SUVmax of the whole data set. PET images were cropped to 12 × 12 × 12 cm3 subvolumes using isotropic voxel spacing of 3 × 3 × 3 mm3 containing the whole tumor and neighboring background including lymph nodes. We used different approaches for data augmentation, including rotation (-15 degrees, +15 degrees), scaling (-20%, 20%), random flipping (3 axes), and elastic deformation (sigma = 1 and proportion to deform = 0.7) to increase the number of training sets. Three state-of-the-art networks, including Dense-VNet, NN-UNet, and Res-Net, with 8 different loss functions, including Dice, generalized Wasserstein Dice loss, Dice plus XEnt loss, generalized Dice loss, cross-entropy, sensitivity-specificity, and Tversky, were used. Overall, 28 different networks were built. Standard image segmentation metrics, including Dice similarity, image-derived PET metrics, first-order, and shape radiomic features, were used for performance assessment of these algorithms. The best results in terms of Dice coefficient (mean ± SD) were achieved by cross-entropy for Res-Net (0.86 ± 0.05; 95% confidence interval [CI], 0.85-0.87), Dense-VNet (0.85 ± 0.058; 95% CI, 0.84-0.86), and Dice plus XEnt for NN-UNet (0.87 ± 0.05; 95% CI, 0.86-0.88). The difference between the 3 networks was not statistically significant (P > 0.05). The percent relative error (RE%) of SUVmax quantification was less than 5% in networks with a Dice coefficient more than 0.84, whereas a lower RE% (0.41%) was achieved by Res-Net with cross-entropy loss. For maximum 3-dimensional diameter and sphericity shape features, all networks achieved a RE ≤ 5% and ≤10%, respectively, reflecting a small variability. Deep learning algorithms exhibited promising performance for automated GTV delineation on HNC PET images. Different loss functions performed competitively when using different networks and cross-entropy for Res-Net, Dense-VNet, and Dice plus XEnt for NN-UNet emerged as reliable networks for GTV delineation. Caution should be exercised for clinical deployment owing to the occurrence of outliers in deep learning-based algorithms.

Robust estimation for multivariate wrapped models

A weighted likelihood technique for robust estimation of multivariate Wrapped distributions of data points scattered on a p-dimensional torus is proposed. The occurrence of outliers in the sample at hand can badly compromise inference for standard techniques such as maximum likelihood method. Therefore, there is the need to handle such model inadequacies in the fitting process by a robust technique and an effective downweighting of observations not following the assumed model. Furthermore, the employ of a robust method could help in situations of hidden and unexpected substructures in the data. Here, it is suggested to build a set of data-dependent weights based on the Pearson residuals and solve the corresponding weighted likelihood estimating equations. In particular, robust estimation is carried out by using a Classification EM algorithm whose M-step is enhanced by the computation of weights based on current parameters’ values. The finite sample behavior of the proposed method has been investigated by a Monte Carlo numerical study and real data examples.

Privacy protection with heavy-tailed noise for linear dynamical systems

Privacy protection in linear dynamical systems is investigated in this paper. A standard mechanism employed in systems and control literature is to mask private data by adding Gaussian noise. A shortcoming of this mechanism is that the occurrence of outliers is vulnerable. The goal of this paper is to present a novel mechanism that can hide outliers. The key idea is to utilize stably distributed noise, which has the following two preferable properties for this purpose; One is its heavy-tailed distribution that is beneficial to hide extreme values including scale-free data. The other is its closedness with respect to addition similarly to Gaussian, which enables us to design filter/controller under privacy requirements based on linear control theory. From a theoretical point of view, we quantify the privacy level of the proposed mechanism in terms of differential privacy. The derivation is nontrivial because the density function of the stable distribution has no analytic expression.

Robust fitting of mixtures of GLMs by weighted likelihood.

Finite mixtures of generalized linear models are commonly fitted by maximum likelihood and the EM algorithm. The estimation process and subsequent inferential and classification procedures can be badly affected by the occurrence of outliers. Actually, contamination in the sample at hand may lead to severely biased fitted components and poor classification accuracy. In order to take into account the potential presence of outliers, a robust fitting strategy is proposed that is based on the weighted likelihood methodology. The technique exhibits a satisfactory behavior in terms of both fitting and classification accuracy, as confirmed by some numerical studies and real data examples.

A multi-physics ensemble approach for short-term precipitation forecasts at convective permitting scales based on sensitivity experiments over southern parts of peninsular India

The southern peninsular India is characterized by unique climatology with rainfall processes throughout the year from land–ocean contrasts. In addition, the complex terrain induces localized effects causing huge spatial and temporal variability in the observed precipitation. This study aims at evaluating the sensitivity of the high-resolution Weather Research and Forecasting (WRF) model (4 km) to multi-physics parameterizations, 3D variational data assimilation, and domain configuration, in the study domain covering southern peninsular India. Furthermore, the study focusses on the formulation of an ensemble method to improve the simulation of precipitation across seasons. A total of 120 experiments were set up across four crucial rainfall events, of varying spatial extent and duration, dominated by different rainfall generation mechanisms. The assessment of the experiments shows that the model’s cumulus and microphysics schemes have the highest impact on the location, intensity, and spread of the simulated 4-day long Quantitative Precipitation Forecasts (QPFs). Applying cumulus schemes at all domains represented the variability in the QPFs, across space and time, for the precipitation events dominated by convective activity. The cases without cumulus schemes at the convective scale domain (4 km), captured the higher intensity rains during organized cyclonic circulations in the north-east monsoon period. Hence, a 10-member multi-physics ensemble approach including members with and without cumulus parameterization at the fine resolution domain was adopted. The preliminary results demonstrate that the mean from the suggested ensemble approach (n-MPP) performed well in capturing the dynamics of QPFs across the rainfall events, as opposed to a single-member deterministic simulation and mean from larger member conventional multi-physics ensemble approach (c-MPP) without cumulus parameterization at the convective scale. The rank histogram, delta semi-variance plots, and outlier statistics at various lead times clearly showed that the suggested n-MPP was able to capture the high-intensity rainfall, increasing the spread of precipitation forecasts and consequently reducing the occurrence of outliers.

Combined Correlation and Cluster Analysis for Long-Term Power Quality Data from Virtual Power Plant

Analysis of the connection between different units that operate in the same area assures always interesting results. During this investigation, the concerned area was a virtual power plant (VPP) that operates in Poland. The main distributed resources included in the VPP are a 1.25 MW hydropower plant and an associated 0.5 MW energy storage system. The mentioned VPP was a source of synchronic, long-term, multipoint power quality (PQ) data. Then, for five related measurement points, the conclusion about the relation in point of PQ was performed using correlation analysis, the global index approach, and cluster analysis. Global indicators were applied in place of PQ parameters to reduce the amount of analyzed data and to check the correlation between phase values. For such a big dataset, the occurrence of outliers is certain, and outliers may affect the correlation results. Thus, to find and exclude them, cluster analysis (k-means algorithm, Chebyshev distance) was applied. Finally, the correlation between PQ global indicators of different measurement points was performed. It assured general information about VPP units’ relation in point of PQ. Under the investigation, both Pearson’s and Spearman’s rank correlation coefficients were considered.

Gaining Outlier Resistance With Progressive Quantiles: Fast Algorithms and Theoretical Studies

Outliers widely occur in big-data applications and may severely affect statistical estimation and inference. In this article, a framework of outlier-resistant estimation is introduced to robustify an arbitrarily given loss function. It has a close connection to the method of trimming and includes explicit outlyingness parameters for all samples, which in turn facilitates computation, theory, and parameter tuning. To tackle the issues of nonconvexity and nonsmoothness, we develop scalable algorithms with implementation ease and guaranteed fast convergence. In particular, a new technique is proposed to alleviate the requirement on the starting point such that on regular datasets, the number of data resamplings can be substantially reduced. Based on combined statistical and computational treatments, we are able to perform nonasymptotic analysis beyond M-estimation. The obtained resistant estimators, though not necessarily globally or even locally optimal, enjoy minimax rate optimality in both low dimensions and high dimensions. Experiments in regression, classification, and neural networks show excellent performance of the proposed methodology at the occurrence of gross outliers. Supplementary materials for this article are available online.

External Validation of Deep Learning Algorithms for Cardiothoracic Ratio Measurement

Recent advances in machine learning have made it possible to create automated systems for medical image diagnosis. Cardiothoracic ratio (CTR) measurement, a common procedure for assessing cardiac abnormality in chest radiographs, has been investigated by several deep learning studies aiming to automate the process. However, of key consideration is whether automated CTR measurements by machine learning models can yield CTR values as accurately and consistently as trained human technicians on unseen data and thereby be considered trustworthy for clinical application. To assess this, we performed external validation of automated CTR algorithms on a dataset of 7,517 images, comparing four variants of U-Net architecture in heart and lung segmentations and CTR calculations: VGG-11, VGG-16, SegNet, and AlbuNet. We then benchmarked their performance against two human experts manually measuring CTR on the same images in a clinical setting such that we could equitably compare model-to-human variation against human-to-human variation. Our analysis shows that AlbuNet demonstrates human-level performance in CTR measurements, achieving MAPE of 2.38%, which is on par with human-to-human inter-rater variability (2.53%) when using the manual measurement method. The other three U-Net variants, particularly VGG-16, also performed similarly well. Additionally, we conducted an extreme outlier analysis on each model architecture, assessing the percentage of samples with higher measurement errors than the maximum error from the manual method. AlbuNet outperformed other architectures with only 0.35% occurrence of extreme outliers, while the other three U-Net variants ranged from 0.64% to 1.06% occurrence. Overall, the deep-learning-based algorithm was demonstrated to be as reliable as the manual method and shows strong potential for assisting radiologists in the CTR measurement process in clinical practice.

Nonparametric Methods in the Analysis of the Quality of Kernel of Selected Winter Wheat Cultivars

Abstract The paper presents a statistical analysis of weight and diameter of kernel obtained for eight investigated winter wheat cultivars. Preliminary analyses of experimental results, such as compliance with a normal distribution, equality of variance for particular cultivars, and occurrence of outliers proved that there is no justification for the use of a traditional one-way analysis of variance for verification of hypotheses according to which weight and diameter of grains of the investigated winter wheat cultivars do not differ. A nonparametric Kruskal-Wallis test was used for verification of hypotheses which based on the experimental results enabled to reject both hypotheses. Further, simultaneous average weight and diameters of grain of the investigated cultivars were compared with the use of Nemenyi-Dunn test. The applied test enabled to conclude which cultivars had a significantly higher weight than the others. Analogous comparisons were carried out for the diameter of kernels of the investigated winter wheat cultivars.

Modified MM Algorithm and Bayesian Expectation Maximization-Based Robust Localization Under NLOS Contaminated Environments

Robust localization methods that employ distance measurements to predict the position of an emitter are proposed in this paper. The occurrence of outliers due to the non-line-of sight (NLOS) propagation of signals can drastically degrade the localization performance in crowded urban areas and indoor situations. Hence, robust positioning methods are considered to mitigate the effects of outliers. Specifically, localization methods based on robust statistics are considered. Modified multi-stage ML-type method (MM) based weighted least squares (WLS), maximum a posteriori (MAP) expectation maximization (EM) WLS and variational Bayes (VB) EM WLS algorithms are developed under various outlier-contaminated environments. Simulation results show that the position estimation accuracy of the proposed modified MM WLS method, which uses the novel weight, is higher than that of the other methods under most outlier-contaminated conditions. Furthermore, the MAP-EM WLS and VB-EM WLS methods are the most accurate among algorithms that do not require statistical testing. Additionally, the mean square error (MSE) and asymptotic unbiasedness of the proposed algorithms are analyzed.

On the drawback of local detrending in universal kriging in conditions of heterogeneously spaced regional TEC data, low-order trends and outlier occurrences

The study intercompares three stochastic interpolation methods originating from the same geostatistical family: least-squares collocation (LSC) known from geodesy, as well as ordinary kriging (OKR) and universal kriging (UKR) known from geology and other geosciences. The objective of this work is to assess advantages and drawbacks of fundamental differences in modeling between these methods in imperfect data conditions. These differences primarily refer to the treatment of the reference field, commonly called ‘mean value’ or ‘trend’ in geostatistical language. The trend in LSC is determined globally before the interpolation, whereas OKR and UKR detrend the observations during the modeling process. The approach to detrending leads to the evident differences between LSC, OKR and UKR, especially in severe conditions such as far from the optimal data distribution. The theoretical comparisons of LSC, OKR and UKR often miss the numerical proof, while numerical prediction examples do not apply cross-validation of the estimates, which is proven to be a reliable measure of the prediction precision and a validation of empirical covariances. Our study completes the investigations with precise parametrization of all these methods by leave-one-out validation. It finds the key importance of the detrending schemes and shows the advantage of LSC prior global detrending scheme in unfavorable conditions of sparse data, data gaps and outlier occurrence. The test case is the modeling of vertical total electron content (VTEC) derived from GNSS station data. This kind of data is a challenge for precise covariance modeling due to weak signal at higher frequencies and existing outliers. The computation of daily set of VTEC maps using the three techniques reveals the weakness of UKR solutions with a local detrending type in imperfect data conditions.

기계학습 알고리즘을 이용한 주택가격감정 시스템의 구축 및 평가: XGBoost, LightGBM, CatBoost 알고리즘에 기반하여

This paper compares the predictive power of a conventional hedonic pricing model and three machine learning algorithm (XGBoost, LightGBM, CatBoost) based models by using 620,617 apartment data in Seoul from 2009 to 2019. The results are summarised as follows; First, the predictive power of the machine learning models are significantly high not only in the comparison to the conventional model but also in the absolute accuracy related to its practical usefulness. The mean percentage error of XGBoost, LightGBM, and CatBoost were only, respectively, 3.7%, 3.8%, and 3.6% while those of the hedonic model was around 11%. Second, we found that CatBoost algorithm is slightely more performative to the other two algorithms in terms of overall predictive power and frequency of outlier occurrences. Third, this paper show that an ensemble model of the three algorithms can raise the predictive power further.

기계학습 알고리즘을 이용한 주택가격감정 시스템의 구축 및 평가: XGBoost, LightGBM, CatBoost 알고리즘에 기반하여

This paper compares the predictive power of a conventional hedonic pricing model and three machine learning algorithm (XGBoost, LightGBM, CatBoost) based models by using 620,617 apartment data in Seoul from 2009 to 2019. The results are summarised as follows; First, the predictive power of the machine learning models are significantly high not only in the comparison to the conventional model but also in the absolute accuracy related to its practical usefulness. The mean percentage error of XGBoost, LightGBM, and CatBoost were only, respectively, 3.7%, 3.8%, and 3.6% while those of the hedonic model was around 11%. Second, we found that CatBoost algorithm is slightely more performative to the other two algorithms in terms of overall predictive power and frequency of outlier occurrences. Third, this paper show that an ensemble model of the three algorithms can raise the predictive power further.

A Distributed Maximum-Likelihood-Based State Estimation Approach for Power Systems

The distribution of measurement noise is commonly considered as an assumed Gaussian model in power systems, but this assumption is not always true in reality. This article introduces a distributed maximum-likelihood-based state estimation approach for multiarea power systems using the student's t-distribution measurement noise model. The t-distribution has the property of “thick tail” to better model the occurrence of outliers and is fairly flexible to model different noise statistics. The finite-time average consensus algorithm is utilized in conjunction with an influence function to realize the proposed distributed approach within a totally distributed framework. Based on the local measurement residuals and the limited information exchanged with neighboring areas, each local area can obtain the global optimum system-wide robust state estimates, while the existing distributed state estimation methods can only get local estimates. Moreover, the communication scheme is more flexible and can be totally different from the transmission lines between local areas. Simulations tested on the IEEE 14-bus and 118-bus systems verify the effectiveness of the proposed distributed approach.

Robust Moving Target Localization in Distributed MIMO Radars via Iterative Lagrange Programming Neural Network

Moving target localization based on the distributed multiple-input multiple-output (MIMO) radar has attracted great research interest recently. However, the occurrence of outliers in the measurements is always unavoidable in many practical situations, which degrades the performances of ordinary algorithms significantly. In this paper, a robust moving target localization method is proposed to tackle this issue. We first present the relevant maximum likelihood (ML) estimation, and recast it to a constrained optimization problem afterwards. We employ the Lagrange programming neural network (LPNN) framework to solve it due to its effectiveness for nonconvex optimization problems. Furthermore, to mitigate the adverse influence caused by outliers, an iterative reweighting scheme is developed and integrated with the LPNN. The target position and velocity estimations can be refined through an iteration process. Simulation results demonstrate that our proposed method not only has an outstanding performance under the Gaussian measurement noise, but is also very robust against outliers.