Outlier Elimination Research Articles

Fast 3D laser scanning of highly reflective surfaces based on a dual-camera system

Existing laser scanning approaches for highly reflective surfaces generally require multiple scans to cope with specular reflection. This paper proposes a dual-camera, line-laser scanning strategy to reconstruct them efficiently, which only needs a single scanning path. To bypass complicated point cloud processing algorithms for outlier elimination, a feasible image processing scheme was designed to delete the disturbed stripe regions. By merging two sets of point clouds computed from residual stripes on a common world frame, these removed noise patches can be compensated by the opposite camera. Therefore, the data fusion result can reconstruct the 3D geometry of targets completely. Experimental results demonstrate good performance with the ability to handle highly reflective surfaces. The scanning period is significantly shortened and scanning accuracy can meet the industrial requirement.

Fusing GNSS/INS/Vision With A Priori Feature Map for High-Precision and Continuous Navigation

Nowadays, the autonomous vehicle has undergone an impressive development, in which one of the most challenging tasks is to pursue high-precision and continuous navigation in complex scenarios. However, GNSS signals are very likely to be blocked and the inertial system suffers from error drift over time. To achieve centimeter-level positioning accuracy, a tightly-coupled model is built via fusing visual, GNSS, and INS measurements, based on a priori visual feature map. The visual map is first generated by a visual mobile surveying system, and the quality of the map is improved by outlier elimination and road lane tracking. This visual feature map consists of two layers that involve 3D coordinates of robust visual features and road lane markings, then two GNSS/INS/Vision fusion models are developed. The first one fuses visual 2D-3D feature matching, GNSS, and INS measurement. The proposed system is effective even when satellites are all out of lock. The second one fuses lane line constraints with GNSS, INS, and distance measurement indicator (DMI). In the experiment part, we show how a visual feature map is generated and refined, then, the performance of the positioning algorithm is evaluated. For the first fusion model, we achieve 10 cm RMS accuracy when only one feature point was matched and all GNSS signals are blocked for 20 minutes. For the other fusion model, the system maintained 10 cm RMS accuracy or even better when a few GNSS satellites are out of lock for 300 seconds.

A Visual SLAM Robust against Dynamic Objects Based on Hybrid Semantic-Geometry Information

A visual localization approach for dynamic objects based on hybrid semantic-geometry information is presented. Due to the interference of moving objects in the real environment, the traditional simultaneous localization and mapping (SLAM) system can be corrupted. To address this problem, we propose a method for static/dynamic image segmentation that leverages semantic and geometric modules, including optical flow residual clustering, epipolar constraint checks, semantic segmentation, and outlier elimination. We integrated the proposed approach into the state-of-the-art ORB-SLAM2 and evaluated its performance on both public datasets and a quadcopter platform. Experimental results demonstrated that the root-mean-square error of the absolute trajectory error improved, on average, by 93.63% in highly dynamic benchmarks when compared with ORB-SLAM2. Thus, the proposed method can improve the performance of state-of-the-art SLAM systems in challenging scenarios.

Stereo Matching Algorithm Based on HSV Color Space and Improved Census Transform

Aiming at the problem that stereo matching accuracy is easily affected by noise and amplitude distortion, a stereo matching algorithm based on HSV color space and improved census transform is proposed. In the cost calculation stage, the color image is first converted from RGB space to HSV space; moreover, the hue channel is used as the matching primitive to establish the hue absolute difference (HAD) cost calculation function, which reduces the amount of calculation and enhances the robustness of matching. Then, to solve the problem of the traditional census transform overrelying on the central pixel and to improve the noise resistance of the algorithm, an improved census method based on neighborhood weighting is also proposed. Finally, the HAD cost and the improved census cost are nonlinearly fused as the initial cost. In the aggregation stage, an outlier elimination method based on confidence interval is proposed. By calculating the confidence interval of the aggregation window, this paper eliminates the cost value that is not in the confidence interval and subsequently filters as well as aggregates the remaining costs to further reduce the noise interference and improve the matching accuracy. Experiments show that the proposed method can not only effectively suppress the influence of noise, but also achieve a more robust matching effect in scenes with changing exposure and lighting conditions.

An adaptive outlier removal aided k-means clustering algorithm

K-means is one of ten popular clustering algorithms. However, k-means performs poorly due to the presence of outliers in real datasets. Besides, a different distance metric makes a variation in data clustering accuracy. Improve the clustering accuracy of k-means is still an active topic among researchers of the data clustering community from outliers removal and distance metrics perspectives. Herein, a novel modification of the k-means algorithm is proposed based on Tukey’s rule in conjunction with a new distance metric. The standard Tukey rule is modified to remove the outliers adaptively by considering whether the data is distributed to the left, right or even to the input data's mean value. The elimination of outliers is applied in the proposed modification of the k-means before calculating the centroids to minimize the outliers' influences. Meanwhile, a new distance metric is proposed to assign each data point to the nearest cluster. In this research, the modified k-means significantly improves the clustering accuracy and centroids convergence. Moreover, the proposed distance metric's overall performance outperforms most of the literature distance metrics. This manuscript's presented work demonstrates the significance of the proposed technique to improve the overall clustering accuracy up to 80.57% on nine standard multivariate datasets.

Development of a Multivariate Model Focused on the Analysis of Water Availability in Mexico

The present study proposes developing a multivariate model that predicts water availability in Mexico through 26 variables related to aquifers, renewable water, demographic characteristics, rivers and basins, dams, and irrigation factors. The information inherent to them was extracted from the platform of the national water system using records from the 13 administrative hydrological regions between 2010 and 2017. The model is based on the multiple linear regression model and the variable selection method. The results show different versions of the model contrasted concerning the statistical assumptions of the multiple regression. Although the findings presented have implications in the development of strategies focused on a better distribution of the vital liquid, in the face of various projected scenarios based on the variables analyzed, it should be noted that the progressive improvement of the model was carried out through the use of techniques such as the transformation of variables, detection, and elimination of outliers. The final result is water availability in the face of various drought conditions explained by a model with 16 relevant variables. Said prediction model is helpful for the generation of drought mitigation strategies.

A combined architecture of multivariate LSTM with Mahalanobis and Z-Score transformations for oil price forecasting

Oil price plays a vital role in a country's economy. Oil price forecasting helps in making better economic planning and decisions. The fluctuation in the oil price occurs due to several factors that make oil price forecasting a challenging task. Deep learning models such as Recurrent Neural Networks (RNN) have emerged as a successful method in solving many nonlinear and complex problems. In this research, a combined architecture of Multivariate Long Short Term Memory (MLSTM) is proposed with Mahalanobis and Z-score transformations. These transformations improve the data to uncorrelated and standardized variance, thus making data more suitable for regression analysis. The available historical time-series data on the West Texas Intermediate (WTI) oil prices and the factors affecting the oil prices are collected to form the data set. The feature selection process using selectKBest and correlation analysis is carried out. We develop six variants of MLSTM models with different features, transformations, and outlier elimination. The MLSTM model with Z-score transformation on the selected features and outlier eliminated data gives an RMSE of 0.212 and an R2 score of 0.954. This indicates that our model performed well.

Prediction of hot metal temperature based on data mining

Abstract Accurately and continuously monitoring the hot metal temperature status of the blast furnace (BF) is a challenging job. To solve this problem, we propose a hot metal temperature prediction model based on the AdaBoost integrated algorithm using the real production data of the BF. We cleaned the raw data using the data analysis technology combined with metallurgical process theory, which mainly included data integration, outliers elimination, and missing value supplement. The redundant features were removed based on Pearson’s thermodynamic diagram analysis, and the input parameters of the model were preliminarily determined by using recursive feature elimination method. We built the hot metal temperature prediction model using the AdaBoost ensemble algorithm on a dataset with selected features as well as derived features by using K-mean clustering tags. The results show that the performance of the hot metal temperature prediction model with K-means clustering tags has been further improved, and the accurate monitoring and forecast of molten iron temperature has been achieved. The model can achieve an accuracy of more than 90% with an error of ±5°C.

Genetic robust kernel sample selection for chemometric data analysis

AbstractIn this work, we propose a new algorithm to improve existing techniques used in the field of spectroscopic data regression analysis. In particular, it combines the power of nonlinear kernel regressors (kernel ridge regression [KRR], kernel principal component regression [KPCR], and Gaussian process regression [GPR]) with an optimization based on nondominated sorting multi‐objective genetic algorithm (NSGAII) to filter the residual outliers in the prediction space and leverage points in the features space. The proposed algorithm, contrary to most existing robust algorithms, simultaneously optimizes many complementary objectives for an automatic adaptation and thus a better outliers detection. It is well known that the elimination of outliers greatly improves the regression model. It is thus the aim of this work to develop a new robust regression algorithm. It has been applied on five different datasets, and the results are compared to both classical nonlinear regression methods and the commonly used robust regression methods robust continuum regression (RCR), partial robust M‐regression (PRM), robust principal component regression (RPCR), robust PLSR (RSIMPLS), and locally weighted regression (LWR). They show that the proposed algorithm outperforms the classical nonlinear regression methods and is a promising competitor to the robust methods outperforming most of them. Even though the results obtained are only from five datasets, this algorithm can be considered an interesting contribution for improving data analysis in the field of chemometrics.

REDE: End-to-End Object 6D Pose Robust Estimation Using Differentiable Outliers Elimination

Object 6D pose estimation is a fundamental task in many applications. Conventional methods solve the task by detecting and matching the keypoints, then estimating the pose. Recent efforts bringing deep learning into the problem mainly overcome the vulnerability of conventional methods to environmental variation due to the hand-crafted feature design. However, these methods cannot achieve end-to-end learning and good interpretability at the same time. In this paper, we propose REDE, a novel end-to-end object pose estimator using RGB-D data, which utilizes network for keypoint regression, and a differentiable geometric pose estimator for pose error back-propagation. Besides, to achieve better robustness when outlier keypoint prediction occurs, we further propose a differentiable outliers elimination method that regresses the candidate result and the confidence simultaneously. Via confidence weighted aggregation of multiple candidates, we can reduce the effect from the outliers in the final estimation. Finally, following the conventional method, we apply a learnable refinement process to further improve the estimation. The experimental results on three benchmark datasets show that REDE slightly outperforms the state-of-the-art approaches and is more robust to object occlusion.

Simulation-Based Transfer Learning for Support Stiffness Identification

The mounting of a rotating machine affects the dynamic behavior of the machine. Typically in large machines, the support structures have lower stiffness on the actual site than in the acceptance tests conducted by the manufacturers. In this research, a method is developed for the support stiffness identification for an in-situ machine using a simulation-data-driven, deep learning algorithm. The novel approach aims to utilize transfer learning to first teach the deep learning algorithm using vibration response data generated from a simulation model of the rotor-bearing-support system, and then test it with measured response. To validate the stiffness estimation of the algorithm for multiple cases, an experimental test rig is used where the horizontal support stiffness can be varied through a range of values. The results from the deep learning algorithm are compared with simpler algorithms such as Linear regression (LR), Artificial Neural Network (ANN), and Support vector regression (SVR) for benchmarking. The models are trained with filtered frequency domain response, and challenges due to measurement uncertainty are analyzed.With proposed pre-processing steps of the frequency domain response and outlier elimination, the deep learning-based virtual sensor can predict the support stiffness with reasonable accuracy, where the limiting factor is the data quality and lack of excitation at critical speed frequencies.

Outliers in Covid 19 data based on Rule representation - the analysis of LOF algorithm

The article concerns the detection of outliers in rule-based knowledge bases containing data on Covid 19 cases. The authors move from the automatic generation of a rule-based knowledge base from source data by clustering rules in the knowledge base to optimize inference processes and to detecting unusual rules allowing for the optimal structure of rule groups. The paper presents a two-phase procedure, wherein in the first phase, we look for the optimal structure of rule clusters when there are outlier rules in the knowledge base. In the second phase, we detect outliers in the rules using the LOF (Local Outlier Factor) algorithm. Then we eliminate the unusual rules from the database and check whether the selected cluster quality measures are responded positively to the elimination of outliers, which would indicate that the rules were rightly considered outliers. The performed experiments confirmed the effectiveness of the LOF algorithm and selected cluster quality measures in the context of detecting atypical rules. The detection of such rules can support knowledge engineers or domain experts in knowledge mining to improve the completeness of the knowledge base, which is usually the basis of the decision support system.

Relationship Marketing Effectiveness Model for an Indian NBFC

The aim of this empirical article was to develop and validate a model to measure relationship marketing effectiveness in an Indian NBFC context that deals with small road transport operators (SRTOs). Data were collected from 302 truckers by five trained data collectors from different locations of India with the help of a structured interview schedule. Customers, whoever came to the branch office during the study period (January – December 2018) and were willing to participate, were interviewed face to face. After scale validation and outlier elimination, finally 255 responses were analyzed with the help of AMOS 23.0 and SPSS 24.0. Path analysis results revealed that perceived effectiveness of RM directly influenced customer satisfaction, loyalty behavior, and quality of life. Further, customer satisfaction influenced loyalty intention. All the fit indices revealed that the proposed model was a fit model. This revealed that satisfaction could create an intention to repeat purchase and spread word of mouth, but it is the perceived relationship marketing effectiveness that makes customers take another loan or continue with the organization. The findings of the study contribute to the concept of RM effectiveness by measuring its impact on the quality of life of the customers. This study provides new prospectives for the companies doing business in this line to develop unique ways to create value for the customers. Future research can be directed at validating the model in the industry context.

Estimating plasma half-lives of dioxin like and non-dioxin like polychlorinated biphenyls after occupational exposure in the German HELPcB cohort.

Polychlorinated biphenyls (PCBs) are carcinogenic persistent organic pollutants that have been used as mixtures in transformers, electrical capacitors or hydraulic oils in underground mining until their ban in the late 1980s in Germany. The widespread use of PCBs has led to an age-dependent body burden in the general population. In order to determine the human half-lives of different PCB-congeners, we have used data collected between 2010 and 2017 from the prospective investigation of the German HELPcB-cohort with high initial occupational exposure to PCBs from a transformer recycling company. PCBs were quantified in plasma of the study participants in six cross-sectional investigations using gas-chromatography coupled to mass-spectrometry. Applying logistic regression on the individual plasma levels after statistical elimination of outliers, the elimination half-lives of 18 PCB-congeners were determined. Further stratifications were performed using adjustment for blood lipids, calculation of the total amount of PCB in the body and by using a statistical model taking into account the individual age-dependent background burden. The calculated plasma half-lives ranged from 0.8 years for PCB 52 until a maximum of 28.9 years for PCB 189, depending on the model applied. E.g. the total body amount related half-live for PCB 28 is 2.41 years, for PCB 74 it is 12.81 years, for PCB 118 it is 6.65 years and for PCB 153=10.75 years. The model with adjustment for age dependent background burden led to shorter half-lives. The analysis revealed structure-related differences in the kinetics for the PCB-congeners examined. Congeners with a chlorine substitution in 2,4,(5)- and 4'-position showed longer half-lives than other isomers with one (or two) free 4-positions. Due to the high number of included cases and repeated measurements, our results provide valid half-live data for a large number of PCB congeners. Using these data, a rough estimation of former occupational exposures from current PCB-levels seems feasible.

Self-Organizing Algorithm for Massive Tractography Datasets Clustering with Outliers Elimination Based on Multiple Species Flocking Model

Background: The study of White Matter (WM) connectivity is of general interest in neuroscience, which is achieved by the analysis and clustering of the streamlines composed of the tractography dataset. The WM streamlines clustering is a challenge, because of the complexity and the vast size of the WM tractography dataset and its composition by various streamlines, in additionally to the presence of outliers.
 Methods: Several WM clustering methods have been proposed in the literature to overcome these constraints. However, these methods stay statics. Once the clustering streamline is performed, it stays in this one. In this paper, we propose a new framework of distributed multiagent, improving, and adapting a bio-inspired model called Multiple Species Flocking (MSF) for WM streamlines clustering and automatic outlier elimination.
 Results: The basic MSF rules are modified and adapted to perform streamlines clustering in higher dimensions. Specifically, each streamline is associated with a mobile agent and move onto a virtual space to form a group following the defined rules. Only the agents assigned to similar streamlines form a flock, whereas the agents assigned to dissimilar streamlines are sidelined and considered as outliers.
 Conclusion: Swarm intelligence features of the approach, such as adaptivity, parallelism, dynamism, and decentralization, make our algorithm scalable to large datasets, very fast and accurate, which are confirmed by experimental results on synthetic and real datasets.

Workplace Attachment as a Mediator of Organizational Tenure and Organizational Citizenship Behavior

The impact of organizational tenure on organizational citizenship behavior (OCB) has been studied previously. However, the role of workplace attachment intervening the link between these two variables has not been researched. Looking at the growing research interest in OCB in academic context, this study aimed to test a mediation model of these three variables in a university context. A structured questionnaire comprising scales on place attachment and OCB was used to collect data from staff of the studied university. Correlation analysis and structural equation modeling of a total of 323 responses (after outlier elimination) was conducted using SPSS 24.0 and AMOS 23.0. It was found out that with the increase in tenure, employees’ attachment to place and their OCB enhanced. It was also found that workplace attachment was a significant mediator of tenure and OCB, implying that tenure enhanced workplace attachment and attachment further enhanced OCB. Based on the previous place attachment literature, it was suggested that there should be meaningful interactions between employees and the physical and group setting, which would foster employees’ emotional bonding with the organization. Theoretically, this study contributes by introducing workplace attachment as a new variable in OCB literature.

Sources of outlier data in the bioanalytical and clinical part of a piroxicam bioequivalence study.

This paper analyzes the potential outliers in the bioanalytical and clinical part of a bioequivalence study, the effect on bioequivalence decisions whether or not it is appropriate to eliminate them from the statistical evaluation of bioequivalence. The clinical part was a cross-over, two periods, two sequences bioequivalence study concerning two piroxicam formulations, on healthy subjects. A simulation study evaluated the influence of 10% errors on the percent bias of calculated concentrations from nominal ones. In bioequivalence studies, it is not possible to distinguish between relevant types of outliers based only on statistical criteria. The "problem" is particularly acute when the omission of outliers leads to a bias in the decision concerning bioequivalence from rejection to acceptance. In such cases, there is the suspicion of subjective analysis and torture of data. The effect of analytical errors at high plasma levels was criticized for the calculated concentrations in the neighborhood of lower limit of quantification. Errors at low concentrations have a less significant effect. In the pharmacokinetic analysis, several types of outliers were shown: single points, curves, pairs of curves corresponding to the same subject, intrasubject ratios of areas under curves and maximum concentrations. These pharmacokinetic outliers could have had, at the same time, bioanalytical, physiological and physicochemical causes. Considering the results, it was proposed the following algorithm in the analysis of outlier data and outlier subjects in bioequivalence studies: evaluation of the implications of the decision concerning elimination of outliers on the decision concerning bioequivalence; application of the statistic tests for detection of outliers data; evaluations from the point of view of physiological pharmacokinetics, final decision concerning elimination of outliers.

Optimizing the Handling of Invasive Fungal Sinusitis Surgical Specimens

Abstract Introduction/Objective Invasive fungal sinusitis (IFS) is an aggressive disease characterized by invasion of fungal hyphae into tissue/neurovascular bundles. This project assessed the handling of IFS specimens and implemented protocols to improve turnaround time (TAT). Methods A retrospective review of cases accessioned with a clinical concern for IFS from 2014-2019 was performed. TAT for each step in the specimen processing was recorded. A flowchart was created using stakeholder interviews and a revised protocol was developed after assessing critical needs. Assessment of interventions was performed following implementation of the new protocol. The protocol will be evaluated by prospective direct case-by-case feedback and after a 6-month interval (projected August/2020). At 6-months, goals are a 24 hours median time between frozen section and sign-out and elimination of outliers (greater than 2 working days). Results We identified 53 specimens from 32 patients in the pre-intervention period (36 cases positive for IFS). Median time from frozen section to final sign-out was 28 (5-312) hours. Four areas for improvements were identified: (1) triaging specimens to different protocols according to arrival time, (2) optimized triaging for available histology processors, (3) standardized GMS ordering, and (4) standardized case delivery/communication with sign-out staff. Interventions include: protocol for processing specimens based on time of day, new histology protocols to expedite GMS performance, an email group for rapid communication with staff pathologists and histology, and a worksheet/checklist to track each case. After implementation of the protocol, 8 cases from 7 patients were received. Median time from frozen section to final sign-out was reduced to 20 (2 – 50) hours. Conclusion The protocol for handling IFS specimens became live on 2/1/2020. It has reduced TAT of suspected IFS cases, from a median of 28 hours to 20 hours. The longest interval to sign-out went from312 hours to 50 hours.

Development of the Scale for Environmental Aesthetic Awareness and investigation of students' environmental aesthetic awareness

Although environmental aesthetic has received considerable attention in recent years, there is a need for measuring and evaluating environmental aesthetic. With considering this gap in the literature, the purpose of this study is to develop the “Scale for Environmental Aesthetic Awareness” for university students and to compare students’ levels of environmental aesthetic awareness according to gender, interest in art and the place where family lives. The study group is comprised of 969 university students. For conducting the validity study of the scale, the study group was randomly divided into two: Exploratory Factor Analysis group and Confirmatory Factor Analysis group. After the elimination of outliers, 329 students remained in the EFA group and 255 students in the CFA group. As a result of EFA, 27 items were gathered under three sub-dimensions. As a result of the first-order and second-order CFA, it was seen that the model-data fit was achieved and the three-factor structure of the scale and its general structure were confirmed. The Cronbach Alpha reliability coefficient of the first, second, third sub-dimensions were calculated respectively as .80, .71 and .71. The Cronbach alfa coefficient value for the whole scale was .82. Three-way variance analysis results showed that the effects of gender and the place where family lives were not found to be statistically significant while the effect of interest in art was found to be significant; thus, the mean of the students stating they were interested in arts was found to be higher.

On the effects of hard and soft equality constraints in the iterative outlier elimination procedure.

Reliability analysis allows for the estimation of a system's probability of detecting and identifying outliers. Failure to identify an outlier can jeopardize the reliability level of a system. Due to its importance, outliers must be appropriately treated to ensure the normal operation of a system. System models are usually developed from certain constraints. Constraints play a central role in model precision and validity. In this work, we present a detailed investigation of the effects of the hard and soft constraints on the reliability of a measurement system model. Hard constraints represent a case in which there exist known functional relations between the unknown model parameters, whereas the soft constraints are employed where such functional relations can be slightly violated depending on their uncertainty. The results highlighted that the success rate of identifying an outlier for the case of hard constraints is larger than soft constraints. This suggested that hard constraints be used in the stage of pre-processing data for the purpose of identifying and removing possible outlying measurements. After identifying and removing possible outliers, one should set up the soft constraints to propagate their uncertainties to the model parameters during the data processing.