Outlier Removal Algorithm Research Articles

Effect and correction of optode coupling errors in breast imaging using diffuse optical tomography.

In diffuse optical tomography (DOT) and spectroscopy (DOS) using handheld probes, tissue curvature can cause bad fiber-to-tissue contact. Understanding and minimizing image artifacts caused by these coupling errors would significantly improve DOT and DOS image quality. In this work, we utilized Monte Carlo simulations and experiments with gelatin-Intralipid phantoms to systematically study the influence of source or detector (optode) coupling errors. Optode coupling errors can increase the amplitude and decrease the phase of the measured diffuse reflectance, creating artifacts in the reconstructed absorption maps, such as hot spots on the edges. We propose an outlier removal algorithm that can correct these image artifacts, and we demonstrate its performance using simulations, phantom experiments, and breast patient data acquired with bad probe contact due to a dense or small breast. Further, we designed and implemented a new resistance-type thin-film force sensor array that provides real-time optode coupling feedback and guides the outlier removal to minimize optode coupling errors. Our approaches and study results have significant implications for reducing image artifacts arising from handheld probes, which are commonly used with mobile and wearable DOT and DOS devices.

Proposed Prediction Framework for Improving the Accuracy of Path loss Models of WiMAX Network

Worldwide interoperability for Microwave Access (WiMAX) is an advanced wireless broadband access technology for delivering high speed voice, video and multimedia access for fixed as well as mobile users. Planning and deployment of this network is considered as a crucial challenge because it depends upon the accurate estimation of parameters such as link budget, coverage area, capacity, capital expenditure and operational expenditure etc. However, estimation of the above mentioned parameters is substantially relied on accuracy of predicted path loss model between the base station and the receiver. From data analysis theory, it is found that the presence of outliers in data set have notable impact on the correctness of predicted models. Hence, in this paper a prediction framework is proposed which suggests the inclusion of outlier removal algorithm along with data collection, statistical analysis and regression analysis for improving the accuracy of path loss model. Several measurement campaigns are conducted in a WiMAX network deployed in a suburban area for collection of data such as received signal strength indicator (RSSI), and carrier to interference plus noise ratio (CINR). Further, the impact of proposed framework in improving the accuracy of predicted RSSI, CINR and path loss models for the concerned scenario is investigated through simulation based study. Also, the impact of proposed algorithm on probability density function of path loss as well as link budget performance is investigated. From the result analysis, it is observed that the estimated path loss model closes to SUI A path loss model and SUI C path loss model without and with using the proposed approach respectively. Further, the path loss exponents of the predicted path loss models are evaluated as 3.78 and 3.47 without and with utilizing the proposed framework respectively.

InSAR Reveals Complex Surface Deformation Patterns Over an 80,000 km2Oil‐Producing Region in the Permian Basin

AbstractHere we used Sentinel‐1 interferometric synthetic aperture radar (InSAR) data acquired between November 2014 to January 2019 to map how the basin's surface has deformed in response to fluid injection and extraction. While our stacking approach has low complexity, its accuracy increases with the Sentinel‐1 data volume. With an automated outlier removal algorithm, we achieved ∼2 mm/year accuracy across the basin in the presence of up to ±15 cm tropospheric noise. We observed numerous subsidence and uplift features near active production and disposal wells, with the maximum deformation rate occurring in 2018 when production peaked. The most important deformation signatures are linear patterns that extend tens of kilometers near Pecos, TX, where a cluster of increased seismic events was cataloged by the Texas Seismological Network (TexNet). Our elastic modeling results demonstrate that fluid extraction and dip slip along normal faults are potential causes for the observed seismicity and deformation patterns.

Defenses Against Byzantine Attacks in Distributed Deep Neural Networks

Large scale deep learning is trending recently, since people find that complicated networks can sometimes reach high accuracy on image recognition, natural language processing, etc. With the increasing complexity and batch size of deep neural networks, the training for such networks is more difficult because of the limited computational power and memory. Distributed machine learning or deep learning provides an efficient solution. However, with the concern of untrusted machines or hardware failures, the distributed system may suffer Byzantine attacks. If some workers are attacked and just upload malicious gradients to the parameter server, they will lead the total training process to a wrong model or even cannot converge. To defend the Byzantine attacks, we propose two efficient algorithms: FABA, a Fast Aggregation algorithm against Byzantine Attacks, and VBOR, a Variance Based Outlier Removal algorithm. FABA conducts the distance information to remove outliers one by one. VBOR uses the variance information to remove outliers with one-pass iteration. Theoretically, we prove the convergence of our algorithms and give an insight of the correctness. In the experiment, we compare FABA and VBOR with the state-of-the-art Byzantine defense algorithms and show our superior performance.

Border Detection in Skin Lesion Images Using an Improved Clustering Algorithm

The incidence of skin cancer has been increasing in recent years and it can become dangerous if not detected early. Computer-aided diagnosis systems can help the dermatologists in assisting with skin cancer detection by examining the features more critically. In this article, a detailed review of pre-processing and segmentation methods is done on skin lesion images by investigating existing and prevalent segmentation methods for the diagnosis of skin cancer. The pre-processing stage is divided into two phases, in the first phase, a median filter is used to remove the artifact; and in the second phase, an improved K-means clustering with outlier removal (KMOR) algorithm is suggested. The proposed method was tested in a publicly available Danderm database. The improved cluster-based algorithm gives an accuracy of 92.8% with a sensitivity of 93% and specificity of 90% with an AUC value of 0.90435. From the experimental results, it is evident that the clustering algorithm has performed well in detecting the border of the lesion and is suitable for pre-processing dermoscopic images.

A Hybrid Machine Learning Model with Cost-function Based Outlier Removal and Its Application on Credit Rating

This paper establishes a methodology to build hybrid machine learning models, aiming to combine the power of different machine learning algorithms on different types of features and hypothesis. A generic cost-based outlier removal algorithm is introduced as a step of pre-process of training data. Using this methodology, an experimental hybrid machine learning model is implemented for a crediting problem, which combines three types of machine learning algorithms SVM, DT and LR. Different metrics of training time, recall, precision, F1 score, and AUC are evaluated and compared with the results from single model of single SVM, DT, and LR. The new hybrid models in general shows improvement in performance on multiple metrics. this study further demonstrated that one learning model may work well on certain type of variables but not as effectively on other variables. Well defined procedures can be followed to create hybrid models to combine the power of different learning models, which are superior in dealing with dataset with complicate feature selection. This methodology can be explored with other algorithms and applications.

Visualization of Shoulder Ligaments Motion by Ultrasound Speckle Tracking Method.

There is no unified consensus on the pathophysiology of adhesive capsulitis which is also called as frozen shoulder. Some studies have suggested that coracohumeral ligament (CHL) played an important role in adhesive capsulitis. These studies showed relation between disease prevalence and CHL thickness by means of ultrasound or MRI. Other possible etiology of the disease is the adhesion of shoulder ligament with each other. In the present study, shoulder ligaments motion is visualized with velocity vector by speckle tracking and temporal outlier removal algorithm to process ultrasound movie during forced movement of shoulder joint. The measurement with a typical subject has demonstrated an improvement of velocity vector deviation of up to 43% by the proposed outlier removal technique.

Clustering With Outlier Removal

Cluster analysis and outlier detection are two continuously rising topics in data mining area, which in fact connect to each other deeply. Cluster structure is vulnerable to outliers; inversely, outliers are the points belonging to none of any clusters. Unfortunately, most existing studies do not notice the coupled relationship between these two tasks and handle them separately. In this article, we consider the joint cluster analysis and outlier detection problem, and propose the Clustering with Outlier Removal (COR) algorithm. Specifically, the original space is transformed into a binary space via generating basic partitions. We employ Holoentropy to measure the compactness of each cluster without involving several outlier candidates. To provide a neat and efficient solution, an auxiliary binary matrix is introduced so that COR completely and efficiently solves the challenging problem via a unified K-means— with theoretical supports. Extensive experimental results on numerous data sets in various domains demonstrate the effectiveness and efficiency of COR significantly over state-of-the-art methods in terms of cluster validity and outlier detection. Some key factors including the basic partition number and generation strategy in COR with an application on abnormal flight trajectory detection are further analyzed for practical use.

An improved copy-move forgery detection based on density-based clustering and guaranteed outlier removal

Copy-move image forgery detection has become a significant research subject in multimedia forensics and security due to its widespread use and its hard detection. In this type of image forging, a region of the image is copied and pasted elsewhere in the same image. Keypoint-based forgery detection approaches use local visual features to identify the duplicated regions. The performance of keypoint-based methods degrades in those cases when the duplicated regions are near to each other and when handling highly textured area. The clustering algorithm that mostly used in keypoint- based methods suffer from high complexity. In this paper, an improved approach for keypoint- based copy-move forgery detection is proposed. The proposed method is based on density-based clustering and Guaranteed Outlier Removal algorithm. Experimental results carried out on various benchmark datasets exhibit that the proposed method surpasses other similar state-of-the-art techniques under different challenging conditions, such as geometric attacks, post-processing attacks, and multiple cloning.

A new data preprocessing method for 3D reconstruction of pavement

ABSTRACT Three-dimensional (3D) reconstruction of pavement surface based on the laser scanning can be used to extract the elevation information for evaluating the damage degree of the pavement. This technology is of great significance to road safety as well as the advances of smart transportation. The preprocessing procedures of the laser-scanned point cloud serve as the basis for accurate surface reconstruction, which includes outlier removal, smoothing and sampling. In view of the limitations of the existing preprocessing methods for pavement applications, this paper proposes a new preprocessing method. First, a local density-based outlier removal algorithm is used to solve the problem of poor adaptability or long computation time of the regular outlier removal algorithms. And then, a continuous algorithm based on the moving least squares method is used to smooth and sample the points, solving the problem of repeated degradation of the data. Finally, the field tests on three types of pavements are carried out to verify the feasibility of the proposed preprocessing method. The accuracy of the reconstructed surfaces of the pavements is improved by the new preprocessing method.

Conceptual space based model fitting for multi-structure data

Abstract In this paper, we propose a novel fitting method, called the Conceptual Space based Model Fitting (CSMF), to fit and segment multi-structure data contaminated with a large number of outliers. CSMF includes two main parts: an outlier removal algorithm and a model selection algorithm. Specifically, we firstly construct a novel conceptual space to measure data points by only considering the good model hypotheses. Then we analyze the conceptual space to effectively remove the gross outliers. Based on the results of outlier removal, we propose to search center points (representing the estimated model instances) in the conceptual space for model selection. CSMF is able to efficiently and effectively remove gross outliers in data, and simultaneously estimate the number and the parameters of model instances without using prior information. Experimental results on both synthetic data and real images demonstrate the advantages of the proposed method over several state-of-the-art fitting methods.

Outlier Detection and Removal Algorithm in K-Means and Hierarchical Clustering

An outlier in a pattern is dissimilar with rest of the pattern in a dataset. Outlier detection is an important issue in data mining. It has been used to detect and remove anomalous objects from data. Outliers occur due to mechanical faults, changes in system behavior, fraudulent behavior, and human errors. This paper describes the methodology or detecting and removing outlier in K-Means and Hierarchical clustering. First apply clustering algorithm K-means and Hierarchical clustering on a data set then find outliers from the each resulting clustering. In K-Means clustering outliers are found by distance based approach and cluster based approach. In case of hierarchical clustering, by using dendrogram outliers are found. The goal of the project is to detect the outlier and remove the outliers to make the clustering more reliable.

Aging Precursor Identification and Lifetime Estimation for Thermally Aged Discrete Package Silicon Power Switches

Thermal/power cycles are widely acknowledged methods to accelerate the package related failures. Many studies have focused on one particular aging precursor at a time and continuously monitored it using custom-built circuits. Due to the difficulties in taking sensitive measurements, the reported findings are more on the quantities requiring less sensitive measurements. In this paper, two custom-designed testbeds are used to age a number of power MOSFETs and insulated gate bipolar transistors. An automated curve tracer is utilized to capture parametric variations in I-V curves, parasitic capacitances, and gate charges at certain time intervals. The results suggest that the only viable aging precursors are the on-state voltage drop/on-state resistance, body diode voltage drop, parasitic capacitances, and gate threshold voltage for die attach solder and gate-oxide degradation mechanisms. Based on the experimental results, gate threshold voltage variation is empirically modeled to estimate the remaining useful lifetime of the switches experiencing gate oxide degradation. The model parameters are found by the least squares method applied to inliers determined by the random sample and consensus outlier removal algorithm.

A Geometrical–Statistical Approach to Outlier Removal for TDOA Measurements

The curse of outlier measurements in estimation problems is a well known issue in a variety of fields. Therefore, outlier removal procedures, which enables the identification of spurious measurements within a set, have been developed for many different scenarios and applications. In this paper, we propose a statistically motivated outlier removal algorithm for time differences of arrival (TDOAs), or equivalently range differences (RD), acquired at sensor arrays. The method exploits the TDOA-space formalism and works by only knowing relative sensor positions. As the proposed method is completely independent from the application for which measurements are used, it can be reliably used to identify outliers within a set of TDOA/RD measurements in different fields (e.g. acoustic source localization, sensor synchronization, radar, remote sensing, etc.). The proposed outlier removal algorithm is validated by means of synthetic simulations and real experiments.

Novel Similarity-Invariant Line Descriptor and Matching Algorithm for Global Motion Estimation

We present a new and fast line descriptor and matching algorithm to geometrically register video frames that are deformed by a similarity transformation and contain line segments but with very low texture details, such as navigation maps. Line segments are extracted from the frames, and each line is described with a novel line descriptor that does not depend on pixel intensities. The described lines are then matched together and the matches are input to an outlier removal algorithm in order to estimate the parameters of the transformation describing the global motion between the frames. We propose a method for fast parameter estimation of the transformation using line segments instead of points. Additionally, we apply this algorithm in the testing and error detection context of navigation systems, in which we show how to detect map jump artifacts that could occur during the development of these systems, leading to a jerky and unsmooth motion between the frames. The proposed descriptor and its matching algorithm are shown to be fast enough for online use and very robust against a wide range of translation, rotation, and scale changes. Furthermore, the error detection algorithm allows to detect almost all map jump artifacts while maintaining a very low number of false alarms.

Modeling the direction-continuous time-of-arrival in head-related transfer functions

Head-related transfer functions (HRTFs) describe the filtering of the incoming sound by the torso, head, and pinna. As a consequence of the propagation path from the source to the ear, each HRTF contains a direction-dependent, broadband time-of-arrival (TOA). TOAs are usually estimated independently for each direction from HRTFs, a method prone to artifacts and limited by the spatial sampling. In this study, a continuous-direction TOA model combined with an outlier-removal algorithm is proposed. The model is based on a simplified geometric representation of the listener, and his/her arbitrary position within the HRTF measurement. The outlier-removal procedure uses the extreme studentized deviation test to remove implausible TOAs. The model was evaluated for numerically calculated HRTFs of sphere, torso, and pinna under various conditions. The accuracy of estimated parameters was within the resolution given by the sampling rate. Applied to acoustically measured HRTFs of 172 listeners, the estimated parameters were consistent with realistic listener geometry. The outlier removal further improved the goodness-of-fit, particularly for some problematic fits. The comparison with a simpler model that fixed the listener position to the center of the measurement geometry showed a clear advantage of listener position as an additional free model parameter.

Processing and interactive editing of huge point clouds from 3D scanners

This paper describes a new out-of-core multi-resolution data structure for real-time visualization, interactive editing and externally efficient processing of large point clouds. We describe an editing system that makes use of the novel data structure to provide interactive editing and preprocessing tools for large scanner data sets. Using the new data structure, we provide a complete tool chain for 3D scanner data processing, from data preprocessing and filtering to manual touch-up and real-time visualization. In particular, we describe an out-of-core outlier removal and bilateral geometry filtering algorithm, a toolset for interactive selection, painting, transformation, and filtering of huge out-of-core point-cloud data sets and a real-time rendering algorithm, which all use the same data structure as storage backend. The interactive tools work in real-time for small model modifications. For large scale editing operations, we employ a two-resolution approach where editing is planned in real-time and executed in an externally efficient offline computation afterwards. We evaluate our implementation on example data sets of sizes up to 63GB, demonstrating that the proposed technique can be used effectively in real-world applications.

Techniques for Video Mosaicing

Video Mosaicing is the stitching of selected frames of a video by estimating the camera motion between the frames and thereby registering successive frames of the video to arrive at the mosaic. Different techniques have been proposed in the literature for video mosaicing. Despite of the large number of papers dealing with techniques to generate mosaic, only a few authors have investigated conditions under which these techniques generate good estimate of motion parameters. In this paper, these techniques are studied under different videos, and the reasons for failures are found. We propose algorithms with incorporation of outlier removal algorithms for better estimation of motion parameters.

Geometric modeling of the human normal cerebral arterial system

We propose an anatomy-based approach for an efficient construction of a three-dimensional human normal cerebral arterial model from segmented and skeletonized angiographic data. The centerline-based model is used for an accurate angiographic data representation. A vascular tree is represented by tubular segments and bifurcations whose construction takes into account vascular anatomy. A bifurcation is defined quantitatively and the algorithm calculating it is given. The centerline is smoothed by means of a sliding average filter. As the vessel radius is sensitive to quality of data as well as accuracy of segmentation and skeletonization, radius outlier removal and radius regression algorithms are formulated and applied. In this way, the approach compensates for some inaccuracies introduced during segmentation and skeletonization. To create the frame of vasculature, we use two different topologies: tubular and B-subdivision based. We also propose a technique to prevent vessel twisting. The analysis of the vascular model is done on a variety of data containing 258 vascular segments and 131 bifurcations. Our approach gives acceptable results from anatomical, topological and geometrical standpoints as well as provides fast visualization and manipulation of the model. The approach is applicable for building a reference cerebrovascular atlas, developing applications for simulation and planning of interventional radiology procedures and vascular surgery, and in education.