Outlier Rejection Research Articles

SCRnet: A Spatial Consistency Guided Network Using Contrastive Learning for Point Cloud Registration

Point cloud registration is used to find a rigid transformation from the source point cloud to the target point cloud. The main challenge in the point cloud registration is in finding correct correspondences in complex scenes that may contain many noise and repetitive structures. At present, many existing methods use outlier rejections to help the network obtain more accurate correspondences, but they often ignore the spatial consistency between keypoints. Therefore, to address this issue, we propose a spatial consistency guided network using contrastive learning for point cloud registration (SCRnet), in which its overall stage is symmetrical. SCRnet consists of four blocks, namely feature extraction block, confidence estimation block, contrastive learning block and registration block. Firstly, we use mini-PointNet to extract coarse local and global features. Secondly, we propose confidence estimation block, which formulate outlier rejection as confidence estimation problem of keypoint correspondences. In addition, the local spatial features are encoded into the confidence estimation block, which makes the correspondence possess local spatial consistency. Moreover, we propose contrastive learning block by constructing positive point pairs and hard negative point pairs and using Point-Pair-INfoNCE contrastive loss, which can further remove hard outliers through global spatial consistency. Finally, the proposed registration block selects a set of matching points with high spatial consistency and uses these matching sets to calculate multiple transformations, then the best transformation can be identified by initial alignment and Iterative Closest Point (ICP) algorithm. Extensive experiments are conducted on KITTI and nuScenes dataset, which demonstrate the high accuracy and strong robustness of SCRnet on point cloud registration task.

Two-Step Registration on Multi-Modal Retinal Images via Deep Neural Networks.

Multi-modal retinal image registration plays an important role in the ophthalmological diagnosis process. The conventional methods lack robustness in aligning multi-modal images of various imaging qualities. Deep-learning methods have not been widely developed for this task, especially for the coarse-to-fine registration pipeline. To handle this task, we propose a two-step method based on deep convolutional networks, including a coarse alignment step and a fine alignment step. In the coarse alignment step, a global registration matrix is estimated by three sequentially connected networks for vessel segmentation, feature detection and description, and outlier rejection, respectively. In the fine alignment step, a deformable registration network is set up to find pixel-wise correspondence between a target image and a coarsely aligned image from the previous step to further improve the alignment accuracy. Particularly, an unsupervised learning framework is proposed to handle the difficulties of inconsistent modalities and lack of labeled training data for the fine alignment step. The proposed framework first changes multi-modal images into a same modality through modality transformers, and then adopts photometric consistency loss and smoothness loss to train the deformable registration network. The experimental results show that the proposed method achieves state-of-the-art results in Dice metrics and is more robust in challenging cases.

Static Calibration for Line-Scan Cameras Based on a Novel Calibration Target

Automated quality control in industrial environments requires sensors to obtain information about the manufactured products. Visual sensors are crucial for this task. In industrial environments, line-scan cameras are commonly used for their ability to acquire images from fast moving objects at high resolution. However, the information in the images must be translated from image coordinates to scene coordinates. This work proposes a calibration procedure for line-scan cameras particularly designed for industrial environments. The proposed approach is based on a novel calibration target that can be used in multi-camera architectures, a common scenario found in many product inspection pipelines. Industrial environments are very challenging for visual sensors, as they are affected by harsh conditions. The calibration procedure integrates an outlier rejection method to deal with these issues. In addition, the calibration target includes a large number of feature points. Consequently, the removal of some of these features does not affect the performance of the calibration. Additional signal processing methods are proposed to extract features from the images. The calibration procedure uses data acquired from multiple observations by optimizing the reprojection error, resulting in an accurate and robust method, suitable for in industrial environments. The design of the calibration target, the increased number of features, the outlier rejection method and other contributions of this work result in an accurate estimation of the camera parameters. The proposed method is validated using synthetic and real data acquired with a commercial line-scan camera.

Kalman Filter-Based Integrity Monitoring for GNSS and 5G Signals of Opportunity Integrated Navigation

A Kalman filter-based receiver autonomous integrity monitoring algorithm (RAIM) is proposed to exploit sequential measurements from global navigation satellite systems (GNSS) and cellular 5G signals of opportunity (SOPs), to ensure safe vehicular navigation in urban environments. To deal with frequent threats caused by multipath and non-line-of-sight conditions, an innovation-based outlier rejection method is introduced. Next, a fault detection technique based on solution separation test is developed, and the quantification of protection levels is derived. Experimental results of a ground vehicle traveling in an urban environment, while making pseudorange measurements to GPS satellites and cellular 5G towers, are presented to demonstrate the efficacy of the proposed method. Incorporating 5G signals from only 2 towers is shown to reduce the horizontal protection level (HPL) by 0.22 m compared to using only GPS. Moreover, the proposed method is shown to reduce the HPL and vertical protection level (VPL) by 84.42% and 69.63%, respectively, over the snapshot advanced RAIM (ARAIM).

Learning Two-View Correspondences and Geometry Using Neighbor-Aware Network

Finding reliable correspondences between two images is a fundamental problem in remote-sensing image registration. In the face of sparse and unordered correspondences, the previous learning-based works often focus on global information and ignore valuable local information. To gather rich local information, we propose a simple and effective approach, the principle of which is to establish a local neighborhood structure for putative correspondences and then extract and aggregate neighborhood context. Specifically, we design a residual block with an innovative normalization operation and then we construct a sub-net, by introducing a competition mechanism in the local neighborhood to enhance the expression ability of features. Finally, we propose a learning-based network, to improve the performance of outlier rejection by extracting neighborhood context and global context. Extensive experiments on relative pose estimation demonstrate that the proposed network surpasses current state-of-the-art approaches on both challenging indoor and outdoor datasets.

Robust Tensor Tracking With Missing Data and Outliers: Novel Adaptive CP Decomposition and Convergence Analysis

Canonical Polyadic (CP) decomposition is a powerful multilinear algebra tool for analyzing multiway (a.k.a. tensor) data and has been used for various signal processing and machine learning applications. When the underlying tensor is derived from data streams, adaptive CP decomposition is required. In this paper, we propose a novel method called robust adaptive CP decomposition (RACP) for dealing with high-order incomplete streaming tensors that are corrupted by outliers. At each time instant, RACP first performs online outlier rejection to accurately detect and remove sparse outliers, and then performs tensor factor tracking to efficiently update the tensor basis. A unified convergence analysis of RACP is also established in that the sequence of generated solutions converges asymptotically to a stationary point of the objective function. Extensive experiments were conducted on both synthetic and real data to demonstrate the effectiveness of RACP in comparison with state-of-the-art adaptive CP algorithms.

Measurement Error Detection for Stereo Visual Odometry Integrity

<h3>Abstract</h3> Integrity, a safety-of-life framework from civil aviation for satellite navigation, is greatly under-explored in visual navigation. A new two-factor approach to rejecting measurement outliers is proposed for navigation integrity in stereo visual odometry (VO). In contrast to other treatments using reprojection error as measurement residuals, our choice of landmark matching error inherently connects navigation solutions and integrity monitoring. We propose two methods to detect large measurement residuals that cannot otherwise be identified by existing outlier rejection methods in state-of-the-art VO pipelines. By rejecting these outliers, measurement residuals can be bounded by the distribution overbounding method that provides fundamental inputs for integrity computations. We evaluate our methods using an open-source data set. Overbounding performance is improved in terms of tightness, computational efficiency, and most important of all, scenario tolerance. This could be a good starting point for developing future integrity monitoring algorithms for visual navigation and in particular, stereo VO.

Enabling Continuous Wearable Reflectance Pulse Oximetry at the Sternum.

In light of the recent Coronavirus disease (COVID-19) pandemic, peripheral oxygen saturation (SpO2) has shown to be amongst the vital signs most indicative of deterioration in persons with COVID-19. To allow for the continuous monitoring of SpO2, we attempted to demonstrate accurate SpO2 estimation using our custom chest-based wearable patch biosensor, capable of measuring electrocardiogram (ECG) and photoplethysmogram (PPG) signals with high fidelity. Through a breath-hold protocol, we collected physiological data with a wide dynamic range of SpO2 from 20 subjects. The ratio of ratios (R) used in pulse oximetry to estimate SpO2 was robustly extracted from the red and infrared PPG signals during the breath-hold segments using novel feature extraction and PPGgreen-based outlier rejection algorithms. Through subject independent training, we achieved a low root-mean-square error (RMSE) of 2.64 ± 1.14% and a Pearson correlation coefficient (PCC) of 0.89. With subject-specific calibration, we further reduced the RMSE to 2.27 ± 0.76% and increased the PCC to 0.91. In addition, we showed that calibration is more efficiently accomplished by standardizing and focusing on the duration of breath-hold rather than the resulting range in SpO2. The accurate SpO2 estimation provided by our custom biosensor and the algorithms provide research opportunities for a wide range of disease and wellness monitoring applications.

A Covid-19's integrated herd immunity (CIHI) based on classifying people vulnerability

Unfortunately, Covid-19 has infected millions of people very quickly, and it continues to infect people and spreads rapidly. Although there are some common symptoms of Covid-19, its effect varies from one individual to another. Estimating the severity of the infection has become a critical need as it can guide the decision makers to take an accurate and timely response. It will be valuable to provide early warning before infection takes place about susceptibility to the disease, especially since the lack of symptoms is a feature of the Covid-19 pandemic. Asymptomatic patients are considered as “silent diffusers” of the virus; hence, detecting people who will be asymptomatic before actual infection takes place will certainly safe the society from the uncontrolled and unseen spread of the virus. People can be classified based on their vulnerability to Covid-19 even before they are infected. Accordingly, precautionary measures can be taken individually based on the persons' Covid-19 susceptibility. This paper introduces a Covid-19's Integrated Herd Immunity (CIHI) strategy. The aim of CIHI is to keep the society safe with the minimal losses even with the existence of Covid-19. This can be accomplished by two basic factors; the first is an accurate prediction of the cases who will be asymptomatic if they were infected by the virus, while the second is to take suitable precautions for those who are predicted to be badly affected by the virus even before the actual infection takes place. CIHI is realized through a new classification strategy called Distance Based Classification Strategy (DBCS) which classifies people based on their vulnerability to Covid-19 infection. The proposed DBCS classifies individuals into six different types, then suitable precautionary measures can be taken for every type. DBCS can also identify future symptomatic and asymptomatic cases. In fact, DBCS consists of three sequential phases, which are; (i) Outlier Rejection Phase (ORP) using Hybrid Outlier Rejection (HOR) method, (ii) Feature Selection Phase (FSP) using Hybrid Feature Selection (HFS) method, and (iii) Classification Phase (CP) using Accumulative K-Nearest Neighbors (AKNN). DBCS has been compared with recent Covid-19 diagnosing techniques based on “NileDS” dataset. Experimental results have proven the efficiency and applicability of the proposed strategy as it provides the best classification accuracy.

Full cardiac cycle asynchronous temporal compounding of 3D echocardiography images.

It is important to improve echocardiography image quality, because the accuracy of echocardiographic assessment and diagnosis relies on image quality. Previous work on 2D temporal image compounding for image frames with matching cardiac phases (synchronous), and for temporally neighbouring image frames (asynchronous) over small ranges of time frames showed good improvement to image quality. Here, we extend this by performing asynchronous temporal compounding to echocardiographic images in 3D, involving all frames within a cardiac cycle, via a robust 3D cardiac motion estimation algorithm to describe the large image deformations. After compounding, the images can be reanimated via the motion model. Various methods of fusing image frames together are tested, including mean, max, and wavelet methods, and outlier rejection algorithms. The compounding algorithm is applied on 3D human adult, porcine adolescent, and human fetal echocardiography images. Results show significant improvements to contrast-to-noise ratio (CNR) and boundary clarity, and significantly decreased variability in manual quantification of cardiac chamber volumes after compounding. Interestingly, compounding can extend the field of view of the echo images, by reconstructing cardiac structures that momentarily exceeded the field of view, using the motion estimation algorithm to calculate their locations outside the field of view during these time periods. Although all compounding methods provide general improvements, the mean method led to blurred boundaries, while the max methods led to high variability of CNR. Outlier rejection algorithms were found to be useful in addressing these weaknesses.

Multi-Detector Heart Rate Extraction Method for Transabdominal Fetal Pulse Oximetry.

Intrapartum fetal well-being assessment relies on fetal heart rate (FHR) monitoring. Studies have shown that FHR monitoring has a high false-positive rate for detecting fetal hypoxia during labor and delivery. A transabdominal fetal pulse oximeter device that measures fetal oxygen saturation non-invasively through NIR light source and photodetectors could increase the accuracy of hypoxia detection. As light travels through both maternal and fetal tissue, photodetectors on the surface of mother's abdomen capture mixed signals comprising fetal and maternal information. The fetal information should be extracted first to enable fetal oxygen saturation calculation. A multi-detector fetal signal extraction method is presented in this paper where adaptive noise cancellation is applied to four mixed signals captured by four separate photodetectors placed at varying distances from the light source. As a result of adaptive noise cancellation, we obtain four separate FHR by peak detection. Weighting, outlier rejection and averaging are applied to these four fetal heart rates and a mean FHR is reported. The method is evaluated in utero on data collected from hypoxic lamb model. Ground truth for FHR is measured through hemodynamics. The results showed that using multi-detector fetal signal extraction gave up to 18.56% lower root-mean-square FHR error, and up to 57.87% lower maximum absolute FHR error compared to single-detector fetal signal extraction.

Perspective Distortion Correction for Multi-Modal Registration between Ultra-Widefield and Narrow-Angle Retinal Images.

Multi-modal retinal image registration between 2D Ultra-Widefield (UWF) and narrow-angle (NA) images has not been well-studied, since most existing methods mainly focus on NA image alignment. The stereographic projection model used in UWF imaging causes strong distortions in peripheral areas, which leads to inferior alignment quality. We propose a distortion correction method that remaps the UWF images based on estimated camera view points of NA images. In addition, we set up a CNN-based registration pipeline for UWF and NA images, which consists of the distortion correction method and three networks for vessel segmentation, feature detection and matching, and outlier rejection. Experimental results on our collected dataset shows the effectiveness of the proposed pipeline and the distortion correction method.

A Convolutional Neural Network Pipeline For Multi-Temporal Retinal Image Registration.

A sequence of images is usually captured to observe the change of health status in medical diagnosis. However, an image sequence taken over year usually suffers from severe deformation, making it time-consuming for physicians to match corresponding patterns. In this paper, we propose a coarse-to-fine pipeline for retinal image registration based on convolutional neural network. By leveraging the three components of the pipeline: feature matching, outlier rejection, and local registration, we recover the deformation and accurately align multi-temporal image sequences. Experimental results show that the proposed network is robust to severe deformation as well as illumination and contrast variations. With the proposed registration pipeline, the change of image patterns over time can be identified through visual analysis.

Evaluating differences in respiratory motion estimates during radiotherapy: a single planning 4DMRI versus daily 4DMRI

BackgroundIn radiotherapy, respiratory-induced tumor motion is typically measured using a single four-dimensional computed tomography acquisition (4DCT). Irregular breathing leads to inaccurate motion estimates, potentially resulting in undertreatment of the tumor and unnecessary dose to healthy tissue. The aim of the research was to determine if a daily pre-treatment 4DMRI-strategy led to a significantly improved motion estimate compared to single planning 4DMRI (with or without outlier rejection).Methods4DMRI data sets from 10 healthy volunteers were acquired. The first acquisition simulated a planning MRI, the respiratory motion estimate (constructed from the respiratory signal, i.e. the 1D navigator) was compared to the respiratory signal in the subsequent scans (simulating 5–29 treatment fractions). The same procedure was performed using the first acquisition of each day as an estimate for the subsequent acquisitions that day (2 per day, 4–20 per volunteer), simulating a daily MRI strategy. This was done for three outlier strategies: no outlier rejection (NoOR); excluding 5% of the respiratory signal whilst minimizing the range (Min95) and excluding the datapoints outside the mean end-inhalation and end-exhalation positions (MeanIE).ResultsThe planning MRI median motion estimates were 27 mm for NoOR, 18 mm for Min95, and 13 mm for MeanIE. The daily MRI median motion estimates were 29 mm for NoOR, 19 mm for Min95 and 15 mm for MeanIE. The percentage of time outside the motion estimate were for the planning MRI: 2%, 10% and 32% for NoOR, Min95 and MeanIE respectively. These values were reduced with the daily MRI strategy: 0%, 6% and 17%. Applying Min95 accounted for a 30% decrease in motion estimate compared to NoOR.ConclusionA daily MRI improved the estimation of respiratory motion as compared to a single 4D (planning) MRI significantly. Combining the Min95 technique with a daily 4DMRI resulted in a decrease of inclusion time of 6% with a 30% decrease of motion. Outlier rejection alone on a planning MRI often led to underestimation of the movement and could potentially lead to an underdosage.Trial registration: protocol W15_373#16.007

Modality-agnostic self-supervised deep feature learning and fast instance optimisation for multimodal fusion in ultrasound-guided interventions

Background and Objective: Fast and robust alignment of pre-operative MRI planning scans to intra-operative ultrasound is an important aspect for automatically supporting image-guided interventions. Thus far, learning-based approaches have failed to tackle the intertwined objectives of fast inference computation time and robustness to unexpectedly large motion and misalignment. In this work, we propose a novel method that decouples deep feature learning and the computation of long ranging local displacement probability maps from fast and robust global transformation prediction. Methods: In our approach, we firstly train a convolutional neural network (CNN) to extract modality-agnostic features with sub-second computation times for both 3D volumes during inference. Using sparsity-based network weight pruning, the model complexity and computation times can be substantially reduced. Based on these features, a large discretized search range of 3D motion vectors is explored to compute a probabilistic displacement map for each control point. These 3D probability maps are employed in our newly proposed, computationally efficient, instance optimisation that robustly estimates the most likely globally linear transformation that best reflects the local displacement beliefs subject to outlier rejection. Results: Our experimental validation demonstrates state-of-the-art accuracy on the challenging CuRIOUS dataset with average target registration errors of 2.50 mm, model size of only 1.2 MByte and run times of approx. 3 seconds for a full 3D multimodal registration. Conclusion: We show that a significant improvement in accuracy and robustness can be gained with instance optimisation and our fast self-supervised deep learning model can achieve state-of-the-art accuracy on challenging registration task in only 3 seconds.

Real-Time Quality Index to Control Data Loss in Real-Life Cardiac Monitoring Applications.

Wearable cardiac sensors pave the way for advanced cardiac monitoring applications based on heart rate variability (HRV). In real-life settings, heart rate (HR) measurements are subject to motion artifacts that may lead to frequent data loss (missing samples in the HR signal), especially for commercial devices based on photoplethysmography (PPG). The current study had two main goals: (i) to provide a white-box quality index that estimates the amount of missing samples in any piece of HR signal; and (ii) to quantify the impact of data loss on feature extraction in a PPG-based HR signal. This was done by comparing real-life recordings from commercial sensors featuring both PPG (Empatica E4) and ECG (Zephyr BioHarness 3). After an outlier rejection process, our quality index was used to isolate portions of ECG-based HR signals that could be used as benchmark, to validate the output of Empatica E4 at the signal level and at the feature level. Our results showed high accuracy in estimating the mean HR (median error: 3.2%), poor accuracy for short-term HRV features (e.g., median error: 64% for high-frequency power), and mild accuracy for longer-term HRV features (e.g., median error: 25% for low-frequency power). These levels of errors could be reduced by using our quality index to identify time windows with few or no data loss (median errors: 0.0%, 27%, and 6.4% respectively, when no sample was missing). This quality index should be useful in future work to extract reliable cardiac features in real-life measurements, or to conduct a field validation study on wearable cardiac sensors.

Learning Two-View Correspondences and Geometry via Local Neighborhood Correlation.

Seeking quality feature correspondences (also known as matches) is a foundational step in computer vision. In our work, a novel and effective network with a stable local constraint, named the Local Neighborhood Correlation Network (LNCNet), is proposed to capture abundant contextual information of each correspondence in the local region, followed by calculating the essential matrix and camera pose estimation. Firstly, the k-Nearest Neighbor () algorithm is used to divide the local neighborhood roughly. Then, we calculate the local neighborhood correlation matrix () between the selected correspondence and other correspondences in the local region, which is used to filter outliers to obtain more accurate local neighborhood information. We cluster the filtered information into feature vectors containing richer neighborhood contextual information so that they can be used to more accurately determine the probability of correspondences as inliers. Extensive experiments have demonstrated that our proposed LNCNet performs better than some state-of-the-art networks to accomplish outlier rejection and camera pose estimation tasks in complex outdoor and indoor scenes.

PESA-Net: Permutation-Equivariant Split Attention Network for correspondence learning

Establishing reliable correspondences by a deep neural network is an important task in computer vision, and it generally requires permutation-equivariant architecture and rich contextual information. In this paper, we design a Permutation-Equivariant Split Attention Network (called PESA-Net), to gather rich contextual information for the feature matching task. Specifically, we propose a novel “Split–Squeeze–Excitation–Union” (SSEU) module. The SSEU module not only generates multiple paths to exploit the geometrical context of putative correspondences from different aspects, but also adaptively captures channel-wise global information by explicitly modeling the interdependencies between the channels of features. In addition, we further construct a block by fusing the SSEU module, Multi-Layer Perceptron and some normalizations. The proposed PESA-Net is able to effectively infer the probabilities of correspondences being inliers or outliers and simultaneously recover the relative pose by essential matrix. Experimental results demonstrate that the proposed PESA-Net relative surpasses state-of-the-art approaches for pose estimation and outlier rejection on both outdoor scenes and indoor scenes (i.e., YFCC100M and SUN3D). Source codes: https://github.com/x-gb/PESA-Net.

The α-minimum convex polygon as a relevant tool for isotopic niche statistics

Ecological (isotopic) niche refers to a surface in a two-dimensional space, where the axes correspond to environmental variables that reflect values of stable isotopes incorporated in an animal's tissues. Carbon and nitrogen stable isotope ratios (δ13C-δ15N) notably provide precious information about trophic ecology, resource and habitat use, and population dynamics. Various metrics allow for isotopic niche size and overlap assessment. In this paper, we advocate α-minimum convex polygons (MCP) - that have long been used for home range estimation – as a relevant tool for isotopic niche size, overlap, and characteristics. The method allows for outlier rejection while being suited to data that are not Gaussian in the bivariate isotopic (δ13C-δ15N) space. The proposed indicators are compared to other existing approaches and are shown to be complementary. Notably an indicator of divergence within the niche is introduced, and allows for comparisons at low (n > 6) and different sample sizes. The R code is made publicly available and will enable ecologists to perform isotopic niche comparison, contraction and expansion assessment, and overlap, based on various methods.

Parametric Surface Fitting on Airborne Lidar Point Clouds for Building Reconstruction

Surface reconstruction is an essential step in most processing pipelines involving point clouds. By constructing a surfacic or volumetric model of the cloud, it is possible to infer large-scale semantic and geometric information required for most applications in computer graphics, simulation and virtual reality. Among the different types of point cloud and the variety of possible problems, we are interested in airborne Lidar data and the problem of building reconstruction for urban planning ranging from flood and light exposure simulation to virtual touristic visits.While most existing reconstruction methods are based on characteristic features extraction in point clouds such as planes, ridges, contours and their combination into a more complex model, we instead adopt a template-based approach relying on a library of complex primitives developed in an industrial context.This is formulated as a global fitting problem between a constrained triangulated mesh (our template) and the point cloud. More precisely, we design an energy function that takes into account the distance between both objects while integrating outliers rejection directly in our numerical optimization through the use of an M-estimator. This energy function being smooth everywhere, it can be efficiently minimized by quasi-Newtonian methods like the L-BFGS algorithm.We demonstrate the reliability of our approach on a collection of diverse roof models and several publicly available Lidar datasets, as well as its robustness and limits in function of initialization, point cloud quality and presence of outliers. By only fitting onto relevant points, this method allows a precise fitting as well as a correct outlier segmentation in a unique step, providing a reasonable initialization close to the barycenter of the cloud.