Confidence Map Research Articles

Multi-Lane Detection and Tracking Using Temporal-Spatial Model and Particle Filtering

Since vision-based methods are vulnerable to challenging environmental conditions such as bad whether, shadows, occlusions and illumination, robustness is important. In this paper, a robust algorithm for detecting and tracking multiple lanes with arbitrary shape is proposed. We extend previous lane detection and tracking processes from the spatial domain to the temporal-spatial domain using our more robust and general multiple lane model. Distinct from traditional image pre-processing procedures, we adopt slice images generated from image sequence decomposition and reconstruction that contains important structured historical information. In this domain, we propose assumptions to reduce the algorithm’s complexity and computation further. First, instead of common binarization, a more general lane marker detector is proposed with weak constraints to obtain cleaner binary image. Then, the lane marker candidates are used as the initialization of following particle filtering tracking process. We propose a confidence map computed from binary slice images by using an improved distance transform for particles sampling and weights calculation. The multi-lane model supports the use of multiple independent particle filters for tracking each lane boundary separately. Finally, a range of experimental results on testing dataset indicate that the proposed algorithm is effective with high performance and comparing with other methods, it has better tolerance to challenging environments.

Using hyperspectral imagery to investigate large-scale seagrass cover and genus distribution in a temperate coast

Seagrasses are regarded as indicators and first line of impact for anthropogenic activities affecting the coasts. The underlying mechanisms driving seagrass cover however have been mostly studied on small scales, making it difficult to establish the connection to seagrass dynamics in an impacted seascape. In this study, hyperspectral airborne imagery, trained from field surveys, was used to investigate broadscale seagrass cover and genus distribution along the coast of Adelaide, South Australia. Overall mapping accuracy was high for both seagrass cover (98%, Kappa = 0.93), and genus level classification (85%, Kappa = 0.76). Spectral separability allowed confident genus mapping in waters up to 10 m depth, revealing a 3.5 ratio between the cover of the dominant Posidonia and Amphibolis. The work identified the absence of Amphibolis in areas historically affected by anthropogenic discharges, which occasionally contained Posidonia and might be recovering. The results suggest hyperspectral imagery as a useful tool to investigate the interplay between seagrass cover and genus distribution at large spatial scales.

A Crowd Counting Framework Combining with Crowd Location

In the past ten years, crowd detection and counting have been applied in many fields such as station crowd statistics, urban safety prevention, and people flow statistics. However, obtaining accurate positions and improving the performance of crowd counting in dense scenes still face challenges, and it is worthwhile devoting much effort to this. In this paper, a new framework is proposed to resolve the problem. The proposed framework includes two parts. The first part is a fully convolutional neural network (CNN) consisting of backend and upsampling. In the first part, backend uses the residual network (ResNet) to encode the features of the input picture, and upsampling uses the deconvolution layer to decode the feature information. The first part processes the input image, and the processed image is input to the second part. The second part is a peak confidence map (PCM), which is proposed based on an improvement over the density map (DM). Compared with DM, PCM can not only solve the problem of crowd counting but also accurately predict the location of the person. The experimental results on several datasets (Beijing-BRT, Mall, Shanghai Tech, and UCF_CC_50 datasets) show that the proposed framework can achieve higher crowd counting performance in dense scenarios and can accurately predict the location of crowds.

Automatic velocity picking from semblances with a new deep-learning regression strategy: Comparison with a classification approach

The physical basis, parameterization, and assumptions involved in root-mean-square (rms) velocity estimation have not significantly changed since they were first developed. However, these three aspects are all good targets for novel application of the recent emergence of machine learning (ML). Therefore, it is useful at this time to provide a tutorial overview of two state-of-the-art ML implementations; we have designed and evaluated classification and regression neural networks for the extraction of apparent rms velocity trajectories from semblance data. Both networks share a similar end-to-end trainable structure, except for the final layer. In the classification network, the velocity picking is performed by finding the largest amplitude trajectory through all velocity bins. The regression network, on the other hand, applies a differentiable soft-argmax function that converts the feature maps directly to apparent rms velocity values as functions of traveltime. Relative confidence maps can also be estimated from both neural networks. A large number of synthetic models with horizontal layers are created, and common-midpoint gathers are simulated from those models as training samples. Transfer learning is applied to fine-tune the networks with a small number of samples for testing with synthetic and field data from more complicated (2D) models. Tests using synthetic data show that the regression and classification networks can give reasonable velocity predictions from semblances, but the regression network gives higher accuracy.

Image based analysis of meibomian gland dysfunction using conditional generative adversarial neural network

ObjectiveMeibomian gland dysfunction (MGD) is a primary cause of dry eye disease. Analysis of MGD, its severity, shapes and variation in the acini of the meibomian glands (MGs) is receiving...

ISTDet: An efficient end-to-end neural network for infrared small target detection

Infrared small target detection has made many breakthroughs in early warning, guidance and battlefield intelligence. However, infrared small target occupies less pixels and lacks color and texture features, which makes infrared small target detection a challenging subject. To achieve the infrared small target detection, an efficient end-to-end network ISTDet is proposed in this paper. ISTDet mainly consists of two modules, including image filtering module and infrared small target detection module. The image filtering module is proposed to obtain the confidence map, aiming to enhance the response of infrared small targets and suppress the response of background. The infrared small target detection module takes the infrared image activated by the confidence map as input, aiming to speculate the category and position of the infrared small targets. Multi-task loss function is used to train the ISTDet in an end-to-end way. Finally, we do comparative experiments on five infrared small target sequences to demonstrate the detection performance of ISTDet. The results show ISTDet has better performance for infrared small target detection compared with other detectors.

Deep Outlier Handling for Image Deblurring.

Outlier handling has attracted considerable attention recently but remains challenging for image deblurring. Existing approaches mainly depend on iterative outlier detection steps to explicitly or implicitly reduce the influence of outliers on image deblurring. However, these outlier detection steps usually involve heuristic operations and iterative optimization processes, which are complex and time-consuming. In contrast, we propose to learn a deep convolutional neural network to directly estimate the confidence map, which can identify reliable inliers and outliers from the blurred image and thus facilitates the following deblurring process. We analyze that the proposed algorithm incorporated with the learned confidence map is effective in handling outliers and does not require ad-hoc outlier detection steps which are critical to existing outlier handling methods. Compared to existing approaches, the proposed algorithm is more efficient and can be applied to both non-blind and blind image deblurring. Extensive experimental results demonstrate that the proposed algorithm performs favorably against state-of-the-art methods in terms of accuracy and efficiency.

Semi-supervised semantic segmentation using an improved generative adversarial network

This paper proposes a better semi-supervised semantic segmentation network using an improved generative adversarial network. It is important for the discriminator on the pixel level to know whether it correctly distinguishes the predicted probability map. However, currently there is no correlation between the actual credibility and the confidence map generated by the pixel-level discriminator. We study this problem and a new network is proposed, which includes one generator and two discriminators. One of the discriminators can output more reliable confidence maps on the pixel level and the other is trained to generate the probability on the image level, which is used as the dynamic threshold in the semi-supervised module instead of being set manually. In addition, the trusted region shared by the two discriminators is used to provide the semi-supervised reference. Through experiments on the PASCAL VOC 2012 and Cityscapes datasets, the proposed network brings better gains, proving the effectiveness of the network.

The Third Generation of Pan-Canadian Wetland Map at 10 m Resolution Using Multisource Earth Observation Data on Cloud Computing Platform

Development of the Canadian Wetland Inventory Map (CWIM) has thus far proceeded over two generations, reporting the extent and location of bog, fen, swamp, marsh, and water wetlands across the country with increasing accuracy. Each generation of this training inventory has improved the previous results by including additional reference wetland data and focusing on processing at the scale of ecozone, which represent ecologically distinct regions of Canada. The first and second generations attained relatively highly accurate results with an average approaching 86% though some overestimated wetland extents, particularly of the swamp class. The current research represents a third refinement of the inventory map. It was designed to improve the overall accuracy (OA) and reduce wetlands overestimation by modifying test and train data and integrating additional environmental and remote sensing datasets, including countrywide coverage of L-band ALOS PALSAR-2, SRTM, and Arctic digital elevation model, nighttime light, temperature, and precipitation data. Using a random forest classification within Google Earth Engine, the average OA obtained for the CWIM3 is 90.53%, an improvement of 4.77% over previous results. All ecozones experienced an OA increase of 2% or greater and individual ecozone OA results range between 94% at the highest to 84% at the lowest. Visual inspection of the classification products demonstrates a reduction of wetland area overestimation compared to previous inventory generations. In this study, several classification scenarios were defined to assess the effect of preprocessing and the benefits of incorporating multisource data for large-scale wetland mapping. In addition, the development of a confidence map helps visualize where current results are most and least reliable given the amount of wetland test and train data and the extent of recent landscape disturbance (e.g., fire). The resulting OAs and wetland areal extent reveal the importance of multisource data and adequate test and train data for wetland classification at a countrywide scale.

Local Semantic Enhanced ConvNet for Aerial Scene Recognition.

Aerial scene recognition is challenging due to the complicated object distribution and spatial arrangement in a large-scale aerial image. Recent studies attempt to explore the local semantic representation capability of deep learning models, but how to exactly perceive the key local regions remains to be handled. In this paper, we present a local semantic enhanced ConvNet (LSE-Net) for aerial scene recognition, which mimics the human visual perception of key local regions in aerial scenes, in the hope of building a discriminative local semantic representation. Our LSE-Net consists of a context enhanced convolutional feature extractor, a local semantic perception module and a classification layer. Firstly, we design a multi-scale dilated convolution operators to fuse multi-level and multi-scale convolutional features in a trainable manner in order to fully receive the local feature responses in an aerial scene. Then, these features are fed into our two-branch local semantic perception module. In this module, we design a context-aware class peak response (CACPR) measurement to precisely depict the visual impulse of key local regions and the corresponding context information. Also, a spatial attention weight matrix is extracted to describe the importance of each key local region for the aerial scene. Finally, the refined class confidence maps are fed into the classification layer. Exhaustive experiments on three aerial scene classification benchmarks indicate that our LSE-Net achieves the state-of-the-art performance, which validates the effectiveness of our local semantic perception module and CACPR measurement.

Light Field View Synthesis via Aperture Disparity and Warping Confidence Map

This paper presents a learning-based approach to synthesize the view from an arbitrary camera position given a sparse set of images. A key challenge for this novel view synthesis arises from the reconstruction process, when the views from different input images may not be consistent due to obstruction in the light path. We overcome this by jointly modeling the epipolar property and occlusion in designing a convolutional neural network. We start by defining and computing the aperture disparity map, which approximates the parallax and measures the pixel-wise shift between two views. While this relates to free-space rendering and can fail near the object boundaries, we further develop a warping confidence map to address pixel occlusion in these challenging regions. The proposed method is evaluated on diverse real-world and synthetic light field scenes, and it shows better performance over several state-of-the-art techniques.

A Confidence Habitats Methodology in MR Quantitative Diffusion for the Classification of Neuroblastic Tumors.

Simple SummaryThere is growing interest in applying quantitative diffusion techniques to magnetic resonance imaging for cancer diagnosis and treatment. These measurements are used as a surrogate marker of tumor cellularity and aggressiveness, although there may be factors that introduce some bias to these approaches. Thus, we explored a novel methodology based on confidence habitats and voxel uncertainty to improve the power of the apparent diffusion coefficient to discriminate between benign and malignant neuroblastic tumor profiles in children. We were able to show this offered an improved sensitivity and negative predictive value relative to standard voxel-based methodologies.Background/Aim: In recent years, the apparent diffusion coefficient (ADC) has been used in many oncology applications as a surrogate marker of tumor cellularity and aggressiveness, although several factors may introduce bias when calculating this coefficient. The goal of this study was to develop a novel methodology (Fit-Cluster-Fit) based on confidence habitats that could be applied to quantitative diffusion-weighted magnetic resonance images (DWIs) to enhance the power of ADC values to discriminate between benign and malignant neuroblastic tumor profiles in children. Methods: Histogram analysis and clustering-based algorithms were applied to DWIs from 33 patients to perform tumor voxel discrimination into two classes. Voxel uncertainties were quantified and incorporated to obtain a more reproducible and meaningful estimate of ADC values within a tumor habitat. Computational experiments were performed by smearing the ADC values in order to obtain confidence maps that help identify and remove noise from low-quality voxels within high-signal clustered regions. The proposed Fit-Cluster-Fit methodology was compared with two other methods: conventional voxel-based and a cluster-based strategy. Results: The cluster-based and Fit-Cluster-Fit models successfully differentiated benign and malignant neuroblastic tumor profiles when using values from the lower ADC habitat. In particular, the best sensitivity (91%) and specificity (89%) of all the combinations and methods explored was achieved by removing uncertainties at a 70% confidence threshold, improving standard voxel-based sensitivity and negative predictive values by 4% and 10%, respectively. Conclusions: The Fit-Cluster-Fit method improves the performance of imaging biomarkers in classifying pediatric solid tumor cancers and it can probably be adapted to dynamic signal evaluation for any tumor.

ROSE: A Retinal OCT-Angiography Vessel Segmentation Dataset and New Model.

Optical Coherence Tomography Angiography (OCTA) is a non-invasive imaging technique that has been increasingly used to image the retinal vasculature at capillary level resolution. However, automated segmentation of retinal vessels in OCTA has been under-studied due to various challenges such as low capillary visibility and high vessel complexity, despite its significance in understanding many vision-related diseases. In addition, there is no publicly available OCTA dataset with manually graded vessels for training and validation of segmentation algorithms. To address these issues, for the first time in the field of retinal image analysis we construct a dedicated Retinal OCTA SEgmentation dataset (ROSE), which consists of 229 OCTA images with vessel annotations at either centerline-level or pixel level. This dataset with the source code has been released for public access to assist researchers in the community in undertaking research in related topics. Secondly, we introduce a novel split-based coarse-to-fine vessel segmentation network for OCTA images (OCTA-Net), with the ability to detect thick and thin vessels separately. In the OCTA-Net, a split-based coarse segmentation module is first utilized to produce a preliminary confidence map of vessels, and a split-based refined segmentation module is then used to optimize the shape/contour of the retinal microvasculature. We perform a thorough evaluation of the state-of-the-art vessel segmentation models and our OCTA-Net on the constructed ROSE dataset. The experimental results demonstrate that our OCTA-Net yields better vessel segmentation performance in OCTA than both traditional and other deep learning methods. In addition, we provide a fractal dimension analysis on the segmented microvasculature, and the statistical analysis demonstrates significant differences between the healthy control and Alzheimer's Disease group. This consolidates that the analysis of retinal microvasculature may offer a new scheme to study various neurodegenerative diseases.

Iterative Refinement of Uniformly Focused Image Set for Accurate Depth from Focus

Estimating the 3D shape of a scene from differently focused set of images has been a practical approach for 3D reconstruction with color cameras. However, reconstructed depth with existing depth from focus (DFF) methods still suffer from poor quality with textureless and object boundary regions. In this paper, we propose an improved depth estimation based on depth from focus iteratively refining 3D shape from uniformly focused image set (UFIS). We investigated the appearance changes in spatial and frequency domains in iterative manner. In order to achieve sub-frame accuracy in depth estimation, optimal location of focused frame in DFF is estimated by fitting a polynomial curve on the dissimilarity measurements. In order to avoid wrong depth values on texture-less regions we propose to build a confidence map and use it to identify erroneous depth estimations. We evaluated our method on public and our own datasets obtained from different types of devices, such as smartphones, medical, and normal color cameras. Quantitative and qualitative evaluations on various test image sets show promising performance of the proposed method in depth estimation.

A workflow for seismic imaging with quantified uncertainty

The interpretation of seismic images faces challenges due to the presence of several uncertainty sources. Uncertainties exist in data measurements, source positioning, and subsurface geophysical properties. Understanding uncertainties’ role and how they influence the outcome is fundamental in the earth sciences and essential in the oil and gas industry decision-making process. Geophysical imaging is time-consuming. When we add uncertainty quantification, it becomes both time and data-intensive. In this work, we propose a workflow for seismic imaging with quantified uncertainty. We build the workflow upon Bayesian tomography, reverse time migration, and image interpretation based on statistical information. The workflow explores an efficient hybrid parallel computational strategy to decrease the reverse time migration execution time. High levels of data compression are applied to reduce data transfer among workflow activities and data storage. We capture and analyze provenance data at runtime to improve workflow execution, monitoring, and debugging with negligible overhead. Numerical experiments on the Marmousi2 Velocity Model Benchmark demonstrate the workflow capabilities. We observe excellent weak and strong scalability, and results suggest that lossy data compression does not hamper the seismic imaging uncertainty quantification. We explore the propagation of the velocity uncertainties to the seismic images by confidence maps and probability density functions in control points. The multi-modal character revealed by such high-order statistics reinforces the need for the machinery we have put together in our workflow.

Online glass confidence map building using laser rangefinder for mobile robots

Accurate localization and mapping are essential for mobile robots. Using laser rangefinders (LRFs), current state-of-the-art indoor Simultaneous Localization and Mapping (SLAM) can provide accurate real-time localization and mapping in most environments. An exemption are those where glass is predominant, as LRFs can not properly detect glass due to glass' transparency and reflectiveness. With such buildings becoming more common, this has become an important issue to address. Failure to detect glass causes two problems for SLAM: incorrectly mapping glass as open space; and, lower localization accuracy due to mismatches between measured and expected range data. In this paper, we propose a glass confidence map that correctly maps glass as occupied, as well as the probability of an object to be glass/non-glass. Our approach consists of four steps: (i) map all objects, even potential dynamic obstacles, as occupied, (ii) compute the probability of scanned objects to be glass/non-glass using a neural network, (iii) online map updates by matching scanned objects to probability map, and (iv) filter dynamic obstacles and noise. We validated our approach in an office with large glass areas, achieving more than 95% of glass areas correctly mapped as occupied with less than 5% glass/non-glass classification error.

AUTOMATIC EXTRACTION OF ROAD CENTERLINES AND EDGE LINES FROM AERIAL IMAGES VIA CNN-BASED REGRESSION

Abstract. Extracting roads from aerial images is a challenging task in the field of remote sensing. Most approaches formulate road extraction as a segmentation problem and use thinning and edge detection to obtain road centerlines and edge lines, which could produce spurs around the extracted centerlines/edge lines. In this study, a novel regression-based method is proposed to extract road centerlines and edge lines directly from aerial images. The method consists of three major steps. First, an end-to-end regression network based on CNN is trained to predict confidence maps for road centerlines and estimate road width. Then, after the CNN predicts the confidence map, non-maximum suppression and road tracking are applied to extract accurate road centerlines and construct road topology. Meanwhile, Road edge lines are generated based on the road width estimated by the CNN. Finally, in order to improve the connectivity of extracted road network, tensor voting is applied to detect road intersections and the detected intersections are used as guidance for the overcome of discontinuities. The experiments conducted on the SpaceNet and DeepGlobe datasets show that our approach achieves better performance than other methods.

Detection and Localization of Ultrasound Scatterers Using Convolutional Neural Networks.

Delay-and-sum (DAS) beamforming is unable to identify individual scatterers when their density is so high that their point spread functions overlap. This paper proposes a convolutional neural network (CNN)-based method to detect and localize high-density scatterers, some of which are closer than the resolution limit of delay-and-sum (DAS) beamforming. A CNN was designed to take radio frequency channel data and return non-overlapping Gaussian confidence maps. The scatterer positions were estimated from the confidence maps by identifying local maxima. On simulated test sets, the CNN method with three plane waves achieved a precision of 1.00 and a recall of 0.91. Localization uncertainties after excluding outliers were ±46 [Formula: see text] (outlier ratio: 4%) laterally and ±26 [Formula: see text] (outlier ratio: 1%) axially. To evaluate the proposed method on measured data, two phantoms containing cavities were 3-D printed and imaged. For the phantom study, the training data were modified according to the physical properties of the phantoms and a new CNN was trained. On an uniformly spaced scatterer phantom, a precision of 0.98 and a recall of 1.00 were achieved with the localization uncertainties of ±101 [Formula: see text] (outlier ratio: 1%) laterally and ±37 [Formula: see text] (outlier ratio: 1%) axially. On a randomly spaced scatterer phantom, a precision of 0.59 and a recall of 0.63 were achieved. The localization uncertainties were ±132 [Formula: see text] (outlier ratio: 0%) laterally and ±44 [Formula: see text] with a bias of 22 [Formula: see text] (outlier ratio: 0%) axially. This method can potentially be extended to detect highly concentrated microbubbles in order to shorten data acquisition times of super-resolution ultrasound imaging.

Shading-aware shadow detection and removal from a single image

Shadow removal is a challenging problem due to its sensitivity to lighting and material conditions. In this paper, we propose a shading-aware shadow processing algorithm, which can automatically detect and remove complex shadows from a single color image. Our framework consists of two key steps. We firstly conduct a shadow-preserving filter upon the image which will effectively remove the image texture while preserving the shadow and shading information. Shadow regions are estimated by establishing a confidence map from the filtered image incorporating depth cue. We then develop a shading-aware optimization framework to remove shadows and recover shading in these regions. The extensive experimental results show that the proposed algorithm produces visually compelling results in a series of challenging images and it can handle complex shadows in both indoor and outdoor scenes. Quantitative and qualitative comparisons with current state-of-the-art methods strongly demonstrate the efficacy of our proposed approach.

An Accurate and Efficient 1-D Barcode Detector for Medium of Deployment in IoT Systems

Camera-based 1-D barcode detectors have a lot of applications in Internet-of-Things (IoT) systems (e.g., retail, air travel, post and parcel services, and manufacturing). Based on the observation that 1-D barcodes always come with a 12-digit product code, this article proposes an end-to-end trainable and fully convoluted model that can detect and output accurate localization results of 1-D barcode and product code simultaneously. Our method uses dilated convolutions-based feature extractors which are then combined with systematic feature merging layers to create a U-shaped network. It predicts multichannel feature maps which later yield localization results after thresholding with a confidence map generated by the model and nonmaximum suppression. Furthermore, we use a Taylor series expansion-based criterion to rank and eliminate a subset of least important convolutional filters of the model, which further increases the inference speed to a great extent. This model can act as a preprocessing module for camera-based barcode decoders. Experimental results on the data set combined from public data sets and self-collected retail products data set obtained in more challenging environment conditions demonstrate that our strategy is effective in increasing the decoding rate of existing commercial barcode decoders efficiently.