Confidence Map Research Articles

Tracking of infrared dim small target in complex sky background

The complex sky cloud background edge and infrared dim small target cannot be distinguished using traditional tracking algorithms, and the deviation problem is produced when tracking the target. To solve this problem, this paper proposes a target tracking algorithm based on Gaussian curvature filter (GCF) and spatio-temporal context (STC) after introducing the STC theory and analyzing the reason of deviation. First of all, the context areas are dealt with GCF which can reserve the image edge and eliminate the noise and the infrared dim small target, so as to obtain the infrared dim small targets accurate confidence map. Then the infrared dim small targets coordinates are confirmed by the confidence map. Four groups of the infrared dim small target sequences in the complex sky background experimental results prove that the proposed algorithm is effective, compared with several classical algorithms, such as TMT, MS-PF and FCT. The proposed algorithm has better performance than the three traditional algorithms on subjective vision and objective evaluating indicator and shows higher tracking accuracy and better real time performance. The infrared dim small target in complex sky cloud background can be effectively tracked with this algorithm.

Generic Workflow for Mapping of Complex Disulfide Bonds Using In-Source Reduction and Extracted Ion Chromatograms from Data-Dependent Mass Spectrometry.

Disulfide bond mapping is a critical task in protein characterization as protein stability, structure, and function is dependent on correct cysteine connectivities. Mass spectrometry (MS) is the method of choice for this, providing fast and accurate characterization of simple disulfide bonds. Disulfide mapping by liquid chromatography tandem mass spectrometry (LC-MS/MS) is performed by identifying disulfide-bonded partner peptides following proteolytic digestion. With the recently introduced ability to assign complex disulfide patterns by online postcolumn partial disulfide reduction by in-source reduction (ISR) in a LC-ISR-MS/MS methodology, the main challenge is data analysis to ensure detection of both expected and unexpected disulfide species. In this study, we introduced a workflow for confident and unbiased mapping of complex disulfide bonds using the powerful combination of extracted ion chromatograms (XICs) of LC-ISR-MS/MS data. With postcolumn partial reduction, identical LC retention times of intact disulfide-bonded species, their constituting free peptides, and partially reduced variants were observed. Subsequent selective MS/MS fragmentation of all reduction products allowed confident identification of free cysteine-containing peptides using a classical shotgun proteomics database search. Matching XICs of the identified cysteine-containing peptides allowed identification of both predicted and unpredicted disulfide species, including unforeseen proteolytic specificities, missed cleavage sites, scrambled disulfide variants, and the presence of disulfide-entangled complexes. Applying this workflow, we successfully mapped the complex disulfide bonds of tertiapin and the epidermal growth factor (EGF) family members transforming growth factor α (TGFα) and EGF. In addition, we were able to characterize the disulfide patterns of the special disulfide fold of the TGFβ superfamily in an all-online methodology.

Tomographic Imaging of Sparse Low-Contrast Targets in Harsh Environments Through Matrix Completion

A new methodology that exploits the matrix completion (MC) paradigm is proposed to image weak and sparse scatterers in heavy noise conditions. The 2-D inverse problem, mathematically formulated under the first-order Born approximation, is addressed with a three-phase algorithm that consists of: 1) an initial estimation step where a preliminary reconstruction of the distribution of the contrast and the associated “confidence map” are computed by means of a Bayesian compressive sensing method; 2) a filtering step devoted to identify and discard the less reliable contrast coefficients; and 3) a final dielectric profile completion step aimed at recovering a faithful image of the whole scattering scenario by exploiting a customized MC procedure. Representative numerical results and comparisons with competitive state-of-the-art inversion techniques are reported and discussed to assess the accuracy, the robustness, and the numerical efficiency of the proposed approach.

Real-time visual tracking by deep reinforced decision making

One of the major challenges of model-free visual tracking problem has been the difficulty originating from the unpredictable and drastic changes in the appearance of objects we target to track. Existing methods tackle this problem by updating the appearance model on-line in order to adapt to the changes in the appearance. Despite the success of these methods however, inaccurate and erroneous updates of the appearance model result in a tracker drift. In this paper, we introduce a novel real-time visual tracking algorithm based on a template selection strategy constructed by deep reinforcement learning methods. The tracking algorithm utilizes this strategy to choose the appropriate template for tracking a given frame. The template selection strategy is self-learned by utilizing a simple policy gradient method on numerous training episodes randomly generated from a tracking benchmark dataset. Our proposed reinforcement learning framework is generally applicable to other confidence map based tracking algorithms. The experiment shows that our tracking algorithm runs in real-time speed of 43 fps and the proposed policy network effectively decides the appropriate template for successful visual tracking.

Multiscale fully convolutional network for image saliency

We focus on saliency estimation in digital images. We describe why it is important to adopt a data-driven model for such an illposed problem, allowing for a universal concept of “saliency” to naturally emerge from data that are typically annotated with drastically heterogeneous criteria. Our learning-based method also involves an explicit analysis of the input at multiple scales, in order to take into account images of different resolutions, depicting subjects of different sizes. Furthermore, despite training our model on binary ground truths only, we are able to output a continuous-valued confidence map, which represents the probability of each image pixel being salient. Every contribution of our method for saliency estimation is singularly tested according to a standard evaluation benchmark, and our final proposal proves to be very effective in a comparison with the state-of-the-art.

SNOW COVER MAPPING AND ICE AVALANCHE MONITORING FROM THE SATELLITE DATA OF THE SENTINELS

Abstract. In order to monitor ice avalanches efficiently under disaster emergency conditions, a snow cover mapping method based on the satellite data of the Sentinels is proposed, in which the coherence and backscattering coefficient image of Synthetic Aperture Radar (SAR) data (Sentinel-1) is combined with the atmospheric correction result of multispectral data (Sentinel-2). The coherence image of the Sentinel-1 data could be segmented by a certain threshold to map snow cover, with the water bodies extracted from the backscattering coefficient image and removed from the coherence segment result. A snow confidence map from Sentinel-2 was used to map the snow cover, in which the confidence values of the snow cover were relatively high. The method can make full use of the acquired SAR image and multispectral image under emergency conditions, and the application potential of Sentinel data in the field of snow cover mapping is exploited. The monitoring frequency can be ensured because the areas obscured by thick clouds are remedied in the monitoring results. The Kappa coefficient of the monitoring results is 0.946, and the data processing time is less than 2 h, which meet the requirements of disaster emergency monitoring.

Learning spatio-temporal context via hierarchical features for visual tracking

Spatio-temporal context (STC) based visual tracking algorithms have demonstrated remarkable tracking capabilities in recent years. In this paper, we propose an improved STC method, which seamlessly integrates capabilities of the powerful feature representations and mappings from the convolutional neural networks (CNNs) based on the theory of transfer learning. Firstly, the dynamic training confidence map, obtained from a mapping neural network using transferred CNN features, rather than the fixed training confidence map is utilized in our tracker to adapt the practical tracking scenes better. Secondly, we exploit hierarchical features from both the original image intensity and the transferred CNN features to construct context prior models. In order to enhance the accuracy and robustness of our tracker, we simultaneously transfer the fine-grained and semantic features from deep networks. Thirdly, we adopt the training confidence index (TCI), reflected from the dynamic training confidence map, to guide the updating process. It can determine whether back propagations should be conducted in the mapping neural network, and whether the STC model should be updated. The introduction of the dynamic training confidence map could effectively deal with the problem of location ambiguity further in our tracker. Overall, the comprehensive experimental results illustrate that the tracking capability of our tracker is competitive against several state-of-the-art trackers, especially the baseline STC tracker, on the existing OTB-2015 and UAV123 visual tracking benchmarks.

Unpaved road detection based on spatial fuzzy clustering algorithm

Vision-based unpaved road detection is a challenging task due to the complex nature scene. In this paper, a novel algorithm is proposed to improve the accuracy and robustness of unpaved road detection and boundary extraction with low computational costs. The novelties of this paper are as follows: (1) We use a normal distribution with infrared images to detect the vanishing line, and a trapezoid prediction model is proposed according to the road shape features. (2) Road recognition based on connected regions is implemented by an improved support vector machine (SVM) classifier with a normalized class feature vector. According to the recognition results, the road probability confidence map is obtained. (3) With the help of fusing continuous information with the trapezoidal forecasting model and the probability from the confidence map, we present a road probability recognition method based on the trapezoidal forecasting model and spatial fuzzy clustering. Furthermore, the histogram backprojection model is used to solve interference problems caused by shadows on the road. It takes approximately 0.012~0.014 s to process one frame of an image for the road recognition, and the accuracy rate can reach 93.2%. The experimental results show that the algorithm can achieve better performance than some state-of-the-art methods in terms of detection accuracy and speed.

MRF-Based Disparity Upsampling Using Stereo Confidence Evaluations

Disparity upsampling methods are derived for restoring high-quality disparity maps from active three-dimensional imaging techniques such as time-of-flight method. Although effective, traditional upsampling methods exhibit certain drawbacks. For example, noisy disparity values in the low-scaled level are taken indiscriminately to carry out the upsampling assignment, which could lead to dramatically weakened results. To solve this problem, we herein present a Markov random field (MRF) based disparity upsampling method using confidence evaluations. We utilize confidence measures under the stereo configuration to evaluate the reliability of disparity value. The confidence maps are then properly integrated into the state-of-the-art MRF-based depth upsampling (MBU) method to constrain the negative effect caused by the noisy disparity. More specifically, the confidence is used to remove unqualified pixels from participating in the disparity upsampling computation. Extensive experiments were performed to validate the proposed new method. In these experiments, initial low-scaled disparity maps were from either ground truth or stereo matching methods. Results verified that our method can obtain significantly better upsampled disparity maps than that from the original MBU or other nonconfidence-based methods. In addition, we implemented our algorithms on general public utilities to improve the execution speed.

Image Quality Assessment for Color Correction Based on Color Contrast Similarity and Color Value Difference

Color correction plays an important role in the image processing field. But substantial research on the assessment of color correction is still insufficient. In this paper, we present an image quality assessment metric for color correction. It assesses the color consistency between the reference and target/result images of color correction according to their color contrast similarity and color value difference. Both the average difference and difference span are considered during the assessment. To compensate for the scene difference between the reference and target/result images of color correction, we propose to use an image registration algorithm to build their matching relationship, upon which a matching image is built. The matching image has the same scene as the target image and the same color feature as the reference image, and thus the matching image is regarded as the real reference image of our color correction assessment. Furthermore, we combine a confidence map of the matching image and a saliency map of the target/result image as a weighting map for assessment, which helps to improve the consistency between the objective and subjective assessment results. The experimental results show that our color correction assessment metric has better correlation, accuracy, and monotonicity with users’ subjective scores than 19 state-of-the-art metrics.

Robust visual tracking using a contextual boosting approach

In recent years, detection-based image trackers have been gaining ground rapidly, thanks to its capacity of incorporating a variety of image features. Nevertheless, its tracking performance might be compromised if background regions are mislabeled as foreground in the training process. To resolve this problem, we propose an online visual tracking algorithm designated to improving the training label accuracy in the learning phase. In the proposed method, superpixels are used as samples, and their ambiguous labels are reassigned in accordance with both prior estimation and contextual information. The location and scale of the target are usually determined by confidence map, which is doomed to shrink since background regions are always incorporated into the bounding box. To address this dilemma, we propose a cross projection scheme via projecting the confidence map for target detecting. Moreover, the performance of the proposed tracker can be further improved by adding rigid-structured information. The proposed method is evaluated on the basis of the OTB benchmark and the VOT2016 benchmark. Compared with other trackers, the results appear to be competitive.

Object tracking in the presence of shaking motions

Visual tracking can be particularly interpreted as a process of searching for targets and optimizing the searching. In this paper, we present a novel tracker framework for tracking shaking targets. We formulate the underlying geometrical relevance between a search scope and a target displacement. A uniform sampling among the search scopes is implemented by sliding windows. To alleviate any possible redundant matching, we propose a double-template structure comprising of initial and previous tracking results. The element-wise similarities between a template and its candidates are calculated by jointly using kernel functions which provide a better outlier rejection property. The STC algorithm is used to improve the tracking results by maximizing a confidence map incorporating temporal and spatial context cues about the tracked targets. For better adaptation to appearance variations, we employ a linear interpolation to update the context prior probability of the STC method. Both qualitative and quantitative evaluations are performed on all sequences that contain shaking motions and are selected from the OTB-50 challenging benchmark. The proposed approach is compared with and outperforms 12 state-of-the-art tracking methods on the selected sequences while running on MATLAB without code optimization. We have also performed further experiments on the whole OTB-50 and VOT 2015 datasets. Although the most of sequences in these two datasets do not contain motion blur that this paper is focusing on, the results of our method are still favorable compared with all of the state-of-the-art approaches.

Abstract P2-04-07: Triple negative breast cancer-specific chromatin conformation links Notch signal to tumor-specific transcriptional program

Abstract Triple-negative breast cancer (TNBC) is an aggressive subtype of breast cancer that is currently incurable with conventional chemotherapy. Some TNBC patients harbor either recurrent somatic gain-of-function mutations or focal amplification in genes coding for Notch receptors. Recent findings suggest that Notch signals, in the absence of Notch mutations are also important to maintain chemotherapy-resistant circulating tumor cells. Importantly, elevated Notch signaling is specifically enriched in the TNBC and is associated with worse survival in this subtype. These findings underscore the importance of Notch signaling in pathogenesis of TNBC and suggest potential new avenue for therapeutic approaches. Despite this, the target genes and oncogenic mechanisms downstream of aberrant Notch signaling are poorly understood. To create a high confidence map of the oncogenic Notch-driven regulome in TNBC, we performed an integrative, comprehensive and functional genome-wide analysis of Notch-driven targets in Notch-mutated TNBC cell lines by combining high-resolution genome-wide chromatin topology data (HiChIP), dynamic mapping of Notch transcriptional complex members and active and repressive chromatin marks (ChIP-seq), as well as evaluation of dynamic transcription (RNAseq) in presence and absence of Notch signals. One important oncogene and critical Notch target in TNBC is MYC. Our integrative analysis identified detailed map of Notch-dependent MYC regulatory network. The TNBC-specific MYC regulatory region, located 5' of MYC, is much more complex compared to other known Notch-driven malignancies and constitutes of a broad domain of active enhancers marked by H3K27ac that are not present in Notch-driven lymphoma and leukemia. Integrative analysis of ChIP-seq and HiChIP revealed multiple enhancers within this broad domain of regulatory elements that are directly bound by the members of Notch transcriptional complex, sensitive to acute changes in Notch activation and that contact MYC promoter via tumor specific chromatin loops. We showed that genetic perturbation of MYC by CRISPR/Cas9 approach was sufficient to kill TNBC cells. Since, MYC is difficult to target pharmacologically an alternative strategy is to use drugs that inhibit factors regulating MYC expression. To this end, elucidating the Notch regulatory landscape, and in particular, the mechanism by which Notch regulates MYC expression in TNBC may reveal new therapeutic strategies to precisely target MYC in Notch-dependent breast cancer. Citation Format: Petrovic J, Zhou Y, Georgakilas G, Vahedi G, Pear WS, Faryabi RB. Triple negative breast cancer-specific chromatin conformation links Notch signal to tumor-specific transcriptional program [abstract]. In: Proceedings of the 2017 San Antonio Breast Cancer Symposium; 2017 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2018;78(4 Suppl):Abstract nr P2-04-07.

Online classification for object tracking based on superpixel

Abstract Treating object tracking as a binary classification problem has been greatly explored in recent years. State-of-the-art classification based trackers perform better robustness than many of the other existing trackers. In this paper, we propose a collaborative model by incorporating the local and holistic models together which is corresponding to discriminative and generative models in tracking-classification-framework. At the local level, an on-line Random Forest (RF) classifier is trained to distinguish the superpixels of the object from the background. A series of local superpixels are used to represent the target, so as to adapt the appearance variances. The discriminative model is used to classify superpixels in the next frame as either belonging to the object or background. A confidence map consisting of dependability and stability is formed to measure the probabilities of superpxiels pertaining to the target from the classifier. A modified mean-shift is proposed to work on the confidence map to find the peak, where is the position of the target. Meanwhile, a separate component for managing the training set dynamically is employed to control the updating of the RF model. At the global level, the target is represented by covariance matrix of multi-scale bounding boxes. The generative model is applied for protection measure, which can effectively reduce the drifts during tracking. Experimental results demonstrate that our method is effective and performs favorably in comparison to the state-of-the-art trackers.

Visual object tracking with multi-scale superpixels and color-feature guided kernelized correlation filters

While discriminative correlation filter based tracking algorithms have achieved competitive results and demonstrated successfully, there still remain some challenges, such as handling the scenarios of scale variation, fast motion, etc. A region of interest with a fixed size, which is usually used by discriminative correlation filter based tracking algorithms to train correlation filter and track object, makes the trackers hard to deal with the challenges of fast motion and scale variation. It also restricts the use of object structure information. In this paper, we propose a Multi-Scale Superpixels and Color Feature Guided Kernelized Correlation Filters (MSSCF-KCF) to deal with the problems mentioned above. Firstly, we treat tracking procedure as optimizing the combination of components of an object, and propose a multi-scale superpixel method to segment object image based on a proposed global confidence mask which determines the center and size of object patches automatically. Then, KCF is embedded into Bayesian filter framework, in which the proposed color guided confidence map is viewed as an observation model, to expand the tracking range and improve the performance when handling fast motion. Finally, we propose a center distance matrix of each object patch to use structure information of object. A novel min–max criterion as well as drop-out strategy is used to search the optimal combination and estimate the state of object, which also helps the tracker to cope with scale variation. Besides, monitoring update strategy is proposed to monitor tracking operation and adjust parameters dynamically. The proposed algorithm is tested on a self-made dataset and some challenging sequences with a novel evaluation criterion. The results demonstrate the feasibility and effectiveness compared with the state-of-the-art methods.

Visual tracking via improving motion model and model updater

Motion model and model updater are two necessary components for online visual tracking. On the one hand, an effective motion model needs to strike the right balance between target processing, to account for the target appearance and scene analysis, and to describe stable background information. Most conventional trackers focus on one aspect out of the two and hence are not able to achieve the correct balance. On the other hand, the admirable model update needs to consider both the tracking speed and the model drift. Most tracking models are updated on every frame or fixed frames, so it cannot achieve the best performance. In this article, we solve the motion model problem by collaboratively using salient region detection and image segmentation. Particularly, the two methods are for different purposes. In the absence of prior knowledge, the former considers image attributes like color, gradient, edges, and boundaries then forms a robust object; the latter aggregates individual pixels into meaningful atomic regions by using the prior knowledge of target and background in the video sequence. Taking advantage of their complementary roles, we construct a more reasonable confidence map. For model update problems, we dynamically update the model by analyzing scene with image similarity, which not only reduces the update frequency of the model but also suppresses the model drift. Finally, we use these improved building blocks not only to do comparative tests but also to give a basic tracker, and extensive experimental results on OTB50 show that the proposed methods perform favorably against the state-of-the-art methods.

Random Walker Framework for Sensor-Based Echocardiography Fusion

Image fusion techniques in 3-D echocardiography attempt to improve the field-of-view by combining multiple 3-D ultrasound (3-DUS) volumes. Echocardiography fusion techniques are mostly based on either image registration or sensor tracking. Compared to registration techniques, sensor tracking approaches are image independent and do not need any spatial overlap between the images. Once the images are spatially aligned the pixel intensities in the overlapping regions are determined using fusion algorithms such as average fusion (AVG) and maximum fusion (MAX). However, averaging generally results in reduced contrast while maximizing results in amplification of noise artifacts in the fused image. Wavelet fusion (WAV) overcomes these issues by selectively enhancing the low-frequency components in the image, but this could result in pixelation artifacts. We propose a new method for image fusion based on a generalized random walker framework (GRW) using ultrasound confidence maps. The maps are based on: 1) focal properties of the transducer and 2) second order image features. The fusion technique was validated on image pairs sampled from 3-DUS volumes acquired from six healthy volunteers. All the images were spatially aligned using optical tracking, and the fusion algorithm was used to determine the pixel intensity in the overlapping region. Comparisons based on quantitative measures showed statistically significant improvements for GRW (p < 0:01) when compared to AVG, MAX, and WAV for contrast-to-noise ratio: 0.85 ± 0.03, signal-to-noise ratio: 7.42 ± 1.98, Wang–Bovik metric (Q0): 0.80 ± 0.15. The Piella metric (Q1): 0.82 ± 0.01 also gave higher values for GRW, but the difference was not statistically significant. Upon visual inspection, the GRW fusion had the lowest amount of stitching and pixelation artifacts. The fusion technique proposed could help in improving the diagnostic accuracy and clinical acceptance of 3-D echocardiography.

Selective Video Object Cutout.

Conventional video segmentation approaches rely heavily on appearance models. Such methods often use appearance descriptors that have limited discriminative power under complex scenarios. To improve the segmentation performance, this paper presents a pyramid histogram-based confidence map that incorporates structure information into appearance statistics. It also combines geodesic distance-based dynamic models. Then, it employs an efficient measure of uncertainty propagation using local classifiers to determine the image regions, where the object labels might be ambiguous. The final foreground cutout is obtained by refining on the uncertain regions. Additionally, to reduce manual labeling, our method determines the frames to be labeled by the human operator in a principled manner, which further boosts the segmentation performance and minimizes the labeling effort. Our extensive experimental analyses on two big benchmarks demonstrate that our solution achieves superior performance, favorable computational efficiency, and reduced manual labeling in comparison to the state of the art.

A Saliency Prior Context Model for Real-Time Object Tracking

Real-time object tracking has wide applications in time-critical multimedia processing areas such as motion analysis and human–computer interaction. It remains a hard problem to balance between accuracy and speed. In this paper, we present a fast real-time context-based visual tracking algorithm with a new saliency prior context (SPC) model. Based on the probability formulation, the tracking problem is solved by sequentially maximizing the computed confidence map of target location in each video frame. To handle the various cases of feature distributions generated from different targets and their contexts, we exploit low-level features as well as fast spectral analysis for saliency to build a new prior context model. Then, based on this model and a spatial context model learned online, a confidence map is computed and the target location is estimated. In addition, under this framework, the tracking procedure can be accelerated by the fast Fourier transform. Therefore, the new method generally achieves a real-time running speed. Extensive experiments show that our tracking algorithm based on the proposed SPC model achieves real-time computation efficiency with overall best performance comparing with other state-of-the-art methods.

A confidence map based damage assessment approach using pulsed thermographic inspection

In the context of non-destructive testing, quantification of uncertainty caused by various factors such as inspection technique, testing environment and the operator is important and challenge. This paper introduces a concept of contour-based confidence map and an application framework for pulsed thermography that offers enhanced flexibility and reliability of inspection. This approach has been successfully applied to detect three flat-bottom holes of diameter 32, 16 and 8 mm drilled onto a 5 mm thick aluminium plate with a high accuracy of damage detection (R > 0.97). Its suitability and effectiveness in assessing impact damage occurring in composites have also been demonstrated.