Image Context Information Research Articles

LCP-Net: A local context-perception deep neural network for medical image segmentation

Automatic image segmentation is an indispensable step in medical image analysis, and it plays an important role in computer-assisted radiotherapy, disease diagnosis and treatment effect evaluation. The difficulty of medical image segmentation is greatly enhanced by the blurry nature of medical image, the complex shape of objects and the existence of noise. In recent years, segmentation methods based on deep learning, especially convolutional neural network, have made great progress in improving the accuracy of medical image segmentation. However, these methods also have poor ability to distinguish similar objects in different environments, because of insufficient use of the local context information of images during the process of feature extraction. To address this problem, this paper proposes a deep neural network (LCP-Net) that can perceive multi-scale context information of images. LCP-Net improves the utilization of context information of feature encoders by using Parallel Dilated Convolution (PDC) and Local Context Embedding (LCE), which are beneficial to get feature map rich in environmental information. In addition, to improve the segmentation accuracy of the model for small objects and alleviate the swing issue during training, we propose a novel improved cross-entropy loss (DDCLoss), which can adaptively adjust the weight of loss according to the certainty and deviation distance of the predicted pixel value and enable the model to focus on optimizing the sample points with low certainty and tend to be mislabeled. Experimental results on three different medical datasets demonstrate that compared with the state-of-the-art medical image segmentation models, our proposed LCP-Net can achieve better segmentation performance.

CoRRN: Cooperative Reflection Removal Network.

Removing the undesired reflections from images taken through the glass is of broad application to various computer vision tasks. Non-learning based methods utilize different handcrafted priors such as the separable sparse gradients caused by different levels of blurs, which often fail due to their limited description capability to the properties of real-world reflections. In this paper, we propose a network with the feature-sharing strategy to tackle this problem in a cooperative and unified framework, by integrating image context information and the multi-scale gradient information. To remove the strong reflections existed in some local regions, we propose a statistic loss by considering the gradient level statistics between the background and reflections. Our network is trained on a new dataset with 3250 reflection images taken under diverse real-world scenes. Experiments on a public benchmark dataset show that the proposed method performs favorably against state-of-the-art methods.

Research of improving semantic image segmentation based on a feature fusion model

The context information of images had been lost due to the low resolution of features, and due to repeated combinations of max-pooling layer and down-sampling layer. When the feature extraction process had been performed using a convolutional network, the result of semantic image segmentation loses sensitivity to the location of the object. The semantic image segmentation based on a feature fusion model with context features layer-by-layer had been proposed. Firstly, the original images had been pre-processed by the Gaussian Kernel Function to generate a series of images with different resolutions to form an image pyramid. Secondly, inputting an image pyramid into the network structure in which the plurality of fully convolutional network was been combined in parallel to obtain a set of initial features with different granularities by expanding receptive fields using Atrous Convolutions, and the initialization of feature fusion with different layer-by-layer granularities in a top-down method. Finally, the score map of feature fusion model had been calculated and sent to the conditional random field, modeling the class correlations between image pixels of the original image by the fully connected conditional random field, and the spatial position information and color vector information of image pixels were jointed to optimize and obtain results. The experiments on the PASCAL VOC 2012 and PASCAL Context datasets had achieved better mean Intersection Over Union than the state-of-the-art works. The proposed method has about 6.3% improved to the conventional methods.

COMPUTATIONAL TIME MODELLING FOR TREE CROWN EXTRACTION FROM HIGH RESOLUTION IMAGERY USING GEOGRAPHIC OBJECT-BASED IMAGE ANALYSIS

Abstract. Image segmentation, unlike most conventional approaches, assesses image pixel information in relation to image contextual information to extract congruent image objects. A comprehensive review of literature during the research conceptualization phase revealed that while a lot of work has been done into the optimization of the segmentation computational time, there was a conspicuous research gap in the area of computational time modelling and pre-analytical projections.The focus of this research was, therefore, to model the computational time requirements of segmentation as a function of variability in image spatial resolutions and pixel counts. The employed methodology integrated structure from motion (SFM), segmentation, and regression modelling into a multiphase workflow to process UAV-acquired imagery into an orthomosaic, which underwent segmentation at multiple resolutions. The segmentation computational time was regressed against the image pixel count and spatial resolution.The results showed that the image pixel count was the main determinant of the computational time, expressing a statistically strong linear relationship. However, when the regressors were tested individually, the spatial resolution showed an exponential relationship with computational time which, however, exhibited saturation. It can be inferred that image spatial resolution is not the key determinant of segmentation computational time but the pixel count of the image.This research recommends the adoption of statistically viable models to thoroughly assess computational time requirements for image segmentation and a pre-analytics estimation of the time requirements such that image quality – image processing time adjustments can be made before rather than after analyses have commenced.

Image captioning with triple-attention and stack parallel LSTM

Abstract Image captioning aims to describe the content of images with a sentence. It is a natural way for people to express their understanding, but a challenging and important task from the view of image understanding. In this paper, we propose two innovations to improve the performance of such a sequence learning problem. First, we give a new attention method named triple attention (TA-LSTM) which can leverage the image context information at every stage of LSTM. Then, we redesign the structure of basic LSTM, in which not only the stacked LSTM but also the paralleled LSTM are adopted, called as PS-LSTM. In this structure, we not only use the stack LSTM but also use the parallel LSTM to achieve the improvement of the performance compared with the normal LSTM. Through this structure, the proposed model can ensemble more parameters on single model and has ensemble ability itself. Through numerical experiments, on the public available MSCOCO dataset, our final TA-PS-LSTM model achieves comparable performance with some state-of-the-art methods.

A context-sensitive deep learning approach for microcalcification detection in mammograms

A challenging issue in computerized detection of clustered microcalcifications (MCs) is the frequent occurrence of false positives (FPs) caused by local image patterns that resemble MCs. We develop a context-sensitive deep neural network (DNN), aimed to take into account both the local image features of an MC and its surrounding tissue background, for MC detection. The DNN classifier is trained to automatically extract the relevant image features of an MC as well as its image context. The proposed approach was evaluated on a set of 292 mammograms using free-response receiver operating characteristic (FROC) analysis on the accuracy both in detecting individual MCs and in detecting MC clusters. The results demonstrate that the proposed approach could achieve significantly higher FROC curves when compared to two MC-based detectors. It indicates that incorporating image context information in MC detection can be beneficial for reducing the FPs in detections.

River segmentation using satellite image contextual information and Bayesian classifier

Satellite-based remote sensing imaging can provide continuous snapshots of the Earth’s surface over long periods. River extraction from remote sensing images is useful for the comprehensive study of dynamic changes of rivers over large areas. This paper presents a new method of extracting rivers by using training samples based on the mathematical morphology, Bayesian classifier and a dynamic alteration filter. The use of a training map from erosion morphology helps to extract the non-predictive river’s curves in the image. The algorithm has two phases: creating the profile to separate river area via evaluated morphological erosion and dilation, namely, a training map; and improving the river’s image segmentation using the Bayesian rule algorithm in which two consecutive filters swipe false positive (non-water area) along the image. The proposed algorithm was tested on the Kuala Terengganu district, Malaysia, an area that includes a river, a bridge, dam and a fair amount of vegetation. The results were compared with two standard methods based on visual perception and on peak signal-to-noise ratio, respectively. The novelty of this approach is the definition of the contextual information filtering technique, which provides an accurate extraction of river segmentation from satellite images.

Image understanding and the web: a state-of-the-art review

The contextual information of Web images is investigated to address the issue of characterizing their content with semantic descriptors and therefore bridge the semantic gap, i.e. the gap between their automated low-level representation in terms of colors, textures, shapes. . . and their semantic interpretation. Such characterization allows for understanding the image content and is crucial in important Web-based tasks such as image indexing and retrieval. Although we are highly motivated by the availability of rich knowledge on the Web and the relative success achieved by commercial search engines in automatically characterizing the image content using contextual information in Web pages, we are aware that the unpredictable quality of the contextual information is a major limiting factor. Among the reasons explaining the difficulty to leverage on the image contextual information, some problems are related to the characterization and extraction of this information. Indeed, the first issue is the lack of large-scale studies to highlight what is considered the relevant contextual information of an image, where it is located in a Web page and whether it is consistent across Web pages of different types, content layouts and domains. Also, the matter related to the extraction of this contextual information is topical as state-of-the-art automated extraction tools are unable to handle the heterogeneous Web. As far as the processing of the contextual information is concerned, problems linked to the syntactic and semantic characterizations of the textual components are important to address in order to tackle the semantic gap. Furthermore, questions pertaining to the organization of these textual components into coherent structures that are usable in image indexing and retrieval frameworks shall arise.

Multifaceted conceptual image indexing on the world wide web

In this paper, we describe a user-centered design of an automated multifaceted concept-based indexing framework which analyzes the semantics of the Web image contextual information and classifies it into five broad semantic concept facets: signal, object, abstract, scene, and relational; and identifies the semantic relationships between the concepts. An important aspect of our indexing model is that it relates to the users’ levels of image descriptions. Also, a major contribution relies on the fact that the classification is performed automatically with the raw image contextual information extracted from any general webpage and is not solely based on image tags like state-of-the-art solutions. Human Language Technology techniques and an external knowledge base are used to analyze the information both syntactically and semantically. Experimental results on a human-annotated Web image collection and corresponding contextual information indicate that our method outperforms empirical frameworks employing tf–idf and location-based tf–idf weighting schemes as well as n-gram indexing in a recall/precision based evaluation framework.

Task-based annotation and retrieval for image information management

Continuing advances in digital image capture and storage are resulting in a proliferation of imagery and associated problems of information overload in image domains. In this work we present a framework that supports image management using an interactive approach that captures and reuses task-based contextual information. Our framework models the relationship between images and domain tasks they support by monitoring the interactive manipulation and annotation of task-relevant imagery. During image analysis, interactions are captured and a task context is dynamically constructed so that human expertise, proficiency and knowledge can be leveraged to support other users in carrying out similar domain tasks using case-based reasoning techniques. In this article we present our framework for capturing task context and describe how we have implemented the framework as two image retrieval applications in the geo-spatial and medical domains. We present an evaluation that tests the efficiency of our algorithms for retrieving image context information and the effectiveness of the framework for carrying out goal-directed image tasks.

Multilevel image segmentation with adaptive image context based thresholding

Neural network based image segmentation techniques primarily focus on the selection of appropriate thresholding points in the image feature space. Research initiatives in this direction aim at addressing this problem of effective threshold selection for activation functions. Multilevel activation functions resort to fixed and uniform thresholding mechanisms. These functions assume homogeneity of the image information content. In this paper, we propose a collection of adaptive thresholding approaches to multilevel activation functions. The proposed thresholding mechanisms incorporate the image context information in the thresholding process. Applications of these mechanisms are demonstrated on the segmentation of real life multilevel intensity images using a self-supervised multilayer self-organizing neural network (MLSONN) and a supervised pyramidal neural network (PyraNet). We also present a bi-directional self-organizing neural network (BDSONN) architecture suitable for multilevel image segmentation. The architecture uses an embedded adaptive thresholding mechanism to a characteristic multilevel activation function. The segmentation efficiencies of the thresholding mechanisms evaluated using four unsupervised measures of merit, are reported for the three neural network architectures considered.

The Use of the Hopfield Neural Network to Measure Sea-Surface Velocities From Satellite Images

The knowledge of ocean surface circulation is of major importance for many applications, including the understanding of global climate, resources exploitation, and containment of chemical spills. In this letter, sea-surface feature tracking based on the Hopfield neural network (NN) is described. The method is based on the minimization of an energy function that represents the feature tracking problem. A Hopfield NN is used to merge cross-correlation information with prior knowledge of sea-surface flows and image contextual information. It has been tested on real satellite images. A set of five Advanced Very High Resolution Radiometer thermal images of the coastal zone of California, along with a data set of coincident surface drifters positions, was used to test the method. Results of the new analysis are compared with in situ data and previous results using other techniques. The method can be used on various kinds of images for tracking and also find other applications in image registration and pattern recognition.

Toward robust analysis of satellite images using map information-application to urban area detection

With the rapid development of remote sensing, digital image processing has become an important tool for the quantitative and statistical analysis of remotely sensed images. These images most often contain complex natural scenes. The robust interpretation of such images requires the use of different sources of information about the scenes under consideration. This paper presents an integrated approach to robust analysis of SPOT images with the aid of map information as well as a priori knowledge about the contextual information of images. Markov random field theory and the Bayes formula are used to formulate the image analysis problem as a problem of optimization of an objective function, which in turn permits the application of various existing optimization algorithms to solve the problem. To increase the robustness of the result, several techniques are proposed to effectively use map information and image contextual information. The first one is concerned with the estimation of the parameters in the objective function with the help of these two sources of information. The second one is the integration of map information in Bayes image modeling using a Markov random field. The third one is a new optimization algorithm which takes into account map information and image contextual information by means of a feedback control scheme. The last technique proposed to increase the robustness of the result is concerned with the fusion of several (intermediate) analysis results by again using map knowledge and image contextual information for the estimation of the reliability of these results.

An adaptive technique to maximize lossless image data compression of satellite images

Data compression will play an increasingly important role in the storage and transmission of image data within the NASA science programs as the Earth Observing System comes into operation. It is important that the science data be preserved at the fidelity the instrument and satellite communication systems were designed to produce. Lossless compression must therefore be applied, at least to archive the processed instrument data. In this paper, we present an analysis of the performance of lossless compression techniques and develop an adaptive approach that applies image remapping, feature-based image segmentation to determine regions of similar entropy, as well as high-order arithmetic coding, to obtain significant improvements over the use of conventional compression techniques alone. Image remapping is used to transform the original image into a lower entropy state. Several techniques were tested on satellite images including differential pulse code modulation, bi-linear interpolation, and block-based linear predictive coding. The results of these experiments are discussed, and trade-offs between computation requirements and entropy reductions are used to identify the optimum approach for a variety of satellite images. Further entropy reduction can be achieved by segmenting the image based on local entropy properties and then applying a coding technique that maximizes compression for the region. Experimental results are presented showing the effect of different coding techniques for regions of different entropy. A rule-base is developed through which the technique giving the best compression is selected. The paper concludes that maximum compression can be achieved cost effectively and at acceptable performance rates with a combination of techniques that are selected based on image contextual information.