Global Feature Map Research Articles

Hierarchical Object Relationship Constrained Monocular Depth Estimation.

Monocular depth estimation has been gaining growing momentum in recent years. Despite significant advances of this task, due to the inherent difficulty of reliably capturing contextual cues from RGB images, it remains challenging to accurately predict depth in scenes with complicated and cluttered spatial arrangement of objects. Instead of naively utilizing the primary features in the single RGB image, in this paper we propose a hierarchical object relationship constrained network for monocular depth estimation, which could enable accurate and smooth depth prediction from monocular RGB image. The key idea of our method is to exploit object-centric hierarchical relationship as contextual constraints to compensate for the regularity of spatial depth changing. In particular, we design a semantics-guided CNN network to encode the original image into a global context feature map and encode the objects’ relationship into a local relationship feature map simultaneously, so that we can leverage such effective and consolidated coding scheme over scenario samples to guide the depth prediction in a more accurate way. Benefiting from the local-to-global context constraints, our method can well respect the global depth changing and preserve the local depth details at the same time. In addition, our approach could make full use of the hierarchical semantic relationship across inner-object components and neighboring objects to define depth changing constraints. We conduct extensive experiments and make comprehensive evaluations on widely-used public datasets, and the experiments confirm that our method outperforms most state-of-the-art depth estimation methods in preserving the local details in depth.

Attention-guided image captioning with adaptive global and local feature fusion

Although attention mechanisms are exploited widely in encoder-decoder neural network-based image captioning framework, the relation between the selection of salient image regions and the supervision of spatial information on local and global representation learning was overlooked, thereby degrading captioning performance. Consequently, we propose an image captioning scheme based on adaptive spatial information attention (ASIA), extracting a sequence of spatial information of salient objects in a local image region or an entire image. Specifically, in the encoding stage, we extract the object-level visual features of salient objects and their spatial bounding-box. We obtain the global feature maps of an entire image, which are fused with local features and the fused features are fed into the LSTM-based language decoder. In the decoding stage, our adaptive attention mechanism dynamically selects the corresponding image regions specified by an image description. Extensive experiments conducted on two datasets demonstrate the effectiveness of the proposed method.

Deep multi-scale attentional features for medical image segmentation

Automatic segmentation of medical images is a difficult task in the field of computer vision owing to the various backgrounds, shapes, size, and colors of polyps or tumors. Despite the success of deep learning (DL)-based encoder–decoder architectures in medical image segmentation, these models have several disadvantages. First, an architecture such as U-Net cannot encode multi-scale semantic information at a different level on the decoder side. Second, it fails to reimpose the feature maps adeptly due to its limited capability on capturing long-range feature dependencies. In this study, we solve this problem by capturing multi-scale global feature maps, which forces the network to learn different semantic information at each scale. Further, we utilize the attention mechanism to suppress noise and the undesirable features, leading to a thorough restoration of contextual feature dependencies. Finally, we propose a novel method which leverages the compound scaled EfficientNet as a encoder backbone for efficient feature extraction and the UNet decoder to reconstruct the fine-grained details. We evaluated the proposed method using three different medical datasets: Kvasir-SEG, nuclei segmentation, and skin-lesion segmentation. The experimental results demonstrate that the proposed method takes an unassailable lead in terms of segmentation accuracy over the baseline models across different datasets and backbone architectures. Further, the proposed method strengthens the segmentation quality of varying shapes, object shapes, suppresses the noise, and leads to a better performance.

Low-Light Image Enhancement via Progressive-Recursive Network

Low-light images have low brightness and contrast, which presents a huge obstacle to computer vision tasks. Low-light image enhancement is challenging because multiple factors (such as brightness, contrast, artifacts, and noise) must be considered simultaneously. In this study, we propose a neural network—a progressive-recursive image enhancement network (PRIEN)—to enhance low-light images. The main idea is to use a recursive unit, composed of a recursive layer and a residual block, to repeatedly unfold the input image for feature extraction. Unlike in previous methods, in the proposed study, we directly input low-light images into the dual attention model for global feature extraction. Next, we use a combination of recurrent layers and residual blocks for local feature extraction. Finally, we output the enhanced image. Furthermore, we input the global feature map of dual attention into each stage in a progressive way. In the local feature extraction module, a recurrent layer shares depth features across stages. In addition, we perform recursive operations on a single residual block, significantly reducing the number of parameters while ensuring good network performance. Although the network structure is simple, it can produce good results for a range of low-light conditions. We conducted experiments on widely adopted datasets. The results demonstrate the advantages of our method compared with other methods, from both qualitative and quantitative perspectives.

CAM: A fine-grained vehicle model recognition method based on visual attention model

Vehicle model recognition (VMR) is a typical fine-grained classification task in computer vision. To improve the representation power of classical CNN networks for this special task, we focus on enhancing the subtle difference of features and their spatial encoding based on the attention mechanism, and then propose a novel architectural unit, which we term the “convolutional attention model” (CAM). It adopts a two-stage attention mechanism for VMR, which includes the global feature map attention (GFMA) algorithm, applied at the lower part of the main network flow to enhance the subtle feature difference from the beginning, and the feature spatial relationship attention (FSRA) algorithm, applied at the higher part to enhance the spatial relationship of features. The experiments are conducted on the benchmark CompCars web-nature and Stanford Car datasets and demonstrate the effectiveness of CAM when integrated with some classical CNN architectures. CAM can improve the top-1 recognition accuracy by an average of 1.15% and top-5 by an average of 0.78%.

Wheat Powdery Mildew Spore Images Segmentation Based on U-Net

In recent years, there have been significant development in deep learning technology, which has also promoted the development of image segmentation. Wheat powdery mildew is a common crop disease, which is very harmful to crops. The segmentation task of wheat powdery mildew spore images is important for the target spores identification and spores counting. In this paper, an improved framework based on U-Net is proposed, comparing with original U-Net, we add the pyramid pooling module after the 1024-channel feature map in the down-sampling part to extract different sizes pooling feature and fuse into a global feature map, and adjust some skip connections in original U-Net to obtain more features that are effective for spore images segmentation, experiment shows that the improved U-Net structure has better segmentation performance than U-Net, and the segmentation miou (mean intersection over union) has reached 91.4% in wheat powdery mildew spore image dataset, which proves that our proposed architecture is effective and competitive in the wheat powdery mildew spore segmentation task.

Generative image inpainting via edge structure and color aware fusion

Very recently, with the widespread research of deep learning, its achievements are increasingly evident in image inpainting tasks. However, many existing methods fail to effectively reconstruct vivid contents and refine structures. In order to solve this issue, in this paper, a novel two-stage generative adversarial network based on the fusion of edge structures and color aware maps is proposed. In the first-stage network, edges with missing regions are employed to train an edge structure generator. Meanwhile, the input image with missing regions is transformed into a global color feature map after the content aware fill algorithm and a large kernel size Gaussian filtering. In the second-stage network, the image fused from the edge map and the color map is used as a label to guide the network to reconstruct the refined image. Qualitative and quantitative experiments conducted on multiple public datasets demonstrate that the method proposed in this paper has superior performance.

Perspective-Adaptive Convolutions for Scene Parsing.

Many existing scene parsing methods adopt Convolutional Neural Networks with receptive fields of fixed sizes and shapes, which frequently results in inconsistent predictions of large objects and invisibility of small objects. To tackle this issue, we propose perspective-adaptive convolutions to acquire receptive fields of flexible sizes and shapes during scene parsing. Through adding a new perspective regression layer, we can dynamically infer the position-adaptive perspective coefficient vectors utilized to reshape the convolutional patches. Consequently, the receptive fields can be adjusted automatically according to the various sizes and perspective deformations of the objects in scene images. Our proposed convolutions are differentiable to learn the convolutional parameters and perspective coefficients in an end-to-end way without any extra training supervision of object sizes. Furthermore, considering that the standard convolutions lack contextual information and spatial dependencies, we propose a context adaptive bias to capture both local and global contextual information through average pooling on the local feature patches and global feature maps, followed by flexible attentive summing to the convolutional results. The attentive weights are position-adaptive and context-aware, and can be learned through adding an additional context regression layer. Experiments on Cityscapes and ADE20K datasets well demonstrate the effectiveness of the proposed methods.

Dense deconvolution net: Multi path fusion and dense deconvolution for high resolution skin lesion segmentation.

BACKGROUND: Dermoscopy imaging has been a routine examination approach for skin lesion diagnosis. Accurate segmentation is the first step for automatic dermoscopy image assessment.OBJECTIVE: The main challenges for skin lesion segmentation are numerous variations in viewpoint and scale of skin lesion region.METHODS: To handle these challenges, we propose a novel skin lesion segmentation network via a very deep dense deconvolution network based on dermoscopic images. Specifically, the deep dense layer and generic multi-path Deep RefineNet are combined to improve the segmentation performance. The deep representation of all available layers is aggregated to form the global feature maps using skip connection. Also, the dense deconvolution layer is leveraged to capture diverse appearance features via the contextual information. Finally, we apply the dense deconvolution layer to smooth segmentation maps and obtain final high-resolution output.RESULTS: Our proposed method shows the superiority over the state-of-the-art approaches based on the public available 2016 and 2017 skin lesion challenge dataset and achieves the accuracy of 96.0% and 93.9%, which obtained a 6.0% and 1.2% increase over the traditional method, respectively.CONCLUSIONS: By utilizing Dense Deconvolution Net, the average time for processing one testing images with our proposed framework was 0.253 s.

Salient object detection using biogeography-based optimization to combine features

Salient object detection aims to automatically localize a foreground object with respect to its background in an image. It plays a crucial role in a wide range of computer vision and multimedia applications. In this work, we propose an improved salient object detection method based on biogeography-based optimization, a relatively new bio-inspired metaheuristic algorithm that searches for the global optimum using a migration model. Our approach consists of two steps. In the first step, a set of local (multi-scale contrast), regional (center-surround histogram), and global (color spatial distribution) salient feature maps are extracted and normalized. In the second step, an optimal weight vector for combining these feature maps into one saliency map is determined using biogeography-based optimization and improved variants of this algorithm. As a result, a salient objects were identified and labeled as distinct from the image background. We implemented our method using three biogeography-based optimization variants, and compared our results for three popular databases against two other state-of-the-art approaches. The experimental results demonstrate that our method exhibits refined and consistent detection of salient objects.

Pruning Rule for kMER-Based Acquisition of the Global Topographic Feature Map

For a kernel-based topographic map formation, kMER (kernel-based maximum entropy learning rule) was proposed by Van Hulle, and some effective learning rules related to kMER have been proposed so far with many applications. However, no discusions have been made concerning the determination of the number of units in kMER. This letter describes a unit-pruning rule, which permits automatic contruction of an appropriate-sized map to acquire the global topographic features underlying the input data. The effectiveness and the validity of the present rule have been confirmed by some preliminary computer simulations.