Features Of Different Levels Research Articles

GRS-Det: An Anchor-Free Rotation Ship Detector Based on Gaussian-Mask in Remote Sensing Images

Ship detection is a significant and challenging task in remote sensing. Due to the arbitrary-oriented property and large aspect ratio of ships, most of the existing detectors adopt rotation boxes to represent ships. However, manual-designed rotation anchors are needed in these detectors, which causes multiplied computational cost and inaccurate box regression. To address the abovementioned problems, an anchor-free rotation ship detector, named GRS-Det, is proposed, which mainly consists of a feature extraction network with selective concatenation module (SCM), a rotation Gaussian-Mask model, and a fully convolutional network-based detection module. First, a U-shape network with SCM is used to extract multiscale feature maps. With the help of SCM, the channel unbalance problem between different-level features in feature fusion is solved. Then, a rotation Gaussian-Mask is designed to model the ship based on its geometry characteristics, which aims at solving the mislabeling problem of rotation bounding boxes. Meanwhile, the Gaussian-Mask leverages context information to strengthen the perception of ships. Finally, multiscale feature maps are fed to the detection module for classification and regression of each pixel. Our proposed method, evaluated on ship detection benchmarks, including HRSC2016 and DOTA Ship data sets, achieves state-of-the-art results.

A Deep Branch-Aggregation Network for Recognition of Gas–Liquid Two-Phase Flow Structure

Gas–liquid two-phase flow widely exists in petroleum, natural gas, and other industries. Recognition of flow structure is an important issue in the study of two-phase flow, and it is of great significance for the optimization of industrial processes. Therefore, how to recognize complex flow structures effectively represents a challenge. To cope with this problem, we develop a novel deep learning network to recognize flow structures under different flow conditions. In particular, we conduct vertical upward gas–liquid two-phase flow experiments to obtain the flow structure data set on the basis of images collected by a high-speed camera. Then, we design a branch-aggregation network (BAN), where the branch structure is utilized to increase the width of the network, and multilevel aggregation structure is used to fuse features of different levels. In recognition of flow structure, the proposed network achieves an accuracy of 99.60% with a fast convergence speed and shows the advantages in antinoise ability, which is significant for online recognition in industrial processes. The results indicate that BAN can be a feasible method to recognize complex flow structures.

Linear Interpolation Based Adaptive Feature-Preserving Filtering Method on Mesh Models

<p indent=0mm>Mesh denoising is an important step to improve the data obtained by scanning devices and it provides an ideal model for subsequent digital geometry processing. The main challenge is to remove the noises while maintaining its geometric features, especially weak features. The denoising methods occurred in recent years oftentimes lack the capability to balance between time consumption and denoising results. Therefore, this paper proposes an efficient feature preserving denoising framework based on linear interpolation. In order to realize feature-preserving (for features of different levels, from salient features to weak features) denoising, this paper uses the interpolation method to jointly analyze the fully polished model that reflects the structural features and the noisy model that contains the comprehensive information. By designing an adaptive interpolation method, the features of different levels are extracted from the noisy model and added to the smooth model. At the same time, an automatic termination judgment for the iterative denoising algorithm is proposed, which provides automatic and reliable termination conditions for different iterative denoising algorithms. The quantitative and qualitative results on synthetic and real data with various levels of noises show that our method can achieve efficient and robust denoising results.

Hierarchical support vector machine for facial micro-expression recognition

The sample category distribution of spontaneous facial micro-expression datasets is unbalanced, due to the experimental environment, collection equipment, and individualization of subjects, which brings great challenges to micro-expression recognition. Therefore, this paper introduces a micro-expression recognition model based on the Hierarchical Support Vector Machine (H-SVM) to reduce the interference of sample category distribution imbalance. First, we calculated the position of the apex frame in the micro-expression image sequence. To keep micro-expression frames balanced, we sparsely sample the images sequence according to the apex frame. Then, the Low-level Descriptors of the region of interest of the micro-expression image sequence and the High-level Descriptors of apex frame are extracted. Finally, the H-SVM model is used to classify the fusion features of different levels. The experimental results on SMIC, CAMSE2, SAMM, and their composite datasets show that our method can achieve superior performance in micro-expression recognition.

PCANet: Pyramid convolutional attention network for semantic segmentation

Pyramid Convolutional Attention Network is proposed to efficiently capture long-range dependency and fuse features from different levels for benefitting semantic segmentation problems. In this paper, we focus on how to extract more representative features for segmentation object recognition and design a decoder to recover details in a more efficient way. Inspired by atrous sampling and attention mechanism, we propose Pyramid Atrous Attention module to capture long-range dependency for learning richer contextual features. We also find that features of different levels have diverse representation so we design Convolutional Attention Refinement module to provide global context for low-level features and local details for high-level features. By combining with these two efficient module, we construct our Pyramid Convolutional Attention Network (PCANet), which achieves state-of-the-art results on Pascal VOC 2012 and Cityscapes benchmark.

Real-time tracking based on deep feature fusion

Deep learning-based methods have recently attracted significant attention in visual tracking community, leading to an increase in state-of-the-art tracking performance. However, due to the utilization of more complex models, it has also been accompanied with a decrease in speed. For real-time tracking applications, a careful balance of performance and speed is required. We propose a real-time tracking method based on deep feature fusion, which combines deep learning with kernel correlation filter. First, hierarchical features are extracted from a lightweight pre-trained convolutional neural network. Then, original features of different levels are fused using canonical correlation analysis. Fused features, as well as some original deep features, are used in three kernel correlation filters to track the target. An adaptive update strategy, based on dispersion analysis of response maps for the correlation filters, is proposed to improve robustness to target appearance changes. Different update frequencies are adopted for the three filters to adapt to severe appearance changes. We perform extensive experiments on two benchmarks: OTB-50 and OTB-100. Quantitative and qualitative evaluations show that the proposed tracking method performs favorably against some state-of-the-art methods – even better than algorithms using complex network model. Furthermore, proposed algorithm runs faster than 20 frame per second (FPS) and hence able to achieve near real-time tracking.

Densely connected GCN model for motion prediction

AbstractHuman motion prediction is a fundamental problem in understanding human natural movements. This task is very challenging due to the complex human body constraints and diversity of action types. Due to the human body being a natural graph, graph convolutional network (GCN)‐based models perform better than the traditional recurrent neural network (RNN)‐based models on modeling the natural spatial and temporal dependencies lying in the motion data. In this paper, we develop the GCN‐based models further by adding densely connected links to increase their feature utilizations and address oversmoothing problem. More specifically, the GCN block is used to learn the spatial relationships between the nodes and each feature map of the GCN block propagates directly to every following block as input rather than residual linked. In this way, the spatial dependency of human motion data is exploited more sufficiently and the features of different level of scale are fused more efficiently. Extensive experiments demonstrate our model achieving the state‐of‐the‐art results on CMU dataset.

Single‐shot detector with enriched semantics for PCB tiny defect detection

Printed circuit board (PCB) defect detection is one of the primary problems in quality control of the most electronic products. Usually, the industrial PCB imagery has high resolution, but defects take up a small proportion (often only ∼10 pixels in size), which makes it difficult to use traditional machine vision methods. To this end, a novel single shot object detector (SSDT) is proposed for tiny defect detection in PCBs in this study. Specifically, a semantic ascending module, which propagates the semantic property of deep layers to shallow layers, is presented by fusing features of different levels. An attention mechanism is utilised to learn the relationship of the features to be fused across channels and a shuffle module is used to eliminate the aliasing effect after fusion. Moreover, the improved non-maximum suppression is proposed to extenuate the overlap effect for testing the whole PCB image. The proposed detector can rapidly detect tiny defects and the results of SSD and SSDT are further compared not only in PCB defect dataset but also the object detection public dataset PASCAL VOC2007 where SSDT achieves 81.3% mAP, better than SSD (79.5%). In final, the proposed detector is validated to be robust to rotation and blur.

Hierarchical Aggregated Deep Features for ALS Point Cloud Classification

Classification of airborne laser scanning (ALS) point clouds is needed in digital cities and 3-D modeling. To efficiently recognize objects in ALS point clouds, we propose a novel hierarchical aggregated deep feature representation method, which can adequately employ spatial association of multilevel structures and deep feature discrimination. In our method, a 3-D deep learning model is constructed to represent the discriminative feature of each point cluster in a hierarchical structure by decreasing the within-class distance and increasing the between-class distance. Our method aggregates the discriminative deep features in different levels into a hierarchical aggregated deep feature that considers the spatial hierarchy and feature distinctiveness. Lastly, we build a multichannel 1-D convolutional neural network to classify the unknown points. Our tests demonstrate that the proposed hierarchical aggregated deep feature method can enhance point cloud classification results. Comparing with seven state-of-the-art methods, those results also verified the superior performance of our method.

A multi-context representation approach with multi-task learning for object counting

Object counting is a fundamental while challenging computer vision task, as it requires the object appearance information as well as semantic understanding of the object. In this paper, we propose an end-to-end multi-context embedding deep network for object counting(MCENet), which observes the object counting task from the three different perspectives to count the number of vehicles in the traffic video frame, or to estimate the number of the pedestrian in the largely congested scene. The first sub-network of MCENet extracts the potential features for the appearance context and the semantic context from different-level layers. The two different-level features from the first sub-network are transferred into the two parallel and complementary sub-networks, which are used to model the appearance context and semantic context for final counting. And thus the multiple contexts are represented and embedded to assist the counting task. Extensive experimental evaluations are reported in this paper, using up to three different object counting benchmarks, which show the proposed approach achieves a competitive performance in all these heterogeneous scenarios.

Attention-guided dual spatial-temporal non-local network for video super-resolution

In this paper, we propose an attention-guided dual spatial-temporal non-local network for video super-resolution (ADNLVSR). We integrate temporal and spatial non-local self-similar contexts from continuous video frames after motion compensation, and merge the features of different levels discriminatively with channel attention mechanism for target frame. During motion compensation, unlike previous methods directly stacking input images or features for merging, we use learnable attention mechanism to guide the merging, which suppresses undesired components caused by misalignment and enhances desirable fine details. During feature fusion, in contrast to most previous approaches where global-level non-local self-similarity existing in space or time is usually considered, we propose region-level spatial and temporal non-local operations for exploiting temporal correlations and enhancing similar spatial structures. The proposed modules can effectively avoid the computational burden caused by existing global-level non-local operations based on our analysis, and enhance correlated structure information. In addition, we propose a channel attention-guided residual dense block (CRDB), in which a second-order channel attention mechanism is applied to adaptively rescale the channel-wise features for more discriminative representations. Extensive experiments on different datasets demonstrate superior performance to state-of-the-art published methods on video super-resolution.

Scale Sensitive Neural Network for Road Segmentation in High-Resolution Remote Sensing Images

Road segmentation in remote sensing images has been widely used in many fields. Semantic segmentation, based on deep learning, has become a hot topic for road segmentation. With the deepening of convolutional neural network (CNN) structures, features in the convolution layer that has more semantic information become more important for road segmentation. However, the spatial resolution of the convolutional layer reduced as the CNN network deepens, which causes the extracted roads to lose some important location information. To solve this problem, this letter proposes a novel end-to-end road segmentation method to effectively utilize the different levels of convolutional layers to enhance the model’s ability to precisely perceive road edges and shapes. The model includes an encoder and a decoder. The encoder encodes the image to obtain the features of different levels and scales. The decoder consists of two modules: scale fusion module and scale sensitive module. In the scale fusion module, features in pooling layers of different scales are fused to obtain a fusion feature. In a scale sensitive module, a weight tensor at the end of the network is learned to evaluate the importance of fusion features. This road segmentation network has been experimentally verified using public data sets, which greatly improves the road segmentation accuracy and achieves good performance.

Rotation-aware and multi-scale convolutional neural network for object detection in remote sensing images

Object detection plays an important role in the field of remote sensing imagery analysis. The most challenging issues in advancing this task are the large variation in object scales and the arbitrary orientation of objects. In this paper, we build a unified framework upon the region-based convolutional neural network for arbitrary-oriented and multi-scale object detection in remote sensing images. To handle the problem of multi-scale object detection, a feature-fusion architecture is proposed to generate a multi-scale feature hierarchy, which augments the features of shallow layers with semantic representations via a top-down pathway and combines the feature maps of top layers with low-level information by a bottom-up pathway. By combining features of different levels, we can form a powerful feature representation for multi-scale objects. Most previous methods locate objects with arbitrary orientations and dense spatial distributions via axis-aligned boxes, which may cover adjacent instances and background areas. We build a rotation-aware object detector that uses oriented boxes to localize objects in remote sensing images. The region proposal network augments the anchors with multiple default angles to cover oriented objects. It utilizes oriented proposal boxes to enclose objects rather than horizontal proposals that coarsely locate oriented objects. The orientation RoI pooling operation is introduced to extract the feature maps of oriented proposals for the following R-CNN subnetwork. We conduct comprehensive experiments on a public dataset for oriented object detection in remote sensing images. Our method achieves state-of-the-art performance, which demonstrates the effectiveness of the proposed methods.

Learning Domain-Specific Features From General Features for Person Re-Identification

Person re-identification (re-id) plays a vital role in surveillance and forensics application. Since the labeled images for person re-id task is limited, the generalization ability of existed person re-id models is poor. On the other hand, images of different classes (pedestrian and non-pedestrian images) share some general features. To this end, this paper aims to improve the performance of person re-id by designing a relearning network which can learn domain-specific features and general features simultaneously. The proposed relearning network consists of a pretrained backbone network which provides the general features, and several attention-based subnetworks that learn domain-specific features from general features of different levels. Besides, we propose a coarse-fine loss to improve the generalization of person re-id model by making full use of the massive labeled non-pedestrian images. Experimental results on the publicly available Market-1501, DukeMTMC-reID and CUHK03 pedestrian re-id datasets demonstrate the effectiveness of the proposed relearning network and coarse-fine loss.

Local-Source Enhanced Residual Network for Steganalysis of Digital Images

Steganalysis refers to the study of identifying hidden messages in images inserted by steganography. Although detection performance is greatly improved when adopting convolutional neural networks (CNNs), they require sophisticated tricks, such as preprocessing for suppression of image content, using absolute and truncated activation functions, and utilizing domain knowledge. These tricks help networks train stably and mitigate the convergence problem of early stages in training, but they also restrict the flexibility of CNNs, which limits their performance. In this paper, we propose a local-source enhanced residual network (LSER) with end-to-end learning. The LSER is simple in its architecture but has two distinct characteristics from previous methods. First, the LSER uses residual blocks without any normalization. We find batch normalization is an unnecessary module in our framework. Second, a local-source skip connection is added to bypass features of different levels, which allows more abundant feature representation. Moreover, the LSER exhibits state-of-the-art results compared with the existing work in both spatial and JPEG domain steganalysis. Furthermore, a simple self-ensemble method further improves its performance without any side information.

Deep Classification Consistency for Person Re-Identification

Person re-identification (Re-ID) has greatly benefited from utilizing features of different levels. Most methods draw support from elegant network designs and elaborate fusion modules. In this paper, different from employing multi-level feature fusion like mainstream skip connection methods, we propose a Deep Classification Consistency(DCC) layer which imposes classification consistency on deep features from different stages in CNNs. DCC regularizes the training process of networks. It profoundly changes the distribution of learned features, making them more discriminative and transferable. Extensive experiments on Market-1501, DukeMTMC-reID, and CUHK03 datasets prove that the proposed method has obtained competitive performances when compared with state-of-the-art approaches, especially those advanced methods merely based on metric learning.

OCT Image Recognition of Cardiovascular Vulnerable Plaque Based on CNN

Optical coherence tomography (OCT) shows an important role in the diagnosis of cardiovascular diseases and the detection and intervention of vulnerable plaques. Clinical diagnosis of vulnerable plaques in cardiovascular system based on optical coherence tomography mainly relies on the manual analysis of vulnerable plaques in images by cardiovascular physicians. This analysis process is prone to subjective misjudgments and heavy workload. Studying the recognition technology of cardiovascular vulnerable plaque image will greatly improve the accuracy of diagnosis and reduce the workload of cardiovascular physicians, which is an effective way to achieve efficient diagnosis and treatment. In view of the recognition of cardiovascular vulnerable plaque medical images, a model based on convolution neural network (CNN) recognition is constructed. The CNN is used to learn the features of different levels from the original input OCT images. At the same time, several decision-making levels are designed. These decision-making levels can classify OCT images according to different feature maps, and finally make final recognition decisions according to the classification results. The experimental results on the clinical data set labeled by doctors show that the classification and recognition model of cardiovascular vulnerable plaque OCT image based on CNN has a high recognition rate. Making full use of the multilevel features of convolutional neural networks can effectively classify and recognize the OCT images of vulnerable cardiovascular plaques, provide support for the clinical diagnosis of cardiovascular diseases, and have great significance for the early intervention and prevention of cardiovascular diseases.

Cross-Camera Person Re-Identification With Body-Guided Attention Network

Various challenges exist throughout person re-identification (ReID) process, including background clutters, illumination variation, pose variation, occlusion, etc. Addressing these problems, this paper explores the incorporation of human attention mechanism in person ReID and proposes an attention-aware model named Body-guided Attention Network (BANet). The proposed attention is based on the body masked images which are obtained by a reliable pixel-level segmentation strategy. To optimize the feature representation learning so as to pay more attention to the discriminative details of human body, BANet is built. It is composed of three attention branches. In order to guide attention learning layer by layer, these branches are applied to the convolution features of different levels. The proposed BANet aims to fully utilize fine-grained information of body region to guide the final process of feature extraction. Extensive experiments on benchmarks including CUHK03, Market1501 and DukeMTMC-reID show that BANet can achieve state-of-the-art performance, which validates the importance of attention mechanism in person ReID.

Correlation Filtering-Based Hashing for Fine-Grained Image Retrieval

The low storage and strong representation capabilities of hash codes for image retrievalhas made hashing technologies very popular. Several existing deep hashing methods focuson the task of general image retrieval, while neglecting the task of fine-grained image retrieval. Recently, some fine-grained hashing methods have been proposed to capture the subtle differences, which mainly utilize the single-modality visual features to solve the discriminative region localization while ignoring the semantic information. In this letter, we propose a correlation filtering hashing (CFH) method to learn discrete binary codes, which can adequately take advantage of the cross-modal correlation between the semantic information and the visual features for discriminative region localization. Specifically, we utilize a feature pyramid network to learn multi-level visual features. Subsequently, the label vector is embedded into the visual space, which can be used as a correlation filter on the feature maps to capture the latent location of objects. Finally, weperform global average pooling over the output maps and concatenate the features of different levels to produce the hash codes of query images. Extensive experiments on two fine-grained datasets show that the proposed CFH outperforms the state-of-the-art hashing methods.

FANet: Features Adaptation Network for 360$^{\circ }$ Omnidirectional Salient Object Detection

Salient object detection (SOD) in 360 $^{\circ }$ omnidirectional images has become an eye-catching problem because of the popularity of affordable 360 $^{\circ }$ cameras. In this paper, we propose a Features Adaptation Network (FANet) to highlight salient objects in 360 $^{\circ }$ omnidirectional images reliably. To utilize the feature extraction capability of convolutional neural networks and capture global object information, we input the equirectangular 360 $^{\circ }$ images and corresponding cube-map 360 $^{\circ }$ images to the feature extraction network (FENet) simultaneously to obtain multi-level equirectangular and cube-map features. Furthermore, we fuse these two kinds of features at each level of the FENet by a projection features adaptation (PFA) module, for selecting these two kinds of features adaptively. Finally, we combine the preliminary adaptation features at different levels by a multi-level features adaptation (MLFA) module, which weights these different-level features adaptively and produces the final saliency maps. Experiments show our FANet outperforms the state-of-the-art methods on the 360 $^{\circ }$ omnidirectional SOD datasets.