Convolutional Neural Network Feature Maps Research Articles

CTHNet: a network for wheat ear counting with local-global features fusion based on hybrid architecture.

Accurate wheat ear counting is one of the key indicators for wheat phenotyping. Convolutional neural network (CNN) algorithms for counting wheat have evolved into sophisticated tools, however because of the limitations of sensory fields, CNN is unable to simulate global context information, which has an impact on counting performance. In this study, we present a hybrid attention network (CTHNet) for wheat ear counting from RGB images that combines local features and global context information. On the one hand, to extract multi-scale local features, a convolutional neural network is built using the Cross Stage Partial framework. On the other hand, to acquire better global context information, tokenized image patches from convolutional neural network feature maps are encoded as input sequences using Pyramid Pooling Transformer. Then, the feature fusion module merges the local features with the global context information to significantly enhance the feature representation. The Global Wheat Head Detection Dataset and Wheat Ear Detection Dataset are used to assess the proposed model. There were 3.40 and 5.21 average absolute errors, respectively. The performance of the proposed model was significantly better than previous studies.

Caching Hybrid Rotation: A Memory Access Optimization Method for CNN on FPGA

Custom computing architectures on field programmable gate array (FPGA) platforms are a viable solution to further accelerate convolutional neural network (CNN) inference. However, due to the large size feature map matrix data, the optimization of CNN feature maps storage computing on FPGA remains a challenge. To overcome these challenges, a FPGA-oriented memory access optimization method for CNN is proposed. Firstly, the feature map partition strategy is used to group the feature map efficiently. Second, the input and the output caching rotation methods are employed in adaptive memory access mode. Third, a caching hybrid rotation method is proposed to optimize memory access performance and can effectively reduce the access time of the CNN feature map. Experimental results based on SkyNet and VGG16 show that the inference speed of the proposed model is accelerated by 7.1 times compared with the previous conventional memory access optimization for CNN on FPGA. Through the evaluation of computational energy efficiency, our method can be improved by 6.4 times compared to the current typical accelerators.

Make U-Net Greater: An Easy-to-Embed Approach to Improve Segmentation Performance Using Hypergraph

Cardiac anatomy segmentation is essential for cardiomyopathy clinical diagnosis and treatment planning. Thus, accurate delineation of target volumes at risk in cardiac anatomy is important. However, manual delineation is a time-consuming and labor-intensive process for cardiologists and has been shown to lead to significant inter-and intra-practitioner variability. Thus, computer-aided or fully automatic segmentation methods are required. They can significantly economize on manpower and improve treatment efficiency. Recently, deep convolutional neural network (CNN) based methods have achieved remarkable successes in various kinds of vision tasks, such as classification, segmentation and object detection. Semantic segmentation can be considered as a pixel-wise task, it requires high-level abstract semantics information while maintaining spatial detail contexts. Long-range context information plays a crucial role in this scenario. However, the traditional convolution kernel only provides the local and small size of the receptive field. To address the problem, we propose a plug-and-play module aggregating both local and global information (aka LGIA module) to capture the high-order relationship between nodes that are far apart. We incorporate both local and global correlations into hypergraph which is able to capture high-order relationships between nodes via the concept of a hyperedge connecting a subset of nodes. The local correlation considers neighborhood nodes that are spatially adjacent and similar in the same CNN feature maps of magnetic resonance (MR) image; and the global correlation is searched from a batch of CNN feature maps of MR images in feature space. The influence of these two correlations on semantic segmentation is complementary. We validated our LGIA module on various CNN segmentation models with the cardiac MR images dataset. Experimental results demonstrate that our approach outperformed several baseline models.

Integrating the Reconstructed Scattering Center Feature Maps With Deep CNN Feature Maps for Automatic SAR Target Recognition

Automatic target recognition has been one of the hottest research in synthetic aperture radar (SAR) data processing. Noticing that popular recognition methods cannot utilize multiple features of SAR complex data, a method fused scattering center feature and deep convolutional neural network (CNN) feature is proposed in this letter. This method contains three key parts, namely, scattering center extraction and reconstruction block, CNN feature extraction block, and final feature fusion and classification block. In this process, the scattering center feature and CNN feature are fused at the level of feature maps, which retain the space information of 2-D feature maps. What is more, the proposed half end-to-end strategy realizes the automatic update of weighting parameters in feature extraction network and subnetwork, which promotes a better recognition efficiency. Experimental results on measured SAR data show that the proposed method can achieve better accuracy than other single feature-based methods and feature fusion methods.

Deep convolutional neural network and sparse least-squares migration

We have recast the forward pass of a multilayered convolutional neural network (CNN) as the solution to the problem of sparse least-squares migration (LSM). The CNN filters and feature maps are shown to be analogous, but not equivalent, to the migration Green’s functions and the quasi-reflectivity distribution, respectively. This provides a physical interpretation of the filters and feature maps in deep CNN in terms of the operators for seismic imaging. Motivated by the connection between sparse LSM and CNN, we adopt the neural network version of sparse LSM. Unlike the standard LSM method that finds the optimal reflectivity image, neural network LSM (NNLSM) finds the optimal quasi-reflectivity image and the quasi-migration Green’s functions. These quasi-migration Green’s functions are also denoted as the convolutional filters in a CNN and are similar to migration Green’s functions. The advantage of NNLSM over standard LSM is that its computational cost is significantly less and it can be used for denoising coherent and incoherent noise in migration images. Its disadvantage is that the NNLSM quasi-reflectivity image is only an approximation to the actual reflectivity distribution. However, the quasi-reflectivity image can be used as an attribute image for high-resolution delineation of geologic bodies.

Deep Learning-Based Real-Time Building Occupancy Detection Using AMI Data

Building occupancy patterns facilitate successful development of the smart grid by enhancing building-to-grid integration efficiencies. Current occupancy detection is limited by the lack of widely deployed non-intrusive sensors and the insufficient learning power of shallow machine learning algorithms. This paper seeks to detect real-time building occupancy from Advanced Metering Infrastructure (AMI) data based on a deep learning architecture. The developed deep learning model consists of a convolutional neural network (CNN) and a long short-term memory (LSTM) network. Specifically, a CNN with convolutional and max-pooling layers extracts spatial features in the AMI data. Then, the forward and backward dependencies within the CNN feature maps are learned by a bidirectional LSTM (BiLSTM) structure with three hidden layers. Case studies based on a publicly available dataset show that the developed CNN-BiLSTM model consistently and robustly outperforms the state-of-the-art machine learning classifiers and other advanced deep learning architectures with around 90% occupancy detection accuracy and high detection confidence.

MAP-Net: Multiple Attending Path Neural Network for Building Footprint Extraction From Remote Sensed Imagery

Accurately and efficiently extracting building footprints from a wide range of remote sensed imagery remains a challenge due to their complex structure, variety of scales and diverse appearances. Existing convolutional neural network (CNN)-based building extraction methods are complained that they cannot detect the tiny buildings because the spatial information of CNN feature maps are lost during repeated pooling operations of the CNN, and the large buildings still have inaccurate segmentation edges. Moreover, features extracted by a CNN are always partial which restricted by the size of the respective field, and large-scale buildings with low texture are always discontinuous and holey when extracted. This paper proposes a novel multi attending path neural network (MAP-Net) for accurately extracting multiscale building footprints and precise boundaries. MAP-Net learns spatial localization-preserved multiscale features through a multi-parallel path in which each stage is gradually generated to extract high-level semantic features with fixed resolution. Then, an attention module adaptively squeezes channel-wise features from each path for optimization, and a pyramid spatial pooling module captures global dependency for refining discontinuous building footprints. Experimental results show that MAP-Net outperforms state-of-the-art (SOTA) algorithms in boundary localization accuracy as well as continuity of large buildings. Specifically, our method achieved 0.68\%, 1.74\%, 1.46\% precision, and 1.50\%, 1.53\%, 0.82\% IoU score improvement without increasing computational complexity compared with the latest HRNetv2 on the Urban 3D, Deep Globe and WHU datasets, respectively. The TensorFlow implementation is available at https://github.com/lehaifeng/MAPNet.

DeepSat V2: feature augmented convolutional neural nets for satellite image classification

ABSTRACTSatellite image classification is a challenging problem that lies at the crossroads of remote sensing, computer vision, and machine learning. Due to the high variability inherent in satellite data, most of the current object classification approaches are not suitable for handling satellite datasets. The progress of satellite image analytics has also been inhibited by the lack of a single labeled high-resolution dataset with multiple class labels. In a preliminary version of this work, we introduced two new high resolution satellite imagery datasets (SAT-4 and SAT-6) and proposed DeepSat framework for classification based on “handcrafted” features and a deep belief network (DBN). The present paper is an extended version, we present an end-to-end framework leveraging an improved architecture that augments a convolutional neural network (CNN) with handcrafted features (instead of using DBN-based architecture) for classification. Our framework, having access to fused spatial information obtained from handcrafted features as well as CNN feature maps, have achieved accuracies of 99.90\\% and 99.84\\% respectively, on SAT-4 and SAT-6, surpassing all the other state-of-the-art results. A statistical analysis based on Distribution Separability Criterion substantiates the robustness of our approach in learning better representations for satellite imagery.

Deep Radiomic Analysis Based on Modeling Information Flow in Convolutional Neural Networks

This paper proposes a novel image feature set based on a principled information theoretic analysis of the convolutional neural network (CNN). The output of convolutional filters is modeled as a random variable conditioned on the object class and network filter bank. The conditional entropy (CENT) of filter outputs is shown in theory and experiments to be a highly compact and class-informative feature that can be computed from the CNN feature maps and used to obtain higher classification accuracy than the original CNN itself. Experiments involve three binary classification tasks using the 3D brain MRI data: Alzheimer’s disease (AD) versus healthy controls (HC), young versus old age, and male versus female, where the area under the curve (AUC) values for the CENT feature classification (93.9%, 96.7%, and 71.9%) are significantly higher than the softmax output of the original CNN classifier trained for the task (81.6%, 79.4%, and 63.1%). A statistical analysis based on the Wilcoxon test identifies CENT features with significant links to brain labels, which could potentially serve as diagnostic biomarkers.

Image-based pencil drawing synthesized using convolutional neural network feature maps

In most cases, the conventional pencil-drawing-synthesized methods were in terms of geometry and stroke, or only used classic edge detection method to extract image edge characters. In this paper, we propose a new method to produce pencil drawing from natural image. The synthesized result can not only generate pencil sketch drawing, but also can save the color tone of natural image and the drawing style is flexible. The sketch and style are learned from the edge of original natural image and one pencil image exemplar of artist’s work. They are accomplished through using the convolutional neural network feature maps of a natural image and an exemplar pencil drawing style image. Large-scale bound-constrained optimization (L-BFGS) is applied to synthesize the new pencil sketch whose style is similar to the exemplar pencil sketch. We evaluate the proposed method by applying it to different kinds of images and textures. Experimental results demonstrate that our method is better than conventional method in clarity and color tone. Besides, our method is also flexible in drawing style.