Set Of Feature Maps Research Articles

FQA-Net: an efficient neural network for blind image quality assessment

Guided by the neural network (NN), NN-based blind image quality assessment (BIQA) methods have improved image quality prediction accuracy. However, the training process of NN will consume a lot of time and training data. To solve the above problems, an efficient NN structure, which is called fast quality assessment network (FQA-Net), is proposed for BIQA. FQA-Net is a very simple NN, which mainly includes a convolution layer, a standard deviation measurement layer, and a regression layer. The convolution kernels in the convolution layer are a set of visual filters with characteristics of visual neurons, which are obtained by training on natural image samples. The output of the convolutional layer is a set of feature maps. Then the standard deviation of each feature map is calculated directly. Finally, a regression model is used for the mapping between the standard deviation values and the quality scores. FQA-Net not only reduces the number of parameters and the output dimensions in the training process, but also prevents network overfitting effectively. The results on seven image databases (i.e., LIVE, CSIQ, TID2013, LIVEMD, KADID-10K, LIVEC, and KonIQ-10K) show that FQA-Net has relatively low computational complexity and high computational speed and accuracy compared with other leading BIQA methods, especially when the number of training samples is very low. Moreover, the FQA-Net can provide stable prediction results across different distortion types.

Needle detection using ultrasound B-mode and power Doppler analyses.

Ultrasound is employed in needle interventions to visualize the anatomical structures and track the needle. Nevertheless, needle detection in ultrasound images is a difficult task, specifically at steep insertionangles. A new method is presented to enable effective needle detection using ultrasound B-mode and power Doppleranalyses. A small buzzer is used to excite the needle and an ultrasound system is utilized to acquire B-mode and power Doppler images for the needle. The B-mode and power Doppler images are processed using Radon transform and local-phase analysis to initially detect the axis of the needle. The detection of the needle axis is improved by processing the power Doppler image using alpha shape analysis to define a region of interest (ROI) that contains the needle. Also, a set of feature maps is extracted from the ROI in the B-mode image. The feature maps are processed using a machine learning classifier to construct a likelihood image that visualizes the posterior needle likelihoods of the pixels. Radon transform is applied to the likelihood image to achieve an improved needle axis detection. Additionally, the region in the B-mode image surrounding the needle axis is analyzed to identify the needle tip using a custom-made probabilistic approach. Our method was utilized to detect needles inserted in ex vivo animal tissues at shallow [ ), moderate [ ), and steep [ ]angles. Our method detected the needles with failure rates equal to 0% and mean angle, axis, and tip errors less than or equal to 0.7°, 0.6mm, and 0.7 mm, respectively. Additionally, our method achieved favorable results compared to two recently introduced needle detectionmethods. The results indicate the potential of applying our method to achieve effective needle detection in ultrasoundimages.

Learning Calibrated-Guidance for Object Detection in Aerial Images

Object detection is one of the most fundamental yet challenging research topics in the domain of computer vision. Recently, the study on this topic in aerial images has made tremendous progress. However, complex background and worse imaging quality are obvious problems in aerial object detection. Most state-of-the-art approaches tend to develop elaborate attention mechanisms for the space-time feature calibrations with arduous computational complexity, while surprisingly ignoring the importance of feature calibrations in channel-wise. In this work, we propose a simple yet effective calibrated-guidance (CG) scheme to enhance channel communications in a feature transformer fashion, which can adaptively determine the calibration weights for each channel based on the global feature affinity correlations. Specifically, for a given set of feature maps, CG first computes the feature similarity between each channel and the remaining channels as the intermediary calibration guidance. Then, rerepresenting each channel by aggregating all the channels weighted together via the guidance operation. Our CG is a general module that can be plugged into any deep neural networks, which is named as CG-Net. To demonstrate its effectiveness and efficiency, extensive experiments are carried out on both oriented object detection task and horizontal object detection task in aerial images. Experimental results on two challenging benchmarks (<i>i</i>.<i>e</i>., DOTA and HRSC2016) demonstrate that our CG-Net can achieve the new state-of-the-art performance in accuracy with a fair computational overhead. The source code has been open sourced at <uri>https://github.com/WeiZongqi/CG-Net</uri>.

Augmented global attention network for image super‐resolution

Convolutional networks dominate many machine vision fields. Nevertheless, a significant drawback of the convolution operation is that it only operates in the local region, so it lacks global information. Self-attention has become the latest technology for capturing long-range interactions, but it is mainly used for generative modeling and sequence modeling tasks. Using self-attention to tackle super-resolution as a substitute for convolution is considered. Therefore, augmented global attention convolution (AGAC) is proposed as an alternative to convolution to use self-attention for super-resolution. The proposed augmented global attention convolution can capture global context to produce more realistic super-resolution results. Due to the most existing works that have not exploited position information, a two-dimensional relative self-attention mechanism is proposed to enhance self-attention. To deal with the super-resolution task, the authors come up with an augmented global attention convolutional network (AGAN) to enhance the convolution operator with the self-attention mechanism through concatenating the convolution pattern map with the generated set of feature maps. Many experiments and analyses are conducted to demonstrate that the proposed model surpasses the advanced models with comparable parameters and performance.

MuRNet: A deep recursive network for super resolution of bicubically interpolated images

In many real-world cases such as printer devices and in-camera interpolation, only the interpolated versions of the low-resolution images are available. In this paper, a new low-complexity high-performance image super resolution network is proposed that starting from the bicubic interpolated version of the low resolution image produces a high quality super resolved image. The main idea in the proposed scheme is the development of a feature generating block that is capable of producing features using multiple local spatial ranges and multiple resolution levels, fusing them in order to provide a rich set of feature maps, and using them in a recursive framework. The objective in designing such a recursive block is not simply to provide a light-weight network, as is traditionally done in the design of such a network, but also to provide a low count on the number of multiply-accumulate operations with high performance. The experimental results are provided to show that the proposed network outperforms other recursive super resolution networks when their super resolution capability, the number of parameters and number of multiply-accumulate operations are simultaneously taken into consideration.

LungSeg-Net: Lung field segmentation using generative adversarial network

Automatic lung segmentation is an essential step towards the Computer-Aided Diagnosis of the lung CT scan. However, in presence of dense abnormalities, existing methods fail in accurate lung segmentation. In this paper, a generative adversarial networks-based approach is proposed for improving the accuracy of lung segmentation. The proposed network effectively segments the lung region from the surrounding chest region hence, named as LungSeg-Net. In the proposed LungSeg-Net, the input lung CT slices are processed through the trail of encoders which encode these slices into a set of feature maps. Further, a multi-scale dense-feature extraction (MSDFE) module is designed for extraction of multi-scale features from the set of encoded feature maps. Finally, the decoders are employed to obtain the lung segmentation map from the multi-scale features. The MSDFE makes the network to learn the relevant features of dense abnormalities whereas the iterative down-sampling followed by the up-sampling makes it invariant to the size of the dense abnormality. The publicly available benchmark ILD dataset is used for the experimental analysis. The qualitative and quantitative analysis has been carried out to compare the performance of the proposed network with the existing state-of-the-art methods for lung segmentation. The experimental analysis show that the performance of the proposed LungSeg-Net is invariant to the presence of dense abnormalities in lung CT scan.

Deep learning-based multi-modal approach using RGB and skeleton sequences for human activity recognition

The deep learning techniques have achieved great success in the application of human activity recognition (HAR). In this paper, we propose a technique for HAR that utilizes the RGB and skeleton information with the help of a convolutional neural network (Convnet) and long short-term memory (LSTM) as a recurrent neural network (RNN). The proposed method has two parts: first, motion representation images like motion history image (MHI) and motion energy image (MEI) have been created from the RGB videos. The convnet has been trained, using these images with feature-level fusion. Second, the skeleton data have been utilized with a proposed algorithm that develops skeleton intensity images, for three views (top, front and side). Each view is first analyzed by a convnet, that generates the set of feature maps, which are fused for further analysis. On top of convnet sub-networks, LSTM has been used to exploit the temporal dependency. The softmax scores from these two independent parts are later combined at the decision level. Apart from the given approach for HAR, this paper also presents a strategy that utilizes the concept of cyclic learning rate to develop a multi-modal neural network by training the model only once to make the system more efficient. The suggested approach privileges for the perfect utilization of RGB and skeleton data available from an RGB-D sensor. The proposed approach has been tested on three famous and challenging multimodal datasets which are UTD-MHAD, CAD-60 and NTU-RGB + D120. Results have shown that the stated method gives a satisfactory result as compared to the other state-of-the-art systems.

An embedded implementation of CNN-based hand detection and orientation estimation algorithm

Hand detection is an essential step to support many tasks including HCI applications. However, detecting various hands robustly under conditions of cluttered backgrounds, motion blur or changing light is still a challenging problem. Recently, object detection methods using CNN models have significantly improved the accuracy of hand detection yet at a high computational expense. In this paper, we propose a light CNN network, which uses a modified MobileNet as the feature extractor in company with the SSD framework to achieve robust and fast detection of hand location and orientation. The network generates a set of feature maps of various resolutions to detect hands of different sizes. In order to improve the robustness, we also employ a top-down feature fusion architecture that integrates context information across levels of features. For an accurate estimation of hand orientation by CNN, we manage to estimate two orthogonal vectors’ projections along the horizontal and vertical axes and then recover the size and orientation of a bounding box exactly enclosing the hand. In order to deploy the detection algorithm on embedded platform Jetson TK1, we optimize the implementations of the building modules in the CNN network. Evaluated on the challenging Oxford hand dataset, our method (the code is available at https://github.com/yangli18/hand_detection ) reaches 83.2% average precision at 139 FPS on a NVIDIA Titan X, outperforming the previous methods both in accuracy and efficiency. The embedded implementation of our algorithm has reached the processing speed of 16 FPS, which basically meets the requirement of real-time processing.

Convolutional sparse coding for demosaicking with panchromatic pixels

Image demosaicking is the problem of reconstructing color images from raw images captured by a digital camera covered by a Color Filter Array (CFA). CFAs allow only part of the incident light to transmit, so they will lose some incident light. Recently, CFAs with panchromatic pixels are used to avoid excessive loss of light. These CFAs’ color is sampled at a sparse set of locations, making demosaicking more challenging. In this paper, we propose a Convolutional Sparse Coding based method for Demosaicking (CSCD) with panchromatic pixels. The CSCD learns a set of filters to decompose training images into color sparse feature maps, then it estimates a set of feature maps of the to-be-reconstructed raw images and gets color images. Convolutional Sparse Coding (CSC) commonly operates on gray images. Naively extending CSC to color images will make the learned filters to be gray, which means that the values of the red, green, and blue channels are almost equal. To address this issue, we add a regularization term to penalize gray filters. We use the Alternating Direction Method of Multipliers (ADMM) to solve the CSCD. We compare the proposed CSCD with the demosaicking methods that are applicable to CFAs with panchromatic pixels. Experimental results validate its advantages in terms of CPSNR and visual quality.

Feature fusion and non-negative matrix factorization based active contours for texture segmentation

Abstract This paper presents a robust and convex active contour model for texture segmentation. Firstly, to achieve more comprehensive feature description, we compute a set of feature maps by combining local variation degree (LVD) of intensity and Gabor features. This feature fusion improves the separability between sub-regions and the robustness against complex textures. Upon these feature maps, we compute local histograms over fixed-size windows to describe the local structures formed by feature values. For each pixel, its feature vector is defined as the concatenation of all computed histograms. Secondly, to localize region boundaries more accurately, we formulate the proposed energy functional via Non-negative Matrix Factorization (NMF), which encourages each pixel to fall into the sub-region that has the largest coverage area in its neighborhood. Finally, the functional is explored further using convex optimization theory. Our segmentation results are therefore insensitive to different initial contours. The experiments performed on synthetic images, histology images and natural images demonstrate that our approach can obtain high-quality object boundaries in the presence of image noise and cluttered scenes.

Deep Attention Neural Network for Multi-Label Classification in Unmanned Aerial Vehicle Imagery

The multi-label classification problem in Unmanned Aerial Vehicle (UAV) images is particularly challenging compared to single-label classification due to its combinatorial nature. To tackle this issue, we propose in this paper a deep learning approach based on encoder-decoder neural network architecture with channel and spatial attention mechanisms. Specifically, the encoder module which is based on a pre-trained convolutional neural network (CNN) has the task to transform the input image to a set of feature maps using an opportune feature combination. To improve the feature representation further, this module incorporates a squeeze excitation (SE) layer for modelling the interdependencies between the channels of the feature maps. The decoder module which is based on a long short terms memory (LSTM) network has the task of generating, in a sequential way, the classes present in the image. At each time step, it predicts the next class-label by aligning its hidden state to the corresponding region in the image by means of an adaptive spatial attention mechanism. The experiments carried out on two UAV datasets with a spatial resolution of 2-cm show that our method is promising in predicting the labels present in the image while attending the relevant objects in the image. Additionally, it is able to provide better classification results compared to state-of-the-art methods.

Remote Sensing Scene Classification by Unsupervised Representation Learning

With the rapid development of the satellite sensor technology, high spatial resolution remote sensing (HSR) data have attracted extensive attention in military and civilian applications. In order to make full use of these data, remote sensing scene classification becomes an important and necessary precedent task. In this paper, an unsupervised representation learning method is proposed to investigate deconvolution networks for remote sensing scene classification. First, a shallow weighted deconvolution network is utilized to learn a set of feature maps and filters for each image by minimizing the reconstruction error between the input image and the convolution result. The learned feature maps can capture the abundant edge and texture information of high spatial resolution images, which is definitely important for remote sensing images. After that, the spatial pyramid model (SPM) is used to aggregate features at different scales to maintain the spatial layout of HSR image scene. A discriminative representation for HSR image is obtained by combining the proposed weighted deconvolution model and SPM. Finally, the representation vector is input into a support vector machine to finish classification. We apply our method on two challenging HSR image data sets: the UCMerced data set with 21 scene categories and the Sydney data set with seven land-use categories. All the experimental results achieved by the proposed method outperform most state of the arts, which demonstrates the effectiveness of the proposed method.

A Framework for Designing the Architectures of Deep Convolutional Neural Networks

Recent advances in Convolutional Neural Networks (CNNs) have obtained promising results in difficult deep learning tasks. However, the success of a CNN depends on finding an architecture to fit a given problem. A hand-crafted architecture is a challenging, time-consuming process that requires expert knowledge and effort, due to a large number of architectural design choices. In this article, we present an efficient framework that automatically designs a high-performing CNN architecture for a given problem. In this framework, we introduce a new optimization objective function that combines the error rate and the information learnt by a set of feature maps using deconvolutional networks (deconvnet). The new objective function allows the hyperparameters of the CNN architecture to be optimized in a way that enhances the performance by guiding the CNN through better visualization of learnt features via deconvnet. The actual optimization of the objective function is carried out via the Nelder-Mead Method (NMM). Further, our new objective function results in much faster convergence towards a better architecture. The proposed framework has the ability to explore a CNN architecture’s numerous design choices in an efficient way and also allows effective, distributed execution and synchronization via web services. Empirically, we demonstrate that the CNN architecture designed with our approach outperforms several existing approaches in terms of its error rate. Our results are also competitive with state-of-the-art results on the MNIST dataset and perform reasonably against the state-of-the-art results on CIFAR-10 and CIFAR-100 datasets. Our approach has a significant role in increasing the depth, reducing the size of strides, and constraining some convolutional layers not followed by pooling layers in order to find a CNN architecture that produces a high recognition performance.

Mandarin tone recognition based on unsupervised feature learning from spectrograms

In tone languages, such as Mandarin Chinese, a syllable with different tones conveys different meanings. Previously, Mandarin tone recognition based on Mel-frequency cepstral coefficients (MFCCs) and Convolutional Neural Networks (CNN) was examined and the results outperformed the model of conventional neural network using manually edited F0 data. In the present study, Mandarin tone recognition based on spectrograms, instead of MFCCs, was explored. Unsupervised feature learning was applied to the unlabeled spectrograms directly with a denoising autoencoder (dAE). Then, the model convolved the labeled spectrograms with the learnt “sound features” and produced a set of feature maps. A dataset that consisted of 4500 monosyllabic words collected from 125 children was used to evaluate the recognition performance. Compared with methods based on MFCCs, there are more parameters to train in the new approach based on spectrograms. As a result, the new model might better capture the statistical distribution in the original data. Therefore, the new approach, with unsupervised feature learning, could perform even better than previous methods based on MFCCs or those based on the extracted F0 information. The advantages and shortcomings of various approaches for lexical tone recognition will be discussed. [Work supported in part by the NIH NIDCD Grant No. R15-DC014587.]

Multiflash Stereopsis: Depth-Edge-Preserving Stereo with Small Baseline Illumination

Traditional stereo matching algorithms are limited in their ability to produce accurate results near depth discontinuities, due to partial occlusions and violation of smoothness constraints. In this paper, we use small baseline multi-flash illumination to produce a rich set of feature maps that enable acquisition of discontinuity preserving point correspondences. First, from a single multi-flash camera, we formulate a qualitative depth map using a gradient domain method that encodes object relative distances. Then, in a multiview setup, we exploit shadows created by light sources to compute an occlusion map. Finally, we demonstrate the usefulness of these feature maps by incorporating them into two different dense stereo correspondence algorithms, the first based on local search and the second based on belief propagation. Experimental results show that our enhanced stereo algorithms are able to extract high quality, discontinuity preserving correspondence maps from scenes that are extremely challenging for conventional stereo methods. We also demonstrate that small baseline illumination can be useful to handle specular reflections in stereo imagery. Different from most existing active illumination techniques, our method is simple, inexpensive, compact, and requires no calibration of light sources.

Global vs. local processing of compressed representations: A computational model of visual search

A novel computational model of a pre-attentive system performing visual-search is presented. The model processes various types of displays, reproduced from three sources of visual-search experimental data: Duncan and Humphreys, Psychol. Rev. 96 (1989) 453–458, Treisman and Sato, J. Exp. Psuchol. 16 (1990) 459–478, and Wolfe, Friedman-Hill, Stewart, O'Connell, J. Exp. Psychol. 18 (1992) 34–49. The response–time-slopes measured in these experiments suggest that some of the displays are searched serially while others are scanned in parallel. Our model operates in two phases. First, the visual-search displays are compressed to overcome assumed biological capacity limitations. Compression is achieved by projecting the tasks’ displays on a small set of feature maps. These features have been extracted from a large set of natural images by means of principal component analysis. Second, the compressed representations are further processed to identify a target in the display. The model succeeds in fast detection of targets in experimentally labeled parallel displays, but fails with serial ones. Analysis of the compressed representations reveals that compressed parallel displays contain global information that enables instantaneous target detection. However, in serial displays’ representations, this global information is obscure and hence, a target detection system should resort to a serial, attentional scan of local features across the display. Our analysis provides a numerical criterion that is strongly correlated with the experimental response–time-slopes. It also provides new insight to the mechanisms of visual-attention, suggesting a self-organized representation of Treisman's feature maps, which may be implemented in other paradigms in the field.