Image Feature Information Research Articles

Entropy-Driven Morphological Top-Hat Transformation for Infrared Small Target Detection

Infrared small target detection is a key technique in an infrared system. In the past decade, many methods have concentrated on traditional top-hat transformation, which relies on the hand-crafted shape and value of structural elements. However, these methods are inevitably challenged by the following two aspects: first, the structural elements cannot suppress heavy clutter because the construction of structural elements is always according to the prior information of the target and unable to consider the feature of clutter. Second, adaptively extracting sufficient local feature information for background suppression is hard for the structural element. In this article, we propose an entropy-driven top-hat transformation with guided filter kernel for considering the features of both the clutters and background. First, we propose an entropy-driven top-hat transformation method with our proposed local mean entropy, which can be used to suppress clutter according to the local complex degree of clutter. Then, an adaptive structural element based on a guided filter kernel is further exploited to capture the local feature information of image for background suppression. Finally, an adaptive threshold is combined with our algorithm to achieve target detection in image sequences. The experimental results show that the proposed algorithm is not only robust for suppressing different kinds of backgrounds but can also obtain a higher value of the signal-to-clutter ratio gain and detection accuracy compared with some popular traditional baseline methods and related top-hat methods.

MMPCANet: An Improved PCANet for Occluded Face Recognition

Principal Component Analysis Network (PCANet) is a lightweight deep learning network, which is fast and effective in face recognition. However, the accuracy of faces with occlusion does not meet the optimal requirement for two reasons: 1. PCANet needs to stretch the two-dimensional images into column vectors, which causes the loss of essential image spatial information; 2. When the training samples are few, the recognition accuracy of PCANet is low. To solve the above problems, this paper proposes a multi-scale and multi-layer feature fusion-based PCANet (MMPCANet) for occluded face recognition. Firstly, a channel-wise concatenation of the original image features and the output features of the first layer is conducted, and then the concatenated result is used as the input of the second layer; therefore, more image feature information is used. In addition, to avoid the loss of image spatial information, a spatial pyramid is used as the feature pooling layer of the network. Finally, the feature vector is sent to the random forest classifier for classification. The proposed algorithm is tested on several widely used facial image databases and compared with other similar algorithms. Our experimental results show that the proposed algorithm effectively improves the efficiency of the network training and the recognition accuracy of occluded faces under the same training and testing datasets. The average accuracies are 98.78% on CelebA, 97.58% on AR, and 97.15% on FERET.

BFENet: A two-stream interaction CNN method for multi-label ophthalmic diseases classification with bilateral fundus images

Background and objectiveEarly fundus screening and timely treatment of ophthalmology diseases can effectively prevent blindness. Previous studies just focus on fundus images of single eye without utilizing the useful relevant information of the left and right eyes. While clinical ophthalmologists usually use binocular fundus images to help ocular disease diagnosis. Besides, previous works usually target only one ocular diseases at a time. Considering the importance of patient-level bilateral eye diagnosis and multi-label ophthalmic diseases classification, we propose a bilateral feature enhancement network (BFENet) to address the above two problems. MethodsWe propose a two-stream interactive CNN architecture for multi-label ophthalmic diseases classification with bilateral fundus images. Firstly, we design a feature enhancement module, which makes use of the interaction between bilateral fundus images to strengthen the extracted feature information. Specifically, attention mechanism is used to learn the interdependence between local and global information in the designed interactive architecture for two-stream, which leads to the reweighting of these features, and recover more details. In order to capture more disease characteristics, we further design a novel multiscale module, which enriches the feature maps by superimposing feature information of different resolutions images extracted through dilated convolution. ResultsIn the off-site set, the Kappa, F1, AUC and Final score are 0.535, 0.892, 0.912 and 0.780, respectively. In the on-site set, the Kappa, F1, AUC and Final score are 0.513, 0.886, 0.903 and 0.767 respectively. Comparing with existing methods, BFENet achieves the best classification performance. ConclusionsComprehensive experiments are conducted to demonstrate the effectiveness of this proposed model. Besides, our method can be extended to similar tasks where the correlation between different images is important.

Identification of two-phase flow regime in the energy industry based on modified convolutional neural network

The identification of the two-phase flow regime is the most fundamental and crucial target of fluid mechanics in the nuclear thermal-hydraulic analysis since the accuracy of fluid and heat transfer models depends on the reasonable prediction of flow regimes. The trend is quantitative identification based on the measurement of quantifiable parameters or the quantitative information of the two-phase flow images. Machine learning is currently the mainstream methodology for flow regime identification. The state-of-the-art method is using flow regime images for identification based on convolutional neural networks (CNNs). To achieve a higher classification accuracy and overcome the disadvantage of the traditional CNN (time-consuming owing to a large number of images required for training), this paper proposes a more efficient methodology for two-phase flow regime identification, which combines the advantages of CNN for image feature information extraction and the support vector machine (SVM) for classification - the CNN-SVM method. Furthermore, the traditional CNN needs a large number of images for training. Considering the limited flow regime dataset, the CNN-SVM method in this paper uses the idea of transfer learning. A small-scale flow regime dataset is used for validating the effectiveness of the proposed method. The accuracies for SVM, CNN and CNN-SVM were 93.9%, 97.2% and 98.8% respectively, which show that the proposed integrated method can realize better classification results since it combines the capability of strong feature extraction of CNN and classification techniques of SVM. The improvement is mainly reflected in the higher classification accuracy of slug flow and churn flow. Additionally, according to the results, the proposed method is also more robust.

Multi‐scale feature combination for person re‐identification

Person re-identification (Re-ID) is an instance-level task of image retrieval, and its identification accuracy depends on the distinguishable features extracted from people. However, most identification methods based on deep learning only mechanically extract distinguishable features of person images, and some important details are frequently overlooked. For scenes with substantial background differences or occlusions, the Re-ID efficiency is not high and the network scalability is not good. Here, the authors propose a multi-scale feature combination network (MFC-Net) model that combines structural feature information with global comprehensive feature information of the person images through a convolution neural network that can effectively retain distinguishing character detail information. The authors also propose a Gaussian stochastic pooling layer to solve the defects of the pooling layer. For the problem of many network parameters and weak performance, the authors propose an attentive feature convolutions layer. The authors perform many comparative experiments on three benchmark datasets. The results prove that our MFC-Net model performs well in person Re-ID and that its identification accuracy is higher than that of other investigated models.

Classification of crop pests based on multi-scale feature fusion

• A multi-scale feature extraction module (MFE) is designed by using multi-layer dilated convolution to obtain the multi-scale feature information of pest image with a large receptive field without increasing the amount of model parameters. • A deep feature extraction module (DFE) is proposed. Firstly, the deep feature information of pest image is obtained by multiple convolution operations, and then the obtained deep feature image is restored to the size of the original image by deconvolution. At the same time, the deep and shallow feature information are fused by multiple skip-connection structure ( Zhang et al., 2019 ) to reduce the loss of feature information. • We propose a network model based on multi-scale feature fusion to accurately identify and classify crop pests. Experiments on 12 types of pest datasets show that our method has better performance in pest identification and classification than other advanced methods, its classification accuracy (ACC) reaches 98.2%, and the model training time is only 197 min. Crop diseases and insect pests are a serious natural disaster, which needs to be predicted and monitored in time to ensure the output of crops. Due to the wide variety of pests and the similar morphology of crops in the early stages of growth, it is difficult for agricultural workers to accurately identify various types of pests. Crop insects have brought huge challenges to the prevention and control of plant diseases and insect pests. In response to this problem, we propose a way of classification of crop pests based on multi-scale feature fusion(MFFNet) to accurately recognizes and classifies crop pests. First, the multi-scale feature extraction module (MFE) is designed by using dilated convolution to obtain the multi-scale feature map of the pest image. At the same time, extracted the deep feature information of the image by the feature extraction module (DFE). Finally, the features extracted separately by the multi-scale feature extraction module (MFE) and the feature extraction module (DFE) were fused thus achieving accurately classified and identified the crops insects by the way of end-to-end. Experiments show that our proposed method has obtained excellent classification performance on the dataset of 12 types of pests, its classification accuracy rate (ACC) reached 98.2%.

RISSNet: Retain low‐light image details and improve the structural similarity net

Images taken under low light conditions cause people's sensory vision to fall into blind areas, and the essential properties of objects are hidden in the dark. There are many excellent algorithms that can adjust the brightness of high and low light region, but they still have room for improvement in realizing the function of denoising and similarity restoration of internal attributes of objects. Inspired by the work principle of human retina, the image is decomposed into two modules, firstly, the enhancement network which is guided by the Attention Mask module completed the enhancement by the U-net network structure. In this structural network, an RISS module is designed here to effectively reduce the loss of image feature information, so as to improve the structural similarity of reconstructed images. Full convolutional network is used to maintain the position relation of illumination feature information. This network is trained on LOL-dataset, and the experimental results prove that the network can recover images with different illuminations. In the later stage, the effectiveness of this method is verified by the subjective and objective comparison with other methods on different datasets. This method can not only realize the function of low-light image enhancement, but also the function of image similarity restoration and image denoising.

A FACIAL EXPRESSION RECOGNITION METHOD USING LOCAL NONLINEAR FEATURES

The purposes are to recognize the facial expression information and establish a fast and accurate facial expression recognition algorithm. First, the noise caused by face image shooting is eliminated by analyzing the primary methods of face recognition and preprocessing the images. Then, the Nonlinear Support Vector Machine (NSVM) is combined with the decision tree to obtain the Nonlinear Support Vector Machine Decision Tree (NSMMD) algorithm. By introducing the binary tree, NSVM has the ability of multi-classification and can extract the feature information of the face image to obtain the characteristic face. Then, the algorithm flow is introduced. Second, the characteristic face data are recognized and classified based on Generalized Regression Neural Network (GRNN), and the expression data contained in the face images are obtained to recognize the expressions of the face image. Finally, the designed face recognition algorithm and facial expression recognition algorithm are tested. The results show that the designed NSMMD algorithm has a weak dependence on feature dimension, and the face recognition rate for different datasets is above 80.1%. The GRNN algorithm will be affected by smooth parameters, its optimal parameter value is 0.1, and its recognition rate of expressions on various databases is above 92%. Therefore, the designed face recognition algorithm and facial expression recognition algorithm can provide sound recognition effects, which can reference the study of facial image feature recognition.

Genetic programming for feature extraction and construction in image classification

Genetic Programming (GP) has been successfully applied to image classification and achieved promising results. However, most existing methods either address binary image classification tasks only or need a predefined classifier to perform multi-class image classification while using GP for feature extraction. This limits their flexibility since it is unknown which combinations of classifiers and features are the most effective for an image classification task. Furthermore, high image variations increase the difficulty of feature extraction and image classification. This paper proposes a GP approach with a new program representation, new functions, and new terminals. The new approach can conduct feature extraction, feature construction, and classification, automatically and simultaneously. It can extract and construct informative image features, select a suitable classification algorithm instead of relying on a predefined classifier, and perform classification for binary and multi-class image classification tasks. In addition, this paper develops a new mutation operator based on fitness of population for dynamically adjusting the size of the evolved GP programs. The experimental results on eight datasets with different variations and difficulties show that the proposed approach achieves higher classification accuracy than most of the benchmark methods. Further analysis shows that the GP evolved programs have appropriate tree sizes and potentially high interpretability.

Extracting Buildings from Remote Sensing Images Using a Multitask Encoder-Decoder Network with Boundary Refinement

Extracting buildings from high-resolution remote sensing images is essential for many geospatial applications, such as building change detection, urban planning, and disaster emergency assessment. Due to the diversity of geometric shapes and the blurring of boundaries among buildings, it is still a challenging task to accurately generate building footprints from the complex scenes of remote sensing images. The rapid development of convolutional neural networks is presenting both new opportunities and challenges with respect to the extraction of buildings from high-resolution remote sensing images. To capture multilevel contextual information, most deep learning methods extract buildings by integrating multilevel features. However, the differential responses between such multilevel features are often ignored, leading to blurred contours in the extraction results. In this study, we propose an end-to-end multitask building extraction method to address these issues; this approach utilizes the rich contextual features of remote sensing images to assist with building segmentation while ensuring that the shape of the extraction results is preserved. By combining boundary classification and boundary distance regression, clear contour and distance transformation maps are generated to further improve the accuracy of building extraction. Subsequently, multiple refinement modules are used to refine each part of the network to minimize the loss of image feature information. Experimental comparisons conducted on the SpaceNet and Massachusetts building datasets show that the proposed method outperforms other deep learning methods in terms of building extraction results.

Two-Stage Deep Neural Network via Ensemble Learning for Melanoma Classification.

Melanoma is a skin disease with a high fatality rate. Early diagnosis of melanoma can effectively increase the survival rate of patients. There are three types of dermoscopy images, malignant melanoma, benign nevis, and seborrheic keratosis, so using dermoscopy images to classify melanoma is an indispensable task in diagnosis. However, early melanoma classification works can only use the low-level information of images, so the melanoma cannot be classified efficiently; the recent deep learning methods mainly depend on a single network, although it can extract high-level features, the poor scale and type of the features limited the results of the classification. Therefore, we need an automatic classification method for melanoma, which can make full use of the rich and deep feature information of images for classification. In this study, we propose an ensemble method that can integrate different types of classification networks for melanoma classification. Specifically, we first use U-net to segment the lesion area of images to generate a lesion mask, thus resize images to focus on the lesion; then, we use five excellent classification models to classify dermoscopy images, and adding squeeze-excitation block (SE block) to models to emphasize the more informative features; finally, we use our proposed new ensemble network to integrate five different classification results. The experimental results prove the validity of our results. We test our method on the ISIC 2017 challenge dataset and obtain excellent results on multiple metrics; especially, we get 0.909 on accuracy. Our classification framework can provide an efficient and accurate way for melanoma classification using dermoscopy images, laying the foundation for early diagnosis and later treatment of melanoma.

Sparse representation optimization of Gaussian mixed feature of image based on convolution neural network

With the rapid development of intelligent algorithm and image processing technology, the limitations of traditional image processing methods are more and more obvious. Based on this, this paper studies a new pattern of sparse representation optimization of image Gaussian mixture feature based on convolution neural network, and designs a sparse representation system model of vehicle detection image based on convolution neural network. The vehicle image data are collected from many aspects, and the convolution neural network is used for comprehensive analysis and evaluation. The model can extract the feature information of the vehicle detection image better by making the scheme of the real-time vehicle detection image and according to the image features and convolution neural network algorithm. The results show that the Gaussian mixture feature sparse representation optimization model based on convolution neural network has the advantages of high feasibility, high data accuracy and high response speed, which can enhance the processing efficiency of vehicle detection image and improve the utilization of local environmental information in the image.

Visual Relationship Detection With Image Position and Feature Information Embedding and Fusion

Visual relationship detection (VRD) is an important direction in the field of image processing, and it is a research task to explore object relationships based on object recognition and localization regression. At the same time, it is also one of the key contents of scene graph generation and construction of multimodal knowledge graph. In order to better improve the visual relationship detection effect, the fusion method of image location and feature information embedding is adopted on the vrd dataset. First, analyze the causes of errors in existing methods, and build a pre-training pre-dataset vrd_P for data enhancement; then, combine the characteristics of the vrd dataset to establish an entities relationship network (Entities-net) to guide target recognition; finally, a visual relationship detection model of LTransE is proposed to achieve the joint representation of target features and target locations for relationship detection. The results show that the method of embedding fusion of image location and feature information can effectively improve the visual relationship detection effect.

Thangka Image Segmentation Method Based on Enhanced Receptive Field

The portrait thangka image is a kind of religious scroll painting that expresses figures’ identity and duties through portraits, sitting platforms, and backlighting. The segmentation of significant semantic objects in the image is one of the essential ways for scholars to study and understand the image’s content. To better understand this content, we elaborately collected a dataset of portrait-like thangkas, which consists of 4086 images covering four object categories. We provide rich annotation for this dataset. In addition, we propose an end-to-end deep learning method that effectively solves the problems of blurred target edges, segmentation errors, and missed segmentation in thangka image segmentation. First, regular convolution and atrous convolution of different sizes are concatenated after the high-level feature output. This method can effectively improve the receptive field of the model while obtaining more image feature information. Then, the attention module is introduced to fully utilize the spatial relationship between the image’s semantic content and enhance the discriminative ability of the feature representation on the thangka image. Finally, cross-layer feature fusion is added to reduce the loss of edge details and improve the accuracy of target edge segmentation. The results show that compared to the base model, the mPA and mIoU indices of the model proposed in this paper reach 90.75% and 85.66%, respectively, which effectively improved the accuracy of the Tangka image segmentation.

Spectrum Sensing Method Based on Residual Cellular Network

The traditional spectrum sensing method based on convolutional neural network (CNN) has the single-branch convolutional network structure and the shallow network structure which limits the ability of extracting the Primary User (PU) feature. Aiming at these problems, a spectrum sensing method based on the residual cellular network (ResCelNet) is proposed in this work, involving a structure of "dual-branch convolution plus summation operation plus residual learning plus dual-branch convolution plus summation operation". Specifically, the dual-branch convolution improves the feature extraction ability, the addition operation enhances the micro-feature information, and the residual learning is adopted to facilitate training the deep spectrum sensing network. This method transforms the spectrum sensing problem into the image binary classification problem. Firstly, the received signals are reshaped into a matrix and normalized to gray levels, which is used as the input of the network. Secondly, the feature information of gray-scale images is extracted and the network is trained through dual-branch convolution and residual learning. Finally, the test data is input into the trained model and spectrum sensing based on image classification is implemented. The experimental results demonstrate that the proposed method exhibits a higher detection probability and a lower false alarm probability as well as better generalization ability as compared with the traditional methods under low signal to noise ratio (SNR) circumstance.

Dual-Stream Convolutional Neural Network With Residual Information Enhancement for Pansharpening

Deep-learning-based pansharpening methods have achieved remarkable results due to their powerful feature representation ability. However, the existing deep-learning-based pansharpening methods not only lack information exchange and sharing between features of different resolutions but also cannot effectively use the residual information at different levels. These disadvantages may lead to the loss of spatial information and spectral information in the pansharpened image. To address the above problems, we propose a novel dual-stream convolutional neural network with residual information enhancement (DSCNN-RIE) for pansharpening. The proposed network is mainly composed of a set of dual-stream information complementation blocks (DSICBs), which can extract various spatial details at two different resolutions using convolutional filters of various sizes simultaneously, and can transfer complementary information effectively between two different resolutions. Furthermore, to improve the learning ability of the network and enhance the feature extraction, an RIE strategy is presented to stack different levels of residuals into the outputs of cascaded DSICBs. The final pansharpened image is obtained by integrating the extracted features using the shallow feature information of the source images. Experimental results on three datasets demonstrate that DSCNN-RIE outperforms ten other state-of-the-art pansharpening methods in both subjective and objective image-quality evaluations.

A Deep Learning-Based Hyperspectral Keypoint Representation Method and Its Application for 3D Reconstruction

In previous studies, the keypoints and their feature descriptors of a hyperspectral image have not received enough attention. Compared with RGB images, the rich feature information of a hyperspectral image can not only constrain the spatial positions of the keypoints, but also provide rich spectral features for their descriptors. In this paper, we propose a deep neural network method called Hyperspectral SuperPoint Network (HSSPN) to detect the keypoints from a hyperspectral image and generate the corresponding feature descriptors. The HSSPN method contains a Shared Encoder (which is responsible for fully extracting the features of the spatial and spectral dimensions) and two decoders: the Keypoint Decoder accurately detects the keypoints according to the feature map outputted by the Shared Encoder, while the Descriptor Decoder is responsible for generating stable feature descriptors. We collected some hyperspectral images with an electronically controlled turntable, and then annotated them with a Hyperspectral Homographic Adaptation (HHA) network. The trained HSSPN gave an excellent keypoint matching performance on the test set. Hyperspectral 3D reconstruction can obtain information in a 3D space and a spectral space at the same time. Most of the previous reconstruction methods are separated for the treatments of 3D reconstruction and hyperspectral imaging, which require complicated registrations. In this study, we propose a SfM-based hyperspectral 3D reconstruction method, in which we first collect hyperspectral images from multiple viewing angles, and then apply the HSSPN method to detect the keypoints and calculate their feature descriptors. These keypoints and feature descriptors are then used to reconstruct each spectral image of the target with the SfM method. The 3D reconstruction models at these wavelengths are simply superimposed to obtain a hyperspectral 3D reconstruction model. Compared with conventional SIFT-based SfM methods, our method not only contains richer spectral information, but also achieves better results in the reconstruction accuracy.

Gradient and Multi Scale Feature Inspired Deep Blind Gaussian Denoiser

In this paper, a novel deep blind Gaussian denoising network is proposed utilizing the concepts of gradient information, multi-scale feature information and feature denoising for removing additive white Gaussian noise (AWGN) from images. The proposed network consists of two modules where in the first module generates an intermediate image whose gradient information is concatenated with the features of second module to generate the final residual image. Subtracting this residual image with the noisy image gives the desired denoised image. The feature denoising block used in the middle of the first module enhances the feature information of the intermediate image. The usage of gradient information of this intermediate denoised image, together with the multi scale feature information block, in the second module further contributes to the quality of the final denoised image. Experimental results show superior denoising performance of the proposed method in comparison to several state of the art classical and learning based blind denoising methods like EPLL, BM3D, WNNM, DnCNN, MemNet, BUIFD, Self2Self and ComplexNet by a decent margin (an improvement of up to 2.4dB in PSNR, 0.07 in SSIM and 0.03 in FOM index with the second best performing model) when experimented over BSD68, Set5, Set14, SunHays80 and Manga109 image databases.

A Dual-Channel Fully Convolutional Network for Land Cover Classification Using Multifeature Information

High-resolution remote sensing images have the advantage of timeliness, and they can display feature information in more detail. Deep learning embodies its unique characteristics in land cover classification, target recognition, and other fields, which can automatically learn the in-depth feature information of images and make accurate classification decisions. However, when deep learning models extract high-dimensional abstract feature information, they often ignore and lose part of the underlying features essential for classification accuracy. This article proposes a dual-channel fully convolutional network (D-FCN), whose two channels, respectively, take image data and low-level features such as color, texture, and shape as the different input data to combine the underlying features with high-dimensional abstract features. To reduce the complexity of the model, we add a large number of skip connections between the model and make full use of the advantages of weight sharing and local connections to connect spatial context information. We used multifeature information as the model input and compared and analyzed the impact of different features on the land cover classification accuracy, and finally obtained the most suitable combination of multifeature information. In addition, we provide a small-scale land cover classification dataset with labels to verify the applicability and transferability of the D-FCN, and use the optimal combination of multifeature information to conduct comparative experiments on the small-scale dataset. The experimental results show that D-FCN has outstanding applicability and transferability. Compared with other state-of-the-art models, D-FCN has a more challenging performance and greatly reduces model complexity.

A novel ResNet101 model based on dense dilated convolution for image classification

Image classification plays an important role in computer vision. The existing convolutional neural network methods have some problems during image classification process, such as low accuracy of tumor classification and poor ability of feature expression and feature extraction. Therefore, we propose a novel ResNet101 model based on dense dilated convolution for medical liver tumors classification. The multi-scale feature extraction module is used to extract multi-scale features of images, and the receptive field of the network is increased. The depth feature extraction module is used to reduce background noise information and focus on effective features of the focal region. To obtain broader and deeper semantic information, a dense dilated convolution module is deployed in the network. This module combines the advantages of Inception, residual structure, and multi-scale dilated convolution to obtain a deeper level of feature information without causing gradient explosion and gradient disappearance. To solve the common feature loss problems in the classification network, the up- down-sampling module in the network is improved, and multiple convolution kernels with different scales are cascaded to widen the network, which can effectively avoid feature loss. Finally, experiments are carried out on the proposed method. Compared with the existing mainstream classification networks, the proposed method can improve the classification performance, and finally achieve accurate classification of liver tumors. The effectiveness of the proposed method is further verified by ablation experiments.HighlightsThe multi-scale feature extraction module is introduced to extract multi-scale features of images, it can extract deep context information of the lesion region and surrounding tissues to enhance the feature extraction ability of the network.The depth feature extraction module is used to focus on the local features of the lesion region from both channel and space, weaken the influence of irrelevant information, and strengthen the recognition ability of the lesion region.The feature extraction module is enhanced by the parallel structure of dense dilated convolution, and the deeper feature information is obtained without losing the image feature information to improve the classification accuracy.