Multi-level Deep Features Research Articles

SAR Target Recognition Method based on Adaptive Weighted Decision Fusion of Deep Features

Background: This paper proposes a synthetic aperture radar (SAR) target recognition method based on adaptive weighted decision fusion of multi-level deep features. Methods: The trained ResNet-18 is employed to extract multi-level deep features from SAR images. Afterwards, based on the joint sparse representation (JSR) model, the multi-level deep features are represented to obtain the corresponding reconstruction error vectors. Considering the differences in the abilities of different levels of features to distinguish the target, the reconstruction error vectors are analyzed based on entropy theory, and their corresponding weights are adaptively obtained. Finally, the fused reconstruction error result is obtained through adaptively weighted fusion, and the target label is determined accordingly. Results: Experiments are conducted on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset under different conditions, and the proposed method is compared with published methods, including multi-feature decision fusion, JSR-based decision fusion and other types of ResNets. Conclusion: The experimental results under standard operating condition (SOC) and extended operating conditions (EOCs) including depression angle variance and noise corruption validate the advantages of the proposed method.

Volumetric Model Genesis in Medical Domain for the Analysis of Multimodality 2-D/3-D Data Based on the Aggregation of Multilevel Features

The automatic and accurate classification of medical imaging data has potential applications in computer-aided disease diagnosis, prognosis, and treatment. However, it remains a challenge to optimize recent deep learning algorithms in medical domain for the accurate classification of large-scale 3D volumetric data. To address these challenges, we propose an efficient deep volumetric classification network based on the aggregation of multilevel deep features for accurate classification of large-scale medical 2D/3D imaging data. To perform a detailed quantitative analysis of our method, 26 different datasets were fused to construct a single large-scale multimodal database that comprises a total of seventy different classes, including 151,095 data samples. Additionally, 15 different baseline methods were configured under the same experimental protocol for volumetric model genesis and extensive performance comparison with our method. The experimental results of our method exhibited promising performance as area under the curve of 93.66% and outperformed various state-of-the-art methods.

CANet: Context-aware Aggregation Network for Salient Object Detection of Surface Defects

Surface defect detection has become more and more important in the industrial manufacture and engineering application in recent years. However, due to the lack of overall perception and interaction among features layers, lots of computer vision-based detection methods cannot grab the complete details of defects when dealing with complex scenes, such as low contrast and irregular shape. Therefore, in this paper, we propose a Context-aware Aggregation Network (CANet) to accurately pop-out the defects, where we focus on the mining of context cues and the fusion of multiple context features. To be specific, embarking on the multi-level deep features extracted by encoder, we first deploy a sufficient exploration to dig the context information by deploying the weighted convolution pyramid (WCP) module, which extracts multi-scale context features, transfers the information flow between different resolution features, and fuses the features with same resolution. By this way, we can obtain the effective context pyramid features. Then, the decoder deploys the weighted context attention (WCA) module to filter the irrelevant information from context features and employs the cascaded fusion structure (CFS) to aggregate the multiple context cues in a hierarchical way. Following this way, the generated high-quality saliency maps can highlight the defects accurately and completely. Extensive experiments are performed on four public datasets, and the results firmly prove the effectiveness and superiority of the proposed CANet under different evaluation metrics.

A two stream face anti-spoofing framework using multi-level deep features and ELBP features

A two stream face anti-spoofing framework using multi-level deep features and ELBP features

Small Sample Hyperspectral Image Classification Based on the Random Patches Network and Recursive Filtering

In recent years, different deep learning frameworks were introduced for hyperspectral image (HSI) classification. However, the proposed network models have a higher model complexity, and do not provide high classification accuracy if few-shot learning is used. This paper presents an HSI classification method that combines random patches network (RPNet) and recursive filtering (RF) to obtain informative deep features. The proposed method first convolves image bands with random patches to extract multi-level deep RPNet features. Thereafter, the RPNet feature set is subjected to dimension reduction through principal component analysis (PCA), and the extracted components are filtered using the RF procedure. Finally, the HSI spectral features and the obtained RPNet-RF features are combined to classify the HSI using a support vector machine (SVM) classifier. In order to test the performance of the proposed RPNet-RF method, some experiments were performed on three widely known datasets using a few training samples for each class, and classification results were compared with those obtained by other advanced HSI classification methods adopted for small training samples. The comparison showed that the RPNet-RF classification is characterized by higher values of such evaluation metrics as overall accuracy and Kappa coefficient.

Edge-Guided Recurrent Positioning Network for Salient Object Detection in Optical Remote Sensing Images.

Optical remote sensing images (RSIs) have been widely used in many applications, and one of the interesting issues about optical RSIs is the salient object detection (SOD). However, due to diverse object types, various object scales, numerous object orientations, and cluttered backgrounds in optical RSIs, the performance of the existing SOD models often degrade largely. Meanwhile, cutting-edge SOD models targeting optical RSIs typically focus on suppressing cluttered backgrounds, while they neglect the importance of edge information which is crucial for obtaining precise saliency maps. To address this dilemma, this article proposes an edge-guided recurrent positioning network (ERPNet) to pop-out salient objects in optical RSIs, where the key point lies in the edge-aware position attention unit (EPAU). First, the encoder is used to give salient objects a good representation, that is, multilevel deep features, which are then delivered into two parallel decoders, including: 1) an edge extraction part and 2) a feature fusion part. The edge extraction module and the encoder form a U-shape architecture, which not only provides accurate salient edge clues but also ensures the integrality of edge information by extra deploying the intraconnection. That is to say, edge features can be generated and reinforced by incorporating object features from the encoder. Meanwhile, each decoding step of the feature fusion module provides the position attention about salient objects, where position cues are sharpened by the effective edge information and are used to recurrently calibrate the misaligned decoding process. After that, we can obtain the final saliency map by fusing all position attention cues. Extensive experiments are conducted on two public optical RSIs datasets, and the results show that the proposed ERPNet can accurately and completely pop-out salient objects, which consistently outperforms the state-of-the-art SOD models.

MrFDDGAN: Multireceptive Field Feature Transfer and Dual Discriminator-Driven Generative Adversarial Network for Infrared and Color Visible Image Fusion

Most of the previous infrared and visible image fusion methods based on deep learning were on the basis of grayscale images and used the single convolution kernel receptive field to extract deep features, which would inevitably cause information loss in the process of feature transfer. Accordingly, this paper proposes a new generative adversarial network based on multi-receptive field feature transfer and dual discriminators, which is named MrFDDGAN. It is applied to infrared and color visible image fusion. Firstly, three different receptive fields are used to extract multi-scale and multi-level deep features of multi-modal images on three channels, so as to obtain significant features of source images from multi-view and multi-perspective in a more comprehensive way. Secondly, a feature interaction module is introduced in the encoder to realize the information interaction and information pre-fusion among the three feature channels. Thirdly, the multi-level features fused in the encoder are cascaded with the deep features in the decoder to enhance feature transfer and feature reuse. Finally, this experiment adopts a dual-discriminator adversarial network structure to keep the balance of infrared image intensity and visible image texture preserved by fusion results. And qualitative and quantitative experimental analyses are carried out on two public datasets of grayscale infrared as well as visible images and one dataset of infrared and color visible images. The experimental results prove that the proposed MrFDDGAN algorithm has a better subjective visual effect and objective performance than the existing state-of-the-art fusion methods.

SRI-Net: Similarity retrieval-based inference network for light field salient object detection

The cutting-edge RGB saliency models are prone to fail for some complex scenes, while RGB-D saliency models are often affected by inaccurate depth maps. Fortunately, light field images can provide a sufficient spatial layout depiction of 3D scenes. Therefore, this paper focuses on salient object detection of light field images, where a Similarity Retrieval-based Inference Network (SRI-Net) is proposed. Due to various focus points, not all focal slices extracted from light field images are beneficial for salient object detection, thus, the key point of our model lies in that we attempt to select the most valuable focal slice, which can contribute more complementary information for the RGB image. Specifically, firstly, we design a focal slice retrieval module (FSRM) to choose an appropriate focal slice by measuring the foreground similarity between the focal slice and RGB image. Secondly, in order to combine the original RGB image and the selected focal slice, we design a U-shaped saliency inference module (SIM), where the two-stream encoder is used to extract multi-level features, and the decoder is employed to aggregate multi-level deep features. Extensive experiments are conducted on two widely used light field datasets, and the results firmly demonstrate the superiority and effectiveness of the proposed SRI-Net.

Efficient and Refined Deep Convolutional Features Network for the Crack Segmentation of Solar Cell Electroluminescence Images

High-quality and fast crack segmentation in solar cell electroluminescence images with heterogeneously textured background disturbance is challenging. We propose an end-to-end Efficient and Refined Deep Convolutional Features Network (ERDCF-Net) for precise and efficient crack segmentation. Firstly, we design a lightweight, efficient deep convolutional features (EDCF) encoder by adopting multilevel deep convolutional features to generate a discriminative feature representation of cracks. Moreover, some complicated background disturbance is restrained by reusing rich crack and background features. Secondly, we present a refined side output (RS) decoder by automatically refining side outputs from the EDCF encoder. The experiments demonstrate that the proposed network excellently performs on the solar cell cracks (MIoU of 92.82%, F - measure of 93.58%, and 89 FPS) and open crack datasets (MIoU of 85.6%, F - measure of 84.1%, and 90 FPS).

White blood cell classification via a discriminative region detection assisted feature aggregation network.

White blood cell (WBC) classification plays an important role in human pathological diagnosis since WBCs will show different appearance when they fight with various disease pathogens. Although many previous white blood cell classification have been proposed and earned great success, their classification accuracy is still significantly affected by some practical issues such as uneven staining, boundary blur and nuclear intra-class variability. In this paper, we propose a deep neural network for WBC classification via discriminative region detection assisted feature aggregation (DRFA-Net), which can accurately locate the WBC area to boost final classification performance. Specifically, DRFA-Net uses an adaptive feature enhancement module to refine multi-level deep features in a bilateral manner for efficiently capturing both high-level semantic information and low-level details of WBC images. Considering the fact that background areas could inevitably produce interference, we design a network branch to detect the WBC area with the supervision of segmented ground truth. The bilaterally refined features obtained from two directions are finally aggregated for final classification, and the detected WBC area is utilized to highlight the features of discriminative regions by an attention mechanism. Extensive experiments on several public datasets are conducted to validate that our proposed DRFA-Net can obtain higher accuracies when compared with other state-of-the-art WBC classification methods.

Automated EEG sentence classification using novel dynamic-sized binary pattern and multilevel discrete wavelet transform techniques with TSEEG database

Electroencephalography (EEG) signal is an important physiological signal commonly used in machine learning to decode brain activities, including imagined words and sentences. We aimed to develop an automated lightweight EEG signal-based sentence classification model using a novel dynamic-sized binary pattern (DSBP) textural feature extractor and iterative multi-classifiers based majority voting (IMCMV) algorithm for iterative voting of results calculated using different classifiers for multi-channel EEG signal inputs. A new Turkish sentence EEG(TSEEG) was prospectively acquired. It comprised of 15-second 14-channel EEG signals recorded when 40 volunteers (for each dataset, we collected EEG signals from 20 participants) were either shown or read corresponding to demonstration or listening modes, respectively. Hence, 20 standardized commonly used sentences were obtained in their native Turkish language. The developed sentence classification model extracted 5,400 multilevel deep features from each channel EEG signal segment using the novel DSBP, statistical features, and multilevel discrete wavelet transform (MDWT). 512 features were then chosen using the neighborhood component analysis selection function. k-nearest neighbor and support vector machine classifiers were used to calculate two prediction vectors from the selected features using tenfold cross-validation, i.e., 28 vectors were generated for each 14-channel EEG recording. Finally, the best general voted results were determined for increasing numbers of iteratively calculated prediction vectors using the novel IMCMV algorithm. Channel-wise and voted results were found to be excellent for sentence classification for the TSEEG dataset in both demonstration and listening modes. The DSBP-IMCMV-based model attained the best general classification rates of 98.81% and 98.19% in the demonstration and listening modes, respectively.

MSAt-GAN: a generative adversarial network based on multi-scale and deep attention mechanism for infrared and visible light image fusion

For the past few years, image fusion technology has made great progress, especially in infrared and visible light image infusion. However, the fusion methods, based on traditional or deep learning technology, have some disadvantages such as unobvious structure or texture detail loss. In this regard, a novel generative adversarial network named MSAt-GAN is proposed in this paper. It is based on multi-scale feature transfer and deep attention mechanism feature fusion, and used for infrared and visible image fusion. First, this paper employs three different receptive fields to extract the multi-scale and multi-level deep features of multi-modality images in three channels rather than artificially setting a single receptive field. In this way, the important features of the source image can be better obtained from different receptive fields and angles, and the extracted feature representation is also more flexible and diverse. Second, a multi-scale deep attention fusion mechanism is designed in this essay. It describes the important representation of multi-level receptive field extraction features through both spatial and channel attention and merges them according to the level of attention. Doing so can lay more emphasis on the attention feature map and extract significant features of multi-modality images, which eliminates noise to some extent. Third, the concatenate operation of the multi-level deep features in the encoder and the deep features in the decoder are cascaded to enhance the feature transmission while making better use of the previous features. Finally, this paper adopts a dual-discriminator generative adversarial network on the network structure, which can force the generated image to retain the intensity of the infrared image and the texture detail information of the visible image at the same time. Substantial qualitative and quantitative experimental analysis of infrared and visible image pairs on three public datasets show that compared with state-of-the-art fusion methods, the proposed MSAt-GAN network has comparable outstanding fusion performance in subjective perception and objective quantitative measurement.

Weakly Supervised Video Object Segmentation via Dual-attention Cross-branch Fusion

Recently, concerning the challenge of collecting large-scale explicitly annotated videos, weakly supervised video object segmentation (WSVOS) using video tags has attracted much attention. Existing WSVOS approaches follow a general pipeline including two phases, i.e., a pseudo masks generation phase and a refinement phase. To explore the intrinsic property and correlation buried in the video frames, most of them focus on the later phase by introducing optical flow as temporal information to provide more supervision. However, these optical flow-based studies are greatly affected by illumination and distortion and lack consideration of the discriminative capacity of multi-level deep features. In this article, with the goal of capturing more effective temporal information and investigating a temporal information fusion strategy accordingly, we propose a unified WSVOS model by adopting a two-branch architecture with a multi-level cross-branch fusion strategy, named as dual-attention cross-branch fusion network (DACF-Net). Concretely, the two branches of DACF-Net, i.e., a temporal prediction subnetwork (TPN) and a spatial segmentation subnetwork (SSN), are used for extracting temporal information and generating predicted segmentation masks, respectively. To perform the cross-branch fusion between TPN and SSN, we propose a dual-attention fusion module that can be plugged into the SSN flexibly. We also pose a cross-frame coherence loss (CFCL) to achieve smooth segmentation results by exploiting the coherence of masks produced by TPN and SSN. Extensive experiments demonstrate the effectiveness of proposed approach compared with the state-of-the-arts on two challenging datasets, i.e., Davis-2016 and YouTube-Objects.

Dense Attention-Guided Cascaded Network for Salient Object Detection of Strip Steel Surface Defects

Recently, more and more researchers have paid attention to the surface defect detection of strip steel. However, the performance of existing methods usually fails to detect the defect regions from some complex scenes, especially with the noise disturbance and diverse defect types. Therefore, this article proposes an end-to-end dense attention-guided cascaded network (DACNet) to detect salient objects (i.e., defects) on the strip steel surface, where the proposed DACNet is a U-shape network including an encoder and a decoder. The encoder first deploys multiresolution convolutional branches (i.e., high/medium/low) in a cascaded way. Concretely, the cascaded feature integration (CFI) unit fuses the deep features from the last convolutional blocks of multiresolution branches, yielding the enhanced high-level deep semantic feature. Subsequently, coupled with the multilevel deep features from high-resolution branch, the new multiscale deep features are capable of characterizing various defects. Then, driven by the dense attention mechanism which enables the deeper attention cues flow into decoding stages, the decoder progressively integrates the multiscale deep features into the final saliency map, where the dense attention is designed to steer deep features pay more concerns to the defect regions. Comprehensive experiments are conducted on the public strip steel datasets, and the experimental results demonstrate that our model consistently outperforms the state-of-the-art models in all evaluation metrics.

Variance-aware attention U-Net for multi-organ segmentation.

With the continuous development of deep learning based medical image segmentation technology, it is expected to attain more robust and accurate performance for more challenging tasks, such as multi-organs, small/irregular areas, and ambiguous boundaryissues. We propose a variance-aware attention U-Net to solve the problem of multi-organ segmentation. Specifically, a simple yet effective variance-based uncertainty mechanism is devised to evaluate the discrimination of each voxel via its prediction probability. The proposed variance uncertainty is further embedded into an attention architecture, which not only aggregates multi-level deep features in a global-level but also enforces the network to pay extra attention to voxels with uncertain predictions duringtraining. Extensive experiments on challenging abdominal multi-organ CT dataset show that our proposed method consistently outperforms cutting-edge attention networks with respect to the evaluation metrics of Dice index (DSC), 95% Hausdorff distance (95HD) and average symmetric surface distance (ASSD). The proposed network provides an accurate and robust solution for multi-organ segmentation and has the potential to be used for improving other segmentationapplications.

Densely connected graph convolutional network for joint semantic and instance segmentation of indoor point clouds

In this paper, a densely connected graph convolutional network is proposed to jointly realize the semantic and instance segmentation of indoor point clouds. We combine a Graph Convolutional Network (GCN) and Multilayer Perceptron (MLP) into a new model (namely GCN-MLP) and design a more efficient attention pooling operation to establish a new and efficient module for extracting point cloud features. Also, we add a point cloud channel aggregation module to aggregate multi-level deep features to better express the discriminative characteristics of indoor point clouds. Next, a framework for joint semantic and instance segmentation is designed on the basis of the above modules. In the framework, the semantic branch and instance branch promote each other and obtain better semantic and instance segmentation effects simultaneously. Besides, a dense connection way among different levels of feature maps is designed, to fully extract the features of indoor scene. We have conducted the experiments on the public datasets (S3DIS and ShapeNet), and the results show that our framework is superior to other methods and also achieve better results in part segmentation. Compared with JSNet (Zhao and Tao, 2020), our method improves the indicators of mCov, mWCov, mRec, and mPrec by 3.3, 3.5, 4.6, and 1.8, respectively, in the instance segmentation of S3DIS (Area 5). Besides, we obtain the best results in 10 of the 13 classes in terms of IoU. For part segmentation of indoor objects, we also outperform ASIS (Wang et al., 2019b) by 0.4 on mIoU.

Multilevel Deep-Aggregated Boosted Network to Recognize COVID-19 Infection from Large-Scale Heterogeneous Radiographic Data.

In the present epidemic of the coronavirus disease 2019 (COVID-19), radiological imaging modalities, such as X-ray and computed tomography (CT), have been identified as effective diagnostic tools. However, the subjective assessment of radiographic examination is a time-consuming task and demands expert radiologists. Recent advancements in artificial intelligence have enhanced the diagnostic power of computer-aided diagnosis (CAD) tools and assisted medical specialists in making efficient diagnostic decisions. In this work, we propose an optimal multilevel deep-aggregated boosted network to recognize COVID-19 infection from heterogeneous radiographic data, including X-ray and CT images. Our method leverages multilevel deep-aggregated features and multistage training via a mutually beneficial approach to maximize the overall CAD performance. To improve the interpretation of CAD predictions, these multilevel deep features are visualized as additional outputs that can assist radiologists in validating the CAD results. A total of six publicly available datasets were fused to build a single large-scale heterogeneous radiographic collection that was used to analyze the performance of the proposed technique and other baseline methods. To preserve generality of our method, we selected different patient data for training, validation, and testing, and consequently, the data of same patient were not included in training, validation, and testing subsets. In addition, fivefold cross-validation was performed in all the experiments for a fair evaluation. Our method exhibits promising performance values of 95.38%, 95.57%, 92.53%, 98.14%, 93.16%, and 98.55% in terms of average accuracy, F-measure, specificity, sensitivity, precision, and area under the curve, respectively and outperforms various state-of-the-art methods.

Spatial context-aware network for salient object detection

Salient Object Detection (SOD) is a fundamental problem in the field of computer vision. This paper presents a novel Spatial Context-Aware Network (SCA-Net) for SOD in images. Compared with other recent deep learning based SOD algorithms, SCA-Net can more effectively aggregate multi-level deep features. A Long-Path Context Module (LPCM) is employed to grant better discrimination ability to feature maps that incorporate coarse global information. Consequently, a more accurate initial saliency map can be obtained to facilitate subsequent predictions. SCA-Net also adopts a Short-Path Context Module (SPCM) to progressively enforce the interaction between local contextual cues and global features. Extensive experiments on five large-scale benchmarks demonstrate that SCA-Net achieves favorable performance against very recent state-of-the-art algorithms.

AMPNet: Average- and Max-Pool Networks for Salient Object Detection

Salient Object Detection aims to detect the most visually distinctive objects in an image. We solve this problem by introducing the average pool to explore the multi-level deep average pool convolution features different from the max pool information. Based on the U-net structure, we propose an Average- and Max-Pool Network (AMPNet) that leverages the average- and max-pool modules to integrate the multi-level complementary contextual features in the spatial and channel-wise dimensions, respectively. The complementary contextual features generated by our network can improve the completeness of detected objects. It has been observed that the non-salient regions are misrecognized as the salient objects because of the redundant information contained in the multi-level convolution features. To address the problem, two top-down feedback paths are introduced based on the above two modules, and their top-level semantic guidance information is fully utilized to improve the accuracy of salient objects detection. Finally, we apply the Feature Fusion Module and Deep Supervision Mechanism to further improve the performance of the network over different datasets. Experimental results on six benchmark datasets show that our network is on par with state-of-the-art approaches. Our method runs at more than 45 FPS (based on VGG) and 35 FPS (based on ResNet) on a single GPU and meets real-time requirements.

A SAR Target Recognition Method via Combination of Multilevel Deep Features

For the problem of synthetic aperture radar (SAR) image target recognition, a method via combination of multilevel deep features is proposed. The residual network (ResNet) is used to learn the multilevel deep features of SAR images. Based on the similarity measure, the multilevel deep features are clustered and several feature sets are obtained. Then, each feature set is characterized and classified by the joint sparse representation (JSR), and the corresponding output result is obtained. Finally, the results of different feature sets are combined using the weighted fusion to obtain the target recognition results. The proposed method in this paper can effectively combine the advantages of ResNet and JSR in feature extraction and classification and improve the overall recognition performance. Experiments and analysis are carried out on the MSTAR dataset with rich samples. The results show that the proposed method can achieve superior performance for 10 types of target samples under the standard operating condition (SOC), noise interference, and occlusion conditions, which verifies its effectiveness.