Multi-layer Feature Research Articles

Underwater Incomplete Target Recognition Network via Generating Feature Module

A complex and changeable underwater archaeological environment leads to the lack of target features in the collected images, affecting the accuracy of target detection. Meanwhile, the difficulty in obtaining underwater archaeological images leads to less training data, resulting in poor generalization performance of the recognition algorithm. For these practical issues, we propose an underwater incomplete target recognition network via generating feature module (UITRNet). Specifically, for targets that lack features, features are generated by dual discriminators and generators to improve target detection accuracy. Then, multilayer features are fused to extract regions of interest. Finally, supervised contrastive learning is introduced into few-shot learning to improve the intraclass similarity and interclass distance of the target and enhance the generalization of the algorithm. The UIFI dataset is produced to verify the effectiveness of the algorithm in this paper. The experimental results show that the mean average precision (mAP) of our algorithm was improved by 0.86% and 1.29% under insufficient light and semiburied interference, respectively. The mAP for ship identification reached the highest level under all four sets of experiments.

Water extraction from optical high-resolution remote sensing imagery: a multi-scale feature extraction network with contrastive learning

ABSTRACT Accurately spatiotemporal distribution of water bodies is of great importance in the fields of ecology and environment. Recently, convolutional neural networks (CNN) have been widely used for this purpose due to their powerful features extraction ability. However, the CNN methods have two limitations in extracting water bodies. First, the large variations in both the spatial and spectral characteristics of water bodies require that the CNN-based methods have the ability of extracting multi-scale features and using multi-layer features. Second, collecting enough samples is a difficult problem in the training phase of CNN. Therefore, this paper proposed a multi-scale features extraction network (MSFENet) for water extraction, and its advantages are contributed to two distinct features: (1) scale features extractor (MSFE) is designed to extract multi-layer multi-scale features of water bodies; (2) contrastive learning (CL) is adopted to reduce the sample size requirement. Experimental results show that MSFE can effectively improve the small water body extraction performance, and the CL can significantly improve the extraction accuracy when the training sample size is small. Compared with other methods, MSFENet achieves the highest F1-score and kappa coefficient in two datasets. Furthermore, spectral variability analysis shows that MSFENet is more robust than other neural networks in a spectrum variation scenario.

Detail enhancement-based vehicle re-identification with orientation-guided re-ranking

Vehicle re-identification (re-ID) has attracted extensive attention in the field of computer vision owing to the development of smart cities. Two issues remain in the task of vehicle re-ID: minor inter-class differences and extreme orientation variations. To address them, we propose a detailed enhancement-based vehicle re-ID method with orientation-guided re-ranking. In the proposed method, a novel enhancement module using stripe-based partition, is presented to avoid the negative influence of minor inter-class difference. The stripe-based partition subdivides the feature map of middle layers in the network into two stripes, and then the module explores more detailed information for vehicle re-ID. Furthermore, a multilayer feature fusion module is proposed to enhance the feature representation of a vehicle. To address the problem of extreme orientation variation in vehicle re-ID, we propose an orientation-guided re-ranking strategy to re-rank the retrieval result based on the orientation information and query expansion algorithm. This strategy can optimize the ranking of vehicles whose orientation is not similar to the query images in the post-processing stage. Extensive experiments on three public datasets confirm the effectiveness of our method.

Multicascaded Feature Fusion-Based Deep Learning Network for Local Climate Zone Classification Based on the So2Sat LCZ42 Benchmark Dataset

A detailed investigation of the microclimate is beneficial for optimizing the urban inner/spatial pattern to enhance thermal comfort or even reduce heatwave disasters, whereas accurately classifying local climate zones (LCZs) severely restricts analysis of thermal characterization. Generally, deep learning-based approaches are effective for adaptive LCZ mapping, yet they often have poor accuracy because inadequate cascade feature extraction patterns may not adapt to the fuzzy LCZ boundaries produced by intricate urban textures, especially when using large-scale datasets. To address these issues, we propose a novel CNN model in which we design a strategy that incorporates a triple feature fusion pattern to enhance LCZ recognition based on the So2sat LCZ 42 large-scale annotated dataset. The approach connects multilayer cascaded information to participate in category judgment, which avoids the loss of effective feature information via continuous cascade transformation as much as possible. The results show that the overall accuracy and kappa coefficient of the proposed model reach 0.70 and 0.68, respectively, manifesting an improvement of approximately 4.47% and 6.25% over advanced LCZ classification approaches. In particular, the accuracy of the proposed approach improves even further after the fusion structure or layer depth is partially removed or reduced, respectively. Finally, we discuss several items, including the effectiveness of different parameters and cascaded feature fusion patterns, the superiority of multilayer cascade feature fusion, the mapping impact of seasons and cloud cover, and even some other issues in LCZ mapping. This article will facilitate improvements in the research precision of urban thermal environments.

M2F2-RCNN: Multi-functional faster RCNN based on multi-scale feature fusion for region search in remote sensing images

In order to realize fast and accurate search of sensitive regions in remote sensing images, we propose a multi-functional faster RCNN based on multi-scale feature fusion model for region search. The feature extraction network is based on ResNet50 and the dilated residual blocks are utilized for multi-layer and multi-scale feature fusion. We add a path aggregation network with a convolution block attention module (CBAM) attention mechanism in the backbone network to improve the efficiency of feature extraction. Then, the extracted feature map is processed, and RoIAlign is used to improve the pooling operation of regions of interest and it can improve the calculation speed. In the classification stage, an improved nonmaximum suppression is used to improve the classification accuracy of the sensitive region. Finally, we conduct cross validation experiments on Google Earth dataset and the DOTA dataset. Meanwhile, the comparison experiments with the state -of the- art methods also prove the high efficiency of the proposed method in region search ability.

MIC-Net: multi-scale integrated context network for automatic retinal vessel segmentation in fundus image.

Accurate retinal vessel segmentation is of great value in the auxiliary screening of various diseases. However, due to the low contrast between the ends of the branches of the fundus blood vessels and the background, and the variable morphology of the optic disc and cup in the retinal image, the task of high-precision retinal blood vessel segmentation still faces difficulties. This paper proposes a multi-scale integrated context network, MIC-Net, which fully fuses the encoder-decoder features, and extracts multi-scale information. First, a hybrid stride sampling (HSS) block was designed in the encoder to minimize the loss of helpful information caused by the downsampling operation. Second, a dense hybrid dilated convolution (DHDC) was employed in the connection layer. On the premise of preserving feature resolution, it can perceive richer contextual information. Third, a squeeze-and-excitation with residual connections (SERC) was introduced in the decoder to adjust the channel attention adaptively. Finally, we utilized a multi-layer feature fusion mechanism in the skip connection part, which enables the network to consider both low-level details and high-level semantic information. We evaluated the proposed method on three public datasets DRIVE, STARE and CHASE. In the experimental results, the Area under the receiver operating characteristic (ROC) and the accuracy rate (Acc) achieved high performances of 98.62%/97.02%, 98.60%/97.76% and 98.73%/97.38%, respectively. Experimental results show that the proposed method can obtain comparable segmentation performance compared with the state-of-the-art (SOTA) methods. Specifically, the proposed method can effectively reduce the small blood vessel segmentation error, thus proving it a promising tool for auxiliary diagnosis of ophthalmic diseases.

Vibration-Adaption Deep Convolutional Transfer Learning Method for Stranded Wire Structural Health Monitoring Using Guided Wave

External vibration is the main disturbance condition in the practical monitoring of outdoor stranded structure using laser ultrasonic guided wave (UGW). It is difficult to extract and identify the real damage state under different vibration conditions due to the variation of the guided-wave feature distribution. At present, there is no effective solution to this practical problem. In this article, a new deep cross-domain adaptive semisupervised damage identification method is proposed by using transfer learning method and combining with the actual demand of stranded guided-wave monitoring. First, a novel laser excitation-piezoelectric receiving sensing method is realized by taking full advantage of the noncontact characteristics, wide frequency band, and high stability of the laser and piezoelectric sensors. Second, a multilayer convolutional neural network (CNN) is constructed to extract the damage features of the guided-wave signals in the source domain and map them to the high-level hidden space. Then, a multicore maximum mean discrepancy (MMD) method is designed to reduce the distribution difference of damage features between the target and source domains by using the optimal multicore selection method, and the essential damage features of UGWs were learned. Finally, different damage states of the target domain are effectively identified by feature identification. The experimental results illustrate that the proposed method can realize automatic extraction of inherent damage features and adaptive matching of multilayer features, connect the source and target domains in the high-level feature space, and learn the invariant features of guided-wave signals under different vibrations. Moreover, the proposed method has a good performance both in the mean between-class average distance and the mean within-class average distance damage degree of feature under various vibration conditions, reaches 100% accuracy in damage degree identification under different vibration conditions, and shows better performance than the comparison methods.

Improving the Accuracy of PCOS’ Data Prediction Model Based on Data Balancing and Multilayer Feature Selection Strategies.

Improving the Accuracy of PCOS’ Data Prediction Model Based on Data Balancing and Multilayer Feature Selection Strategies.

Attention-Guided Pyramid Context Network for Polyp Segmentation in Colonoscopy Images

Recently, deep convolutional neural networks (C-NNs) have provided us an effective tool for automated polyp segmentation in colonoscopy images. However, most CNN-based methods do not fully consider the feature interaction among different layers and often cannot provide satisfactory segmentation performance. In this paper, a novel attention-guided pyramid context network (APCNet) is proposed for accurate and robust polyp segmentation in colonoscopy images. Specifically, considering that different network layers represent the polyp in different aspects, APCNet first extracts multi-layer features in a pyramid structure, then utilizes an attention-guided multi-layer aggregation strategy to refine the context features of each layer by utilizing the complementary information of different layers. To obtain abundant context features, APCNet employs a context extraction module that explores the context information of each layer via local information retainment and global information compaction. Through the top-down deep supervision, our APCNet implements a coarse-to-fine polyp segmentation and finally localizes the polyp region precisely. Extensive experiments on two in-domain and four out-of-domain experiments show that APCNet is comparable to 19 state-of-the-art methods. Moreover, it holds a more appropriate trade-off between effectiveness and computational complexity than these competing methods.

M2FL-CCC: Multibranch Multilayer Feature Leaning and Comprehensive Classification Criterion for Gas Sensor Drift Compensation

M²FL-CCC: Multibranch Multilayer Feature Leaning and Comprehensive Classification Criterion for Gas Sensor Drift Compensation

Landslide Mapping Using Multilevel-Feature-Enhancement Change Detection Network

Landslide mapping (LM) from bitemporal remote sensing images is essential for disaster prevention and mitigation. Bitemporal change detection technology has been applied for LM, however, there remains room for improvement in its accuracy and automation. In this paper, a multi-level feature enhancement network (MFENet) is proposed for LM based on modules built in convolutional neural network (CNN) like CNN-Attention. MFENet mainly consists of three modules: post-event feature enhancement module (PFEM), bi-feature difference enhancement module (BFDEM) and flow direction calibration module (FDCM). Specifically, the main role of PFEM selectively fuses post-event multi-layer features to provide discriminative post-event features. BFDEM fuses the multi-layer differences of both pre-event and post-event features to generate high-quality change detection features, sufficiently powerful to distinguish foreground from background. FDCM uses a digital elevation model (DEM) to calibrate the flow direction of each pixel of the landslide detection results to complete the LM task. Experiments were conducted to test the effectiveness of MFENet on two real-world regions, Lantau Island and Sharp Peak, Hong Kong, in which landslides occurred after rainstorms. Compared with other state-of-the-art general change detection methods and landslide-specific change detection methods, the proposed method outperforms on all metrics with its intersection over union (IoU) reaching 87.23%. The availability of additional features and the generalization performance of MFENet are demonstrated experimentally. It is anticipated that the proposed network will further contribute to disaster prevention and mitigation.

Automated polyp segmentation based on a multi-distance feature dissimilarity-guided fully convolutional network.

Colorectal malignancies often arise from adenomatous polyps, which typically begin as solitary, asymptomatic growths before progressing to malignancy. Colonoscopy is widely recognized as a highly efficacious clinical polyp detection method, offering valuable visual data that facilitates precise identification and subsequent removal of these tumors. Nevertheless, accurately segmenting individual polyps poses a considerable difficulty because polyps exhibit intricate and changeable characteristics, including shape, size, color, quantity and growth context during different stages. The presence of similar contextual structures around polyps significantly hampers the performance of commonly used convolutional neural network (CNN)-based automatic detection models to accurately capture valid polyp features, and these large receptive field CNN models often overlook the details of small polyps, which leads to the occurrence of false detections and missed detections. To tackle these challenges, we introduce a novel approach for automatic polyp segmentation, known as the multi-distance feature dissimilarity-guided fully convolutional network. This approach comprises three essential components, i.e., an encoder-decoder, a multi-distance difference (MDD) module and a hybrid loss (HL) module. Specifically, the MDD module primarily employs a multi-layer feature subtraction (MLFS) strategy to propagate features from the encoder to the decoder, which focuses on extracting information differences between neighboring layers' features at short distances, and both short and long-distance feature differences across layers. Drawing inspiration from pyramids, the MDD module effectively acquires discriminative features from neighboring layers or across layers in a continuous manner, which helps to strengthen feature complementary across different layers. The HL module is responsible for supervising the feature maps extracted at each layer of the network to improve prediction accuracy. Our experimental results on four challenge datasets demonstrate that the proposed approach exhibits superior automatic polyp performance in terms of the six evaluation criteria compared to five current state-of-the-art approaches.

Remote Sensing Image Scene Classification via Regional Growth-Based Key Area Fine Location and Multilayer Feature Fusion

Remote sensing image scene classification (RSISC) is important for analyzing and interpreting remote sensing images (RSIs). However, the intraclass difference and interclass similarity problems caused by complex backgrounds and variable scales bring great challenges for the effective classification of RSI scenes. In this letter, we solve the above problem mainly from two aspects. First, a multilayer feature fusion (MLFF) module is proposed to improve the classification performance of the networks by adaptively fusing multilayer semantic information. Then, to learn more discriminative local fine-grained, a regional growth-based key area (RGKA) fine location algorithm is proposed to accurately obtain local key areas. Finally, a two-branch network is utilized to complete the classification. Experiments are conducted on three publicly available datasets. The experimental results show that the proposed method outperforms most state-of-the-art methods and can considerably improve the accuracy of RSISC.

Point Cloud Classification Method Based on Graph Convolution and Multilayer Feature Fusion

Point Cloud Classification Method Based on Graph Convolution and Multilayer Feature Fusion

Investigation of a dual-mode algorithm for joint processing of infrared and visible images based on a Residual Network

A dual-mode algorithm for processing images of infrared and visible light based on the ResNet152 network to implement the process of recognizing the external environment of unmanned vehicles in low light conditions is developed. It is proposed to split the input image into low-frequency and high-frequency parts by using medium filtering and then combine the low-frequency parts. In this case, high-frequency parts are introduced into the ResNet network to obtain maps of multilayer features. The experimental results show that the target detection and recognition technology based on the processing of dual-mode images of infrared and visible light has higher accuracy and reliability than single-mode image detection and recognition. Keywords image processing, infrared imaging, visible light imaging, residual network

Bearing Fault Diagnosis Based on Multisensor Information Coupling and Attentional Feature Fusion

The effective fault diagnosis of bearing can guarantee the safety of rotating machinery and is very important for its stable operation. The information fusion of multi-sensor data has been a feasible method to enhance the performance of fault diagnosis. However, how to fuse the joint information from different channels or even different kinds of sensors is still an important challenge. The present study proposes a novel multi-sensor information coupling network (MICN) for bearing fault diagnosis, which handles the signals from the same or different types of sensors, and the deeper features can be extracted from multi-sensors independently and simultaneously fused layer by layer. Especially, during the multi-layer feature fusion process, a novel feature-level information coupling model is developed based on the mutual attention mechanism. Finally, to validate the efficiency of the proposed method, several different experiments are designed, and the results show validity and superiority.

Consistency and Diversity Induced Human Motion Segmentation.

Subspace clustering is a classical technique that has been widely used for human motion segmentation and other related tasks. However, existing segmentation methods often cluster data without guidance from prior knowledge, resulting in unsatisfactory segmentation results. To this end, we propose a novel Consistency and Diversity induced human Motion Segmentation (CDMS) algorithm. Specifically, our model factorizes the source and target data into distinct multi-layer feature spaces, in which transfer subspace learning is conducted on different layers to capture multi-level information. A multi-mutual consistency learning strategy is carried out to reduce the domain gap between the source and target data. In this way, the domain-specific knowledge and domain-invariant properties can be explored simultaneously. Besides, a novel constraint based on the Hilbert Schmidt Independence Criterion (HSIC) is introduced to ensure the diversity of multi-level subspace representations, which enables the complementarity of multi-level representations to be explored to boost the transfer learning performance. Moreover, to preserve the temporal correlations, an enhanced graph regularizer is imposed on the learned representation coefficients and the multi-level representations of the source data. The proposed model can be efficiently solved using the Alternating Direction Method of Multipliers (ADMM) algorithm. Extensive experimental results on public human motion datasets demonstrate the effectiveness of our method against several state-of-the-art approaches.

A Novel Pressure Relief Hole Recognition Method of Drilling Robot Based on SinGAN and Improved Faster R-CNN

The drilling robot is the key equipment for pressure relief in rockburst mines, and the accurate recognition of a pressure relief hole is the premise for optimizing the layout of pressure relief holes and intelligent drilling. In view of this, a pressure relief hole recognition method for a drilling robot, based on single-image generative adversarial network (SinGAN) and improved faster region convolution neural network (Faster R-CNN), is proposed. Aiming at the problem of insufficient sample generation diversity and poor performance of the traditional SinGAN model, some improvement measures including image size adjustment, multi-stage training, and dynamically changing iteration times are designed as an improved SinGAN for the generation of pressure relief hole images. In addition, to solve the problem that the traditional depth neural network is not ideal for small-size target recognition, an improved Faster R-CNN based on multi-scale image input and multi-layer feature fusion is designed with the improved SqueezeNet as the framework, and the sample data collected from ground experiments are used for comparative analysis. The results indicate that the improved SinGAN model can improve the diversity of generated images on the premise of ensuring the quality of image samples, and can greatly improve the training speed of the model. The accuracy and recall rate of the improved Faster R-CNN model were able to reach 90.09% and 98.32%, respectively, and the average detection time was 0.19 s, which verifies the superiority of the improved Faster R-CNN model. To further verify the practicability of the proposed method, some field images were collected from the underground rockburst relief area in the coal mine, and a corresponding test analysis was carried out. Compared with three YOLO models, the accuracy and recall rate of improved Faster R-CNN model improved significantly, although the training time and recognition time increased to a certain extent, which proves the feasibility and effectiveness of the proposed method.

A Deep Learning Workflow for Mass-Forming Intrahepatic Cholangiocarcinoma and Hepatocellular Carcinoma Classification Based on MRI.

Precise classification of mass-forming intrahepatic cholangiocarcinoma (MF-ICC) and hepatocellular carcinoma (HCC) based on magnetic resonance imaging (MRI) is crucial for personalized treatment strategy. The purpose of the present study was to differentiate MF-ICC from HCC applying a novel deep-learning-based workflow with stronger feature extraction ability and fusion capability to improve the classification performance of deep learning on small datasets. To retain more effective lesion features, we propose a preprocessing method called semi-segmented preprocessing (Semi-SP) to select the region of interest (ROI). Then, the ROIs were sent to the strided feature fusion residual network (SFFNet) for training and classification. The SFFNet model is composed of three parts: the multilayer feature fusion module (MFF) was proposed to extract discriminative features of MF-ICC/HCC and integrate features of different levels; a new stationary residual block (SRB) was proposed to solve the problem of information loss and network instability during training; the attention mechanism convolutional block attention module (CBAM) was adopted in the middle layer of the network to extract the correlation of multi-spatial feature information, so as to filter the irrelevant feature information in pixels. The SFFNet model achieved an overall accuracy of 92.26% and an AUC of 0.9680, with high sensitivity (86.21%) and specificity (94.70%) for MF-ICC. In this paper, we proposed a specifically designed Semi-SP method and SFFNet model to differentiate MF-ICC from HCC. This workflow achieves good MF-ICC/HCC classification performance due to stronger feature extraction and fusion capabilities, which provide complementary information for personalized treatment strategy.

Crowd Density Level Estimation and Anomaly Detection Using Multicolumn Multistage Bilinear Convolution Attention Network (MCMS-BCNN-Attention)

The detection of crowd density levels and anomalies is a hot topic in video surveillance. Especially in human-centric action and activity-based movements. In some respects, the density level variation is considered an anomaly in the event. Crowd behaviour identification relies on a computer-vision-based approach and basically deals with spatial information of foreground video information. In this work, we focused on a deep-learning-based attention-oriented classification system for identifying several basic movements in public places, especially, human flock movement, sudden motion changes and panic events in several indoor and outdoor places. The important spatial features were extracted from a bilinear CNN and a multicolumn multistage CNN with preprocessed morphological video frames from videos. Finally, the abnormal and crowd density estimation was distinguished by using an attention feature combined with a multilayer CNN feature by modifying the fully connected layer for several categories (binary and multiclass). We validate the proposed method on several video surveillance datasets including PETS2009, UMN and UCSD. The proposed method achieved an accuracy of 98.62, 98.95, 96.97, 99.10 and 98.38 on the UCSD Ped1, UCSD Ped2, PETS2009, UMN Plaza1 and UMN Plaza2 datasets, respectively, with the different pretrained models. We compared the performance between recent modern approaches and the proposed method (MCMS-BCNN-Attention) and achieved the highest accuracy. The anomaly detection performance on the UMN and PETS2009 datasets was compared with that of a state-of-the-art method and achieved the best AUC results as 0.9953 and 1.00 for both scenarios, respectively, with a binary classification.