Features Of Different Scales Research Articles

RelationRS: Relationship Representation Network for Object Detection in Aerial Images

Object detection is a basic and important task in the field of aerial image processing and has gained much attention in computer vision. However, previous aerial image object-detection approaches have insufficient use of scene semantic information between different regions of large-scale aerial images. In addition, complex background and scale changes make it difficult to improve detection accuracy. To address these issues, we propose a relationship representation network for object detection in aerial images (RelationRS): (1) Firstly, multi-scale features are fused and enhanced by a dual relationship module (DRM) with conditional convolution. The dual relationship module learns the potential relationship between features of different scales and learns the relationship between different scenes from different patches in a same iteration. In addition, the dual relationship module dynamically generates parameters to guide the fusion of multi-scale features. (2) Secondly, the bridging visual representations module (BVR) is introduced into the field of aerial images to improve the object detection effect in images with complex backgrounds. Experiments with a publicly available object detection dataset for aerial images demonstrate that the proposed RelationRS achieves a state-of-the-art detection performance.

Recognition of abnormal car door noise based on multi-scale feature fusion

To accurately identify the abnormal door-closing noise, we propose a method to recognize the time-frequency image of door closing sound based on a multi-scale feature fusion network model. The door-closing sound signal is transformed into a time-frequency image through wavelet analysis, and a classification model based on multi-scale feature fusion is designed. The model introduces multi-scale filters and dilated convolution and adds two improved inception modules to keep the model lightweight. At the same time, richer spatial features can be obtained. The features of different scales are spliced and input to the fully connected layer, and a dropout layer is added to the fully connected layer to suppress overfitting. By comparing the loss and accuracy rate, adjusting different hyperparameters, the optimal model is obtained. The experimental results show that the multi-scale feature fusion network model has a higher accuracy rate than the transfer learning model. Test accuracy rate is 86% and can effectively recognize abnormal door-closing noise. It provides a feasible theoretical basis for the direction of abnormal door noise recognition.

Deep Compressed Sensing-Based Cascaded Channel Estimation for RIS-Aided Communication Systems

To reduce the pilot overhead of cascaded channel estimation for RIS-aided Massive MIMO communication system, we proposed a deep compressed sensing-based channel estimation scheme, where U-shaped network (U-Net), an encoder-decoder with skip connection, is used to recover the high-dimensional cascaded channel matrix from limited pilot overhead. The skip connections between encoder and decoder can fuse features of different scales and semantic by concatenating the feature map, which enhance the reconstruction performance of cascaded channel. To further improve the feature extraction ability of U-Net, we design a ResU-Net architecture with stacked residual units to increase the depth of network. Simulation results show the channel estimation of ResU-Net is more accurate than conventional algorithm and other network model. Meanwhile, ResU-Net has good generalization and robustness for different pilot lengths and phase quantization errors.

Closed-Loop Residual Attention Network for Single Image Super-Resolution

Recent research on single image super-resolution (SISR) using convolutional neural networks (CNNs) with the utilization of residual structures and attention mechanisms to utilize image features has demonstrated excellent performance. However, previous SISR techniques mainly integrated extracted image features within a deep or wide network architecture, ignoring the interaction between multiscale features and the diversity of features. At the same time, SISR is also a typical ill-posed problem in that it allows for several predictions for a given LR image. These problems limit the great learning ability of CNNs. To solve these problems, we propose a closed-loop residual attention network (CLRAN) to extract and interact with all the available diversity of features features efficiently and limit the space of possible function solutions. Specifically, we design an enhanced residual attention block (ERA) to extract features, and it dynamically assigns weight to the internal attention branches. The ERA combines multi-scale block (MSB) and enhanced attention mechanism (EAM) base on the residual module. The MSB adaptively detects multiscale image features of different scales by using different 3 × 3 convolution kernels. The EAM combines multi-spectral channel attention (MSCA) and spatial attention (SA). Therefore, the EAM extracts different frequency component information and spatial information to utilize the diversity features. Furthermore, we apply the progressive network architecture and learn an additional map for model monitoring, which forms a closed-loop with the mapping already learned by the LR to HR function. Extensive experiments demonstrate that our CLRAN outperforms the state-of-the-art SISR methods on public datasets for both ×4 and ×8, proving its accuracy and visual perception.

End-to-end residual attention mechanism for cataractous retinal image dehazing

Background and objectiveCataract is one of the most common causes of vision loss. Light scattering due to clouding of the lens in cataract patients makes it extremely difficult to image the retina of cataract patients with fundus cameras, resulting in a serious decrease in the quality of the retinal images taken. Furthermore, the age of cataract patients is generally too old, in addition to cataracts, the patients often have other retinal diseases, which brings great challenges to experts in the clinical diagnosis of cataract patients using retinal imaging. MethodsIn this paper, we present the End-to-End Residual Attention Mechanism (ERAN) for Cataractous Retinal Image Dehazing, which it includes four modules: encoding module, multi-scale feature extraction module, feature fusion module, and decoding module. The encoding module encodes the input cataract haze image into an image, facilitating subsequent feature extraction and reducing memory usage. The multi-scale feature extraction module includes a hole convolution module, a residual block, and an adaptive skip connection, which can expand the receptive field and extract features of different scales through weighted screening for fusion. The feature fusion module uses adaptive skip connections to enhance the network's ability to extract haze density images to make haze removal more thorough. Furthermore, the decoding module performs non-linear mapping on the fused features to obtain the haze density image, and then restores the haze-free image. ResultsThe experimental results show that the proposed method has achieved better objective and subjective evaluation results, and has a better dehazing effect. ConclusionWe proposed ERAN method not only provides visually better images, but also helps experts better diagnose other retinal diseases in cataract patients, leading to better care and treatment.

Kidney Tumor Segmentation Based on FR2PAttU-Net Model.

The incidence rate of kidney tumors increases year by year, especially for some incidental small tumors. It is challenging for doctors to segment kidney tumors from kidney CT images. Therefore, this paper proposes a deep learning model based on FR2PAttU-Net to help doctors process many CT images quickly and efficiently and save medical resources. FR2PAttU-Net is not a new CNN structure but focuses on improving the segmentation effect of kidney tumors, even when the kidney tumors are not clear. Firstly, we use the R2Att network in the “U” structure of the original U-Net, add parallel convolution, and construct FR2PAttU-Net model, to increase the width of the model, improve the adaptability of the model to the features of different scales of the image, and avoid the failure of network deepening to learn valuable features. Then, we use the fuzzy set enhancement algorithm to enhance the input image and construct the FR2PAttU-Net model to make the image obtain more prominent features to adapt to the model. Finally, we used the KiTS19 data set and took the size of the kidney tumor as the category judgment standard to enhance the small sample data set to balance the sample data set. We tested the segmentation effect of the model at different convolution and depths, and we got scored a 0.948 kidney Dice and a 0.911 tumor Dice results in a 0.930 composite score, showing a good segmentation effect.

Research on 3D Point Cloud Object Detection Algorithm for Autonomous Driving

In autonomous driving, lidar has become the main vehicle sensor due to its advantages such as long-range measurement and high accuracy. However, the collected point cloud data is sparse and unevenly distributed, and it lacks characterization capabilities when facing objects with missing or similar shapes, so that the detection accuracy is low while detecting long-distance small targets with similar shapes and a small number of point clouds. In order to improve the detection accuracy of small targets represented by point clouds, this paper adopts a method that fuses point cloud and RGB image to construct a 3D object detection network architecture based on two-stage complementary fusion. In the first stage of the fusion, we use the FPS method to select some points from the raw point cloud data as key points. Then, we voxelize the raw point cloud and use the 3D sparse convolutional neural network to extract multi-scale points cloud features, which would fuse features of different scales with key points. In the second stage of fusion, a 2D object detector is used to obtain the 2D bounding box and category information of the target in the image and take the camera as the origin to extend along the direction of the 2D bounding box to form a frustum; then the point cloud and target category information within the frustum are fused into the key points. This paper uses key points as a bridge to effectively combine the semantic information of the image such as texture and color with the point cloud features. The experimental results show that the network proposed in this paper has excellent performance on small objects.

Semantic Segmentation of High-Resolution Remote Sensing Images Using Multiscale Skip Connection Network

Semantic segmentation of remote sensing images plays a vital role in land resource management, yield estimation, and economic evaluation. Therefore, this paper proposes a multi-scale skip connection network with the Atrous convolution to deal with the segmentation problems of the multi-modal and multi-scale high-resolution remote sensing images. Firstly, we applied the Atrous convolution in the encoder to enlarge the convolution kernel’s receptive field. Secondly, based on the U-Net network, we merged the light and deep features of different scales by redesigning the skip connection and combining multi-scale features in each U-Net layer. Finally, we applied a pixel-by-pixel classification method and obtained the semantic segmentation results of remote sensing images. The effectiveness of the proposed algorithm is verified. The experimental results show that the mF1 scores are 89.4% and 90.3% on the open dataset of ISPRS Vaihingen and ISPRS Potsdam, respectively, which are better than the state-of-the-art algorithms.

Multi-Discriminator Adversarial Convolutional Network for Nerve Fiber Segmentation in Confocal Corneal Microscopy Images.

Quantitative measurements of corneal sub-basal nerves are biomarkers for many ocular surface disorders and are also important for early diagnosis and assessment of progression of neurodegenerative diseases. This paper aims to develop an automatic method for nerve fiber segmentation from in vivo corneal confocal microscopy (CCM) images, which is fundamental for nerve morphology quantification. A novel multi-discriminator adversarial convolutional network (MDACN) is proposed, where both the generator and the two discriminators emphasize multi-scale feature representations. The generator is a U-shaped fully convolutional network with multi-scale split and concatenate blocks, and the two discriminators have different effective receptive fields, sensitive to features of different scales. A novel loss function is also proposed which enables the network to pay more attention to thin fibers. The MDACN framework was evaluated on four datasets. Experiment results show that our method has excellent segmentation performance for corneal nerve fibers and outperforms some state-of-the-art methods.

Seismic random noise suppression by using deep residual U-Net

Pre-stack seismic denoising is one of most important processing steps in seismic exploration, which can significantly enhance the signal-to-noise ratio (SNR) and resolution of real seismic data. Seismic random noise often presents some complicated characteristics (non-gaussian, non-stationary, non-linear) and may overlap with desired signals in frequency domain. Thus, we need to explore the corresponding approach to attenuate the unwanted random noise and simultaneously recover the signals. Recently, some methods based on convolutional neural network (CNN) have shown excellent performance in seismic data denoising. These CNN-based denoising methods can learn the potential features of labeled data, thereby establishing the mapping relationship between noisy seismic data and signals. However, these existing CNN-based methods only consider the single-scale potential features and neglect some useful multiscale features, leading to their performance degradation when processing some seismic data with low SNR. In view of above drawback, we propose a novel CNN by introducing the residual theory and reconstruction block into the conventional U-Net, called deep residual U-Net (DRUN). In DRUN, some residual blocks are utilized to obtain the multiscale features of noisy seismic data and the followed reconstruction block can incorporate features of different scales, thereby distinguishing signals and random noise. Both the synthetic and field seismic data are processed to verify the effectiveness of DRUN. Compared with the original U-Net, the experimental results demonstrate that the proposed DRUN has superiority in noise attenuation and signal preservation.

Hyperspectral image classification based on octave convolution and multi-scale feature fusion

Hyperspectral Image (HSI) classification is one of the main research directions of remote sensing applications. With the high dimension, strong correlation, and a large amount of HSI data, conventional classification methods often have problems like computation complexity, low classification accuracy, and poor generalization ability while extracting remote sensing image features. Moreover, the extraction of efficient features will impact the classification results. This work proposed an HSI classification method based on octave convolution and multi-scale feature fusion. The low-level features of HSI were extracted by octave convolution, and the attention mechanism was introduced into the spatial and spectral dimensions to focus on the area of interest. Then, the spectral-spatial fusion features were extracted for the classification task. Based on 300 training samples, several experiments were carried out, and the results showed that the classification accuracy of the proposed method was 99.63%, 97.90%, and 98.69% on the widely used WHU-Hi dataset. The notable observation was that fusion of features of different scales helped improve the classification performance.

Face photo-sketch synthesis based on multi-scale feature fusion

The synthesis of face sketches from real face photos and its inverse process have a wide range of applications, such as digital entertainment and assisting in the investigation of criminal cases. However, due to the significant discrepancies in texture between photos and sketches, the transformation between them is still a challenging problem. Recently, the methods based on generative adversarial nets (GANs) have shown encouraging results in the transformation between images, especially the transformation from photos to sketches. However, most of them produce different deformations or blur in the key components of the face, which affects the realism of the synthetic image. To address this challenge, we propose a novel face photo-sketch synthesis algorithm based on multi-scale feature fusion to improve the structural integrity and texture fidelity of the synthetic image. Firstly, an encoder is utilized to extract the multi-scale coding features of the input image. Then the bottom coding features are passed by a dilated convolution module and transmitted to the decoder for decoding. In the decoding process, the decoding features of different scales and the coding features of corresponding scales are concatenated in the channel dimension to obtain the multi-scale coding and decoding fusion features. Finally, the coding and decoding fusion features of different scales are further fused at the output of the decoder and the final synthesis result is generated through a convolution layer. By combining the features of different scales in the encoding and decoding processes in the channel dimension at the same time, we can maintain good image structure and texture details and generate realistic face sketch/photo images. We verify the effectiveness of the proposed method in several challenging datasets. The quantitative and qualitative evaluation shows that the proposed method is superior to other state-of-the-art methods in generating face sketches (or photos) with high visual quality.

COVID-19 CT image recognition algorithm based on transformer and CNN

Novel corona virus pneumonia (COVID-19) broke out in 2019, which had a great impact on the development of world economy and people's lives. As a new mainstream image processing method, deep learning network has been constructed to extract medical features from chest CT images, and has been used as a new detection method in clinical practice. However, due to the medical characteristics of COVID-19 CT images, the lesions are widely distributed and have many local features. Therefore, it is difficult to diagnose directly by using the existing deep learning model. According to the medical features of CT images in COVID-19, a parallel bi-branch model (Trans-CNN Net) based on Transformer module and Convolutional Neural Network module is proposed by making full use of the local feature extraction capability of Convolutional Neural Network and the global feature extraction advantage of Transformer. According to the principle of cross-fusion, a bi-directional feature fusion structure is designed, in which features extracted from two branches are fused bi-directionally, and the parallel structures of branches are fused by a feature fusion module, forming a model that can extract features of different scales. To verify the effect of network classification, the classification accuracy on COVIDx-CT dataset is 96.7%, which is obviously higher than that of typical CNN network (ResNet-152) (95.2%) and Transformer network (Deit-B) (75.8%). These results demonstrate accuracy is improved. This model also provides a new method for the diagnosis of COVID-19, and through the combination of deep learning and medical imaging, it promotes the development of real-time diagnosis of lung diseases caused by COVID-19 infection, which is helpful for reliable and rapid diagnosis, thus saving precious lives.

Multichannel Two-Dimensional Convolutional Neural Network Based on Interactive Features and Group Strategy for Chinese Sentiment Analysis.

In Chinese sentiment analysis tasks, many existing methods tend to use recurrent neural networks (e.g., long short-term memory networks and gated recurrent units) and standard one-dimensional convolutional neural networks (1D-CNN) to extract features. This is because a recurrent neural network can deal with the order dependence of the data to a certain extent and the one-dimensional convolution can extract local features. Although these methods have good performance in sentiment analysis tasks, recurrent neural networks (RNNs) cannot be parallelized, resulting in time-inefficiency, and the standard 1D-CNN can only extract a single sample feature, with the result that the feature information cannot be fully utilized. To this end, in this paper, we propose a multichannel two-dimensional convolutional neural network based on interactive features and group strategy (MCNN-IFGS) for Chinese sentiment analysis. Firstly, we no longer use word encoding technology but use character-based integer encoding to retain more fine-grained information. Besides, in character-level vectors, the interactive features of different elements are introduced to improve the dimensionality of feature vectors and supplement semantic information so that the input matches the model network. In order to ensure that more sentiment features are learned, group strategies are used to form several feature mapping groups, so the learning object is converted from the traditional single sample to the learning of the feature mapping group, so as to achieve the purpose of learning more features. Finally, multichannel two-dimensional convolutional neural networks with different sizes of convolution kernels are used to extract sentiment features of different scales. The experimental results on the Chinese dataset show that our proposed method outperforms other baseline and state-of-the-art methods.

Sonar Image Target Detection Based on Adaptive Global Feature Enhancement Network

Automatic underwater target detection plays a vital role in sonar image processing and analysis, and its core task is to discriminate target categories and achieve precise positioning. However, the sonar image is interfered by the seafloor reverberation noise and complex background, which brings more significant challenges to the accurate detection of sonar target. To achieve accurate detection of different categories targets in sonar image, we proposed an adaptive global feature enhancement network (AGFE-Net), which uses multi-scale convolution and attention mechanisms with global receptive field to obtain sonar image multi-scale semantic feature and enhance the correlation between features. Specifically, we use the multi-scale receptive field feature extraction block (MSFF-Block) and the self-attention mechanism block (SAM-Block) to enhance model feature extraction ability; the bidirectional feature pyramid network (Bi-FPN) and the global pyramid pooling block (GPP-Block) are used to obtain the deep semantic feature and suppress background noise interference; the adaptive feature fusion block (AFF-Block) is used to effectively fuse features of different scales. Experimental results on the presented sonar target detection dataset WH-Dataset and QD-Dataset validate the advantage of AGFE-Net over other state-of-the-art target detection methods.

MSFusion: Multistage for Remote Sensing Image Spatiotemporal Fusion Based on Texture Transformer and Convolutional Neural Network

Due to the limitations of current technology and budget, a single satellite sensor can not obtain high spatiotempo- ral resolution remote sensing images. Therefore, remote sensing image spatio-temporal fusion technology is considered as an effective solution and has attracted extensive attention. In the field of deep learning, due to the fixed size of the perception field of convolutional neural network, it is impossible to model the correlation of global features, and the features extracted only through convolution operation lack the ability to capture long- distance features, At the same time, complex fusion methods can not better integrate temporal and spatial features. In order to solve these problems, we propose a multi-stage remote sensing image spatio-temporal fusion model based on Texture Trans- former and convolutional neural network. The model combines the advantages of Transformer and convolutional network, uses a lightweight convolution network to extract spatial features and temporal discrepancy features, uses Transformer to learn global temporal correlation, and finally fuses temporal features with spatial features. In order to make full use of the features obtained in different stages, we design a cross-stage adaptive fusion module CSAFM. The module adopts the self attention mechanism to adaptively integrate the features of different scales while considering the temporal and spatial characteristics. To test the robustness of the model, the experiments are carried out on three datasets of CIA, LGC and DX. Compared with five typical spatio-temporal fusion algorithms, we obtain excellent results, which prove the superiority of MSFusion model.

Multiscale Alternately Updated Clique Network for Hyperspectral Image Classification

Recently, deep learning has drawn significant attention in hyperspectral image (HSI) classification. With the growth of network depth and feature integration, deep learning demands abundant labeled samples to optimize many parameters. Unfortunately, most hyperspectral data are unlabeled and the available labeled samples are extremely limited. How to obtain richer features under limited training samples is a challenge for HSI classification. To tackle this issue, a new supervised multiscale alternately updated clique network (MSCN) is proposed for HSI classification to fully employ HSI features in different scales. Based on the Clique Block, we design the multiscale alternately updated clique block (MSCB) that applies convolution kernels of various sizes to adaptively exploit the multiscale HSI information and merge them within the block. Meanwhile, the recurrent feedback architecture is introduced to reuse high-level visual information and network parameters. The proposed MSCN includes two MSCBs to capture the multiscale spectral and spatial information in turn. The MSCN improves the information flow and the efficiency of parameter tuning through the feedback mechanism and the cross-utilization of multiscale feature. It not only obtains more abstract HSI information, but also reduces the network depth and the number of parameters, thereby improving the classification accuracy under limited samples. To certify the validity of the proposed MSCN, experiments are conducted on three real HSI datasets and compared with multiple state-of-the-art deep learning-based approaches. The experimental results demonstrate that the presented multiscale network achieves superior performance, especially in the case of a small number of training samples.

Multi-scale attention and deep supervision-based 3D UNet for automatic liver segmentation from CT.

Automatic liver segmentation is a prerequisite for hepatoma treatment; however, the low accuracy and stability hinder its clinical application. To alleviate this limitation, we deeply mine the context information of different scales and combine it with deep supervision to improve the accuracy of liver segmentation in this paper. We proposed a new network called MAD-UNet for automatic liver segmentation from CT. It is grounded in the 3D UNet and leverages multi-scale attention and deep supervision mechanisms. In the encoder, the downsampling pooling in 3D UNet is replaced by convolution to alleviate the loss of feature information. Meanwhile, the residual module is introduced to avoid gradient vanishment. Besides, we use the long-short skip connections (LSSC) to replace the ordinary skip connections to preserve more edge detail. In the decoder, the features of different scales are aggregated, and the attention module is employed to capture the spatial context information. Moreover, we utilized the deep supervision mechanism to improve the learning ability on deep and shallow information. We evaluated the proposed method on three public datasets, including, LiTS17, SLiver07, and 3DIRCADb, and obtained Dice scores of 0.9727, 0.9752, and 0.9691 for liver segmentation, respectively, which outperform the other state-of-the-art (SOTA) methods. Both qualitative and quantitative experimental results demonstrate that the proposed method can make full use of the feature information of different stages while enhancing spatial data's learning ability, thereby achieving high liver segmentation accuracy. Thus, it proved to be a promising tool for automatic liver segmentation in clinical assistance.

MSFSA-GAN: Multi-Scale Fusion Self Attention Generative Adversarial Network for Single Image Deraining

Bad weather such as rainy days will seriously affect the image quality and the accuracy of visual processing algorithm. In order to improve the image deraining quality, a multi-scale fusion self attention generation adversarial network (MSFSA-GAN) is proposed. This network uses different scales to extract input characteristics of rain lines. First, Gaussian pyramid rain maps with different scales are generated by Gaussian algorithm. Then, in order to extract the features of rain lines with different scales, the coarse fusion module and fine fusion module are designed respectively. Next, the extracted features are fused at different scales. In this process, the self attention mechanism is introduced to make the network focus on the extracted features of different scales. And before the fusion, the rain pattern reconstruction operation is also carried out, so that the network can reproduce the input image more perfectly. Finally, it is input into the discriminator network with dense blocks to obtain the image that removes the rain lines. We used R100H and R100L datasets to train and test our network. The results show that our method as high as 27.79 in PSNR and UQI is 0.94, which is superior to the existing methods in performance. Meanwhile, we also compared the cost of time, the result of our network is only 0.02s.

AMFAN: Adaptive Multiscale Feature Attention Network for Hyperspectral Image Classification

Recently, convolutional neural networks (CNNs) have been widely used in hyperspectral image (HSI) classification with appreciable performance. However, the current CNN-based HSI classification methods have limitations in exploiting the multiscale features and extracting sufficiently discriminative features, and usually adopted dimensionality reduction method such as PCA leads to some or all of the physical information of the original band may be lost. To address the above problems, in this letter, we propose an adaptive multiscale feature attention network (AMFAN) for HSI classification. First, we use a band selection algorithm to perform data dimensionality reduction, which helps maintain the original characteristics of the image. Second, different from existing multiscale feature extraction methods that give features of different scales the same degree of importance, we propose an adaptive multiscale feature residual module (AMFRM) to give multiscale features different importance. Finally, due to the input of the HSI classification model based on deep learning being the patch cube, the only available initial information is the category of the center pixel. However, the patch often contains pixels different from the center pixel category, and existing attention mechanisms do not consider the impact of such pixels on HSI classification, so we design a novel position attention module (PAM) to calculate the similarity between the center (target) pixel and surrounding pixels, and then pay more attention to the pixels with high similarity to the center pixel. Besides, we also use a spectral attention module (SAM) to obtain more discriminative spectral features. Experimental results show that the proposed AMFAN effectively improves the classification accuracy, and outperforms the state-of-the-art CNNs.