Multi-scale Receptive Fields Research Articles

DSMFFNet: Depthwise separable multiscale feature fusion network for bridge detection in very high resolution satellite images

Bridge detection methods based on deep learning have many parameters, complex calculations, and serious errors and missed detections for multiscale bridges. To solve the above problems, a depth-wise separable multiscale feature fusion network (DSMFFNet) is proposed for efficient and accurate bridge detection in very high resolution satellite images (VHR). First, depth-wise separable convolution was used to build a backbone feature extraction network to extract the bridge features. Second, to better match bridges of different scales, multiscale receptive fields were obtained by multibranch parallel dilated convolution at the last layer of the feature map. Then, to make full use of the details and semantic information of the bridges at different depths, the three effective feature layers of the bridges at different levels are fused by a multiscale feature pyramid. The experimental results showed that the mean average precision (mAP) and frame per second (FPS) of the proposed method reach 94.26% and 60.04%, which can lead most of the mainstream object detection networks in accuracy and speed and can be integrated into the mobile end to complete the task of high-precision and fast bridge detection.

A dual-branch hybrid dilated CNN model for the AI-assisted segmentation of meningiomas in MR images

Background and objective:Treatment for meningiomas usually includes surgical removal, radiation therapy, and chemotherapy. Accurate segmentation of tumors significantly facilitates complete surgical resection and precise radiotherapy, thereby improving patient survival. In this paper, a deep learning model is constructed for magnetic resonance T1-weighted Contrast Enhancement (T1CE) images to develop an automatic processing scheme for accurate tumor segmentation. Methods:In this paper, a novel Convolutional Neural Network (CNN) model is proposed for the accurate meningioma segmentation in MR images. It can extract fused features in multi-scale receptive fields of the same feature map based on MR image characteristics of meningiomas. The attention mechanism is added as a helpful addition to the model to optimize the feature information transmission. Results and conclusions:The results were evaluated on two internal testing sets and one external testing set. Mean Dice Similarity Coefficient (DSC) values of 0.886, 0.851, and 0.874 are demonstrated, respectively. In this paper, a deep learning approach is proposed to segment tumors in T1CE images. Multi-center testing sets validated the effectiveness and generalization of the method. The proposed model demonstrates state-of-the-art tumor segmentation performance.

AIE-YOLO: Auxiliary Information Enhanced YOLO for Small Object Detection.

Small object detection is one of the key challenges in the current computer vision field due to the low amount of information carried and the information loss caused by feature extraction. You Only Look Once v5 (YOLOv5) adopts the Path Aggregation Network to alleviate the problem of information loss, but it cannot restore the information that has been lost. To this end, an auxiliary information-enhanced YOLO is proposed to improve the sensitivity and detection performance of YOLOv5 to small objects. Firstly, a context enhancement module containing a receptive field size of 21×21 is proposed, which captures the global and local information of the image by fusing multi-scale receptive fields, and introduces an attention branch to enhance the expressive ability of key features and suppress background noise. To further enhance the feature expression ability of small objects, we introduce the high- and low-frequency information decomposed by wavelet transform into PANet to participate in multi-scale feature fusion, so as to solve the problem that the features of small objects gradually disappear after multiple downsampling and pooling operations. Experiments on the challenging dataset Tsinghua-Tencent 100 K show that the mean average precision of the proposed model is 9.5% higher than that of the original YOLOv5 while maintaining the real-time speed, which is better than the mainstream object detection models.

Position Detection of Doors and Windows Based on DSPP-YOLO

Autonomous exploration of autonomous mobile robots in unknown environments is a hot topic at present. Object detection is an important research direction in improving the autonomous capability of autonomous mobile robots in unknown environments. In object detection, doors and windows have many similar features and are difficult to distinguish. Therefore, improving the detection accuracy of doors and windows is helpful to improve the autonomous ability of autonomous mobile robots. Aiming at the problem of insufficient doors and windows detection accuracy caused by the large difference between the receptive fields of doors and windows, this paper proposes DSPP-YOLO (DenseNet SPP) algorithm. Firstly, on the basis of deepening the network addition, to prevent the loss of shallow location feature information, some residual blocks in YOLOV3 are improved to dense blocks by using the idea of DenseNet. Secondly, the spatial pyramid pooling (SPP) structure is fused into the YOLOV3 feature extraction network to realize multiscale receptive field fusion. Finally, K-means ++ algorithm is used to re-cluster the size of candidate boxes to reduce the error caused by candidate boxes. DSPP-YOLO realizes the position detection of doors and windows by an autonomous robot in an unknown, complex environment. This method is tested. Under the condition of the same data set, the detection accuracy of the DSPP-YOLO algorithm is 77.4% for doors and 38.1% for windows. Compared with YOLOV3 algorithm, the calculation consumption time of the DSPP-YOLO algorithm does not increase, and the detection accuracy of doors is improved by 3.3%, the detection accuracy of windows is improved by 8.8%, and the average accuracy of various types is improved by 6.05%.

A lightweight convolutional neural network model with receptive field block for C-shaped root canal detection in mandibular second molars

Rapid and accurate detection of a C-shaped root canal on mandibular second molars can assist dentists in diagnosis and treatment. Oral panoramic radiography is one of the most effective methods of determining the root canal of teeth. There are already some traditional methods based on deep learning to learn the characteristics of C-shaped root canal tooth images. However, previous studies have shown that the accuracy of detecting the C-shaped root canal still needs to be improved. And it is not suitable for implementing these network structures with limited hardware resources. In this paper, a new lightweight convolutional neural network is designed, which combined with receptive field block (RFB) for optimizing feature extraction. In order to optimize the hardware resource requirements of the model, a lightweight, multi-branch, convolutional neural network model was developed in this study. To improve the feature extraction ability of the model for C-shaped root canal tooth images, RFB has been merged with this model. RFB has achieved excellent results in target detection and classification. In the multiscale receptive field block, some small convolution kernels are used to replace the large convolution kernels, which allows the model to extract detailed features and reduce the computational complexity. Finally, the accuracy and area under receiver operating characteristics curve (AUC) values of C-shaped root canals on the image data of our mandibular second molars were 0.9838 and 0.996, respectively. The results show that the deep learning model proposed in this paper is more accurate and has lower computational complexity than many other similar studies. In addition, score-weighted class activation maps (Score-CAM) were generated to localize the internal structure that contributed to the predictions.

MSNet-4mC: learning effective multi-scale representations for identifying DNA N4-methylcytosine sites.

N4-methylcytosine (4mC) is an essential kind of epigenetic modification that regulates a wide range of biological processes. However, experimental methods for detecting 4mC sites are time-consuming and labor-intensive. As an alternative, computational methods that are capable of automatically identifying 4mC with data analysis techniques become a reasonable option. A major challenge is how to develop effective methods to fully exploit the complex interactions within the DNA sequences to improve the predictive capability. In this work, we propose MSNet-4mC, a lightweight neural network building upon convolutional operations with multi-scale receptive fields to perceive cross-element relationships over both short and long ranges of given DNA sequences. With strong imbalances in the number of candidates in different species in mind, we compute and apply class weights in the cross-entropy loss to balance the training process. Extensive benchmarking experiments show that our method achieves a significant performance improvement and outperforms other state-of-the-art methods. The source code and models are freely available for download at https://github.com/LIU-CT/MSNet-4mC, implemented in Python and supported on Linux and Windows. Supplementary data are available at Bioinformatics online.

Remote sensing aircraft object detection algorithm based on memory CenterNet

In order to solve the problem of poor detection accuracy of CenterNet in remote sensing aircraft with dense targets and complex backgrounds, an improved memory CenterNet object detection algorithm is proposed. First, the multiscale receptive field module and residual attention module are introduced to improve ResNet50, which is used as a new backbone feature extraction network. It has a strong feature extraction ability and detection speed at the same time. Different dilated ratios are used in the multiscale receptive field to expand the receptive field and extract multiscale features, and the multiscale target features are highlighted through the attention mechanism. Then the last set of residual networks in ResNet50 is deleted, and the residual attention network is introduced and stacked in the same way. The feature parameters of different channels in the residual attention network are calculated by two-dimensional cosine transform and weighted with the original feature map, and the target features are highlighted to enhance the ability to locate the target at the end of the network. Second, three times deconvolution is replaced by a memory feature fusion module. The shallow detail information and the deep semantic information are fully fused. The key information in the output feature map is further enriched, which is beneficial to the improvement of model detection performance. Finally, the experimental results show that the average accuracy of remote sensing aircraft dataset is 88.46%, which is 18.21% higher than that of the original algorithm and effectively improves the detection accuracy of remote sensing aircraft targets.

GM-TCNet: Gated Multi-scale Temporal Convolutional Network using Emotion Causality for Speech Emotion Recognition

In human-computer interaction, Speech Emotion Recognition (SER) plays an essential role in understanding the user's intent and improving the interactive experience. While similar sentimental speeches own diverse speaker characteristics but share common antecedents and consequences, an essential challenge for SER is how to produce robust and discriminative representations through causality between speech emotions. In this paper, we propose a Gated Multi-scale Temporal Convolutional Network (GM-TCNet) to construct a novel emotional causality repre- sentation learning component with a multi-scale receptive field. GM-TCNet deploys a novel emotional causality representation learning component to capture the dynamics of emotion across the time domain, constructed with dilated causal convolutions layer and gating mechanism. Besides, it utilizes skip connection fusing high-level fea- tures from different Gated Convolution Blocks (GCB) to capture abundant and subtle emotion changes in human speech. GM-TCNet first uses a single type of feature, Mel-Frequency Cepstral Coefficients (MFCC), as inputs and then passes them through the Gated Temporal Convolutional Module (GTCM) to generate the high-level fea- tures. Finally, the features are fed to the emotion classifier to accomplish the SER task. The experimental results show that our model maintains the highest performance in most cases, with +0.90% to +18.50% and +0.55% to +20.15% average relative improvement on the weighted average recall and unweighted average recall compared to state-of-the-art techniques. The source code is available at: https://github.com/Jiaxin-Ye/GM-TCNet for SER.

Learning Balance Feature for Object Detection

In the field of studying scale variation, the Feature Pyramid Network (FPN) replaces the image pyramid and has become one of the most popular object detection methods for detecting multi-scale objects. State-of-the-art methods have FPN inserted into a pipeline between the backbone and the detection head to enable shallow features with more semantic information. However, FPN is insufficient for object detection on various scales, especially for small-scale object detection. One of the reasons is that the features are extracted at different network depths, which introduces gaps between features. That is, as the network becomes deeper and deeper, the high-level features have more semantics but less content description. This paper proposes a new method that includes a multi-scale receptive fields extraction module, a feature constructor module, and an attention module to improve the detection efficiency of FPN for objects of various scales and to bridge the gap in content description and semantics between different layers. Together, these three modules make the detector capable of selecting the most suitable feature for objects. Especially for the attention module, this paper chooses to use a parallel structure to simultaneously extract channel and spatial attention from the same features. When we use Adopting Adaptive Training Sample Selection (ATSS) and FreeAnchor as the baseline and ResNet50 as the backbone, the experimental results on the MS COCO dataset show that our algorithm can enhance the mean average precision (mAP) by 3.7% and 2.4% compared to FPN, respectively.

Recognition of fatigue and severe sliding wear particles using a CNN model with multi-scale feature extractor

PurposeThe purpose of this study is to provide a new convolutional neural network (CNN) model with multi-scale feature extractor to segment and recognize wear particles in complex ferrograph images, especially fatigue and severe sliding wear particles, which are similar in morphology while different in wear mechanism.Design/methodology/approachA CNN model named DWear is proposed to semantically segment fatigue, severe sliding particles and four other types of particles, that is, chain, spherical, cutting and oxide particles, which unifies segmentation and recognition together. DWear is constructed using four modules, namely, encoder, densely connected atrous spatial pyramid pooling, decoder and fully connected conditional random field. Different from the architectures of ordinary semantic segmentation CNN models, a multi-scale feature extractor using cascade connections and a coprime atrous rate group is incorporated into the DWear model to obtain multi-scale receptive fields and better extract features of wear particles. Moreover, fully connected conditional random field module is adopted for post-processing to smooth coarse prediction and obtain finer results.FindingsDWear is trained and verified on the ferrograph image data set, and experimental results show that the final Mean Pixel Accuracy is 95.6% and the Mean Intersection over Union is 92.2%, which means that the recognition and segmentation accuracy is higher than those of previous works.Originality/valueDWear provides a promising approach for wear particle analysis and can be further developed in equipment condition monitoring applications.

Stacked dilated convolutions and asymmetric architecture for U-Net-based medical image segmentation

Deep learning has been widely utilized for medical image segmentation. The most commonly used U-Net and its variants often share two common characteristics but lack solid evidence for the effectiveness. First, each block (i.e., consecutive convolutions of feature maps of the same resolution) outputs feature maps from the last convolution, limiting the variety of the receptive fields. Second, the network has a symmetric structure where the encoder and the decoder paths have similar numbers of channels. We explored two novel revisions: a stacked dilated operation that outputs feature maps from multi-scale receptive fields to replace the consecutive convolutions; an asymmetric architecture with fewer channels in the decoder path. Two novel models were developed: U-Net using the stacked dilated operation (SDU-Net) and asymmetric SDU-Net (ASDU-Net). We used both publicly available and private datasets to assess the efficacy of the proposed models. Extensive experiments confirmed SDU-Net outperformed or achieved performance similar to the state-of-the-art while using fewer parameters (40% of U-Net). ASDU-Net further reduced the model parameters to 20% of U-Net with performance comparable to SDU-Net. In conclusion, the stacked dilated operation and the asymmetric structure are promising for improving the performance of U-Net and its variants.

Deep image compression based on multi-scale deformable convolution

Deep image compression efficiency has been improved in the past years. However, to fully exploit context information for compressing image objects of different scales and shapes, more adaptive geometric structure of inputs should be considered. In this paper, we novelly introduce deformable convolution and its spatial attention extension into deep image compression task to fully exploit the context information. Specifically, a novel deep image compression network with Multi-Scale Deformable Convolution and Spatial Attention, named MS-DCSA, is proposed to better extract compact and efficient latent representation as well as reconstruct higher-quality images. First, multi-scale deformable convolution is presented to provide multi-scale receptive fields for learning spatial sampling offsets in deformable operations. Subsequently, multi-scale deformable spatial attention module is developed to generate attention masks to re-weight extracted features according to their importance. In addition, the multi-scale deformable convolution is applied to design delicate up/down sampling modules. Extensive experiments demonstrate that the proposed MS-DCSA network achieves improved performance on both PSNR and MS-SSIM quality metrics, compared to conventional as well as competing deep image compression methods.

Real-time grasping detection method based on attention residual block and multi-scale receptive field

In order to achieve high-precision and high-speed grasping detection, in this paper, we propose a lightweight grasping network model with the combination of attention residual block and multi-scale receptive field, which directly predicts the grasping pose of the manipulator from n-channel grasping scene images. The proposed grasping network structure extracts features from the input image based on the residual block introduced with the attention mechanism, and combines the receptive field module to improve the discriminability of the multi-scale features of the image. The network model is trained and verified on the Cornell grasping dataset. The results show that the multi-modal image information can improve the accuracy of grasping detection. With RGB-D image as input, the grasping accuracy rate of both image-wise and object-wise can reach 97.7%, which is better than other current grasping detection algorithms. Meanwhile, the detection speed of the algorithm is 17ms, which ensures the real-time performance of grasping detection tasks.

Deep Trident Decomposition Network for Single License Plate Image Glare Removal

Deep convolutional neural networks have achieved state-of-the-art performance for the removal of atmospheric obscuration. However, most relevant studies have focused on eliminating the effects of atmospheric obscuration but not on the glare in images caused by reflected sunlight. On the basis of a glare image formation model, we propose a deep trident decomposition network with a large-scale sun glare image dataset for glare removal from single images. Specifically, the proposed network is designed and implemented with a trident decomposition module for decomposing an input glare image into occlusion, foreground, and coarse glare-free images by exploring background features from spatial locations. Moreover, a residual refinement module is adopted to refine the coarse glare-free image into fine glare-free image by learning the residuals from features of multiscale receptive field. The experimental results indicated that the proposed network significantly outperforms state-of-the-art atmospheric obscuration removal networks on the built dataset.

Attentional Gated Res2Net for Multivariate Time Series Classification

Multivariate time series classification is a critical problem in data mining with broad applications. It requires harnessing the inter-relationship of multiple variables and various ranges of temporal dependencies to assign the correct classification label of the time series. Multivariate time series may come from a wide range of sources and be used in various scenarios, bringing the classifier challenge of temporal representation learning. We propose a novel convolutional neural network architecture called Attentional Gated Res2Net for multivariate time series classification. Our model uses hierarchical residual-like connections to achieve multi-scale receptive fields and capture multi-granular temporal information. The gating mechanism enables the model to consider the relations between the feature maps extracted by receptive fields of multiple sizes for information fusion. Further, we propose two types of attention modules, channel-wise attention and block-wise attention, to better leverage the multi-granular temporal patterns. Our experimental results on 14 benchmark multivariate time-series datasets show that our model outperforms several baselines and state-of-the-art methods by a large margin. Our model outperforms the SOTA by a large margin, the classification accuracy of our model is 10.16% better than the SOTA model. Besides, we demonstrate that our model improves the performance of existing models when used as a plugin. Further, based on our experiments and analysis, we provide practical advice on applying our model to a new problem.

Encoder-Decoder Structure with Multiscale Receptive Field Block for Unsupervised Depth Estimation from Monocular Video

Monocular depth estimation is a fundamental yet challenging task in computer vision as depth information will be lost when 3D scenes are mapped to 2D images. Although deep learning-based methods have led to considerable improvements for this task in a single image, most existing approaches still fail to overcome this limitation. Supervised learning methods model depth estimation as a regression problem and, as a result, require large amounts of ground truth depth data for training in actual scenarios. Unsupervised learning methods treat depth estimation as the synthesis of a new disparity map, which means that rectified stereo image pairs need to be used as the training dataset. Aiming to solve such problem, we present an encoder-decoder based framework, which infers depth maps from monocular video snippets in an unsupervised manner. First, we design an unsupervised learning scheme for the monocular depth estimation task based on the basic principles of structure from motion (SfM) and it only uses adjacent video clips rather than paired training data as supervision. Second, our method predicts two confidence masks to improve the robustness of the depth estimation model to avoid the occlusion problem. Finally, we leverage the largest scale and minimum depth loss instead of the multiscale and average loss to improve the accuracy of depth estimation. The experimental results on the benchmark KITTI dataset for depth estimation show that our method outperforms competing unsupervised methods.

Multi-scale receptive field fusion network for lightweight image super-resolution

In recent years, with the rapid development of deep learning, quantities of convolutional neural network (CNN) based methods are applied to the field of single image super-resolution (SISR), which have excellent performance compared with traditional ones. However, these methods have a large amount of calculation and memory consumption, limiting their application in edge devices. To address this problem, we propose a more lightweight multi-scale receptive field fusion network (MRFN) for fast and accurate SISR. Specifically, we first propose a multi-scale receptive field fusion block to fuse different scales of spatial information and enlarge the overall receptive field simultaneously. Secondly, based on the non-local sparse attention, we introduce a hash-learnable cheap non-local attention to capture long-range dependencies with much fewer parameters and calculations. Moreover, channel attention is also very important for SISR, we further combine directional second-order information and spatial coordinate information into it to enhance its capability, which is called second-order coordinate attention. The cooperation of these components leads to the success of our MRFN. Extensive experimental results show that our method achieves a better trade-off against other state-of-the-art methods in terms of SISR performance and model complexity.

UAV remote sensing monitoring of pine forest diseases based on improved Mask R-CNN

ABSTRACT Timely and accurate monitoring of pine forest diseases is greatly important to pine forest disease prevention and forest protection. A method for unmanned aerial vehicle (UAV) remote sensing monitoring of pine forest diseases is proposed based on improved Mask region-based convolutional neural network (R-CNN) to solve the problem of low accuracy of the existing methods due to complex background and nonevident characteristics of pine forest diseases. The Mask R-CNN with multitask capabilities is utilized as backbone network to build the monitoring model. An improved multiscale receptive field block module is added to the Mask R-CNN to extract the detailed features of pine forest diseases and reduce the missed detection of pine forest diseases. Meanwhile, a multilevel fusion residual pyramid structure is adopted to integrate low-level and high-level features to obtain distinguishable features of pine forest diseases to reduce the misdetection and misidentification of complex backgrounds. In addition, the proposed method uses cutting mixing splicing method to construct training data set to increase the number of background objects in the training images and reduce the impact of complex backgrounds on the monitoring results. Experimental results show that the proposed method can monitor pine forest diseases more accurately than methods of SSD, Faster R-CNN, MaskScoring R-CNN, and the original Mask R-CNN. Compared with the method of backbone network, the precision of the proposed method has increased by 22.4%, the recall has increased by 3.5%, and the F1-score has increased by 14.4%.

Small Object Detection Method Based on Adaptive Spatial Parallel Convolution and Fast Multi-Scale Fusion

As one type of object detection, small object detection has been widely used in daily-life-related applications with many real-time requirements, such as autopilot and navigation. Although deep-learning-based object detection methods have achieved great success in recent years, they are not effective in small object detection and most of them cannot achieve real-time processing. Therefore, this paper proposes a single-stage small object detection network (SODNet) that integrates the specialized feature extraction and information fusion techniques. An adaptively spatial parallel convolution module (ASPConv) is proposed to alleviate the lack of spatial information for target objects and adaptively obtain the corresponding spatial information through multi-scale receptive fields, thereby improving the feature extraction ability. Additionally, a split-fusion sub-module (SF) is proposed to effectively reduce the time complexity of ASPConv. A fast multi-scale fusion module (FMF) is proposed to alleviate the insufficient fusion of both semantic and spatial information. FMF uses two fast upsampling operators to first unify the resolution of the multi-scale feature maps extracted by the network and then fuse them, thereby effectively improving the small object detection ability. Comparative experimental results prove that the proposed method considerably improves the accuracy of small object detection on multiple benchmark datasets and achieves a high real-time performance.

Sonar Image Target Detection Based on Adaptive Global Feature Enhancement Network

Automatic underwater target detection plays a vital role in sonar image processing and analysis, and its core task is to discriminate target categories and achieve precise positioning. However, the sonar image is interfered by the seafloor reverberation noise and complex background, which brings more significant challenges to the accurate detection of sonar target. To achieve accurate detection of different categories targets in sonar image, we proposed an adaptive global feature enhancement network (AGFE-Net), which uses multi-scale convolution and attention mechanisms with global receptive field to obtain sonar image multi-scale semantic feature and enhance the correlation between features. Specifically, we use the multi-scale receptive field feature extraction block (MSFF-Block) and the self-attention mechanism block (SAM-Block) to enhance model feature extraction ability; the bidirectional feature pyramid network (Bi-FPN) and the global pyramid pooling block (GPP-Block) are used to obtain the deep semantic feature and suppress background noise interference; the adaptive feature fusion block (AFF-Block) is used to effectively fuse features of different scales. Experimental results on the presented sonar target detection dataset WH-Dataset and QD-Dataset validate the advantage of AGFE-Net over other state-of-the-art target detection methods.