Dynamic Convolution Research Articles

Monocular depth estimation using multi-dimensional dynamic convolution

To address the issue of inaccurate prediction results and low accuracy at the object edges in existing self-supervised monocular depth estimation algorithms, this study proposes a self-supervised monocular depth estimation algorithm based on dynamic convolution (OE-Depth). The algorithm employs a multi-dimensional dynamic convolution feature extraction network to acquire more comprehensive feature representations, thereby enhancing the predictive capability. Additionally, the algorithm is optimized by integrating a triplet loss term and employing metric learning techniques to refine the networks performance at the object edges. Experimental evaluations conducted on the KITTI dataset validate the effectiveness of the proposed algorithm, demonstrating an optimization of the error class index by nearly 10%. Notably, the most stringent criterion < 1.25 achieves an accuracy of 0.908 for depth prediction.

THE IMPROVED ROAD DEFECT DETECTION ALGORITHM BASED ON YOLOv8s

Road defect detection is a key task to ensure road safety and repair damage in a timely manner. However, traditional manual inspection methods are inefficient and costly. To address such problems, a BL-YOLOv8 enhanced road defect detection algorithm is proposed. By integrating the BiFPN concept, the neck structure of the YOLOv8s model is reconstructed, reducing model parameters, computational load and overall size, and introducing a dynamic large convolution kernel attention mechanism (LSK-attention) to expand the model's receptive field and improve the accuracy of target detection. The experimental results show that on the road defect data set (KITTI), the number of parameters is reduced by 23.03 %, the amount of calculation is reduced by 5.85 %, and the average accuracy of mAP@0.5 is increased by 2.1 %, verifying the effectiveness of this method for automatic road defect detection technology effectiveness.

Task-Sensitive Efficient Feature Extraction Network for Oriented Object Detection in Remote Sensing Images

The widespread application of convolutional neural networks (CNNs) has led to significant advancements in object detection. However, challenges remain in achieving efficient and precise extraction of critical features when applying typical CNN-based methods to remote sensing detection tasks: (1) The convolutional kernels sliding horizontally in the backbone are misaligned with the features of arbitrarily oriented objects. Additionally, the detector shares the features extracted from the backbone, but the classification task requires orientation-invariant features while the regression task requires orientation-sensitive features. The inconsistency in feature requirements makes it difficult for the detector to extract the critical features required for each task. (2) The use of deeper convolutional structures can improve the detection accuracy, but it also results in substantial convolutional computations and feature redundancy, leading to inefficient feature extraction. To address this issue, we propose a Task-Sensitive Efficient Feature Extraction Network (TFE-Net). Specifically, we propose a special mixed fast convolution module for constructing an efficient network architecture that employs cheap transform operations to replace some of the convolution operations, generating more features with fewer parameters and computation resources. Next, we introduce the task-sensitive detection module, which first aligns the convolutional features with the targets using adaptive dynamic convolution based on the orientation of the targets. The task-sensitive feature decoupling mechanism is further designed to extract orientation-sensitive features and orientation-invariant features from the aligned features and feed them into the regression and classification branches, respectively, which provide the critical features needed for different tasks, thus improving the detection performance comprehensively. In addition, in order to make the training process more stable, we propose a balanced loss function to balance the gradients generated by different samples. Extensive experiments demonstrate that our proposed TFE-Net can achieve superior performance and obtain an effective balance between detection speed and accuracy on DOTA, UCAS-AOD, and HRSC2016.

MDC-RHT: Multi-Modal Medical Image Fusion via Multi-Dimensional Dynamic Convolution and Residual Hybrid Transformer.

The fusion of multi-modal medical images has great significance for comprehensive diagnosis and treatment. However, the large differences between the various modalities of medical images make multi-modal medical image fusion a great challenge. This paper proposes a novel multi-scale fusion network based on multi-dimensional dynamic convolution and residual hybrid transformer, which has better capability for feature extraction and context modeling and improves the fusion performance. Specifically, the proposed network exploits multi-dimensional dynamic convolution that introduces four attention mechanisms corresponding to four different dimensions of the convolutional kernel to extract more detailed information. Meanwhile, a residual hybrid transformer is designed, which activates more pixels to participate in the fusion process by channel attention, window attention, and overlapping cross attention, thereby strengthening the long-range dependence between different modes and enhancing the connection of global context information. A loss function, including perceptual loss and structural similarity loss, is designed, where the former enhances the visual reality and perceptual details of the fused image, and the latter enables the model to learn structural textures. The whole network adopts a multi-scale architecture and uses an unsupervised end-to-end method to realize multi-modal image fusion. Finally, our method is tested qualitatively and quantitatively on mainstream datasets. The fusion results indicate that our method achieves high scores in most quantitative indicators and satisfactory performance in visual qualitative analysis.

Dynamic Convolution Neural Networks with Both Global and Local Attention for Image Classification

Dynamic Convolution Neural Networks with Both Global and Local Attention for Image Classification

Improved YOLOv8-Seg Based on Multiscale Feature Fusion and Deformable Convolution for Weed Precision Segmentation

Laser-targeted weeding methods further enhance the sustainable development of green agriculture, with one key technology being the improvement of weed localization accuracy. Here, we propose an improved YOLOv8 instance segmentation based on bidirectional feature fusion and deformable convolution (BFFDC-YOLOv8-seg) to address the challenges of insufficient weed localization accuracy in complex environments with resource-limited laser weeding devices. Initially, by training on extensive datasets of plant images, the most appropriate model scale and training weights are determined, facilitating the development of a lightweight network. Subsequently, the introduction of the Bidirectional Feature Pyramid Network (BiFPN) during feature fusion effectively prevents the omission of weeds. Lastly, the use of Dynamic Snake Convolution (DSConv) to replace some convolutional kernels enhances flexibility, benefiting the segmentation of weeds with elongated stems and irregular edges. Experimental results indicate that the BFFDC-YOLOv8-seg model achieves a 4.9% increase in precision, an 8.1% increase in recall rate, and a 2.8% increase in mAP50 value to 98.8% on a vegetable weed dataset compared to the original model. It also shows improved mAP50 over other typical segmentation models such as Mask R-CNN, YOLOv5-seg, and YOLOv7-seg by 10.8%, 13.4%, and 1.8%, respectively. Furthermore, the model achieves a detection speed of 24.8 FPS on the Jetson Orin nano standalone device, with a model size of 6.8 MB that balances between size and accuracy. The model meets the requirements for real-time precise weed segmentation, and is suitable for complex vegetable field environments and resource-limited laser weeding devices.

WSS-YOLO: An improved industrial defect detection network for steel surface defects

Surface defects in steel manufacturing can compromise product quality and safety. Detecting these diverse and complex defects under industrial conditions is challenging. This paper proposes WSS-YOLO, a YOLOv8-based model, for accurate defect detection on industrial steel. Firstly, the dynamic non-monotonic focusing mechanism based on WIoU loss was employed to focus on anchor boxes with ordinary quality, thereby improving the overall performance of the detector. Secondly, the C2f-DSC module based on dynamic snake convolution is designed to enable the model to adaptively adjust the receptive field. Finally, GSConv and VOV-GSCSP modules are introduced into the neck network to reduce the computational complexity and parameter quantity while ensuring the accuracy of the model. This paper conducts extensive experiments on the public datasets NEU-DET and GC10-DET, achieving mAP of 82.3% and 72.0%, respectively, outperforming other excellent models. Moreover, the effectiveness of the proposed method in industrial defect detection is also validated.

MST-DGCN: A Multi-Scale Spatio-Temporal and Dynamic Graph Convolution Fusion Network for Electroencephalogram Recognition of Motor Imagery

The motor imagery brain-computer interface (MI-BCI) has the ability to use electroencephalogram (EEG) signals to control and communicate with external devices. By leveraging the unique characteristics of task-related brain signals, this system facilitates enhanced communication with these devices. Such capabilities hold significant potential for advancing rehabilitation and the development of assistive technologies. In recent years, deep learning has received considerable attention in the MI-BCI field due to its powerful feature extraction and classification capabilities. However, two factors significantly impact the performance of deep-learning models. The size of the EEG datasets influences how effectively these models can learn. Similarly, the ability of classification models to extract features directly affects their accuracy in recognizing patterns. In this paper, we propose a Multi-Scale Spatio-Temporal and Dynamic Graph Convolution Fusion Network (MST-DGCN) to address these issues. In the data-preprocessing stage, we employ two strategies, data augmentation and transfer learning, to alleviate the problem of an insufficient data volume in deep learning. By using multi-scale convolution, spatial attention mechanisms, and dynamic graph neural networks, our model effectively extracts discriminative features. The MST-DGCN mainly consists of three parts: the multi-scale spatio-temporal module, which extracts multi-scale information and refines spatial attention; the dynamic graph convolution module, which extracts key connectivity information; and the classification module. We conduct experiments on real EEG datasets and achieve an accuracy of 77.89% and a Kappa value of 0.7052, demonstrating the effectiveness of the MST-DGCN in MI-BCI tasks. Our research provides new ideas and methods for the further development of MI-BCI systems.

KepSalinst: Using Peripheral Points to Delineate Salient Instances.

Salient instance segmentation (SIS) is an emerging field that evolves from salient object detection (SOD), aiming at identifying individual salient instances using segmentation maps. Inspired by the success of dynamic convolutions in segmentation tasks, this article introduces a keypoints-based SIS network (KepSalinst). It employs multiple keypoints, that is, the center and several peripheral points of an instance, as effective geometrical guidance for dynamic convolutions. The features at peripheral points can help roughly delineate the spatial extent of the instance and complement the information inside the central features. To fully exploit the complementary components within these features, we design a differentiated patterns fusion (DPF) module. This ensures that the resulting dynamic convolutional filters formed by these features are sufficiently comprehensive for precise segmentation. Furthermore, we introduce a high-level semantic guided saliency (HSGS) module. This module enhances the perception of saliency by predicting a map for the input image to estimate a saliency score for each segmented instance. On four SIS datasets (ILSO, SOC, SIS10K, and COME15K), our KepSalinst outperforms all previous models qualitatively and quantitatively.

TransConv: Transformer Meets Contextual Convolution for Unsupervised Domain Adaptation.

Unsupervised domain adaptation (UDA) aims to reapply the classifier to be ever-trained on a labeled source domain to a related unlabeled target domain. Recent progress in this line has evolved with the advance of network architectures from convolutional neural networks (CNNs) to transformers or both hybrids. However, this advance has to pay the cost of high computational overheads or complex training processes. In this paper, we propose an efficient alternative hybrid architecture by marrying transformer to contextual convolution (TransConv) to solve UDA tasks. Different from previous transformer based UDA architectures, TransConv has two special aspects: (1) reviving the multilayer perception (MLP) of transformer encoders with Gaussian channel attention fusion for robustness, and (2) mixing contextual features to highly efficient dynamic convolutions for cross-domain interaction. As a result, TransConv enables to calibrate interdomain feature semantics from the global features and the local ones. Experimental results on five benchmarks show that TransConv attains remarkable results with high efficiency as compared to the existing UDA methods.

ODGNet: Robotic Grasp Detection Network Based on Omni-Dimensional Dynamic Convolution

In this article, to further improve the accuracy and speed of grasp detection for unknown objects, a new omni-dimensional dynamic convolution grasp detection network (ODGNet) is proposed. The ODGNet includes two key designs. Firstly, it integrates omni-dimensional dynamic convolution to enhance the feature extraction of the graspable region. Secondly, it employs a grasping region feature enhancement fusion module to refine the features of the graspable region and promote the separation of the graspable region from the background. The ODGNet attained an accuracy of 98.4% and 97.8% on the image-wise and object-wise subsets of the Cornell dataset, respectively. Moreover, the ODGNet’s detection speed can reach 50 fps. A comparison with previous algorithms shows that the ODGNet not only improves the grasp detection accuracy, but also satisfies the requirement of real-time grasping. The grasping experiments in the simulation environment verify the effectiveness of the proposed algorithm.

An Image Dehazing Algorithm for Underground Coal Mines Based on gUNet.

Aiming at the problems of incomplete dehazing, color distortion, and loss of detail and edge information encountered by existing algorithms when processing images of underground coal mines, an image dehazing algorithm for underground coal mines, named CAB CA DSConv Fusion gUNet (CCDF-gUNet), is proposed. First, Dynamic Snake Convolution (DSConv) is introduced to replace traditional convolutions, enhancing the feature extraction capability. Second, residual attention convolution blocks are constructed to simultaneously focus on both local and global information in images. Additionally, the Coordinate Attention (CA) module is utilized to learn the coordinate information of features so that the model can better capture the key information in images. Furthermore, to simultaneously focus on the detail and structural consistency of images, a fusion loss function is introduced. Finally, based on the test verification of the public dataset Haze-4K, the Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Mean Squared Error (MSE) are 30.72 dB, 0.976, and 55.04, respectively, and on a self-made underground coal mine dataset, they are 31.18 dB, 0.971, and 49.66, respectively. The experimental results show that the algorithm performs well in dehazing, effectively avoids color distortion, and retains image details and edge information, providing some theoretical references for image processing in coal mine surveillance videos.

Modeling and predicting energy consumption of chiller based on dynamic spatial-temporal graph neural network

As an important equipment in the HVAC system, the chiller accounts for 50–60 % of the total energy consumption. According to the characteristics of chiller operation data and using the multivariate time series (MTS) forecasting method, a novel chiller energy consumption prediction model (T-CADGCN) is proposed, which is based on the temporal convolution (TC) and the dynamic graph convolution (DGC) network with a channel attention (CA) mechanism. Two datasets are used to verify the prediction performance of different models. Compared with the actual energy consumption, the average MAPEs of the T-CADGCN model on the measured dataset and simulated dataset are 0.92 % and 0.08 %, respectively. Whereas they are 2.8 % and 0.91 % for LSTM, 11.45 % and 11.74 % for MTGNN_GCA, 12.3 % and 12.63 % for MTGNN, respectively. The results demonstrate that the T-CADGCN model can effectively enhance the prediction performance of chiller energy consumption, followed by LSTM, then MTGNN_GCA, and finally MTGNN. Besides, the prediction performance on different future time steps is also conducted on two datasets. It can be found that all models decrease in prediction accuracy with the increase of the prediction time steps, but the T-CADGCN model can maintain a relatively good prediction performance with high robustness. The changes of in-degree and out-degree in the chiller energy consumption node in the dynamic graph construction module indicate that the factors affecting and being affected by chiller energy consumption change over time. This study can help to predict the energy consumption of chiller accurately and thus contributes to building energy management.

FlexDTI: flexible diffusion gradient encoding scheme-based highly efficient diffusion tensor imaging using deep learning

Objective. Most deep neural network-based diffusion tensor imaging methods require the diffusion gradients’ number and directions in the data to be reconstructed to match those in the training data. This work aims to develop and evaluate a novel dynamic-convolution-based method called FlexDTI for highly efficient diffusion tensor reconstruction with flexible diffusion encoding gradient scheme. Approach. FlexDTI was developed to achieve high-quality DTI parametric mapping with flexible number and directions of diffusion encoding gradients. The method used dynamic convolution kernels to embed diffusion gradient direction information into feature maps of the corresponding diffusion signal. Furthermore, it realized the generalization of a flexible number of diffusion gradient directions by setting the maximum number of input channels of the network. The network was trained and tested using datasets from the Human Connectome Project and local hospitals. Results from FlexDTI and other advanced tensor parameter estimation methods were compared. Main results. Compared to other methods, FlexDTI successfully achieves high-quality diffusion tensor-derived parameters even if the number and directions of diffusion encoding gradients change. It reduces normalized root mean squared error by about 50% on fractional anisotropy and 15% on mean diffusivity, compared with the state-of-the-art deep learning method with flexible diffusion encoding gradient scheme. Significance. FlexDTI can well learn diffusion gradient direction information to achieve generalized DTI reconstruction with flexible diffusion gradient scheme. Both flexibility and reconstruction quality can be taken into account in this network.

A self-attention dynamic graph convolution network model for traffic flow prediction

A self-attention dynamic graph convolution network model for traffic flow prediction

Backbone-based Dynamic Spatio-Temporal Graph Neural Network for epidemic forecasting

Accurate epidemic forecasting is a critical task in controlling epidemic spread. Many deep learning-based models focus only on static or dynamic graphs when dealing with spatial information, ignoring their relationship. Additionally, these models often rely on recurrent structures, which can lead to error accumulation and computational time consumption. To address the aforementioned problems, we propose a novel model called Backbone-based Dynamic Spatio-Temporal Graph Neural Network (BDSTGNN). Intuitively, the continuous and smooth changes in graph structure make adjacent graph structures share a basic pattern. To capture this property, we use adaptive methods to generate static backbone graphs containing the primary information, and use temporal models to generate dynamic temporal graphs, and then fuse them to generate a backbone-based dynamic graph. To overcome potential limitations associated with recurrent structures, we introduce a linear model DLinear to handle temporal dependencies, and combine it with dynamic graph convolution for epidemic forecasting. Extensive experiments on two datasets demonstrate that BDSTGNN outperforms baseline models, and ablation comparison further verifies the effectiveness of model components. Furthermore, we analyze and measure the significance of backbone and temporal graphs by using information metrics from different aspects. Finally, we verify the superior efficiency of the BDSTGNN.

ADDGCN: A Novel Approach with Down-Sampling Dynamic Graph Convolution and Multi-Head Attention for Traffic Flow Forecasting

An essential component of autonomous transportation system management and decision-making is precise and real-time traffic flow forecast. Predicting future traffic conditionsis a difficult undertaking because of the intricate spatio-temporal relationships involved. Existing techniques often employ separate modules to model spatio-temporal features independently, thereby neglecting the temporally and spatially heterogeneous features among nodes. Simultaneously, many existing methods overlook the long-term relationships included in traffic data, subsequently impacting prediction accuracy. We introduce a novel method to traffic flow forecasting based on the combination of the feature-augmented down-sampling dynamic graph convolutional network and multi-head attention mechanism. Our method presents a feature augmentation mechanism to integrate traffic data features at different scales. The subsampled convolutional network enhances information interaction in spatio-temporal data, and the dynamic graph convolutional network utilizes the generated graph structure to better simulate the dynamic relationships between nodes, enhancing the model’s capacity for capturing spatial heterogeneity. Through the feature-enhanced subsampled dynamic graph convolutional network, the model can simultaneously capture spatio-temporal dependencies, and coupled with the process of multi-head temporal attention, it achieves long-term traffic flow forecasting. The findings demonstrate that the ADDGCN model demonstrates superior prediction capabilities on two real datasets (PEMS04 and PEMS08). Notably, for the PEMS04 dataset, compared to the best baseline, the performance of ADDGCN is improved by 2.46% in MAE and 2.90% in RMSE; for the PEMS08 dataset, compared to the best baseline, the ADDGCN performance is improved by 1.50% in RMSE, 3.46% in MAE, and 0.21% in MAPE, indicating our method’s superior performance.

Fine-grained recognition of bitter gourd maturity based on Improved YOLOv5-seg model

Bitter gourd, being perishable, requires timely harvesting. Delayed harvesting can result in a substantial reduction in fruit quality. while premature harvesting leads to underdeveloped fruit and decreased yields, the continuous flowering pattern in bitter gourd underscores the significance of accurately assessing fruit growth and ensuring timely harvesting for subsequent fruit setting and development. The current reliance on the experience of production personnel represents a substantial inefficiency. We present an improved real-time instance segmentation model based on YOLOv5-seg. The utilization of dynamic snake convolution enables the extraction of morphological features from the curved and elongated structure of bitter gourd. Diverse branch blocks enhance feature space diversity without inflating model size and inference time, contributing to improved recognition of expansion stages during bitter gourd growth. Additionally, the introduction of Focal-EIOU loss accurately locates the boundary box and mask, addressing sample imbalances in the L2 stage. Experimental results showcase remarkable accuracy rates of 99.3%, 93.8%, and 98.3% for L1, L2, and L3 stages using mAP@0.5. In comparison, our model outperforms other case segmentation models, excelling in both detection accuracy and inference speed. The improved YOLOv5-seg model demonstrates strong performance in fine-grained recognition of bitter gourd during the expansion stage. It efficiently segments bitter gourd in real-time under varying lighting and occlusion conditions, providing crucial maturity information. This model offers reliable insights for agricultural workers, facilitating precise harvesting decisions.

FS_YOLOv8: A Deep Learning Network for Ground Fissures Instance Segmentation in UAV Images of the Coal Mining Area

Abstract. The ground fissures caused by coal mining have seriously affected the ecological environment of the land. Timely and accurate identification and landfill treatment of ground fissures can avoid secondary geological disasters in coal mine areas. At present, the fissure identification methods based on deep learning show excellent performance on roads and walls, etc. Nevertheless, the automatic and reliable segmentation of ground fissures in remote sensing images poses a challenge for deep learning networks, due to the diverse and complex texture information included in the mining ground fissures and background. To overcome these challenges, we propose an improved YOLOv8 instance segmentation network to automatically and efficiently segment the ground fissures in coal mining areas. In detail, a model called FS_YOLOv8 is proposed. The DSPP (Dynamic Snake convolutional Pyramid Pooling) module is incorporated into the FS_YOLOv8 model to establish a multi-scale dynamic snake convolution feature aggregation structure. This module replaces the conventional convolution found in the SPPF module of YOLOv8 and aims to enhance the model's ability to extract features related to fissures with tubular structures. Furthermore, the D-LKA (Deformable Large Kernel Attention) module is employed to autonomously collect fissure context information. To enhance the detection capability of challenging samples in remote sensing images with intricate background and fissure texture, we employ a Slide Loss function. Ultimately, the ground fissure dataset of unmanned aerial vehicle (UAV) images in coal mine areas is subjected to experimental analysis. The experimental findings demonstrate that FS_YOLOv8 exhibits exceptional proficiency in segmenting ground fissures within intricate and expansive mining areas.

A lightweight dynamic dual-damped wavelet-based convolutional neural network for interpretable bearing fault diagnosis

Wavelet-based convolutional neural networks (CNNS) have attracted widespread attention because they can improve the interpretability of intelligent fault diagnosis methods. However, the fault feature representation capability of typical wavelet-based convolution kernel frameworks must be strengthened to improve the diagnostic accuracy of complex faults. In the meantime, the large number of network parameters leads to high computational costs. To address these issues, a lightweight wavelet-based dynamic CNN, which comprises a dual-damping wavelet-based dynamic CNN (DWDC) block and a discrete wavelet transformation (DWT) enhancement (DWTE) block, is put forward. In the DWDC block, a wavelet convolution layer is initially designed, where a dual-damping wavelet is used as the kernel function to improve the match of the convolution kernel with fault impulses. Subsequently, a dynamic convolution layer with multiple parallel small-size convolutional kernels is designed to screen the fault features instead of a multilayer network structure, thereby greatly reducing the number of network parameters. Finally, the DWTE block is constructed by combining the DWT and residual dense block, and it can mine more fault information from the previously extracted features. The experiments on the variable speed bearing dataset, locomotive bearing dataset with constant speed and the Case Western Reserve University dataset prove that the proposed approach outperforms five classical CNN models and six advanced wavelet-based CNN models. In addition, it can effectively solve the issue of data imbalance because of its powerful feature extraction capability.