Depthwise Separable Convolution Research Articles

DP-GAN+B: A lightweight generative adversarial network based on depthwise separable convolutions for generating CT volumes

X-rays, commonly used in clinical settings, offer advantages such as low radiation and cost-efficiency. However, their limitation lies in the inability to distinctly visualize overlapping organs. In contrast, Computed Tomography (CT) scans provide a three-dimensional view, overcoming this drawback but at the expense of higher radiation doses and increased costs. Hence, from both the patient's and hospital's standpoints, there is substantial medical and practical value in attempting the reconstruction from two-dimensional X-ray images to three-dimensional CT images. In this paper, we introduce DP-GAN+B as a pioneering approach for transforming two-dimensional frontal and lateral lung X-rays into three-dimensional lung CT volumes. Our method innovatively employs depthwise separable convolutions instead of traditional convolutions and introduces vector and fusion loss for superior performance. Compared to prior models, DP-GAN+B significantly reduces the generator network parameters by 21.104 M and the discriminator network parameters by 10.82 M, resulting in a total reduction of 31.924 M (44.17%). Experimental results demonstrate that our network can effectively generate clinically relevant, high-quality CT images from X-ray data, presenting a promising solution for enhancing diagnostic imaging while mitigating cost and radiation concerns.

Enhanced multimodal emotion recognition in healthcare analytics: A deep learning based model-level fusion approach

Deep learning techniques have drawn considerable interest in emotion recognition due to recent technological developments in healthcare analytics. Automatic patient emotion recognition can assist healthcare analytics by providing feedback to the stakeholders of competent healthcare about the conditions of the patients and their satisfaction levels. In this paper, we propose a novel model-level fusion technique based on deep learning for enhanced emotion recognition from multimodal signals to monitor patients in connected healthcare. The representative visual features from the video signals are extracted through the Depthwise Separable Convolution Neural Network, and the optimized temporal attributes are derived from the multiple physiological data utilizing Bi-directional Long Short-Term Memory. A soft attention method fused the high multimodal features obtained from the two data modalities to retrieve the most significant features by focusing on emotionally salient parts of the features. We exploited two face detection methods, Histogram of Oriented Gradients and Convolutional Neural Network-based face detector (ResNet-34), to observe the effects of facial features on emotion recognition. Lastly, extensive experimental evaluations have been conducted using the widely used Bio Vid Emo DB multimodal dataset to verify the performance of the proposed architecture. Experimental results show that the developed fusion architecture improved the accuracy of emotion recognition from multimodal signals and outperformed the performance of both state-of-the-art techniques and baseline methods.

Offline Handwriting Writer Identification using Depth-wise Separable Convolution with Siamese Network

Offline handwriting writer identification has significant implications for forensic investigations and biometric authentication. Handwriting, as a distinctive biometric trait, provides insights into individual identity. Despite advancements in handcrafted algorithms and deep learning techniques, the persistent challenges related to intra-variability and inter-writer similarity continue to drive research efforts. In this study, we build on well-separated convolution architectures like the Xception architecture, which has proven to be robust in our previous research comparing various deep learning architectures such as MobileNet, EfficientNet, ResNet50, and VGG16, where Xception demonstrated minimal training-validation disparities for writer identification. Expanding on this, we use a model based on similarity or dissimilarity approaches to identify offline writers' handwriting, known as the Siamese Network, that incorporates the Xception architecture. Similarity or dissimilarity measurements are based on the Manhattan or L1 distance between representation vectors of each input pair. We train publicly available IAM and CVL datasets; our approach achieves accuracy rates of 99.81% for IAM and 99.88% for CVL. The model was evaluated using evaluation metrics, which revealed only two error predictions in the IAM dataset, resulting in 99.75% accuracy, and five error predictions for CVL, resulting in 99.57% accuracy. These findings modestly surpass existing achievements, highlighting the potential inherent in our methodology to enhance writer identification accuracy. This study underscores the effectiveness of integrating the Siamese Network with depth-wise separable convolution, emphasizing the practical implications for supporting writer identification in real-world applications.

Potato leaf disease detection with a novel deep learning model based on depthwise separable convolution and transformer networks

Early diagnosis of plant diseases is essential in reducing economic losses for farmers and increasing production efficiency. Therefore, Computer-Aided Diagnosis (CAD) systems supported by artificial intelligence technologies can be developed to help diagnose diseases quickly and accurately by examining the symptoms and signs in plant leaves. In this study, Multi-head Attention Mechanism Depthwise Separable Convolution Inception Reduction Network (MDSCIRNet) architecture, an image-based deep convolutional neural network, is proposed for classifying potato leaf diseases. The main components of the MDSCIRNet architecture are depthwise separable convolution (DSC) and a multi-head attention mechanism. The proposed architecture has been compared with modern algorithms developed with DSC technology, such as Xception, MobileNet, and deep learning algorithms, such as ResNet101, InceptionV3, and EfficientNetB2, to evaluate its performance in the classification process. In addition, hybrid methods developed with the classical machine learning algorithms Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), Adaptive Boosting (AdaBoost), and MDSCIRNet model, integrated deep learning model, hard voting ensemble learning model. Suggested methods such as these were also used in the experimental process. Moreover, techniques such as Contrast Limited Adaptive Histogram Equalization (CLAHE), Enhanced Super-Resolution Generative Adversarial Networks (ESRGAN), and Hypercolumn were used to improve the image quality of the data set. In the experimental process, the MDSCIRNet deep learning architecture achieved 99.24% accuracy in the study using the original dataset. While a 99.11% accuracy rate was achieved with the integrated deep learning model and hard voting ensemble learning model, the highest success rate of 99.33% was performed in the study conducted with the MDSCIRNet + SVM method. This study contributes to developing new and effective strategies in the agricultural industry for the early diagnosis and control of potato plant diseases. Machine learning-based approaches offer the potential to minimize economic losses and increase productivity in production by allowing farmers to intervene early.

A new frontier in electronics manufacturing: Optimized deep learning techniques for PCB image reconstruction

In the rapidly evolving electronics manufacturing sector, maintaining quality control and conducting failure analysis of Printed Circuit Boards (PCBs) are critical yet challenging tasks. This study presents a groundbreaking self-supervised learning framework to address existing gaps in the reconstruction of encoded or blurred Printed Circuit Board images. By leveraging a customized DeepLabV3+ architecture with depth-wise separable convolutions, our model is engineered to autonomously learn intrinsic Printed Circuit Board features, eliminating the need for manual data labeling. This not only alleviates computational burden but also ensures robust performance. Augmented by feature quantization and channel reduction techniques, our model stands out as both lightweight and resilient, making it highly adaptable for Printed Circuit board imaging. To validate the framework, a tailored dataset comprising raw and encoded Printed Circuit board images from diverse sources was assembled and further refined to match real-world industrial standards. Our model demonstrates unparalleled efficacy in Printed Circuit board image reconstruction, establishing a new benchmark for the field.

A novel Dual-Branch Asymmetric Encoder–Decoder Segmentation Network for accurate colonic crypt segmentation

Colorectal cancer (CRC) is a leading cause of cancer-related deaths, with colonic crypts (CC) being crucial in its development. Accurate segmentation of CC is essential for decisions CRC and developing diagnostic strategies. However, colonic crypts’ blurred boundaries and morphological diversity bring substantial challenges for automatic segmentation. To mitigate this problem, we proposed the Dual-Branch Asymmetric Encoder–Decoder Segmentation Network (DAUNet), a novel and efficient model tailored for confocal laser endomicroscopy (CLE) CC images. In DAUNet, we crafted a dual-branch feature extraction module (DFEM), employing Focus operations and dense depth-wise separable convolution (DDSC) to extract multiscale features, boosting semantic understanding and coping with the morphological diversity of CC. We also introduced the feature fusion guided module (FFGM) to adaptively combine features from both branches using cross-group spatial and channel attention to improve the model representation in focusing on specific lesion features. These modules are seamlessly integrated into the encoder for effective multiscale information extraction and fusion, and DDSC is further introduced in the decoder to provide rich representations for precise segmentation. Moreover, the local multi-layer perceptron (LMLP) module is designed to decouple and recalibrate features through a local linear transformation that filters out the noise and refines features to provide edge-enriched representation. Experimental evaluations on two datasets demonstrate that the proposed method achieves Intersection over Union (IoU) scores of 81.54% and 84.83%, respectively, which are on par with state-of-the-art methods, exhibiting its effectiveness for CC segmentation. The proposed method holds great potential in assisting physicians with precise lesion localization and region analysis, thereby improving the diagnostic accuracy of CRC.

Advanced Deep Learning Model for Plant Diseases Detection in Precision Agriculture

Plant disease detection is becoming a vital research area due to the need to achieve sustainable development goals. This study aims to introduce a new deep learning technique based on inception and a depthwise-separable convolution layer. This approach aims to reduce computational complexity, size, and parameter set without compromising performance. The proposed model was evaluated on two datasets to classify different crop diseases. The proposed model achieved the highest accuracy of 99.1 in the plant village and 98.5 in the potato dataset with the compared studies.

Advanced Deep Learning Model for Plant Diseases Detection in Precision Agriculture

Plant disease detection is becoming a vital research area due to the need to achieve sustainable development goals. This study aims to introduce a new deep learning technique based on inception and a depthwise-separable convolution layer. This approach aims to reduce computational complexity, size, and parameter set without compromising performance. The proposed model was evaluated on two datasets to classify different crop diseases. The proposed model achieved the highest accuracy of 99.1 in the plant village and 98.5 in the potato dataset with the compared studies.

DSCNet: lightweight and efficient self-supervised network via depthwise separable cross convolution blocks for speckle image matching.

Speckle structured light has become a research hotspot due to its ability to acquire target three-dimensional information with single image projection in recent years. To address the challenges of a low number of extracted speckle feature points, high mismatch rate and poor real-time performance in traditional algorithms, as well as the obstacle of requiring expensive annotation data in deep learning-based methods, a lightweight and efficient self-supervised convolutional neural network (CNN) is proposed to achieve high-precision and rapid matching of speckle images. First, to efficiently utilize the speckle projection information, a feature extraction backbone based on the depthwise separable cross convolution blocks is proposed. Second, in the feature detection module, a softargmax detection head is designed to refine the coordinates of speckle feature points to sub-pixel accuracy. In the feature description module, a coarse-to-fine module is presented to further refine matching accuracy. Third, we adopt strategies of transfer learning and self-supervised learning to improve the generalization and feature representation capabilities of the model. Data augmentation and real-time training techniques are used to improve the robustness of the model. The experimental results show that the proposed method achieves a mean matching accuracy of 91.62% for speckle feature points on the pilot's helmet, with mere 0.95% mismatch rate. The full model runs at 42ms for a speckle image pair on an RTX 3060.

Depth_ASPP_UNet++: a novel 2D ESPI phase unwrapping based on convolutional neural network

Phase unwrapping is a key step in optical metrology and physical optics to obtain accurate phase distributions. In practice, phase images obtained from electronic speckle pattern interferometry (ESPI) exhibit diverse and complex morphology, with significant shape variations and non-uniform densities among different individuals. This takes challenges for accurately extracting phase information and unwrapping the phase. With the progress of deep learning technology in optical image processing, real-time performance and accuracy have become concerned issues. In this paper, an ESPI phase unwrapping method based on convolutional neural network UNet++ is proposed. The proposed network combines the depthwise separable convolution (DSC), atrous spatial pyramid pooling (ASPP), defined as Depth_ASPP_UNet++. In this model, the use of DSC improves network computational efficiency and provides better feature representation capability. In addition, ASPP is introduced to pay more attention to the phase information of the phase image, and then obtain better phase unwrapping results. The experimental results show that our proposed method can obtain excellent results, especially with various of variable density, different noise levels, and different speckle sizes.

A Multi-Source Data Fusion Network for Wood Surface Broken Defect Segmentation.

Wood surface broken defects seriously damage the structure of wooden products, these defects have to be detected and eliminated. However, current defect detection methods based on machine vision have difficulty distinguishing the interference, similar to the broken defects, such as stains and mineral lines, and can result in frequent false detections. To address this issue, a multi-source data fusion network based on U-Net is proposed for wood broken defect detection, combining image and depth data, to suppress the interference and achieve complete segmentation of the defects. To efficiently extract various semantic information of defects, an improved ResNet34 is designed to, respectively, generate multi-level features of the image and depth data, in which the depthwise separable convolution (DSC) and dilated convolution (DC) are introduced to decrease the computational expense and feature redundancy. To take full advantages of two types of data, an adaptive interacting fusion module (AIF) is designed to adaptively integrate them, thereby generating accurate feature representation of the broken defects. The experiments demonstrate that the multi-source data fusion network can effectively improve the detection accuracy of wood broken defects and reduce the false detections of interference, such as stains and mineral lines.

Fault Diagnosis for Wind Turbine Flange Bolts Based on One-Dimensional Depthwise Separable Convolutions

In this paper, a new bolt fault diagnosis method is developed to solve the fault diagnosis problem of wind turbine flange bolts using one-dimensional depthwise separable convolutions. The main idea is to use a one-dimensional convolutional neural network model to classify and identify the acoustic vibration signals of bolts, which represent different bolt damage states. Through the method of knock test and modal simulation, it is concluded that the damage state of wind turbine flange bolt is related to the natural frequency distribution of acoustic vibration signal. It is found that the bolt damage state affects the modal shape of the structure, and then affects the natural frequency distribution of the bolt vibration signal. Therefore, the damage state can be identified by identifying the natural frequency distribution of the bolt acoustic vibration signal. In the present one-dimensional depth-detachable convolutional neural network model, the one-dimensional vector is first convolved into multiple channels, and then each channel is separately learned by depth-detachable convolution, which can effectively improve the feature quality and the effect of data classification. From the perspective of the realization mechanism of convolution operation, the depthwise separable convolution operation has fewer parameters and faster computing speed, making it easier to build lightweight models and deploy them to mobile devices.

Mask wearing detection algorithm based on improved Yolov7

Manual inspection of the mask is too time-consuming and laborious. In order to detect whether a mask is worn in a crowded public place, a mask-wearing detection method based on improved YOLOV7 is proposed, which uses Depth wise separable convolution instead of conventional convolution, in order to integrate the local feature information and the whole image information deeply, Dilated Convolution was used to improve the Pyramid Pooling Module (DC-PPM) , at last, the loss function of target location is optimized, which makes it not only have the ability of feature extraction to fuse the whole and local information, but also have the ability of not losing the detail information. The experimental results show that the detection accuracy and speed of the algorithm are 95.07% and 79 frames/s respectively, which are 3.4% and 14 frames/s higher than the original YOLOV7 algorithm, very good to meet the actual application needs.

Person identification with arrhythmic ECG signals using deep convolution neural network

Over the past decade, the use of biometrics in security systems and other applications has grown in popularity. ECG signals in particular are attracting increased attention due to their characteristics, which are required for a trustworthy identification system. The majority of ECG-based person identification systems are evaluated without considering the health-state of the individuals. Few person identification systems consider person-by-person health-state annotation. This paper proposes a person identification system considering the health-state annotated ECG signals where each person’s beats overlap among variant arrhythmia classes. This overlapping between the normal class and other arrhythmia classes grants the ability to isolate normal beats in the train set from the Arrhythmic beats in the test set. Therefore, this paper investigates the effect of arrhythmic heartbeats on biometric recognition. An effective lightweight CNN based on depth-wise separable convolution (DWSC) is proposed to enhance the performance of person identification for several common arrhythmia types using the MITBIH dataset. The proposed methodology has been tested on nine arrhythmia types and presents how different types of arrhythmia affect ECG-based biometric systems differently. The experimental results show excellent recognition performance (99.28%) on normal heartbeats and (93.81%) on arrhythmic heartbeats, outperforming other models in terms of mean accuracy.

FV-MViT: Mobile Vision Transformer for Finger Vein Recognition.

In addressing challenges related to high parameter counts and limited training samples for finger vein recognition, we present the FV-MViT model. It serves as a lightweight deep learning solution, emphasizing high accuracy, portable design, and low latency. The FV-MViT introduces two key components. The Mul-MV2 Block utilizes a dual-path inverted residual connection structure for multi-scale convolutions, extracting additional local features. Simultaneously, the Enhanced MobileViT Block eliminates the large-scale convolution block at the beginning of the original MobileViT Block. It converts the Transformer's self-attention into separable self-attention with linear complexity, optimizing the back end of the original MobileViT Block with depth-wise separable convolutions. This aims to extract global features and effectively reduce parameter counts and feature extraction times. Additionally, we introduce a soft target center cross-entropy loss function to enhance generalization and increase accuracy. Experimental results indicate that the FV-MViT achieves a recognition accuracy of 99.53% and 100.00% on the Shandong University (SDU) and Universiti Teknologi Malaysia (USM) datasets, with equal error rates of 0.47% and 0.02%, respectively. The model has a parameter count of 5.26 million and exhibits a latency of 10.00 milliseconds from the sample input to the recognition output. Comparison with state-of-the-art (SOTA) methods reveals competitive performance for FV-MViT.

Lightweight land cover classification via semantic segmentation of remote sensing imagery and analysis of influencing factors

Land cover classification is of great value and can be widely used in many fields. Earlier land cover classification methods used traditional image segmentation techniques, which cannot fully and comprehensively extract the ground information in remote sensing images. Therefore, it is necessary to integrate the advanced techniques of deep learning into the study of semantic segmentation of remote sensing images. However, most of current high-resolution image segmentation networks have disadvantages such as large parameters and high network training cost. In view of the problems above, a lightweight land cover classification model via semantic segmentation, DeepGDLE, is proposed in this paper. The model DeepGDLE is designed on the basis of DeeplabV3+ network and utilizes the GhostNet network instead of the backbone feature extraction network in the encoder. Using Depthwise Separable Convolution (DSC) instead of dilation convolution. This reduces the number of parameters and increases the computational speed of the model. By optimizing the dilation rate of parallel convolution in the ASPP module, the “grid effect” is avoided. ECANet lightweight channel attention mechanism is added after the feature extraction module and the pyramid pooling module to focus on the important weights of the model. Finally, the loss function Focal Loss is utilized to solve the problem of category imbalance in the dataset. As a result, the model DeepGDLE effectively reduces the parameters of the network model and the network training cost. And extensive experiments compared with several existing semantic segmentation algorithms such as DeeplabV3+, UNet, SegNet, etc. show that DeepGDLE improves the quality and efficiency of image segmentation. Therefore, compared to other networks, the DeepGDLE network model can be more effectively applied to land cover classification. In addition, in order to investigate the effects of different factors on the semantic segmentation performance of remote sensing images and to verify the robustness of the DeepGDLE model, a new remote sensing image dataset, FRSID, is constructed in this paper. This dataset takes into account more influences than the public dataset. The experimental results show that on the WHDLD dataset, the experimental metrics mIoU, mPA, and mRecall of the proposed model, DeepGDLE, are 62.29%, 72.85%, and 72.46%, respectively. On the FRSID dataset, the metrics mIoU, mPA, and mRecall are 65.89%, 74.43%, and 74.08%, respectively. For the future scope of research in this field, it may focus on the fusion of multi-source remote sensing data and the intelligent interpretation of remote sensing images.

An N-Shaped Lightweight Network with a Feature Pyramid and Hybrid Attention for Brain Tumor Segmentation.

Brain tumor segmentation using neural networks presents challenges in accurately capturing diverse tumor shapes and sizes while maintaining real-time performance. Additionally, addressing class imbalance is crucial for achieving accurate clinical results. To tackle these issues, this study proposes a novel N-shaped lightweight network that combines multiple feature pyramid paths and U-Net architectures. Furthermore, we ingeniously integrate hybrid attention mechanisms into various locations of depth-wise separable convolution module to improve efficiency, with channel attention found to be the most effective for skip connections in the proposed network. Moreover, we introduce a combination loss function that incorporates a newly designed weighted cross-entropy loss and dice loss to effectively tackle the issue of class imbalance. Extensive experiments are conducted on four publicly available datasets, i.e., UCSF-PDGM, BraTS 2021, BraTS 2019, and MSD Task 01 to evaluate the performance of different methods. The results demonstrate that the proposed network achieves superior segmentation accuracy compared to state-of-the-art methods. The proposed network not only improves the overall segmentation performance but also provides a favorable computational efficiency, making it a promising approach for clinical applications.

A Lightweight convolutional medical segmentation algorithm based on ConvNeXt to improve UNet

In recent years, UNet and its derivative networks have gained widespread recognition as major methods of medical image segmentation. However, networks like UNet often struggle with Point-of-Care (POC) healthcare applications due to their high number of parameters and computational complexity. To tackle these challenges, this paper introduces an efficient network designed for medical image segmentation called MCU-Net, which leverages ConvNeXt to enhance UNet. 1) Based on ConvNeXt, MCU-Net proposes the MCU Block, which employs techniques such as large kernel convolution, depth-wise separable convolution, and an inverted bottleneck design. To ensure stable segmentation performance, it also integrates global response normalization (GRN) layers and Gaussian Error Linear Unit (GELU) activation functions. 2) Additionally, MCU-Net introduces an enhanced Multi-Scale Convolution Attention (MSCA) module after the original UNet’s skip connections, emphasizing medical image features and capturing semantic insights across multiple scales. 3)The downsampling process replaces pooling layers with convolutions, and both upsampling and downsampling stages incorporate batch normalization (BN) layers to enhance model stability during training. The experimental results demonstrate that MCU-Net, with a parameter count of 2.19 million and computational complexity of 19.73 FLOPs, outperforms other segmentation models. The overall performance of MCU-Net in medical image segmentation surpasses that of other models, achieving a Dice score of 91.8% and mIoU of 84.7% on the GlaS dataset. When compared to UNet on the BUSI dataset, MCU-Net shows an improvement of 2% in Dice and 2.9% in mIoU.

An Efficient FPGA-based Depthwise Separable Convolutional Neural Network Accelerator with Hardware Pruning

Convolutional neural networks (CNNs) have been widely deployed in computer vision tasks. However, the computation and resource intensive characteristics of CNN bring obstacles to its application on embedded systems. This article proposes an efficient inference accelerator on Field Programmable Gate Array (FPGA) for CNNs with depthwise separable convolutions. To improve the accelerator efficiency, we make four contributions: (1) an efficient convolution engine with multiple strategies for exploiting parallelism and a configurable adder tree are designed to support three types of convolution operations; (2) a dedicated architecture combined with input buffers is designed for the bottleneck network structure to reduce data transmission time; (3) a hardware padding scheme to eliminate invalid padding operations is proposed; and (4) a hardware-assisted pruning method is developed to support online tradeoff between model accuracy and power consumption. Experimental results show that for MobileNetV2 the accelerator achieves 10× and 6× energy efficiency improvement over the CPU and GPU implementation, and 302.3 frames per second and 181.8 GOPS performance that is the best among several existing single-engine accelerators on FPGAs. The proposed hardware-assisted pruning method can effectively reduce 59.7% power consumption at the accuracy loss within 5%.

MetaFormer Baselines for Vision.

MetaFormer, the abstracted architecture of Transformer, has been found to play a significant role in achieving competitive performance. In this paper, we further explore the capacity of MetaFormer, again, by migrating our focus away from the token mixer design: we introduce several baseline models under MetaFormer using the most basic or common mixers, and demonstrate their gratifying performance. We summarize our observations as follows: (1) MetaFormer ensures solid lower bound of performance. By merely adopting identity mapping as the token mixer, the MetaFormer model, termed IdentityFormer, achieves [Formula: see text]80% accuracy on ImageNet-1K. (2) MetaFormer works well with arbitrary token mixers. When specifying the token mixer as even a random matrix to mix tokens, the resulting model RandFormer yields an accuracy of [Formula: see text]81%, outperforming IdentityFormer. Rest assured of MetaFormer's results when new token mixers are adopted. (3) MetaFormer effortlessly offers state-of-the-art results. With just conventional token mixers dated back five years ago, the models instantiated from MetaFormer already beat state of the art. (a) ConvFormer outperforms ConvNeXt. Taking the common depthwise separable convolutions as the token mixer, the model termed ConvFormer, which can be regarded as pure CNNs, outperforms the strong CNN model ConvNeXt. (b) CAFormer sets new record on ImageNet-1K. By simply applying depthwise separable convolutions as token mixer in the bottom stages and vanilla self-attention in the top stages, the resulting model CAFormer sets a new record on ImageNet-1K: it achieves an accuracy of 85.5% at 224 ×224 resolution, under normal supervised training without external data or distillation. In our expedition to probe MetaFormer, we also find that a new activation, StarReLU, reduces 71% FLOPs of activation compared with commonly-used GELU yet achieves better performance. Specifically, StarReLU is a variant of Squared ReLU dedicated to alleviating distribution shift. We expect StarReLU to find great potential in MetaFormer- like models alongside other neural networks. Code and models are available at https://github.com/sail-sg/metaformer.