Lightweight Convolutional Neural Network Research Articles

Application of Lightweight Deep Learning Model-Based Shear Wave Elastic Imaging with Abdominal Probe to Diagnose Prostate Cancer: A Biomedical Approach

We aimed to investigate the diagnostic value of lightweight convolutional neural network (CNN) model abdominal probe shear wave elastic imaging (SWE) in the perineal diagnosis and guided puncture biopsy of prostate cancer (PCa), and to provide reference for the clinical diagnosis of PCa. 100 PCa patients were assigned to group I (malignant) and group II (benign), with 50 cases in each. Ultrasonic elastic imaging based on lightweight convolutional neural network denoising model was adopted for detection. In both systolic and diastolic blood pressure (SBP/DBP), there was not a significant intergroup difference (P > 0.05). The levels of prostate specific antigen (PSA) and its free variant (fPSA) in group II were markedly lower (P < 0.05). Patients in group II had obviously more cystic components and fewer solid components. Patients with hyperechogenicity was more in group II. Patients had clearly fewer irregular margins and outward margin spread in group II. Patients without focal hyperechogenicity and punctate hyperechogenicity was more in group II, and the number of calcifications in group II was less. Patients with type 0 and type I was more and patients with type IIa and type IIb was less in group II. The Emean level of patients in group II was clearly higher, and the Emax level and Esd level of patients in group II were clearly lower. The SI level of patients was clearly lower in group II TTP was higher in group II (P < 0.05). Multivariate logistic regression analysis of abdominal probe SWE for transperineal diagnosis of PCa and guided puncture biopsy showed that internal echoes had the greatest OR and were associated with the occurrence of PCa. Ultrasonic elastic imaging index based on the lightweight convolutional neural network denoising model can be used for the benign and malignant diagnosis of PCa patients.

Intrusion detection system for cloud environment based on convolutional neural networks and PSO algorithm

Authentication of clients and their applications to cloud services is a major concern. Network security and the identification of hostile activities are greatly aided by intrusion detection systems (IDS). In general, optimisation strategies can be applied to improve IDS model performance. Convolutional neural networks (CNN) and other deep learning (DL) algorithms is utilised to enhance IDS’s capability to identify and categories intrusions. IDSs can identify prior attacks, adapt to changing threats, and minimise false positives by utilising these strategies. In this work, a lightweight CNN is proposed for intrusion detection in cloud environment. The main contribution of this research is to use particle swarm optimization (PSO), ametaheuristic algorithm to find the CNNs optimal parameters that comprise the number of convolutional layers, the size of the filter utilized in the convolutional procedure, the number of convolutional filters, and the batch size. Heuristic based searches are useful for solving these kinds of problems. The experimental outcomes demonstrate that the proposed method reaches 91.70% of accuracy, 91.82% of precision, 91.99% of recall and 91.90% of F1-score. Cloud providers can gain from improved security measures by incorporating the proposed IDS paradigm into cloud settings, thereby minimizing unauthorized access and any data breaches.

Quaternion-based 2D-DOST and stacked principal component analysis network for multimodal face recognition

Face recognition is widely utilized and has become a ubiquitous part of daily lives. However, original face images are often sensitive and unauthorized access could imperil personal information. To protect privacy and security of multimodal face data, we propose an original approach that integrates differential privacy and a lightweight convolutional neural network with unsupervised predefined quaternion-type filters. Firstly, different modalities of face images are banded together by means of full quaternion matrix representation for simultaneously processing. Then the proposed quaternion two-dimensional discrete orthogonal Stockwell transform is exploited for implementing difference privacy, which is disturbed by virtue of Laplace and exponential mechanism. Afterwards, we devise a stacked quaternion two-dimensional principal component analysis network with three-stages for learning representative features. The proposed network model can enhance the discriminative characteristics through inserting nonlinearity, together with union of feature maps in different levels to achieve information complementation. Experiments conducted on four multimodal face datasets unveil that the proposed method achieves superior recognition performance in comparison with other state-of-the-art approaches.

Fast detection and obstacle avoidance on UAVs using lightweight convolutional neural network based on the fusion of radar and camera

Fast detection and obstacle avoidance on UAVs using lightweight convolutional neural network based on the fusion of radar and camera

Early classification and recognition algorithm for sudden cardiac arrest based on limited electrocardiogram data trained with a two-stages convolutional neural network

Sudden cardiac arrest (SCA) is a lethal cardiac arrhythmia that poses a serious threat to human life and health. However, clinical records of sudden cardiac death (SCD) electrocardiogram (ECG) data are extremely limited. This paper proposes an early prediction and classification algorithm for SCA based on deep transfer learning. With limited ECG data, it extracts heart rate variability features before the onset of SCA and utilizes a lightweight convolutional neural network model for pre-training and fine-tuning in two stages of deep transfer learning. This achieves early classification, recognition and prediction of high-risk ECG signals for SCA by neural network models. Based on 16 788 30-second heart rate feature segments from 20 SCA patients and 18 sinus rhythm patients in the international publicly available ECG database, the algorithm performance evaluation through ten-fold cross-validation shows that the average accuracy (Acc), sensitivity (Sen), and specificity (Spe) for predicting the onset of SCA in the 30 minutes prior to the event are 91.79%, 87.00%, and 96.63%, respectively. The average estimation accuracy for different patients reaches 96.58%. Compared to traditional machine learning algorithms reported in existing literatures, the method proposed in this paper helps address the requirement of large training datasets for deep learning models and enables early and accurate detection and identification of high-risk ECG signs before the onset of SCA.

Machine learning enabled classification of lung cancer cell lines co-cultured with fibroblasts with lightweight convolutional neural network for initial diagnosis

BackgroundIdentification of lung cancer subtypes is critical for successful treatment in patients, especially those in advanced stages. Many advanced and personal treatments require knowledge of specific mutations, as well as up- and down-regulations of genes, for effective targeting of the cancer cells. While many studies focus on individual cell structures and delve deeper into gene sequencing, the present study proposes a machine learning method for lung cancer classification based on low-magnification cancer outgrowth patterns in a 2D co-culture environment.MethodsUsing a magnetic well plate holder, circular pattern lung cancer cell clusters were generated among fibroblasts, and daily images were captured to monitor cancer outgrowth over a 9-day period. These outgrowth images were then augmented and used to train a convolutional neural network (CNN) model based on the lightweight TinyVGG architecture. The model was trained with pairs of classes representing three subtypes of NSCLC: A549 (adenocarcinoma), H520 (squamous cell carcinoma), and H460 (large cell carcinoma). The objective was to assess whether this lightweight machine learning model could accurately classify the three lung cancer cell lines at different stages of cancer outgrowth. Additionally, cancer outgrowth images of two patient-derived lung cancer cells, one with the KRAS oncogene and the other with the EGFR oncogene, were captured and classified using the CNN model. This demonstration aimed to investigate the translational potential of machine learning-enabled lung cancer classification.ResultsThe lightweight CNN model achieved over 93% classification accuracy at 1 day of outgrowth among A549, H460, and H520, and reached 100% classification accuracy at 7 days of outgrowth. Additionally, the model achieved 100% classification accuracy at 4 days for patient-derived lung cancer cells. Although these cells are classified as Adenocarcinoma, their outgrowth patterns vary depending on their oncogene expressions (KRAS or EGFR).ConclusionsThese results demonstrate that the lightweight CNN architecture, operating locally on a laptop without network or cloud connectivity, can effectively create a machine learning-enabled model capable of accurately classifying lung cancer cell subtypes, including those derived from patients, based upon their outgrowth patterns in the presence of surrounding fibroblasts. This advancement underscores the potential of machine learning to enhance early lung cancer subtyping, offering promising avenues for improving treatment outcomes in advanced stage-patients.

MnasNet-SimAM: An Improved Deep Learning Model for the Identification of Common Wheat Diseases in Complex Real-Field Environments.

Deep learning approaches have been widely applied for agricultural disease detection. However, considerable challenges still exist, such as low recognition accuracy in complex backgrounds and high misjudgment rates for similar diseases. This study aimed to address these challenges through the detection of six prevalent wheat diseases and healthy wheat in images captured in a complex natural context, evaluating the recognition performance of five lightweight convolutional networks. A novel model, named MnasNet-SimAM, was developed by combining transfer learning and an attention mechanism. The results reveal that the five lightweight convolutional neural networks can recognize the six different wheat diseases with an accuracy of more than 90%. The MnasNet-SimAM model attained an accuracy of 95.14%, which is 1.7% better than that of the original model, while only increasing the model's parameter size by 0.01 MB. Additionally, the MnasNet-SimAM model reached an accuracy of 91.20% on the public Wheat Fungi Diseases data set, proving its excellent generalization capacity. These findings reveal that the proposed model can satisfy the requirements for rapid and accurate wheat disease detection.

Research on facial expression recognition algorithm based on improved MobileNetV3

Aiming at the problem that face images are easily interfered by occlusion factors in uncontrollable environments, and the complex structure of traditional convolutional neural networks leads to low expression recognition rates, slow network convergence speed, and long network training time, an improved lightweight convolutional neural network is proposed for facial expression recognition algorithm. First, the dilation convolution is introduced into the shortcut connection of the inverted residual structure in the MobileNetV3 network to expand the receptive field of the convolution kernel and reduce the loss of expression features. Then, the channel attention mechanism SENet in the network is replaced by the two-dimensional (channel and spatial) attention mechanism SimAM introduced without parameters to reduce the network parameters. Finally, in the normalization operation, the Batch Normalization of the backbone network is replaced with Group Normalization, which is stable at various batch sizes, to reduce errors caused by processing small batches of data. Experimental results on RaFD, FER2013, and FER2013Plus face expression data sets show that the network reduces the training times while maintaining network accuracy, improves network convergence speed, and has good convergence effects.

A two-stage accelerated search strategy for large-scale multi-objective evolutionary algorithm

Since large-scale multi-objective problems (LSMOPs) have huge decision variables, the traditional evolutionary algorithms are facing difficulties of low exploitation efficiency and high exploration costs in solving LSMOPs. Therefore, this paper proposes an evolutionary strategy based on two-stage accelerated search optimizers (ATAES). Specifically, a convergence optimizer is devised in the first stage, while a three-layer lightweight convolutional neural network model is built, and the population is homogenized into two subsets, the diversity subset, and the convergence subset, which serve as input nodes and the expected output nodes of the neural network, respectively. Then, by constantly backpropagating the gradient, a satisfactory individual will be produced. Once exploitation stagnation is discovered in the first phase, the second phase will be run, where a diversity optimizer using a differential optimization algorithm with opposite learning is suggested to increase the exploration range of candidate solutions and thereby increase the population's diversity. Finally, to validate the algorithm's performance, on multi-objective LSMOP and DTLZ benchmark suits with decision variable quantities of 100, 300, 500, and 1000, the ATAES demonstrated its superiority with other advanced multi-objective evolutionary algorithms.

Lightweight CNN combined with knowledge distillation for the accurate determination of black tea fermentation degree

Black tea is the second most common type of tea in China. Fermentation is one of the most critical processes in its production, and it affects the quality of the finished product, whether it is insufficient or excessive. At present, the determination of black tea fermentation degree completely relies on artificial experience. It leads to inconsistent quality of black tea. To solve this problem, we use machine vision technology to distinguish the degree of fermentation of black tea based on images, this paper proposes a lightweight convolutional neural network (CNN) combined with knowledge distillation to discriminate the degree of fermentation of black tea. After comparing 12 kinds of CNN models, taking into account the size of the model and the performance of discrimination, as well as the selection principle of teacher models, Shufflenet_v2_x1.0 is selected as the student model, and Efficientnet_v2 is selected as the teacher model. Then, CrossEntropy Loss is replaced by Focal Loss. Finally, for Distillation Loss ratios of 0.6, 0.7, 0.8, 0.9, Soft Target Knowledge Distillation (ST), Masked Generative Distillation (MGD), Similarity-Preserving Knowledge Distillation (SPKD), and Attention Transfer (AT) four knowledge distillation methods are tested for their performance in distilling knowledge from the Shufflenet_v2_x1.0 model. The results show that the model discrimination performance after distillation is the best when the Distillation Loss ratio is 0.8 and the MGD method is used. This setup effectively improves the discrimination performance without increasing the number of parameters and computation volume. The model’s P, R and F1 values reach 0.9208, 0.9190 and 0.9192, respectively. It achieves precise discrimination of the fermentation degree of black tea. This meets the requirements of objective black tea fermentation judgment and provides technical support for the intelligent processing of black tea.

Intra-Pulse Modulation Recognition of Radar Signals Based on Efficient Cross-Scale Aware Network.

Radar signal intra-pulse modulation recognition can be addressed with convolutional neural networks (CNNs) and time-frequency images (TFIs). However, current CNNs have high computational complexity and do not perform well in low-signal-to-noise ratio (SNR) scenarios. In this paper, we propose a lightweight CNN known as the cross-scale aware network (CSANet) to recognize intra-pulse modulation based on three types of TFIs. The cross-scale aware (CSA) module, designed as a residual and parallel architecture, comprises a depthwise dilated convolution group (DDConv Group), a cross-channel interaction (CCI) mechanism, and spatial information focus (SIF). DDConv Group produces multiple-scale features with a dynamic receptive field, CCI fuses the features and mitigates noise in multiple channels, and SIF is aware of the cross-scale details of TFI structures. Furthermore, we develop a novel time-frequency fusion (TFF) feature based on three types of TFIs by employing image preprocessing techniques, i.e., adaptive binarization, morphological processing, and feature fusion. Experiments demonstrate that CSANet achieves higher accuracy with our TFF compared to other TFIs. Meanwhile, CSANet outperforms cutting-edge networks across twelve radar signal datasets, providing an efficient solution for high-precision recognition in low-SNR scenarios.

A Lightweight Convolutional Neural Network for Salient Object Detection

A Lightweight Convolutional Neural Network for Salient Object Detection

Comparative Analysis of CNN Algorithms for Mushroom Classification with Proposed Lightweight CNN Model

The classification of mushroom species presents significant ecologic and health-related challenges; advancement in classification techniques is required to gain reliable identifications. This study aims to explain a methodology that was devised and evaluated in the development of a novel, lightweight Convolutional Neural Network (CNN) designed specifically for the task of mushroom classification. The paper provides a custom CNN model that is computationally cost-effective and capable of high-precision classification, fit for real-time usage. Hence, the proposed model was evaluated on this dataset of curated mushroom images with traditional classifiers and state-of-the-art CNN architectures, such as EfficientNet-B7, ResNet50, InceptionV3, and MobileNetV2. The custom model is depth-wise separations engineered in such a way that while they reduce the computational load, they don't compromise the effectiveness of the model. The custom model achieved a test score of 0.68, which is moderate compared to more established models such as EfficientNet-B7 or ResNet50. This approach helps the model function effectively even on platforms having low computational resources. A comprehensive evaluation reveals that a custom CNN has reasonable accuracy in the identification of different mushroom species vis-à-vis existing models, but also significantly lightens the classification process.

Fault detection and diagnosis of grid-connected photovoltaic systems using energy valley optimizer based lightweight CNN and wavelet transform

Early fault detection and diagnosis of grid-connected photovoltaic systems (GCPS) is imperative to improve their performance and reliability. Low-cost edge devices have emerged as innovative solutions for real-time monitoring, reducing latency, and improving response times. In this work, a lightweight Convolutional Neural Network (CNN) is designed and fine-tuned using Energy Valley Optimizer (EVO) for fault diagnosis. The CNN input consists of two-dimensional scalograms generated using Continuous Wavelet Transform (CWT). The proposed diagnosis technique demonstrated superior performance compared to benchmark architectures, namely MobileNet, NASNetMobile, and InceptionV3, achieving higher test accuracies and lower losses on binary and multi-fault classification tasks on balanced, unbalanced, and noisy datasets. Further, a quantitative comparison is conducted with similar recent studies. The obtained results indicate good performance and high reliability of the proposed fault diagnosis method.

Intelligent Fault Diagnosis Method for Rotating Machinery Based on Recurrence Binary Plot and DSD-CNN.

To tackle the issue of the traditional intelligent diagnostic algorithm's insufficient utilization of correlation characteristics within the time series of fault signals and to meet the challenges of accuracy and computational complexity in rotating machinery fault diagnosis, a novel approach based on a recurrence binary plot (RBP) and a lightweight, deep, separable, dilated convolutional neural network (DSD-CNN) is proposed. Firstly, a recursive encoding method is used to convert the fault vibration signals of rotating machinery into two-dimensional texture images, extracting feature information from the internal structure of the fault signals as the input for the model. Subsequently, leveraging the excellent feature extraction capabilities of a lightweight convolutional neural network embedded with attention modules, the fault diagnosis of rotating machinery is carried out. The experimental results using different datasets demonstrate that the proposed model achieves excellent diagnostic accuracy and computational efficiency. Additionally, compared with other representative fault diagnosis methods, this model shows better anti-noise performance under different noise test data, and it provides a reliable and efficient reference solution for rotating machinery fault-classification tasks.

DiRecNetV2: A Transformer-Enhanced Network for Aerial Disaster Recognition

The integration of Unmanned Aerial Vehicles (UAVs) with artificial intelligence (AI) models for aerial imagery processing in disaster assessment, necessitates models that demonstrate exceptional accuracy, computational efficiency, and real-time processing capabilities. Traditionally Convolutional Neural Networks (CNNs), demonstrate efficiency in local feature extraction but are limited by their potential for global context interpretation. On the other hand, Vision Transformers (ViTs) show promise for improved global context interpretation through the use of attention mechanisms, although they still remain underinvestigated in UAV-based disaster response applications. Bridging this research gap, we introduce DiRecNetV2, an improved hybrid model that utilizes convolutional and transformer layers. It merges the inductive biases of CNNs for robust feature extraction with the global context understanding of Transformers, maintaining a low computational load ideal for UAV applications. Additionally, we introduce a new, compact multi-label dataset of disasters, to set an initial benchmark for future research, exploring how models trained on single-label data perform in a multi-label test set. The study assesses lightweight CNNs and ViTs on the AIDERSv2 dataset, based on the frames per second (FPS) for efficiency and the weighted F1 scores for classification performance. DiRecNetV2 not only achieves a weighted F1 score of 0.964 on a single-label test set but also demonstrates adaptability, with a score of 0.614 on a complex multi-label test set, while functioning at 176.13 FPS on the Nvidia Orin Jetson device.

ALKU-Net: Adaptive Large Kernel Attention Convolution Network for Lung Nodule Segmentation

The accurate segmentation of lung nodules in computed tomography (CT) images is crucial for the early screening and diagnosis of lung cancer. However, the heterogeneity of lung nodules and their similarity to other lung tissue features make this task more challenging. By using large receptive fields from large convolutional kernels, convolutional neural networks (CNNs) can achieve higher segmentation accuracies with fewer parameters. However, due to the fixed size of the convolutional kernel, CNNs still struggle to extract multi-scale features for lung nodules of varying sizes. In this study, we propose a novel network to improve the segmentation accuracy of lung nodules. The network integrates adaptive large kernel attention (ALK) blocks, employing multiple convolutional layers with variously sized convolutional kernels and expansion rates to extract multi-scale features. A dynamic selection mechanism is also introduced to aggregate the multi-scale features obtained from variously sized convolutional kernels based on selection weights. Based on this, we propose a lightweight convolutional neural network with large convolutional kernels, called ALKU-Net, which integrates the ALKA module in a hierarchical encoder and adopts a U-shaped decoder to form a novel architecture. ALKU-Net efficiently utilizes the multi-scale large receptive field and enhances the model perception capability through spatial attention and channel attention. Extensive experiments demonstrate that our method outperforms other state-of-the-art models on the public dataset LUNA-16, exhibiting considerable accuracy in the lung nodule segmentation task.

Fire-Net: Rapid Recognition of Forest Fires in UAV Remote Sensing Imagery Using Embedded Devices

Forest fires pose a catastrophic threat to Earth’s ecology as well as threaten human beings. Timely and accurate monitoring of forest fires can significantly reduce potential casualties and property damage. Thus, to address the aforementioned problems, this paper proposed an unmanned aerial vehicle (UAV) based on a lightweight forest fire recognition model, Fire-Net, which has a multi-stage structure and incorporates cross-channel attention following the fifth stage. This is to enable the model’s ability to perceive features at various scales, particularly small-scale fire sources in wild forest scenes. Through training and testing on a real-world dataset, various lightweight convolutional neural networks were evaluated on embedded devices. The experimental outcomes indicate that Fire-Net attained an accuracy of 98.18%, a precision of 99.14%, and a recall of 98.01%, surpassing the current leading methods. Furthermore, the model showcases an average inference time of 10 milliseconds per image and operates at 86 frames per second (FPS) on embedded devices.

MobileAmcT: A Lightweight Mobile Automatic Modulation Classification Transformer in Drone Communication Systems

With the rapid advancement of wireless communication technology, automatic modulation classification (AMC) plays a crucial role in drone communication systems, ensuring reliable and efficient communication in various non-cooperative environments. Deep learning technology has demonstrated significant advantages in the field of AMC, effectively and accurately extracting and classifying modulation signal features. However, existing deep learning models often have high computational costs, making them difficult to deploy on resource-constrained drone communication devices. To address this issue, this study proposes a lightweight Mobile Automatic Modulation Classification Transformer (MobileAmcT). This model combines the advantages of lightweight convolutional neural networks and efficient Transformer modules, incorporating the Token and Channel Conv (TCC) module and the EfficientShuffleFormer module to enhance the accuracy and efficiency of the automatic modulation classification task. The TCC module, based on the MetaFormer architecture, integrates lightweight convolution and channel attention mechanisms, significantly improving local feature extraction efficiency. Additionally, the proposed EfficientShuffleFormer innovatively improves the traditional Transformer architecture by adopting Efficient Additive Attention and a novel ShuffleConvMLP feedforward network, effectively enhancing the global feature representation and fusion capabilities of the model. Experimental results on the RadioML2016.10a dataset show that compared to MobileNet-V2 (CNN-based) and MobileViT-XS (ViT-based), MobileAmcT reduces the parameter count by 74% and 65%, respectively, and improves classification accuracy by 1.7% and 1.09% under different SNR conditions, achieving an accuracy of 62.93%. This indicates that MobileAmcT can maintain high classification accuracy while significantly reducing the parameter count and computational complexity, clearly outperforming existing state-of-the-art AMC methods and other lightweight deep learning models.

A Lightweight Convolutional Spiking Neural Network for Fires Detection Based on Acoustics

A Lightweight Convolutional Spiking Neural Network for Fires Detection Based on Acoustics