Feature Extraction Part Research Articles

Improved YOLOv7 Electric Work Safety Belt Hook Suspension State Recognition Algorithm Based on Decoupled Head

Safety is the eternal theme of power systems. In view of problems such as time-consuming and poor real-time performance in the correct use of seat belt hooks by manual supervision operators in the process of power operation, this paper proposes an improved YOLOv7 seat belt hook suspension state recognition algorithm. Firstly, the feature extraction part of the YOLOv7 backbone network is improved, and the M-Spatial Pyramid Pooling Concurrent Spatial Pyramid Convolution (M-SPPCSPC) feature extraction module is constructed to replace the Spatial Pyramid Pooling Concurrent Spatial Pyramid Convolution (SPPCSPC) module of the backbone network, which reduces the amount of computation and improves the detection speed of the backbone network while keeping the sensory field of the backbone network unchanged. Second, a decoupled head, which realizes the confidence and regression frames separately, is introduced to alleviate the negative impact of the conflict between the classification and regression tasks, consequently improving the network detection accuracy and accelerating the network convergence. Ultimately, a dynamic non-monotonic focusing mechanism is introduced in the output layer, and the Wise Intersection over Union (WioU) loss function is used to reduce the competitiveness of high-quality anchor frames while reducing the harmful gradient generated by low-quality anchor frames, which ultimately improves the overall performance of the detection network. The experimental results show that the mean Average Precision (mAP@0.5) value of the improved network reaches 81.2%, which is 7.4% higher than that of the original YOLOv7, therefore achieving better detection results for multiple-state recognition of hooks.

TF-REF-RNN: Time-Frequency and Reference Signal Feature Fusion Recurrent Neural Network for Underwater Backscatter Signal Separation

Underwater wireless sensor networks play an important role in exploring the oceans as part of an integrated space–air–ground–ocean network. Because underwater energy is limited, the equipment’s efficiency is significantly impacted by the battery duration. Underwater backscatter technology does not require batteries and has significant potential in positioning, navigation, communication, and sensing due to its passive characteristics. However, underwater backscatter signals are susceptible to being swamped by the excitation signal. Additionally, the signals from different reflection signals share the same frequency and overlap, and contain fewer useful features, leading to significant challenges in detection. In order to solve the above problems, this paper proposes a recurrent neural network that introduces time-frequency and reference signal features for underwater backscatter signal separation (TF-REF-RNN). In the feature extraction part, we design an encoder that introduces time-frequency domain features to learn more about the frequency details. Additionally, to improve performance, we designed a separator that incorporates the reference signal’s pure global information features. The proposed TF-REF-RNN network model achieves metrics of 28.55 dB SI-SNRi and 19.51 dB SDRi in the dataset that includes shipsEar noise data and underwater simulated backscatter signals, outperforming similar classical methods.

Multimodal Fusion for Talking Face Generation Utilizing Speech-Related Facial Action Units

Talking face generation is to synthesize a lip-synchronized talking face video by inputting an arbitrary face image and corresponding audio clips. The current talking face model can be divided into four parts: visual feature extraction, audio feature processing, multimodal feature fusion, and rendering module. For the visual feature extraction part, existing methods face the challenge of complex learning task with noisy features, this article introduces an attention-based disentanglement module to disentangle the face into Audio-face and Identity-face using speech-related facial action unit (AU) information. For the multimodal feature fusion part, existing methods ignore not only the interaction and relationship of cross-modal information but also the local driving information of the mouth muscles. This study proposes a novel generative framework that incorporates a dilated non-causal temporal convolutional self-attention network as a multimodal fusion module to enhance the learning of cross-modal features. The proposed method employs both audio- and speech-related facial AUs as driving information. Speech-related AU information can facilitate more accurate mouth movements. Given the high correlation between speech and speech-related AUs, we propose an audio-to-AU module to predict speech-related AU information. Finally, we present a diffusion model for the synthesis of talking face images. We verify the effectiveness of the proposed model on the GRID and TCD-TIMIT datasets. An ablation study is also conducted to verify the contribution of each component. The results of quantitative and qualitative experiments demonstrate that our method outperforms existing methods in terms of both image quality and lip-sync accuracy. Code is available at https://mftfg-au.github.io/Multimodal_Fusion/.

Lung-YOLO: Multiscale feature fusion attention and cross-layer aggregation for lung nodule detection

ObjectiveLung cancer is a significant public health problem worldwide, and its mortality and morbidity rates are among the highest of cancers. At the same time, early diagnosis of nodules can significantly improve the survival rate of patients. Therefore, this paper proposes the Lung-YOLO algorithm for lung CT image detection based on YOLOv6. MethodsFirst, to enable the network to detect nodules of different sizes and to minimize missed detections, we introduce the Multi-scale Dual-branch Attention (MSDA) mechanism in the feature extraction part of the network. The input features undergo continual dilation convolutions, effectively establish remote dependencies, and the fused multiscale contextual information expands the receptive field while enhancing the model’s ability to detect targets of different sizes, which not only contains precise category and location information but also enables the allocation of attentional weights to generate more pixel-level attention. Then, the fused features access the dual-branch attention module to shift the model’s attention to the target nodules, and the composed dual-branch structure captures cross-dimensional interactions and realizes inter-dimensional dependencies, effectively improving the detection performance. Second, during the feature transfer process, the original Bidirectional Feature Pyramid structure (RepBiFPAN) suffers from the loss of detailed information such as texture and color, making it challenging to localize target nodules accurately. To address this, we propose the Cross-layer Aggregation Module (CLAM), by cross-layer aggregating the multi-level feature layer of the backbone with the multi-level detection layer of the head, which preserves the multi-level fine-grained information that may be lost during the feature transfer process, which is crucial for the detection of small targets. Finally, the module proposed in this paper can be easily incorporated into any detection framework for plug-and-play. ResultsOur method achieves accuracy, precision, recall, and mAP of 97.5 %, 96.5 %, 96.9 %, and 97.9 % on the LUNA16 dataset, and 95.1 %, 94.3 %, 93.4 %, and 95.9 % on the LIDC-IDRI dataset, respectively, surpassing many existing state-of-the-art detection methods. Moreover, the inference speed is 22.8 ms and 28.4 ms per image with 30.6 M parameters, respectively.

Multi-level interactive fusion network based on adversarial learning for fusion classification of hyperspectral and LiDAR data

With the growing awareness of the limitations of unimodal data, research on joint classification using multimodal remote sensing data has garnered increasing attention. However, existing methods still have certain deficiencies in dealing with feature extraction and feature fusion of multimodal remote sensing data. In order to obtain more accurate classification results, it is crucial to fully exploit and utilize the complementary information in multimodal remote sensing data. In this paper, a multi-level interactive fusion network based on adversarial learning (MLIF-AL) is proposed for hyperspectral image (HSI) and light detection and ranging (LiDAR) data fusion classification. First, the feature extraction part based on generative adversarial network (GAN) utilizes adversarial training between the generator and the discriminator to guide the network to learn more discriminative and distinguished feature representations. Meanwhile, a cross-modal interactive information extraction (CMIIE) module is also designed for the generator encoding part to promote the interaction between multimodal data from a global perspective and realize the full utilization of multimodal complementary information. In addition, in the multi-level feature fusion classification (MLFFC) part, the complementarities between the features learned by the neural networks at each layer are comprehensively utilized to obtain higher-level multimodal fusion semantic information. Finally, the adversarial loss and classification loss are combined for network training. Extensive experiments on three commonly used HSI and LiDAR datasets and comparisons with state-of-the-art methods demonstrate the effectiveness and superiority of the model.

IEA-Net: Internal and External Dual-Attention Medical Segmentation Network with High-Performance Convolutional Blocks.

Currently, deep learning is developing rapidly in the field of image segmentation, and medical image segmentation is one of the key applications in this field. Conventional CNN has achieved great success in general medical image segmentation tasks, but it has feature loss in the feature extraction part and lacks the ability to explicitly model remote dependencies, which makes it difficult to adapt to the task of human organ segmentation. Although methods containing attention mechanisms have made good progress in the field of semantic segmentation, most of the current attention mechanisms are limited to a single sample, while the number of samples of human organ images is large, ignoring the correlation between the samples is not conducive to image segmentation. In order to solve these problems, an internal and external dual-attention segmentation network (IEA-Net) is proposed in this paper, and the ICSwR (interleaved convolutional system with residual) module and the IEAM module are designed in this network. The ICSwR contains interleaved convolution and hopping connection, which are used for the initial extraction of the features in the encoder part. The IEAM module (internal and external dual-attention module) consists of the LGGW-SA (local-global Gaussian-weighted self-attention) module and the EA module, which are in a tandem structure. The LGGW-SA module focuses on learning local-global feature correlations within individual samples for efficient feature extraction. Meanwhile, the EA module is designed to capture inter-sample connections, addressing multi-sample complexities. Additionally, skip connections will be incorporated into each IEAM module within both the encoder and decoder to reduce feature loss. We tested our method on the Synapse multi-organ segmentation dataset and the ACDC cardiac segmentation dataset, and the experimental results show that the proposed method achieves better performance than other state-of-the-art methods.

TROG: a fast and robust scene-matching algorithm for geo-referenced images

ABSTRACT In a GNSS-denied/challenged environment, the scene-matching navigation system (SMNS) is a vital autonomous technology for unmanned aerial vehicles (UAV). This paper proposes a novel template-matching framework for multimodal images since UAV navigation requires precision and real-time performance while utilizing onboard computers with low computational power. Specifically, first, the local descriptor is extracted to form a pixel-wise feature representation of an image. Then, the Fast Fourier Transform is applied to measure the similarity based on the feature representation. In the feature extraction part, a gradient-like pixel-level feature descriptor is designed, which is reconstructed by weighting it according to the gradient-like angles, named the three-dimensional reconstruction oriented gradient-like (TROG) descriptor. An optimized similarity measurement template is introduced in the matching part, which improves the traditional feature-based similarity measurement algorithm defined using Fast Fourier Transform (FFT) in the frequency domain. This strategy eliminates redundant computations during the matching process. To verify the effectiveness of the proposed algorithm, satellite imagery data from Google Earth are used as a reference images, and sensed images (including optical, IR, SAR, and Hyperspectral) are captured by UAV and satellites for image matching to testify to the registration accuracy, robustness, and computational efficiency. The experiment demonstrates that TROG is accurate, robust, and attains high real-time performance, making it applicable to UAV navigation and positioning. Additionally, field-flight experiments evaluate scene-matching navigation under satellite-denied conditions and low computational power conditions for UAVs, demonstrating that our scene-matching navigation system can achieve precise positioning with a positioning error of less than 1.637 m, which is comparable to satellite/inertial navigation systems. The experimental results from outdoor flight experiments highlight the value of our proposed algorithm in engineering applications under satellite denial conditions.

A high speed inference architecture for multimodal emotion recognition based on sparse cross modal encoder

In recent years, multimodal emotion recognition models are using pre-trained networks and attention mechanisms to pursue higher accuracy, which increases the training burden and slows down the training and inference speed. In order to strike a balance between speed and accuracy, this paper proposes a speed-optimized multimodal emotion recognition architecture for speech and text emotion recognition. In the feature extraction part, a lightweight residual graph convolutional network (ResGCN) is selected as the speech feature extractor, and an efficient RoBERTa pre-trained network is used as the text feature extractor. Then, an algorithm complexity-optimized sparse cross-modal encoder (SCME) is proposed and used to fuse these two types of features. Finally, a new gated fusion module (GF) is used to weight multiple results and input them into a fully connected layer (FC) for classification. The proposed method is tested on the IEMOCAP dataset and the MELD dataset, achieving weighted accuracies (WA) of 82.4% and 65.0%, respectively. This method achieves higher accuracy than the listed methods while having an acceptable training and inference speed.

Research on efficient classification algorithm for coal and gangue based on improved MobilenetV3-small

ABSTRACT Aiming at the problems of insufficient attention to spatial information, long classification training time, and high model complexity in current gangue image classification algorithms, a lightweight fast recognition model for gangue based on improved MobileNetV3-small is proposed. First, we optimize the feature extraction part by introducing the CeLU activation function to alleviate the neuron death and gradient vanishing problems, thus improving the performance of the model. Second, Bneck is further optimized using an improved Triplet Attention Module to achieve almost parameter-free spatial and channel dimension interactions, which drastically reduces the complexity of the model. Finally, we construct a new Efficient Last Stage structure, which improves the model performance while successfully reducing the computation and model size by replacing the traditional convolution with Ghost Module and introducing Mix-pool as a pooling layer. In addition, the effectiveness of each component was fully demonstrated through ablation experiments, visualization analysis, and comparative experiments. The experimental results show that training the homemade gangue dataset using the improved MobilenetV3-small model improves Accuracy and F1-Score on the validation set by 0.48% and 0.42%, respectively, compared to the original network, and in terms of model complexity, we can see that FLOPs, number of parameters, and Weight size are reduced by 2.8%, 33.0%, and 32.3%, respectively, and FPS reaches 28 frames per second. The improved algorithm greatly reduces the model complexity and decreases the dependence on hardware devices while improving the classification accuracy and anti-interference capability. This provides an important reference basis for deploying automated underground coal gangue sorting, which helps to realize intelligent integrated mining.

A single-frame infrared small target detection method based on joint feature guidance

Single-frame infrared small target detection is affected by the low image resolution and small target size, and is prone to the problems of small target feature loss and positional offset during continuous downsampling; at the same time, the sparse features of the small targets do not correlate well with the global-local linkage of the background features. To solve the above problems, this paper proposes an efficient infrared small target detection method. First, this paper incorporates BlurPool in the feature extraction part, which reduces the loss and positional offset of small target features in the process of convolution and pooling. Second, this paper designs an interactive attention deep feature fusion module, which acquires the correlation information between the target and the background from a global perspective, and designs a compression mechanism based on deep a priori knowledge, which reduces the computational difficulty of the self-attention mechanism. Then, this paper designs the context local feature enhancement and fusion module, which uses deep semantic features to dynamically guide shallow local features to realize enhancement and fusion. Finally, this paper proposes an edge feature extraction module for shallow features, which utilizes the complete texture and location information in the shallow features to assist the network to initially locate the target position and edge shape. Numerous experiments show that the method in this paper significantly improves nIoU, F1-Measure and AUC on IRSTD-1k Datasets and NUAA-SIRST Datasets.

Pulmonary Nodule Segmentation Network Based on Res Select Kernel Contextual U-Net

Abstract U-Net network is widely used in the field of medical image segmentation. The automatic segmentation and detection of lung nodules can help in the early detection of lung cancer. Therefore, in this paper, to solve the problems of small proportion of nodules in computer tomography (CT) images, complex features, and insufficient segmentation accuracy, an improved U-Net network based on residual network and attention mechanism was proposed. The feature extraction part of Res select Kernel Contextual U-Net (RkcU-Net) network is based on Res2net, a variant of Resnet, and on which a feature extraction module with automatic selection of convolution kernel size is designed to perform multiscale convolution inside the feature layer to form perceptual fields of different sizes. This module selects the appropriate convolution kernel size to extract lung nodule features in the face of different fine-grained lung nodules. Second, the contextual supplementary (CS) block is designed to use the information of adjacent upper and lower layers to correct for the upper layer features, eliminating the discrepancy in the fusion of features at different levels. In this paper, the LUNA16 dataset was selected as the basis for lung nodule segmentation experiments. The method used in this dataset can achieve an intersection ratio (IoU) of 80.59% and a dice similarity coefficient (DSC) score of 89.25%. The network effectively improves the accuracy of lung nodule segmentation compared with other models. The results show that the method enhances the feature extraction ability of the network and improves the segmentation effect. In addition, the contribution of jump connections to information recovery should be noted.

Handwritten Character Recognition using Deep Learning Algorithm with Machine Learning Classifier

Handwritten character recognition is a problem that has been worked on for many mainstream languages. Handwritten letter recognition has been proven to achieve promising results. Several studies using deep learning models have been conducted to achieve better accuracies. In this paper, the authors conducted two experiments on the EMNIST Letters dataset: Wavemix-Lite and CoAtNet. The Wavemix-Lite model uses Two-Dimensional Discrete Wavelet Transform Level 1 to reduce the parameters and speed up the runtime. The CoAtNet is a combined model of CNN and Visual Transformer where the image is broken down into fixed-size patches. The feature extraction part of the model is used to embed the input image into a feature vector. From those two models, the authors hooked the value of the features of the Global Average Pool layer using EMNIST Letters data. The features hooked from the training results of the two models, such as SVM, Random Forest, and XGBoost models, were used to train the machine learning classifier. The experiments conducted by the authors show that the best machine-learning model is the Random Forest, with 96.03% accuracy using the Wavemix-Lite model and 97.90% accuracy using the CoAtNet model. These results showcased the benefit of using a machine learning model for classifying image features that are extracted using a deep learning model.

TSIC-CLIP: Traffic Scene Image Captioning Model Based on Clip

Image captioning in traffic scenes presents several challenges, including imprecise caption generation, lack of personalization, and an unwieldy number of model parameters. We propose a new image captioning model for traffic scenes to address these issues. The model incorporates an adapter-based fine-tuned feature extraction part to enhance personalization and a caption generation module using global weighted attention pooling to reduce model parameters and improve accuracy. The proposed model consists of four main stages. In the first stage, the Image-Encoder extracts the global features of the input image and divides it into nine sub-regions, encoding each sub-region separately. In the second stage, the Text-Encoder encodes the text dataset to obtain text features. It then calculates the similarity between the image sub-region features and encoded text features, selecting the text features with the highest similarity. Subsequently, the pre-trained Faster RCNN model extracts local image features. The model then splices together the text features, global image features, and local image features to fuse the multimodal information. In the final stage, the extracted features are fed into the Captioning model, which effectively fuses the different features using a novel global weighted attention pooling layer. The Captioning model then generates natural language image captions. The proposed model is evaluated on the MS-COCO dataset, Flickr 30K dataset, and BUUISE-Image dataset, using mainstream evaluation metrics. Experiments demonstrate significant improvements across all evaluation metrics on the public datasets and strong performance on the BUUISE-Image traffic scene dataset.

Research on Handwriting Text Generation Algorithm Based on Generative Adversarial Network

The main task of this paper is to study the handwriting text generation method based on deep learning. Through understanding the development status of the research, It can be found that the current research on the generation of different handwriting styles still has some obvious defects, such as the need for manual intervention in character segmentation, failure to capture the global handwriting style, style collapse, failure to generate arbitrary length characters, and text. Finally, this paper proposes a handwritten text generation algorithm combining advantages of convolutional network and Transformer. Specifically, this paper first constructs a lightweight backbone network and uses lightweight MobileNetv3 network as the backbone network to realize feature extraction of input images. The Efficient Channel Attention module is introduced to replace the SE attention module of MobileNetv3, which makes the network pay more attention to the global and local style features of handwritten images. In the feature extraction part, the network reduces the number of parameters, calculation amount and video memory occupation. It can also extract rich feature information. The FID and KID indexes of this algorithm obtained 20.28 and 9.07×10-3 respectively, and the generation effect of handwritten pictures was excellent, which could effectively imitate the writing style of writers.

3D Point Cloud Object Detection Method Based on Multi-Scale Dynamic Sparse Voxelization.

Perception plays a crucial role in ensuring the safety and reliability of autonomous driving systems. However, the recognition and localization of small objects in complex scenarios still pose challenges. In this paper, we propose a point cloud object detection method based on dynamic sparse voxelization to enhance the detection performance of small objects. This method employs a specialized point cloud encoding network to learn and generate pseudo-images from point cloud features. The feature extraction part uses sliding windows and transformer-based methods. Furthermore, multi-scale feature fusion is performed to enhance the granularity of small object information. In this experiment, the term "small object" refers to objects such as cyclists and pedestrians, which have fewer pixels compared to vehicles with more pixels, as well as objects of poorer quality in terms of detection. The experimental results demonstrate that, compared to the PointPillars algorithm and other related algorithms on the KITTI public dataset, the proposed algorithm exhibits improved detection accuracy for cyclist and pedestrian target objects. In particular, there is notable improvement in the detection accuracy of objects in the moderate and hard quality categories, with an overall average increase in accuracy of about 5%.

Performance analysis of genetically optimized 1D-convolutional neural network architecture for rotor system fault detection and diagnosis

Rotor mass imbalance is a critical problem in industries, which can lead to a breakdown or a catastrophic failure if unattended. Hence, Industry 4.0 uses deep learning approaches like convolutional neural network (CNN) for rotor fault detection and diagnosis of Industrial machines and turbomachinery where multifunctional structures are used. In machinery fault diagnosis, manual tuning of CNN hyperparameters is widely used, which is a tedious, time-consuming, and challenging task to achieve effective feature extraction. Additionally, overfitting of CNN during its training is an obstacle to achieving higher prediction accuracy. Hence, to address these issues, this research focuses on multi-class mass imbalance fault diagnosis using genetically optimized deep learning architecture of 1D-CNN to achieve higher diagnosis capability. The performance of the proposed methodology is evaluated through various case studies using experimental data from the in-house developed test rig. The test results are compared against various 1D-CNN architectures in which the hyperparameters and effective dropout layer positioning are genetically optimized. Also, the performance is benchmarked against standard machine learning algorithms. The results show that genetic algorithm (GA)-optimized 1D-CNN with dropout achieves highest fault prediction accuracy of 97.47% with reduced depth of CNN (with three convolution layers) compared to 84.16% of manually tunned CNN architecture (with seven convolutional layers) and outperformed standard machine learning algorithms. Best drop positioning (end of feature extraction part) reduced the learnable parameters to 95.9% with the highest prediction accuracy of 97.47% compared to adding dropout at all the CNN layers. The proposed GA-optimized 1D-CNN with effective dropout positioning eliminates human intervention and reduces the depth of CNN architecture; this, in turn, reduces computational load and time by reducing learnable parameters of CNN (network weights) with the highest prediction accuracy. Hence, the proposed approach shows promise in enhancing the performance and contributing to the advancement of 1D-CNN for rotor system fault detection and diagnosis.

A novel interpretable deep transfer learning combining diverse learnable parameters for improved T2D prediction based on single-cell gene regulatory networks

Accurate deep learning (DL) models to predict type 2 diabetes (T2D) are concerned not only with targeting the discrimination task but also with learning useful feature representation. However, existing DL tools are far from perfect and do not provide appropriate interpretation as a guideline to explain and promote superior performance in the target task. Therefore, we provide an interpretable approach for our presented deep transfer learning (DTL) models to overcome such drawbacks, working as follows. We utilize several pre-trained models including SEResNet152, and SEResNeXT101. Then, we transfer knowledge from pre-trained models via keeping the weights in the convolutional base (i.e., feature extraction part) while modifying the classification part with the use of Adam optimizer to deal with classifying healthy controls and T2D based on single-cell gene regulatory network (SCGRN) images. Another DTL models work in a similar manner but just with keeping weights of the bottom layers in the feature extraction unaltered while updating weights of consecutive layers through training from scratch. Experimental results on the whole 224 SCGRN images using five-fold cross-validation show that our model (TFeSEResNeXT101) achieving the highest average balanced accuracy (BAC) of 0.97 and thereby significantly outperforming the baseline that resulted in an average BAC of 0.86. Moreover, the simulation study demonstrated that the superiority is attributed to the distributional conformance of model weight parameters obtained with Adam optimizer when coupled with weights from a pre-trained model.

A food quality detection method based on electronic nose technology

Food quality detection is of great importance for human health and industrial production. Currently, the common detection methods are difficult to achieve the need for fast, accurate, and non-destructive detection. In this work, an electronic nose (E-nose) detection method based on the combination of convolutional neural network combined with wavelet scattering network (CNN-WSN) and improved seahorse optimizes kernel extreme learning machine (ISHO-KELM) is proposed for identifying the quality level of a variety of food products. In the feature extraction part, the abstract features of CNN are fused with the scattering features of WSN, and the obtained CNN-WSN fusion features can characterize the original information of the food quality effectively. In the classifier design and decision-making section, chaotic mapping is used to initialize the population in the seahorse optimisation algorithm (SHO), avoiding the problem that SHO may fall into local optimal solutions. The kernel parameters and regularisation coefficients of the KELM model were then optimized by improving the locomotion, predation, and reproduction behaviors of the hippocampal populations, which solved the problem of the difficult selection of the key parameters in the model, and thus improved the accuracy and generalization of the overall model. To validate the effectiveness of the proposed food quality detection model, the E-nose system was first built and milk quality data were collected independently, and then tested on two publicly available food quality datasets as well as a self-collected milk quality dataset, respectively. The experimental results show that the food quality detection method proposed in this work has good quality assessment effect on different datasets.

Plant Diseased Lesion Image Segmentation and Recognition Based on Improved Multi-Scale Attention Net

Fallen leaf disease can lead to a decrease in leaf area, a decrease in photosynthetic products, insufficient accumulation of fruit sugar, poor coloring and flavor, and a large number of fruits developing sunburn. To address the aforementioned issue, this article introduces a deep learning algorithm designed for the segmentation and recognition of agricultural disease images, particularly those involving leaf lesions. The essence of this algorithm lies in enhancing the Multi-scale Attention Net (MA-Net) encoder and attention mechanism to improve the model’s performance when processing agricultural disease images. Firstly, an analysis was conducted on MA-Net, and its limitations were identified. Compared to res-block, Mix Vision Transformer (MiT) consumes relatively less time during the training process, can better capture global and contextual information in images, and has better robustness and scalability. Then, the feature extraction parts of different networks were used as encoders to join the MA-Net network. Compared to a Position-wise Attention Block (PAB), which has higher computational complexity and requires a larger amount of computing resources, Effective Channel Attention net (ECANet) reduces the number of model parameters and computation by learning the correlation between channels, as well as having a better denoising ability. The experimental results show that the proposed solution has high accuracy and stability in agricultural disease image segmentation and recognition. The mean Intersection over Union (mIoU) is 98.1%, which is 0.2% higher than traditional MA-Net; Dice Loss is 0.9%, which is 0.1% lower than traditional MA-Net.

Auxiliary input-enhanced siamese neural network: A robust tool wear prediction framework with improved feature extraction and generalization ability

Tool wear monitoring is essential for automated and resilient manufacturing, as it can prevent catastrophic failures caused by severe wear on cutting edges during machining. The conventional tool wear monitoring approaches depend on features extracted from signals, which require sensitive signals and consistent tool wear. In practical scenarios, however, neither the sensitivity of collected signals to the tool wear status nor the consistency of the actual tool wear evolution is hard to meet the requirement of the tool condition monitoring algorithm, which greatly limit the wide spread of its industrial applications. To overcome this challenge, we propose an Auxiliary Input-enhanced Siamese Neural Network (AISNN) framework by incorporating a Siamese structure into the feature extraction part of a convolutional neural network (CNN), and introducing an auxiliary input to its nonlinear regression part. The Siamese structure, instead of extracting features directly from signals, distinguishes the difference between the features extracted from signals of the examined cut and the first cut, and uses this difference as the indicator of tool wear status. Moreover, the auxiliary input provides an additional feature that has heavy dependence on the tool wear, which enables the model learning the general wear evolution of the examined cutting tool. The effectiveness of the proposed AISNN framework is verified in a set of milling experiments where input signal is insensitive to the flank wear of cutting tool and different tools’ wear evolution exhibits obvious inconsistency. Compared to the traditional CNN, the proposed AISNN significantly improves the accuracy on the verification set from 63% to 95% and on the testing set from 50% to 81%. The results demonstrate that the AISNN framework achieves significant improvement in feature extraction and generalization ability. The proposed AISNN, as a universal framework, can empower most existing deep learning-based tool wear prediction methods, enhancing their robustness in handling insensitive signals and inconsistent wear evolution and thereby promoting more industrial applications.