Parallel Branches Research Articles

Optimizing flotation froth image segmentation via parallel branch network and hybrid loss supervision

Flotation is a crucial technology for fine coal separation, and accurately acquiring bubble size information during the flotation process is essential for monitoring flotation conditions and achieving intelligent control. However, existing semantic segmentation models encountered issues with boundary disconnection when segmenting flotation bubbles, resulting in deviations between the extracted bubble sizes and their true values. To address the aforementioned challenges, a semantic segmentation model was proposed to maintain high-resolution feature maps throughout the network by designing a parallel branch network structure. Additionally, a ConvTranspose module was proposed to preserve the detailed feature information of images while gradually enhancing the resolution of feature maps. In the model training phase, a hybrid loss function combining pixel classification loss with shape similarity loss was proposed to alleviate the sample imbalance problem caused by the substantial difference in the number of pixels between bubble boundaries and the interior of bubbles. Moreover, since traditional semantic segmentation evaluation metrics, such as MIoU, lack a mechanism for measuring bubble boundary continuity and cannot effectively penalize the problem of boundary disconnection, this paper proposed a new evaluation method for assessing the segmentation performance of flotation froth images. To comprehensively evaluate the effectiveness of the proposed method, this paper conducted tests using flotation froth images collected from actual production processes. Compared with existing methods, the segmentation model proposed in this paper exhibited clear superiority in mitigating the problem of bubble boundary disconnection. The prediction error for the number of bubbles was 6.38 %, which is significantly better than other methods.

Brain-Inspired Architecture for Spiking Neural Networks.

Spiking neural networks (SNNs), using action potentials (spikes) to represent and transmit information, are more biologically plausible than traditional artificial neural networks. However, most of the existing SNNs require a separate preprocessing step to convert the real-valued input into spikes that are then input to the network for processing. The dissected spike-coding process may result in information loss, leading to degenerated performance. However, the biological neuron system does not perform a separate preprocessing step. Moreover, the nervous system may not have a single pathway with which to respond and process external stimuli but allows multiple circuits to perceive the same stimulus. Inspired by these advantageous aspects of the biological neural system, we propose a self-adaptive encoding spike neural network with parallel architecture. The proposed network integrates the input-encoding process into the spiking neural network architecture via convolutional operations such that the network can accept the real-valued input and automatically transform it into spikes for further processing. Meanwhile, the proposed network contains two identical parallel branches, inspired by the biological nervous system that processes information in both serial and parallel. The experimental results on multiple image classification tasks reveal that the proposed network can obtain competitive performance, suggesting the effectiveness of the proposed architecture.

Influence of external AC transverse magnetic field on air arc oscillation patterns and its application in DC forced current zero techniques

Abstract Direct current (DC) circuit breakers are essential for maintaining the stability of the electric energy transmission and safeguarding the power equipment from the damage caused by the fault currents in DC power systems. The characteristics of the switching arc can be utilized in DC forced current zero techniques. This paper investigates the influence of an external alternating current (AC) transverse magnetic field on the oscillation characteristics of air arc. By confining the arc between two insulating walls and applying a transverse AC magnetic field, the arc exhibits "bidirectional oscillation" and "unidirectional oscillation" modes. The effects of magnetic field amplitude and frequency on the air arc oscillation characteristics are analyzed through experiments, from the aspects of the arc voltages and arc motions. It can be found that an external AC magnetic field can stabilize the arc voltage at a specific oscillation frequency. By connecting an inductor-capacitor (LC) branch in parallel with the arc and setting the LC branch's resonant frequency equal to the arc voltage oscillation frequency, a current resonance process between the arc and the LC branch can be achieved. This resonance facilitates the creation of the arc current zero-crossing point. A test platform based on the arc oscillation characteristics is established for current interruption experiments. For system currents of 1 kA and 3 kA, the arc current zero-crossing time is controlled within 0.41 ms after applying the external AC transverse magnetic field. The experimental results verify the rationality of the proposed DC breaking scheme.

A clothing classification algorithm based on self-attention information compensation

Traditional content-based clothing classification has high requirements for image features. When there are many clothing styles, its accuracy is difficult to meet the application requirements of clothing classification. A parallel self-attention classification network based on deep learning method is proposed. The network adds a parallel self-attention compensation branch on the basis of ResNet50. This branch can improve the feature extraction quality in clothing classification tasks and gradually supplement the shallow detail information missing in the deep network. A comparative experiment was carried out on the DeepFashion dataset, and the experimental results proved the effectiveness of this method.

Local-global feature fusion network for hyperspectral image classification

ABSTRACT Hyperspectral images (HSIs) are rich in spatial and spectral information, making them crucial for accurate classification. While existing convolutional neural network (CNN) methods have shown promise in HSI classification, they primarily focus on local features and struggle with establishing long-range dependencies. To address these limitations, we propose a Local-Global Feature Fusion Network (LGFFN). LGFFN effectively learns joint local-global information from HSIs. Firstly, we employ a group-general convolution parallel module (GGCPM) to extract both local and global spectral features. Next, we utilize parallel branches: a CNN branch for capturing fine-grained spatial information and an improved transformer encoder (ITE) branch for capturing long-distance spatial-spectral features and exploiting semantic information. Finally, we elegantly integrate the diverse features obtained from these branches to obtain classification results. We validate our method on four datasets and compare it with six state-of-the-art hyperspectral classification methods. Our experimental results demonstrate that LGFFN achieves high classification accuracy, outperforming existing methods.

Spatio-temporal fusion with motion masks for the moving small target detection from remote-sensing videos

Moving small target detection is an eye-catching research topic, full of great challenges in object detection field. Its primary difficulties exist in (i) the very small size proportion of the target region in background images, (ii) the quite weak contrast to background and (iii) the perception of slight target motion. Currently-existing detection methods mainly utilize the image features from spatial domain. This kind of features is quite weak to small targets. More new feature kinds need to be concerned. In view of these, besides traditional image clues, motion features are currently attracting more and more attention in small target detection. In order to promote detection performance, this paper proposes a Spatio-Temporal Fusion Network (STFNet) with two parallel feature extraction branches for detecting moving small targets. In this network, one branch is designed to capture the traditional semantic features from spatial domain, and the other is devised to perceive the slight target motion features hidden in temporal domain. Meanwhile, to optimize the extraction of spatio-temporal features, a group of motion masks is specially designed to guide feature extractors to pay more precise attention to the positions where small targets appear. Moreover, we design a new bridging module to enhance the cross-domain and cross-scale fusion of spatio-temporal features. Through extensive comparison and ablation experiments on seven sub-datasets, it is proved that our new STFNet is effective and it is obviously superior to the compared ones in detecting moving small targets from satellite sequence images. It achieves a Precision of 0.93, an mAP50 of 71.10% and an F1 of 0.84, evidently higher than existing state-of-the-art methods. Our codes are available at https://github.com/UESTC-nnLab/STF.

M-adapter: Multi-level image-to-video adaptation for video action recognition

With the growing size of visual foundation models, training video models from scratch has become costly and challenging. Recent attempts focus on transferring frozen pre-trained Image Models (PIMs) to video fields by tuning inserted learnable parameters such as adapters and prompts. However, these methods require saving PIM activations for gradient calculations, leading to limited savings of GPU memory. In this paper, we propose a novel parallel branch that adapts the multi-level outputs of the frozen PIM for action recognition. It avoids passing gradients through the PIMs, thus naturally owning much lower GPU memory footprints. The proposed adaptation branch consists of hierarchically combined multi-level output adapters (M-adapters), comprising a fusion module and a temporal module. This design digests the existing discrepancies between the pre-training task and the target task with lower training costs. We show that when using larger models or on scenarios with higher demands for temporal modelling, the proposed method performs better than those with the full-parameter tuning manner. Finally, despite only tuning fewer parameters, our method achieves superior or comparable performance against current state-of-the-art methods.

Interior Crisis Route to Extreme Events in a Memristor-Based 3D Jerk System

In dynamical systems, events that deviate significantly from usual or expected behavior are referred to as extreme events. This paper investigates the mechanism of extreme event generation in a 3D jerk system based on a generalized memristive device. In addition, regions of coexisting parallel bifurcation branches are explored as a way of investigating the multistability of the memristive system. The system is examined using bifurcation diagrams, Lyapunov exponents, time series, probability density functions of events, and inter-event intervals. It is found that extreme events occur via a period-doubling route and are due to an interior crisis that manifests itself as a sudden shift from low-amplitude to high-amplitude oscillations. Multistability is also identified when both control parameters and initial values are modified. Finally, an analog circuit based on the memristive jerk system is designed and experimentally realized. To our knowledge, this is the first time that extreme events have been reported in a memristive jerk system in particular and in jerk systems in general.

Classification of infection grade for anthracnose in mango leaves under complex background based on CBAM-DBIRNet

Anthracnose is one of the most harmful fungal diseases in mango orchards. Accurate and rapid detection of anthracnose infection grade for mango leaves in complex natural environments is a prerequisite of precise variable pesticide spraying by robots in intelligent orchards. In this paper, A lightweight double branch inverted residual attention network (CBAM-DBIRNet) model was proposed by combining the inverted residual network and the attention mechanism, which was used to detect the infection grade of anthracnose for mango leaves in natural environment. The model designed two parallel residual branch structures to extract multi-scale features of mango leaves with different infection grades through different dilation coefficients. The inverted residual structure and depthwise separable convolution were used to reduce the complexity of the model. The CBAM attention module was integrated into the model to suppress the interference of complex background information. The mango leaves collected in the natural orchard were used for experiments. The results showed that the accuracy, recall and F1 of the CBAM-DBIRNet model for the classification of anthracnose infection grade were 98.42 %, 97.80 % and 97.89 %, respectively. The model size and parameter were 0.64 MB and 0.15 M, respectively. This study provides technical support for smart orchard management and robotic precision variable application operations.

Temporal convolutional network with soft threshold and contractile self-attention mechanism for remaining useful life prediction of rolling bearings

Abstract RUL prediction is an effective approach to prevent system failures and cut maintenance expenditures. Due to the wide receptive field and the avoidance of future information leakage, temporal convolutional neural network (TCN) is widely applied for RUL estimation of bearings. However, the predictive performance of TCN is limited by the loss of degradation features and the breakdown of continuity of timing information. To overcome the above defects, hybrid temporal convolutional neural network with soft threshold and contractile self-attention mechanism (HTCN-SC) is presented. Firstly, the adaptive threshold is determined by the contraction self-attention mechanism with higher interpretability, which captures the contribution of different features to the estimation of RUL. Then, the soft threshold is employed to activate the degraded features. On the one hand, the degeneracy features endowed by the dilated causal convolution with obvious negative values are fully preserved. On the other hand, the noise components that are given low weights are completely suppressed compared to the original TCN. Finally, parallel branch composed of one-dimensional convolutional networks are used to supplement the continuity of time series. Degradation signals from different working conditions and bearings are employed to verify the performance of the HTCN-SC. The results indicate that HTCN-SC with accurate RUL estimation and generalization ability is an effective tool for rolling bearing health monitoring.

HIRI-ViT: Scaling Vision Transformer With High Resolution Inputs.

The hybrid deep models of Vision Transformer (ViT) and Convolution Neural Network (CNN) have emerged as a powerful class of backbones for vision tasks. Scaling up the input resolution of such hybrid backbones naturally strengthes model capacity, but inevitably suffers from heavy computational cost that scales quadratically. Instead, we present a new hybrid backbone with HIgh-Resolution Inputs (namely HIRI-ViT), that upgrades prevalent four-stage ViT to five-stage ViT tailored for high-resolution inputs. HIRI-ViT is built upon the seminal idea of decomposing the typical CNN operations into two parallel CNN branches in a cost-efficient manner. One high-resolution branch directly takes primary high-resolution features as inputs, but uses less convolution operations. The other low-resolution branch first performs down-sampling and then utilizes more convolution operations over such low-resolution features. Experiments on both recognition task (ImageNet-1K dataset) and dense prediction tasks (COCO and ADE20 K datasets) demonstrate the superiority of HIRI-ViT. More remarkably, under comparable computational cost ( ∼5.0 GFLOPs), HIRI-ViT achieves to-date the best published Top-1 accuracy of 84.3% on ImageNet with 448×448 inputs, which absolutely improves 83.4% of iFormer-S by 0.9% with 224×224 inputs.

Robust-EQA: Robust Learning for Embodied Question Answering With Noisy Labels.

Embodied question answering (EQA) is a recently emerged research field in which an agent is asked to answer the user's questions by exploring the environment and collecting visual information. Plenty of researchers turn their attention to the EQA field due to its broad potential application areas, such as in-home robots, self-driven mobile, and personal assistants. High-level visual tasks, such as EQA, are susceptible to noisy inputs, because they have complex reasoning processes. Before the profits of the EQA field can be applied to practical applications, good robustness against label noise needs to be equipped. To tackle this problem, we propose a novel label noise-robust learning algorithm for the EQA task. First, a joint training co-regularization noise-robust learning method is proposed for noisy filtering of the visual question answering (VQA) module, which trains two parallel network branches by one loss function. Then, a two-stage hierarchical robust learning algorithm is proposed to filter out noisy navigation labels in both trajectory level and action level. Finally, by taking purified labels as inputs, a joint robust learning mechanism is given to coordinate the work of the whole EQA system. Empirical results demonstrate that, under extremely noisy environments (45% of noisy labels) and low-level noisy environments (20% of noisy labels), the robustness of deep learning models trained by our algorithm is superior to the existing EQA models in noisy environments.

Accurate Measurement of Tower Grounding Resistance for Single-Tower and Multi-Tower Parallel Scenarios Based on the Clamp Meter Method: For the Sustainable Operation of Towers

Transmission tower grounding safety is a critical element that significantly impacts the reliability and sustainability of power grids. Tower grounding resistance is a significant grounding characteristic parameter. In order to accurately measure the grounding resistance of towers using the clamp meter method in both single-tower and multi-tower parallel scenarios, this paper establishes theoretical calculation models for measuring tower grounding resistance using the clamp meter method. Considering the interaction between the artificial grounding device and the tower foundation, this paper models and simulates transmission towers and lightning shield wires, analyzing the influencing factors on the grounding resistance measurement results using the clamp meter method in both single- and multi-tower parallel scenarios. The results show that for single towers, the clamp meter measurement results increase with decreasing foundation root spacing and increasing length of the artificial device’s extension lines, but the changes are relatively small. In multi-tower grounding scenarios, due to the shunting effect of the parallel branches formed by the towers and lightning shield wires, the value is smaller than in single-tower grounding scenarios. Changing the number of parallel towers and the type of lightning shield wire produces little effect on the measurement results. However, changes in soil resistivity have the most significant impact. Therefore, correction formulas for the impact of soil resistivity on clamp meter measurements are proposed and then verified by applying them to field tests.

A facial expression recognition network based on attention double branch enhanced fusion.

The facial expression reflects a person's emotion, cognition, and even physiological or mental state to a large extent. It has important application value in medical treatment, business, criminal investigation, education, and human-computer interaction. Automatic facial expression recognition technology has become an important research topic in computer vision. To solve the problems of insufficient feature extraction, loss of local key information, and low accuracy in facial expression recognition, this article proposes a facial expression recognition network based on attention double branch enhanced fusion. Two parallel branches are used to capture global enhancement features and local attention semantics respectively, and the fusion and complementarity of global and local information is realized through decision-level fusion. The experimental results show that the features extracted by the network are made more complete by fusing and enhancing the global and local features. The proposed method achieves 89.41% and 88.84% expression recognition accuracy on the natural scene face expression datasets RAF-DB and FERPlus, respectively, which is an excellent performance compared with many current methods and demonstrates the effectiveness and superiority of the proposed network model.

Sensitive Information Detection Based on Deep Learning Models

Currently, a large number of pornographic images on the Internet severely affect the growth of adolescents. In order to create a healthy and benign online environment, it is necessary to recognize and detect these sensitive images. Current techniques for detecting pornographic content are still in an immature stage, with the key issue being low detection accuracy. To address this problem, this paper proposes a method for detecting pornographic content based on an improved YOLOv8 model. Firstly, InceptionNeXt is introduced into the backbone network to enhance the model’s adaptability to images of different scales and complexities by optimizing feature extraction through parallel branches and deep convolution. Simultaneously, the SPPF module is simplified into the SimCSPSPPF module, which further enhances the effectiveness and diversity of features through improved spatial pyramid pooling and cross-layer feature fusion. Secondly, switchable dilated convolutions are incorporated to improve the adaptability of the C2f enhancement model and enhance the model’s detection capability. Finally, SEAattention is introduced to enhance the model’s ability to capture spatial details. The experiments demonstrate that the model achieves an mAP@0.5 of 79.7% on our self-made sensitive image dataset, which is a significant improvement of 5.9% compared to the previous YOLOv8n network. The proposed method excels in handling complex backgrounds, targets of varying scales, and resource-constrained scenarios, while simultaneously improving the model’s computational efficiency without compromising detection accuracy, making it more advantageous for practical applications.

TPANet: A novel triple parallel attention network approach for remaining useful life prediction of lithium-ion batteries

Accurate remaining useful life (RUL) prediction for lithium-ion batteries (LIBs) is important for proper equipment operation. Among the numerous existing studies on battery research, data-driven approaches have gained significant attention because they obviate the need for complex chemical and physical modeling of battery processes. However, in most current data-driven methods, the sliding window size values are determined empirically, which affects both the time required for model prediction and the prediction accuracy. To address this issue, this paper proposes a two-stage prediction method for battery capacity aging trajectories. In the first stage, a false nearest neighbor (FNN) technique is utilized to infer the sliding window size of the battery, reducing the error associated with manually selecting the sliding window size. In the second stage, a novel triple parallel attention network (TPANet) based on convolutional neural networks (CNNs) and attention units is developed, which extracts the input battery capacity degradation features through a parallel mechanism. The introduction of attention units in each parallel branch allows the model to enhance the capture of capacity regeneration phenomena as well as long-term dependence on input data. The proposed approach is validated on two publicly available datasets as well as a battery developed by ourselves. Diverse simulation results demonstrate the ability of the proposed approach to accurately predict the RUL of LIBs in a short time with broad generalization capabilities.

A 3D boundary-guided hybrid network with convolutions and Transformers for lung tumor segmentation in CT images

—Accurate lung tumor segmentation from Computed Tomography (CT) scans is crucial for lung cancer diagnosis. Since the 2D methods lack the volumetric information of lung CT images, 3D convolution-based and Transformer-based methods have recently been applied in lung tumor segmentation tasks using CT imaging. However, most existing 3D methods cannot effectively collaborate the local patterns learned by convolutions with the global dependencies captured by Transformers, and widely ignore the important boundary information of lung tumors. To tackle these problems, we propose a 3D boundary-guided hybrid network using convolutions and Transformers for lung tumor segmentation, named BGHNet. In BGHNet, we first propose the Hybrid Local-Global Context Aggregation (HLGCA) module with parallel convolution and Transformer branches in the encoding phase. To aggregate local and global contexts in each branch of the HLGCA module, we not only design the Volumetric Cross-Stripe Window Transformer (VCSwin-Transformer) to build the Transformer branch with local inductive biases and large receptive fields, but also design the Volumetric Pyramid Convolution with transformer-based extensions (VPConvNeXt) to build the convolution branch with multi-scale global information. Then, we present a Boundary-Guided Feature Refinement (BGFR) module in the decoding phase, which explicitly leverages the boundary information to refine multi-stage decoding features for better performance. Extensive experiments were conducted on two lung tumor segmentation datasets, including a private dataset (HUST-Lung) and a public benchmark dataset (MSD-Lung). Results show that BGHNet outperforms other state-of-the-art 2D or 3D methods in our experiments, and it exhibits superior generalization performance in both non-contrast and contrast-enhanced CT scans.

Tex-Net: texture-based parallel branch cross-attention generalized robust Deepfake detector

Tex-Net: texture-based parallel branch cross-attention generalized robust Deepfake detector

An interpretable TFAFI-1DCNN-LSTM framework for UGW-based pre-stress identification of steel strands

Steel strands serve as the key load-bearing components of pre-stressed bridges, yet the identification of effective pre-stress for steel strands is a challenging task. In this study, a time and frequency adaptive fusion input-one dimensional convolutional neural network-long short-term memory (TFAFI-1DCNN-LSTM) framework was proposed for ultrasonic guided wave (UGW)-based effective pre-stress identification of steel strands. In this framework, the time and frequency domain signals of UGW were input into the network as two parallel branches, CNN and LSTM were utilized to extract spatial and serial-related features, and a trainable weight coefficient was introduced for adaptive fusion of time and frequency domain features. Firstly, ablation studies were conducted to investigate the influence of each key component in the proposed framework on the prediction results. Secondly, deep learning (DL) models of Res Net, Dense Net and Inception Net were introduced as comparisons, and the prediction results based on TFAFI-1DCNN-LSTM were compared with those based on the above DL models. Thirdly, the noise-robustness and performance under a few trainable samples of TFAFI-1DCNN-LSTM were also investigated. Finally, the outputs of various key layers of the proposed framework were subjected to dimensionality reduction and visualization to provide an intuitive demonstration of the feature extraction process, and the inherent mechanisms of the proposed framework were interpreted using local interpretable model-agnostic explanations (LIME). Results show that the architecture of TFAFI-1DCNN-LSTM is reasonably designed and all key components are necessary. The weights of features in time and frequency domain branches are rationally assigned due to the addition of an adaptive weight coefficient. Compared with other DL models, the proposed TFAFI-1DCNN-LSTM exhibits higher pre-stress identification accuracy, excellent noise-robustness, and superior performance under only a few trainable samples. The correlation between features extracted by the network and the pre-stress labels significantly increases with the deepening of the network, and the correlation between input UGW signals and predicted pre-stress values is effectively interpreted utilizing LIME, proving that the architecture of the proposed framework is reasonable and effective, the predicted results are reliable and credible.

DMWMNet: A novel dual-branch multi-level convolutional network for high-performance mixed-type wafer map defect detection in semiconductor manufacturing

Wafer map defect detection plays an important role in semiconductor manufacturing by identifying root causes and accelerating process adjustments to ensure product quality and reduce unnecessary expenditures. However, existing methods have some limitations, such as low accuracy in mixed-type defect detection and poor recognition of similar defects and weak features. In this article, a novel dual-branch multi-level convolutional network (DMWMNet) is proposed for high-performance mixed-type wafer map defect detection. By fully considering the interrelationships between basic defects, defect number, and defect type, the network is designed to include two efficient parallel Branches and a Fusion classifier. Detecting defect types using basic defect discrimination and defect number detection is helpful for ameliorating problems with high complexity and low accuracy caused by multiple defect categories and feature overlaps. Furthermore, a composite loss function based on focal loss is employed to improve the network’s capacity to recognize weak features and similar defects. Experimental results on the MixedWM38 dataset show that DMWMNet has favorable mixed-type defect detection performance compared to other methods, with accuracy, precision, recall, F1 score, and MCC of 98.99%, 98.94%, 99.03%, 98.98%, and 98.97%, respectively.