Feature Information Research Articles

Bidirectional Mamba with Dual-Branch Feature Extraction for Hyperspectral Image Classification

The hyperspectral image (HSI) classification task is widely used in remote sensing image analysis. The HSI classification methods based on convolutional neural networks (CNNs) have greatly improved the classification performance. However, they cannot well utilize the sequential properties of spectral features and face the challenge of increasing computational cost with the increase in network depth. To address these shortcomings, this paper proposes a novel network with a CNN-Mamba architecture, called DBMamba, which uses a bidirectional Mamba to process spectral feature sequences at a linear computational cost. In the DBMamba, principal component analysis (PCA) is first used to extract the main features of the data. Then, a dual-branch CNN structure, with the fused features from spectral–spatial features by 3D-CNN and spatial features by 2D-CNN, is used to extract shallow spectral–spatial features. Finally, a bidirectional Mamba is used to effectively capture global contextual information in features and significantly enhance the extraction of spectral features. Experimental results on the Indian Pines, Salinas, and Pavia University datasets demonstrate that the classification performance surpasses that of many cutting-edge methods, improving by 1.04%, 0.15%, and 0.09%, respectively, over the competing SSFTT method. The research in this paper enhances the existing knowledge on HSI classification and provides valuable insights for future research in this field.

A Pseudo-Labeling Multi-Screening-Based Semi-Supervised Learning Method for Few-Shot Fault Diagnosis

In few-shot fault diagnosis tasks in which the effective label samples are scarce, the existing semi-supervised learning (SSL)-based methods have obtained impressive results. However, in industry, some low-quality label samples are hidden in the collected dataset, which can cause a serious shift in model training and lead to the performance of SSL-based method degradation. To address this issue, the latest prototypical network-based SSL techniques are studied. However, most prototypical network-based scenarios consider that each sample has the same contribution to the class prototype, which ignores the impact of individual differences. This article proposes a new SSL method based on pseudo-labeling multi-screening for few-shot bearing fault diagnosis. In the proposed work, a pseudo-labeling multi-screening strategy is explored to accurately screen the pseudo-labeling for improving the generalization ability of the prototypical network. In addition, the AdaBoost adaptation-based weighted technique is employed to obtain accurate class prototypes by clustering multiple samples, improving the performance that deteriorated by low-quality samples. Specifically, the squeeze and excitation block technique is used to enhance the useful feature information and suppress non-useful feature information for extracting accuracy features. Finally, three well-known bearing datasets are selected to verify the effectiveness of the proposed method. The experiments illustrated that our method can receive better performance than that of the state-of-the-art methods.

YOLO-DHGC: Small Object Detection Using Two-Stream Structure with Dense Connections

Small object detection, which is frequently applied in defect detection, medical imaging, and security surveillance, often suffers from low accuracy due to limited feature information and blurred details. This paper proposes a small object detection method named YOLO-DHGC, which employs a two-stream structure with dense connections. Firstly, a novel backbone network, DenseHRNet, is introduced. It innovatively combines a dense connection mechanism with high-resolution feature map branches, effectively enhancing feature reuse and cross-layer fusion, thereby obtaining high-level semantic information from the image. Secondly, a two-stream structure based on an edge-gated branch is designed. It uses higher-level information from the regular detection stream to eliminate irrelevant interference remaining in the early processing stages of the edge-gated stream, allowing it to focus on processing information related to shape boundaries and accurately capture the morphological features of small objects. To assess the effectiveness of the proposed YOLO-DHGC method, we conducted experiments on several public datasets and a self-constructed dataset. Exceptionally, a defect detection accuracy of 96.3% was achieved on the Market-PCB public dataset, demonstrating the effectiveness of our method in detecting small object defects for industrial applications.

Uncertainty-driven dynamic ensemble framework for rotating machinery fault diagnosis under time-varying working conditions

Multi-scale ensemble learning combines different scales of feature resolution, thereby improving fault diagnostic accuracy. However, the effectiveness of different information scales in characterizing fault features under time-varying speed conditions varies with speed. It is difficult for existing ensemble strategies to ensure the effectiveness of feature information when ensemble multi-scale feature information is involved. Accordingly, we propose an uncertainty-driven dynamic ensemble Bayesian convolutional neural network (DEBCNN) framework. The uncertainty of the results of different scale models was used to dynamically determine their weights in the ensemble framework, which reduced the influence of irrelevant features on the diagnostic results. By employing the proposed dynamic ensemble strategy, the ensemble framework can utilize fault feature information corresponding to different rotational speeds in the final diagnostic results. Experiments on motor and bearing datasets illustrate the superiority of this strategy over other techniques. This study provides useful insights for further research in the field of fault diagnosis of rotating machinery at time-varying speeds.

SOD-YOLO: A lightweight small object detection framework

Currently, lightweight small object detection algorithms for unmanned aerial vehicles (UAVs) often employ group convolutions, resulting in high Memory Access Cost (MAC) and rendering them unsuitable for edge devices that rely on parallel computing. To address this issue, we propose the SOD-YOLO model based on YOLOv7, which incorporates a DSDM-LFIM backbone network and includes a small object detection branch. The DSDM-LFIM backbone network, which combines Deep-Shallow Downsampling Modules (DSD Modules) and Lightweight Feature Integration Modules (LFI Modules), avoids excessive use of group convolutions and element-wise operations. The DSD Module focuses on extracting both deep and shallow features from feature maps using fewer parameters to obtain richer feature representations. The LFI Module, is a dual-branch feature integration module designed to consolidate feature information. Experimental results demonstrate that the SOD-YOLO model achieves an AP50 of 50.7% and a FPS of 72.5 on the VisDrone validation set. Compared to YOLOv7, our model reduces computational costs by 20.25% and decreases the number of parameters by 17.89%. After scaling the number of channels in the model, it achieves an AP50 of 33.4% with an inference time of 27.3ms on the Atlas 200I DK A2. These experimental results indicate that the SOD-YOLO model can effectively perform small object detection tasks in a large number of aerial images captured by UAVs.

Text-Enhanced Graph Attention Hashing for Cross-Modal Retrieval

Deep hashing technology, known for its low-cost storage and rapid retrieval, has become a focal point in cross-modal retrieval research as multimodal data continue to grow. However, existing supervised methods often overlook noisy labels and multiscale features in different modal datasets, leading to higher information entropy in the generated hash codes and features, which reduces retrieval performance. The variation in text annotation information across datasets further increases the information entropy during text feature extraction, resulting in suboptimal outcomes. Consequently, reducing the information entropy in text feature extraction, supplementing text feature information, and enhancing the retrieval efficiency of large-scale media data are critical challenges in cross-modal retrieval research. To tackle these, this paper introduces the Text-Enhanced Graph Attention Hashing for Cross-Modal Retrieval (TEGAH) framework. TEGAH incorporates a deep text feature extraction network and a multiscale label region fusion network to minimize information entropy and optimize feature extraction. Additionally, a Graph-Attention-based modal feature fusion network is designed to efficiently integrate multimodal information, enhance the affinity of the network for different modes, and retain more semantic information. Extensive experiments on three multilabel datasets demonstrate that the TEGAH framework significantly outperforms state-of-the-art cross-modal hashing methods.

Subject-aware PET Denoising with Contrastive Adversarial Domain Generalization.

Recent advances in deep learning (DL) have greatly improved the performance of positron emission tomography (PET) denoising performance. However, DL model performance can vary a lot across subjects, due to the large variability of the count levels and spatial distributions. A generalizable DL model that mitigates the subject-wise variations is highly expected toward a reliable and trustworthy system for clinical application. In this work, we propose a contrastive adversarial learning framework for subject-wise domain generalization (DG). Specifically, we configure a contrastive discriminator in addition to the UNet-based denoising module to check the subject-related information in the bottleneck feature, while the denoising module is adversarially trained to enforce the extraction of subject-invariant features. The sampled low-count realizations from the list-mode data are used as anchor-positive pairs to be close to each other, while the other subjects are used as negative samples to be distributed far away. We evaluated on 97 18F-MK6240 tau PET studies, each having 20 noise realizations with 25% fractions of events. Training, validation, and testing were implemented using 1400, 120, and 420 pairs of 3D image volumes in a subject-independent manner. The proposed contrastive adversarial DG demonstrated superior denoising performance than conventional UNet without subject-wise DG and cross-entropy-based adversarial DG.

VCU-Net: a vascular convolutional network with feature splicing for cerebrovascular image segmentation.

Cerebrovascular image segmentation is one of the crucial tasks in the field of biomedical image processing. Due to the variable morphology of cerebral blood vessels, the traditional convolutional kernel is weak in perceiving the structure of elongated blood vessels in the brain, and it is easy to lose the feature information of the elongated blood vessels during the network training process. In this paper, a vascular convolutional U-network (VCU-Net) is proposed to address these problems. This network utilizes a new convolution (vascular convolution) instead of the traditional convolution kernel, to extract features of elongated blood vessels in the brain with different morphologies and orientations by adaptive convolution. In the network encoding stage, a new feature splicing method is used to combine the feature tensor obtained through vascular convolution with the original tensor to provide richer feature information. Experiments show that the DSC and IOU of the proposed method are 53.57% and 69.74%, which are improved by 2.11% and 2.01% over the best performance of the GVC-Net among several typical models. In image visualization, the proposed network has better segmentation performance for complex cerebrovascular structures, especially in dealing with elongated blood vessels in the brain, which shows better integrity and continuity.

Fast feature- and category-related parafoveal previewing support free visual exploration.

While humans typically saccade every ∼250 ms in natural settings, studies on vision tend to prevent or restrict eye movements. As it takes ∼50 ms to initiate and execute a saccade, this leaves only ∼200 ms to identify the fixated object and select the next saccade goal. How much detail can be derived about parafoveal objects in this short time interval, during which foveal processing and saccade planning both occur? Here, we had male and female human participants freely explore a set of natural images while we recorded magnetoencephalography and eye movements. Using multivariate pattern analysis, we demonstrate that future parafoveal images could be decoded at the feature and category level with peak decoding at ∼110 ms and ∼165 ms respectively, while the decoding of fixated objects at the feature and category level peaked at ∼100 ms and ∼145 ms. The decoding of features and categories was contingent on the objects being saccade goals. In sum, we provide insight on the neuronal mechanism of pre-saccadic attention by demonstrating that feature and category specific information of foveal and parafoveal objects can be extracted in succession within a ∼200 ms intersaccadic interval. These findings rule out strict serial or parallel processing accounts but are consistent with a pipeline mechanism in which foveal and parafoveal objects are processed in parallel but at different levels in the visual hierarchy.Significance Statement We provide neural evidence that future parafoveal saccade goals are processed surprisingly quickly at the feature and the category level before we saccade to them. Specifically, using multivariate pattern analysis applied to magnetoencephalography and eye-tracking data, we found that information about the colour and the category of parafoveal objects emerged at ∼110 ms and ∼165 ms respectively, with the same information about foveal objects emerging ∼100 ms and ∼145 ms. Our findings provide novel insight into the neuronal dynamics of parafoveal previewing during free visual exploration. The dynamics rule out strict serial or parallel processing, but are consistent with a pipelining mechanism in which foveal and parafoveal objects are processed in parallel but at different levels in the visual hierarchy.

An Extended Method Based on the Geometric Position of Salient Image Features: Solving the Dataset Imbalance Problem in Greenhouse Tomato Growing Scenarios

Machine vision has significant advantages in a wide range of agricultural applications; however, acquiring a large number of high-quality image resources is often challenging in actual agricultural production due to environmental and equipment conditions. Therefore, crop image augmentation techniques are particularly important in crop growth analysis. In this paper, greenhouse tomato plants were used as research subjects to collect images of their different fertility stages with flowers and fruits. Due to the different durations of each fertility period, there is a significant difference in the number of images collected. For this reason, this paper proposes a method for balanced amplification of significant feature information in images based on geometric position. Through the geometric position information of the target in the image, different segmentation strategies are used to process the image and supervised and unsupervised methods are applied to perform balanced augmentation of the image, which is combined with the YOLOv7 algorithm to verify the augmentation effect. In terms of the image dataset, the mixed image dataset (Mix) is supplemented with mobile phone images on top of in situ monitoring images, with precision increased from 70.33% to 82.81% and recall increased from 69.15% to 81.25%. In terms of image augmentation, after supervised balanced amplification, the detection accuracy is improved from 70.33% to 77.29%, which is suitable for supervised balanced amplification. For the mobile phone dataset (MP), after amplification, it was found that better results could be achieved without any amplification method. The detection accuracy of the mixed dataset with different data sources matching the appropriate amplification method increased slightly from 82.81% to 83.59%, and accurate detection could be achieved when the target was shaded by the plant, and in different environments and light conditions.

Order-difference of normalized square envelope spectrum and its applications in early chatter detection of roll grinder

Roll grinder is widely demanded in grinding the surface of rolls that will be used to mill steel strips for high-quality automobile outer panels. The grinding chatter generally leads to poor machining accuracy and massive economic losses. The significance of roll grinder vibration monitoring is to detect the early grinding chatter. However, in harsh working conditions, some chatter detection methods inevitably confuse the fault components and interference components in vibration signals and the chatter feature information cannot be extracted exactly. The dynamical model of roll grinder system reveals that the spectrums of vibration signals contain intrinsic frequency components and chatter frequency components under health and chatter conditions. Based on the analytical modelling, a new health index order-difference of normalized square envelope spectrum (ODNSES) is proposed to detect early chatter and evaluate the health degradation of the roll grinder. The normalized square envelope spectrum (SES) is used as the pretreatment of ODNSES to compare the frequency components in vibration signals under health and early chatter conditions. Furthermore, order-difference of normalized SES is proposed to suppress the noise interference and to enable ODNSES to characterize the grinding chatter process. The detailed simulation research is performed to verify three important properties of ODNSES. The experimental results show that ODNSES would not be interfered by background noise significantly and it is on average 29.12% more sensitivity to impulse amplitudes than classical health indexes. The testing results on actual roll grinder exhibit that ODNSES can detect the early chatter and quantify the abnormal vibration in the grinding process.

Global induced local network for infrared: dim small target detection

Abstract It is challenging to detect infrared dim targets submerged in complicated backgrounds due to their small size and faint intensity. The previous attention-based detection networks frequently require global long-range dependence. Significant calculations are required to determine the target's sparse but meaningful position. To prevent wasting calculations on the background, this paper offers a detection network guided by global context for local feature learning, named GILNet (Global Induced Local Network). It designs a global location module (GLM) and a local feature interaction module (LFIM) to capture the global position and features of targets, respectively. More specifically, using global context interaction, the GLM finds the region that might contain dim small targets, that is, the coarse location. In the coarsely located regions, the LFIM further acquires feature information about targets. Next, we also design an eight-directional attention operation to obtain the contour information of targets in the low feature map. It is fused with the high feature map in the multi-directional feature fusion module (MFFM), which retains more semantic and spatial information about targets. Finally, quantitative and qualitative analysis show that the GILNet performs better than eight comparison methods on two public datasets.

MFE-SSNet: Multi-Modal Fusion-Based End-to-End Steering Angle and Vehicle Speed Prediction Network

AbstractIn the field of autonomous vehicles, accurately predicting steering angle and speed is a pivotal task. This task affects the accuracy of the final decision of the autonomous vehicle and is the basis for ensuring the safe and efficient operation of the autonomous vehicle. Previous studies have often relied on data from only one or two modalities to make predictions for steering angle and vehicle speed, which were often inadequate. In this paper, the authors propose a Multi-Modal Fusion-Based End-to-End Steering Angle and Vehicle Speed Prediction Network (MFE-SSNet). The network innovatively extends the one-stream and two-stream structure to a three-stream structure and cleverly extracts features of images, steering angles, and vehicle speeds using HRNet and LSTM layers. In addition, in order to fully fuse the feature information of different modal data, this paper also proposes a local attention-based feature fusion module. This module improves the fusion of different modal feature vectors by capturing the interdependencies in the local channels. Experimental results demonstrate that MFE-SSNet outperforms the current state-of-the-art model on the publicly available Udacity dataset.

AEFF-SSC: an attention-enhanced feature fusion for 3D semantic scene completion

Abstract Three-dimensional (3D) occupancy perception technology aims to enable autonomous vehicles to observe and understand dense 3D environments. Estimating the complete geometry and semantics of a scene solely from visual images is challenging. However, humans can easily conceive the complete form of objects based on partial key information and their own experience. This ability is crucial for recognizing and interpreting the surrounding environment. To equip 3D occupancy perception systems with a similar capability, a 3D semantic scene completion method called AEFF-SSC is proposed. This method deeply explores boundary and multi-scale information in voxels, aiming to reconstruct 3D geometry more accurately. We have specifically designed an attention-enhanced feature fusion module that effectively fuses image feature information from different scales and focuses on feature boundary information, thereby more efficiently extracting voxel features. Additionally, we introduce a semantic segmentation module driven by a 3D attention-UNet network. This module combines a 3D U-Net network with a 3D attention mechanism. Through feature fusion and feature weighting, it aids in restoring 3D spatial information and significantly improves the accuracy of segmentation results. Experimental verification on the SemanticKITTI dataset demonstrates that AEFF-SSC significantly outperforms other existing methods in terms of both geometry and semantics. Specifically, within a 12.8 m × 12.8 m area ahead, our geometric occupancy accuracy has achieved a significant improvement of 71.58%, and at the same time, the semantic segmentation accuracy has also increased remarkably by 54.20%.

Research on X-ray security contraband identification technology based on lightweight YOLOv8

Aiming at the difficulty of identification and localization in the detection of contraband targets in X-ray images, as well as the demand for upgrading and deployment of security equipment during the peak flow stage, a lightweight X-ray security contraband detection algorithm is proposed and optimized for YOLOv8. Firstly, the number of channels in the original network is reduced according to a predetermined ratio, which effectively reduces model parameters and computational complexity and achieves better results than the current mainstream lightweight methods, such as model pruning and detection backbone optimization. Secondly, due to the presence of objects of various scales and sizes in X-ray security inspection images, we introduced the BiFPN feature extraction strategy in the neck. We designed a simplified version of BiFPN based on the YOLOv8 network structure to effectively integrate feature information from different levels and scales. Finally, we incorporated the idea of Focal Loss to address the issue of imbalance between positive and negative samples and proposed a new regression loss function, Focal-GIoU Loss, which combines GIoU Loss. This increases the loss weight for important but difficult-to-detect samples, such as small targets and overlapping targets, making the model more focused on these critical samples. Results show that our proposed YOLO-CBF improves mAP by 1.5%, 0.9%, and 0.5% on three datasets with different levels of occlusion in PIDray, while reducing the parameter count and computational cost from the original 3.0M and 8.1G to 2.1M and 6.3G.Moreover, it performs well in comparative experiments between different models and in generalization experiments across different datasets.

Research on visual features and identification of coal and gangue in simulated underground mine environments

ABSTRACT In underground mine environments, factors such as low light intensity, diffusion of dust particles, and coal slurry coverage interfere with coal gangue identification. To further realize coal-gangue sorting underground, we analyzed the coal-gangue visual characteristics coupled with the above factors. The analysis reveals that both smoke environments and coal slurry coverage tend to make the coal-gangue visual features more consistent. There are still differences in coal-gangue visual features under smoke environments, while coal slurry coverage leads to gangue exhibiting coal-related feature information. To improve the detection capability of coal gangue in underground environments, this study adopts a coal gangue recognition method based on the DG module and YOLOX-PSB network: Utilize the DG module to achieve dehazing and then use the YOLOX-PSB model for object detection. The proposed method was validated through ablation experiments, and the results showed that the detection accuracy reached 99.6%, with an FPS of 98. The proposed method enables real-time detection and exhibits the best robustness in complex environments, realizing reliable and efficient performance.

Small Target Detection Algorithm Based on Improved YOLOv5

Small targets exist in large numbers in various fields. They are broadly used in aerospace, video monitoring, and industrial detection. However, because of its tiny dimensions and modest resolution, the precision of small-target detection is low, and the erroneous detection rate is high. Therefore, based on YOLOv5, an improved small-target detection model is proposed. First, in order to improve the number of tiny targets detected while enhancing small-target detection performance, an additional detection head is added. Second, involution is used between the backbone and neck to increase the channel information of feature mapping. Third, the model introduces the BiFormer, wherein both the global and local feature information are captured simultaneously by means of its double-layer routing attention mechanism. Finally, a context augmentation module (CAM) is inserted into the neck in order to maximize the structure of feature fusion. In addition, in order to consider among the required real frame as well as the prediction frame simultaneously, YOLOv5’s original loss function is exchanged. The experimental results using the public dataset VisDrone2019 show that the proposed model has P increased by 13.43%, R increased by 11.28%, and mAP@.5 and mAP@[.5:.95] increased by 13.88% and 9.01%, respectively.

Weak saliency ensemble network for person Re-identification using infrared light images

In recent years, person re-identification (re-id) has primarily been studied using visible light (VL) images. However, the challenges of employing VL images in nighttime environments have prompted research into using infrared light (IR) images. Yet, the utilization of both VL and IR images in person re-id has resulted in increased computational cost and processing time in multi-modality systems, leading to studies focusing solely on IR images. Nevertheless, IR images, lacking color and texture information, generally yield lower recognition performance in existing person re-id studies. In addition, previous studies have shown that person re-id performance suffers in the presence of complex background noise. To tackle these challenges, this study proposes a new weak saliency ensemble network (WSE-Net) for person re-id using IR images. WSE-Net incorporates a channel reduction of feature (CRF) method to reduce computational cost in the ensemble network, a technique for converting input images into group of patch images and feeding them into the ensemble model to enhance the reduced feature information, and a grouped convolution ensemble network (GCE-Net) that enables the fusion of features extracted from original and attention-guided ensemble models.The performance of person re-id using WSE-Net was evaluated on the Dongguk body-based person recognition database version 1 (DBPerson-Recog-DB1) and the Sun Yat-sen university multiple modality re-identification version 1 (SYSU-MM01). Experimental results demonstrated that on DBPerson-Recog-DB1, WSE-Net achieved 93.65% in rank 1, 95.28% in mean average precision (mAP), and 93.52% in the harmonic mean of precision and recall. Additionally, on SYSU-MM01, WSE-Net achieved 86.85% in rank 1, 44.58% in mAP, and 40.06% in the harmonic mean of precision and recall. Furthermore, the accuracy of WSE-Net on both datasets surpassed that of state-of-the-art methods.

Digital signal detection and recognition in the communication field combining DAEN and CNN

In communication, the detection and recognition of digital signals have always faced problems such as low adaptability and high misclassification rate. To address these issues, this study innovatively fused the deep auto-encoder network with convolutional neural networks to construct a novel digital signal detection and recognition model. Firstly, this study utilized the powerful feature extraction capabilities of the deep auto-encoder network to extract key feature information from massive amounts of data. Then these features were combined with convolutional neural networks to construct a detection and recognition model. These results confirmed that the constructed digital signal detection and recognition model had values of 93.75% and 94.18% in data detection recognition rate and average classification accuracy, respectively. Meanwhile, this model also performed well in terms of data processing accuracy and recall. In the comparison of data processing, the accuracy and recall rates were 93.59% and 94.67%, respectively, and the performance of data detection and recognition was better than that of the comparison methods. This indicates that the constructed digital signal detection and recognition model can significantly improve the reliability and robustness of signal detection and recognition. This paper brings new breakthroughs to the development of digital signal detection and recognition technology in communication.

Optimizing flotation froth image segmentation via parallel branch network and hybrid loss supervision

Flotation is a crucial technology for fine coal separation, and accurately acquiring bubble size information during the flotation process is essential for monitoring flotation conditions and achieving intelligent control. However, existing semantic segmentation models encountered issues with boundary disconnection when segmenting flotation bubbles, resulting in deviations between the extracted bubble sizes and their true values. To address the aforementioned challenges, a semantic segmentation model was proposed to maintain high-resolution feature maps throughout the network by designing a parallel branch network structure. Additionally, a ConvTranspose module was proposed to preserve the detailed feature information of images while gradually enhancing the resolution of feature maps. In the model training phase, a hybrid loss function combining pixel classification loss with shape similarity loss was proposed to alleviate the sample imbalance problem caused by the substantial difference in the number of pixels between bubble boundaries and the interior of bubbles. Moreover, since traditional semantic segmentation evaluation metrics, such as MIoU, lack a mechanism for measuring bubble boundary continuity and cannot effectively penalize the problem of boundary disconnection, this paper proposed a new evaluation method for assessing the segmentation performance of flotation froth images. To comprehensively evaluate the effectiveness of the proposed method, this paper conducted tests using flotation froth images collected from actual production processes. Compared with existing methods, the segmentation model proposed in this paper exhibited clear superiority in mitigating the problem of bubble boundary disconnection. The prediction error for the number of bubbles was 6.38 %, which is significantly better than other methods.