Multi Feature Research Articles

SML: A Skeleton-based multi-feature learning method for sign language recognition

Sign language recognition (SLR) is an effective solution to communication barriers experienced by hearing and vocally impaired individuals with other communities. Its applications extend to human–robot interaction (HRI), virtual reality (VR), and augmented reality (AR). However, the diverse nature of sign languages, stemming from varying user habits and geographical regions, poses significant challenges. To address these challenges, we propose the Skeleton-based Multi-feature Learning method (SML). This method comprises a Multi Feature Aggregation (MFA) module, designed to capture the inherent relationships between different skeleton-based features, enabling effective fusion of complementary information. Furthermore, we propose the Self knowledge distillation Guided Adaptive Residual Decoupled Graph Convolutional Network (SGAR-DGCN) for feature extraction. SGAR-DGCN consists of three components: a Self Knowledge Distillation (SKD) mechanism to enhance model training, convergence, and accuracy; a DGCN-Block, incorporating Decoupled GCN and Spatio Temporal Channel attention (STC) for efficient feature extraction; and an Adaptive Residual Block (ARes-Block) for cross-layer information fusion. Experimental results demonstrate that our SML method outperforms state-of-the-art approaches on the WLASL (55.85%) and AUTSL (96.85%) datasets, solely utilizing skeleton data. Code is available at https://github.com/DzwFine37/SML.

Multi-stage coarse-to-fine progressive enhancement network for single-image HDR reconstruction

Compared with traditional imaging, high dynamic range (HDR) imaging technology can record scene information more accurately, thereby providing users higher quality of visual experience. Inverse tone mapping is a direct and effective way to realize single-image HDR reconstruction, but it usually suffers from some problems such as detail loss, color deviation and artifacts. To solve the problems, this paper proposes a multi-stage coarse-to-fine progressive enhancement network (named MSPENet) for single-image HDR reconstruction. The entire multi-stage network architecture is designed in a progressive manner to obtain higher-quality HDR images from coarse-to-fine, where a mask mechanism is used to eliminate the effects of over-exposure regions. Specifically, in the first two stages, two asymmetric U-Nets are constructed to learn the multi-scale information of input image and perform coarse reconstruction. In the third stage, a residual network with channel attention mechanism is constructed to learn the fusion of progressively transferred multi-level features and perform fine reconstruction. In addition, a multi-stage progressive detail enhancement mechanism is designed, including progressive gated recurrent unit fusion mechanism and multi-stage feature transfer mechanism. The former fuses the progressively transferred features with coarse HDR features to reduce the error stacking effect caused by multi-stage networks. Meanwhile, the latter fuses early features to supplement the lost information during each stage of feature delivery and combines features from different stages. Extensive experimental results show that the proposed method can reconstruct higher quality HDR images and effectively recover texture and color information in over-exposure regions compared to the state-of-the-art methods.

SPD Manifold Deep Metric Learning for Image Set Classification.

By characterizing each image set as a nonsingular covariance matrix on the symmetric positive definite (SPD) manifold, the approaches of visual content classification with image sets have made impressive progress. However, the key challenge of unhelpfully large intraclass variability and interclass similarity of representations remains open to date. Although, several recent studies have mitigated the two problems by jointly learning the embedding mapping and the similarity metric on the original SPD manifold, their inherent shallow and linear feature transformation mechanism are not powerful enough to capture useful geometric features, especially in complex scenarios. To this end, this article explores a novel approach, termed SPD manifold deep metric learning (SMDML), for image set classification. Specifically, SMDML first selects a prevailing SPD manifold neural network (SPDNet) as the backbone (encoder) to derive an SPD matrix nonlinear representation. To counteract the degradation of structural information during multistage feature embedding, we construct a Riemannian decoder at the end of the encoder, trained by a reconstruction error term (RT), to induce the generated low-dimensional feature manifold of the hidden layer to capture the pivotal information about the visual data describing the imaged scene. We demonstrate through theory and experiments that it is feasible to replace the Riemannian metric with Euclidean distance in RT. Then, the ReCov layer is introduced into the established Riemannian network to regularize the local statistical information within each input feature matrix, which enhances the effectiveness of the learning process. The theoretical analysis of the activation function used in the ReCov layer in terms of continuity and conditions for generating positive definite matrices is beneficial for network design. Inspired by the fact that the single cross-entropy loss used for training is unable to effectively parse the geometric distribution of the deep representations, we finally endow the suggested model with a novel metric learning regularization term. By explicitly incorporating the encoding and processing of the data variations into the network learning process, this term can not only derive a powerful Riemannian representation but also train an effective classifier. The experimental results show the superiority of the proposed approach on three typical visual classification tasks.

Optimizing product design and development of engine carbon cleaning maintenance tools using reverse engineering and VDI 2222 methods

This research article explains the product design and development optimization of multi feature engine carbon cleaning maintenance tools. The implementation of the Reverse Engineering method was integrated with VDI 2222 in the research process. The results of the optimal design and development of this tool obtained a component structure that was able to meet 14 types of consumer needs with a total tool cost of IDR 1.137.900 with a total tool weight of 6.9 kg. The results of the load test simulation concluded that the housing case was able to withstand the overall weight of the components that make up the tool. Depending on the liquid chemical used, the results of tool tests on a limited scale have proven to be able in save fuel consumption by ±12.5% per hour with engine workload which can also be reduced by 3,14%.

MOX-NET: Multi-stage deep hybrid feature fusion and selection framework for monkeypox classification

Background:Monkeypox virus has quickly expanded throughout several nations, raising serious public health concerns. Lack of precautionary measures raises concerns about the possibility of global pandemic. As a result, early detection of monkeypox is essential for efficient diagnosis and treatment before broad community transmission. Artificial intelligence (AI) and machine learning (ML) have recently shown tremendous promise in early monkeypox diagnosis using image-based diagnoses methods. But, challenges such as low contrast lesions, overloaded datasets, high memory complexity, and redundant feature extraction still need to be addressed. Methods:This work proposes a coalesced CAD system for monkeypox classification using a deep learning framework. The proposed approach includes multiple stages: First, the given dataset images are pre-processed by a proposed fusion-based contrast enhancement method. Second, six pre-trained deep learning models i.e., Vision Transformers (ViT), Shifted Windows (Swin) Transformers, ResNet-50, ResNet-101, EfficientNetV2, and ConvNeXt-V2 are modified and transfer learning is used for training. Third, from all the deep learning networks, the deep feature vectors are acquired and integrated through the convolutional sparse image decomposition fusion strategy. Subsequently, an entropy-controlled firefly feature selection approach is applied to identify the most suitable features for classification. Lastly, the multi-class support vector machine is utilized to classify the input data based on the selected features. Results:The proposed algorithm utilized the MSLID dataset and obtained an accuracy of 98.64% which is better than other state-of-the-art works. Conclusions:Experimental outcomes reveals that the proposed monkeypox classification algorithm outperforms well-established existing algorithms in both visual and quantitative evaluations, resulting in improved accuracy.

LithoSegNet: Regional attention-based deep fusion of multi-scale and cross-stage features for real-time lithology segmentation

Lithology identification plays an important role in engineering construction, disaster prediction, reservoir evaluation, and other fields. However, with the development of mechanization and automation, traditional lithology recognition methods are gradually unable to meet existing demands in terms of recognition speed and efficiency. In order to achieve real-time, fast, and accurate identification of lithology, we designed a lightweight model backbone, proposed a multi-scale feature extraction module and a cross-stage feature fusion module based on regional attention, developed a step-wise fusion scheme for multi-stage feature, and developed a real-time lithology segmentation model. The experimental results show that the feature extraction module improves the mIoU of the model by 0.0021; The cross-stage feature fusion module improves the mIoU of the model by 0.0104; The step-wise fusion scheme can effectively fuse shallow spatial features and deep semantic features. The model mIoU using this scheme is 0.0246 and 0.0145 higher than that using SegFormer and SegNeXt fusion schemes, respectively. Our model LithoSegNet can achieve a mIoU of 0.9583 with a speed of 116.25 images per second, comprehensively surpassing many state-of-the-art models. The research results can provide technical support for automated geological sketching in field exploration and on-site construction, and lay the foundation for intelligent construction of underground engineering, which has important scientific and engineering application value.

MultiTumor Analyzer (MTA-20–55): A network for efficient classification of detected brain tumors from MRI images

Brain cancer, one of the leading causes of mortality worldwide, is caused by brain tumors. Early diagnosis of tumors and predicting their progression can help doctors to save lives. In this article, we have designed an automated approach for locating and classifying tumors from MRI images. The novelties of the research work include the following two stages: Developing an encoder-decoder type 20-Layered deep neural network (DNN) named MultiTumor Analyzer (MTA-20) with 15 down-sampling layers and 4 up-sampling layers, the segmentation is performed in the initial stage. Here, we have adhered a Leaky ReLU activation function instead of ReLU which learn a parameter with negative values that may have valuable information which is essential specifically for image segmentation. Further, a 55-layered DNN using multistage feature fusion is developed in the second stage of the work for the classification of localized tumors. The classification is performed using developed MultiTumor Analyzer (MTA-55) DNN with Softmax classifier. The efficacy of the designed network is validated using highly cited quantitative measures such as accuracy, sensitivity, specificity, dice similarity coefficient (DSC), precision, and F1-measure. It is observed that the proposed MTA-20 DNN attains the average accuracy, sensitivity, specificity, DSC, and precision of 99.2 %, 94.6 %, 99.3 %, 88 %, and 82.5 % respectively against seven state-of-the-art techniques. Also, it is found that, the proposed MTA-55 DNN provides the overall accuracy, recall, specificity, F1-measure, precision, and DSC of 99.8 %, 99.633 %, 99.844 %, 99.659 %, 99.689 %, and 99.656 % respectively as compared to thirteen state-of-the-art techniques. These results corroborate the superiority of the proposed technique.

SDP-MTF: A Composite Transfer Learning and Feature Fusion for Cross-Project Software Defect Prediction

Software defect prediction is critical for improving software quality and reducing maintenance costs. In recent years, Cross-Project software defect prediction has garnered significant attention from researchers. This approach leverages transfer learning to apply the knowledge from existing projects to new ones, thereby enhancing the universality of predictive models. It provides an effective solution for projects with limited historical defect data. Nevertheless, current methodologies face two main challenges: first, the inadequacy of feature information mining, where code statistical information or semantic information is used in isolation, ignoring the benefits of their integration; second, the substantial feature disparity between different projects, which can lead to insufficient effect during transfer learning, necessitating additional efforts to narrow this gap to improve precision. Addressing these challenges, this paper proposes a novel methodology, SDP-MTF (Software Defect Prediction using Multi-stage Transfer learning and Feature fusion), that combines code statistical features, deep semantic features, and multiple feature transfer learning methods to enhance the predictive effect. The SDP-MTF method was empirically tested on single-source cross-project software defect prediction across six projects from the PROMISE dataset, benchmarked against five baseline algorithms that employ distinct features and transfer methodologies. Our findings indicate that SDP-MTF significantly outperforms five classical baseline algorithms, improving the F1-Score by 8% to 15.2%, thereby substantively advancing the precision of cross-project software defect prediction.

FCAN : Speech emotion recognition network based on focused contrastive learning

Traditional speech emotion recognition (SER) models predominantly utilize cross-entropy loss for supervised optimization. Yet, given the limited size of SER datasets, relying solely on cross-entropy loss does not adequately capture the intricacies within the data. Moreover, conventional contrastive learning techniques, though improving performance, do not completely harness the inherent data structure across samples. To surmount these limitations, we introduce an innovative emotion recognition network founded on focused contrastive learning. Our model employs bidirectional gated recurrent unit (GRU) networks to seize critical contextual dependencies. By integrating an attention mechanism, we adeptly merge the contextual features of dialogues with individual utterance attributes, forming a comprehensive multi-stage feature fusion framework. This strategy extracts more elaborate emotional features from the constrained dataset. Additionally, by amalgamating the focal loss function into our training protocol, we mitigate imbalanced sample issues to a degree. We further refine our contrastive loss function by enacting a weighted strategy for intra-dialogue category assignments, where higher penalty weights are ascribed to harder-to-identify categories, thus incentivizing the model to concentrate on distinguishing subtler category differences. This refinement significantly bolsters the model’s proficiency in learning various emotional feature distributions, thereby enhancing its emotion recognition capabilities. Experimental validation on IEMOCAP and MELD benchmark datasets underpins our model’s superiority.

Millimeter wave gesture recognition using multi-feature fusion models in complex scenes

As a form of body language, the gesture plays an important role in smart homes, game interactions, and sign language communication, etc. The gesture recognition methods have been carried out extensively. The existing methods have inherent limitations regarding user experience, visual environment, and recognition granularity. Millimeter wave radar provides an effective method for the problems lie ahead gesture recognition because of the advantage of considerable bandwidth and high precision perception. Interfering factors and the complexity of the model raise an enormous challenge to the practical application of gesture recognition methods as the millimeter wave radar is applied to complex scenes. Based on multi-feature fusion, a gesture recognition method for complex scenes is proposed in this work. We collected data in variety places to improve sample reliability, filtered clutters to improve the signal’s signal-to-noise ratio (SNR), and then obtained multi features involves range-time map (RTM), Doppler-time map (DTM) and angle-time map (ATM) and fused them to enhance the richness and expression ability of the features. A lightweight neural network model multi-CNN-LSTM is designed to gestures recognition. This model consists of three convolutional neural network (CNN) for three obtained features and one long short-term memory (LSTM) for temporal features. We analyzed the performance and complexity of the model and verified the effectiveness of feature extraction. Numerous experiments have shown that this method has generalization ability, adaptability, and high robustness in complex scenarios. The recognition accuracy of 14 experimental gestures reached 97.28%.

Multistage feature fusion knowledge distillation

Generally, the recognition performance of lightweight models is often lower than that of large models. Knowledge distillation, by teaching a student model using a teacher model, can further enhance the recognition accuracy of lightweight models. In this paper, we approach knowledge distillation from the perspective of intermediate feature-level knowledge distillation. We combine a cross-stage feature fusion symmetric framework, an attention mechanism to enhance the fused features, and a contrastive loss function for teacher and student models at the same stage to comprehensively implement a multistage feature fusion knowledge distillation method. This approach addresses the problem of significant differences in the intermediate feature distributions between teacher and student models, making it difficult to effectively learn implicit knowledge and thus improving the recognition accuracy of the student model. Compared to existing knowledge distillation methods, our method performs at a superior level. On the CIFAR100 dataset, it boosts the recognition accuracy of ResNet20 from 69.06% to 71.34%, and on the TinyImagenet dataset, it increases the recognition accuracy of ResNet18 from 66.54% to 68.03%, demonstrating the effectiveness and generalizability of our approach. Furthermore, there is room for further optimization of the overall distillation structure and feature extraction methods in this approach, which requires further research and exploration.

Enhancing Predictive Models for Assessing 5G Exposure Effects on Human Health and Cognition through Supervised Machine Learning: A Multi-Stage Feature Selection Approach

Enhancing Predictive Models for Assessing 5G Exposure Effects on Human Health and Cognition through Supervised Machine Learning: A Multi-Stage Feature Selection Approach

Priority-based Multi-feature Vector Model Using Convolution Neural Network for Biometric Authentication

Many societal institutions have increased standards for the efficacy and dependability of identification systems in response to the ever-increasing sophistication of computer technology. Biometric identity systems have mostly supplanted the usage of conventional key and smart card systems in highly secure industries. There has been a dramatic increase in the adoption of biometric authentication systems in the last decade. While many biometric systems have relied on exterior physiological traits like fingerprints, iris scans, palm prints, and faces, very few have investigated the possibility of using interior physiological traits as a biometric. It is challenging for the benefits of unimodal biometric technology to be realized in real-world applications because to its inherent constraints, such as the fact that it only proposes a single piece of information and that environmental factors can impact data verification. While passwords have served their purpose of authenticating users and controlling access, they have also revealed their weaknesses. Concerns about forging or impersonating authentication have prompted the proposal of various Artificial Intelligence (AI)-based solutions. A lot of people are interested in user authentication based on ECGs these days. Unlike other biometrics, an electrocardiogram verifies that the person is actual and alive, making it one of the most trustworthy advanced authentication methods. The P, Q, R, S, and T characteristic points of an ECG signal are responsible for its most salient features. Feature selection is a statistical method that can be implemented in a supervised or unsupervised setting via regression or classification. A Convolution Neural Network (CNN) is a type of neural network that extracts and feeds into another neural network, which then classifies those extracted features. The ECG signal serves as input to a feature extraction network. The neural network performs categorization based on the retrieved feature signals. To overcome the problem that the recognition accuracy of traditional ECG identification methods declines with the rise in the number of testing samples at various moments or throughout different heartbeat cycles, this research offers a multi feature reuse model. In this research, an Associated Priority-based Weighted Multi-Feature Vector model using Convolution Neural Network (APbWMFV-CNN) is proposed for ECG signal-based authentication. The proposed model selects the most appropriate features used for accurate biometric model. The comparison results represent that the proposed model performance in user detection is high.

Self-supervised monocular depth estimation with self-distillation and dense skip connection

Monocular depth estimation (MDE) is crucial in a wide range of applications, including robotics, autonomous driving and virtual reality. Self-supervised monocular depth estimation has emerged as a promising MDE approach without requiring hard-to-obtain depth labels during training, and multi-scale photometric loss is widely used for self-supervised monocular depth estimation as the self-supervised signal. However, multi-photometric loss is a weak training signal and might disturb the good intermediate features representation. In this paper, we propose a successive depth map self-distillation(SDM-SD) loss, which combines with the single-scale photometric loss to replace the multi-scale photometric loss. Moreover, considering that multi-stage feature representations are essential for dense prediction tasks such as depth estimation, we also propose a dense skip connection, which can efficiently fuse the intermediate features of the encoder and fully utilize them in each stage of the decoder in our encoder–decoder architecture. By applying successive depth map self-distillation loss and dense skip connection, our proposed method can achieve state-of-the-art performance on the KITTI benchmark, and exhibit the best generalization ability on the challenging indoor dataset NYUv2 dataset.

An Optical Study on the Ignition and Flame Development Initiated by a Combination of Passive Pre-Chamber and Spark Ignition

ABSTRACT Pre-chamber jet ignition combined with spark ignition in main chamber has advantages in adjusting the combustion phase and improving the burning rate for spark ignition engines. In this study, optical experiments based on a constant volume combustion chamber and high-speed schlieren imaging were conducted to investigate the ignition and flame development under the combined ignition mode with a passive pre-chamber and spark plug using methane fuel. Two modes of flame development were observed with different ignition time interval of pre-chamber ignition and spark ignition (Δt ign). At relatively short Δt ign, the “jet-disturbed flame” mode occurs with a three-stage combustion process: laminar flame; flame acceleration by jet-flame interaction and re-ignition at the jet root; flame deceleration. The maximum global flame speed can be increased from 2 m/s to 9 m/s under λ = 1 and from 1 m/s to 5.5 m/s under λ = 1.3. The ignition delay is relatively short and the heat release shows a slow and multi-stage feature. Moreover, the ignition delay and combustion duration in main chamber are relatively not sensitive to the variation of nozzle diameter. As the spark ignition becomes later (longer Δt ign), the “spark-initiated turbulent flame” mode occurs. In this mode, a rapidly developing turbulent flame is initiated by the spark plug, with a global flame speed up to 16 ~ 18 m/s under λ = 1 and 6 ~ 11 m/s under λ = 1.3. Compared with pre-chamber ignition, a shorter combustion duration is obtained at certain Δt ign and misfire can be avoided because of the synergy of the combined ignition mode.

Detection of Road Crack Images Based on Multistage Feature Fusion and a Texture Awareness Method.

Structural health monitoring for roads is an important task that supports inspection of transportation infrastructure. This paper explores deep learning techniques for crack detection in road images and proposes an automatic pixel-level semantic road crack image segmentation method based on a Swin transformer. This method employs Swin-T as the backbone network to extract feature information from crack images at various levels and utilizes the texture unit to extract the texture and edge characteristic information of cracks. The refinement attention module (RAM) and panoramic feature module (PFM) then merge these diverse features, ultimately refining the segmentation results. This method is called FetNet. We collect four public real-world datasets and conduct extensive experiments, comparing FetNet with various deep-learning methods. FetNet achieves the highest precision of 90.4%, a recall of 85.3%, an F1 score of 87.9%, and a mean intersection over union of 78.6% on the Crack500 dataset. The experimental results show that the FetNet approach surpasses other advanced models in terms of crack segmentation accuracy and exhibits excellent generalizability for use in complex scenes.

PMA-Net: Progressive multi-stage adaptive feature learning for two-view correspondence

Establishing high-quality correspondences is a fundamental step for many computer vision tasks. Leveraging the local consistency of correct correspondences (i.e., inliers) has been a prevalent approach to distinguish them from incorrect correspondences (i.e., outliers). However, the random distribution of a significant proportion of outliers complicates the local neighborhood construction of inliers, making it inevitably contain many outliers, which compromises the reliability of the consistency information. In this paper, we propose a Progressive Multi-Stage Adaptive Feature Learning Network (PMA-Net) to solve this problem, which progressively and adaptively learns over multiple stages to remove random outliers embedded in the initial consistency. Specifically, we propose an Adaptive Dynamic Graph Construction (ADGC) module to construct a global topological graph with a high inlier ratio by calculating the affinity between the corresponding neighbors. Additionally, we also design a Feature Mapping Processing (FMP) block and Multi-Stage Prediction (MSP) block to improve the accuracy of subsequent local neighborhood information aggregation, as well as information loss due to channel dimension compression in the prediction phase. Experimental results in camera pose estimation, remote sensing image registration, and homography estimation demonstrate that the proposed PMA-Net performs better than other state-of-the-art methods on various public datasets. Code: https://github.com/XiaojieLi11/PMA-Net

Deep learning DGA malicious domain name detection based on multi-stage feature fusion

Deep learning DGA malicious domain name detection based on multi-stage feature fusion

Self-supervised learning for interventional image analytics: toward robust device trackers.

The accurate detection and tracking of devices, such as guiding catheters in live X-ray image acquisitions, are essential prerequisites for endovascular cardiac interventions. This information is leveraged for procedural guidance, e.g., directing stent placements. To ensure procedural safety and efficacy, there is a need for high robustness/no failures during tracking. To achieve this, one needs to efficiently tackle challenges, such as device obscuration by the contrast agent or other external devices or wires and changes in the field-of-view or acquisition angle, as well as the continuous movement due to cardiac and respiratory motion. To overcome the aforementioned challenges, we propose an approach to learn spatio-temporal features from a very large data cohort of over 16million interventional X-ray frames using self-supervision for image sequence data. Our approach is based on a masked image modeling technique that leverages frame interpolation-based reconstruction to learn fine inter-frame temporal correspondences. The features encoded in the resulting model are fine-tuned downstream in a light-weight model. Our approach achieves state-of-the-art performance, in particular for robustness, compared to ultra optimized reference solutions (that use multi-stage feature fusion or multi-task and flow regularization). The experiments show that our method achieves a 66.31% reduction in the maximum tracking error against the reference solutions (23.20% when flow regularization is used), achieving a success score of 97.95% at a faster inference speed of 42 frames-per-second (on GPU). In addition, we achieve a 20% reduction in the standard deviation of errors, which indicates a much more stable tracking performance. The proposed data-driven approach achieves superior performance, particularly in robustness and speed compared with the frequently used multi-modular approaches for device tracking. The results encourage the use of our approach in various other tasks within interventional image analytics that require effective understanding of spatio-temporal semantics.

SAMT-generator: A second-attention for image captioning based on multi-stage transformer network

In recent years, Transformer has been widely used in the crossing task of computer vision (CV) and natural language processing (NLP), e.g., image captioning. The prior works of image captioning based on Transformer have achieved remarkable progress. In order to further improve the model’s ability to describe detail features and generate high-quality sentences, in the paper, we first construct a Multi-stage Transformer Feature Enhancement Network (MT-FEN), which obtains more semantics by fusing features at different scales. Furthermore, we crucially propose a novel second-attention (SA) that can focus more on valuable features and filter out noises in MT-FEN. Besides that, to generate more sensible sentences, we use multiple decoding layers to choose the best-described word via the Max-out module. The overall model is dubbed as SAMT-Generator. After conducting extensive experiments on two well-known datasets: MS COCO and Flickr8k, the results demonstrate the effectiveness of our proposed model, and it is comparable with SOTA methods.