Feature Learning Techniques Research Articles

Temporal denoising and deep feature learning for enhanced defect detection in thermography using stacked denoising convolution autoencoder

Thermal wave imaging uses the temporal temperature distribution over the object’s surface for subsurface analysis. However, the noise generated during experimentation corrupts this temporal history and hampers the detection of defect signatures. As denoising of the temporal thermal history enhances the defect detectability, this study offers a stacked denoising convolution autoencoder (SDCAE) in frequency-modulated thermal wave imaging with one-dimensional convolution layers to reduce noise in temporal thermal evolution and train high-level features resulting in improved defect signs. Experimental results on mild steel and carbon fiber reinforced polymer specimens with different sizes of defects at various depths demonstrate that integrating temporal denoising and deep feature learning techniques into a single novel framework significantly improved defect detectability. In addition, defect signal-to-noise ratios of the denoised thermal data and latent space of the proposed model compared to conventional autoencoder and dimensionality reduction techniques recommend the superiority of the proposed method.

A survey on feature extraction and learning techniques for link prediction in homogeneous and heterogeneous complex networks

Complex networks are commonly observed in several real-world areas, such as social, biological, and technical systems, where they exhibit complicated patterns of connectedness and organised clusters. These networks have intricate topological characteristics that frequently elude conventional characterization. Link prediction in complex networks, like data flow in telecommunications networks, protein interactions in biological systems, and social media interactions on platforms like Facebook, etc., is an essential element of network analytics and presents fresh research challenges. Consequently, there is a growing emphasis in research on creating new link prediction methods for different network applications. This survey investigates several strategies related to link prediction, ranging from feature extraction based to feature learning based techniques, with a specific focus on their utilisation in dynamic and developing network topologies. Furthermore, this paper emphasises on a wide variety of feature learning techniques that go beyond basic feature extraction and matrix factorization. It includes advanced learning-based algorithms and neural network techniques specifically designed for link prediction. The study also presents evaluation results of different link prediction techniques on homogeneous and heterogeneous network datasets, and provides a thorough examination of existing methods and potential areas for further investigation.

A Temporal Fusion Memory Network-Based Method for State-of-Health Estimation of Lithium-Ion Batteries

As energy storage technologies and electric vehicles evolve quickly, it becomes increasingly difficult to precisely gauge the condition (SOH) of lithium-ion batteries (LiBs) during rapid charging scenarios. This paper introduces a novel Time-Fused Memory Network (TFMN) for SOH estimation, integrating advanced feature extraction and learning techniques. Both directly measured and computationally derived features are extracted from the charge/discharge curves to simulate real-world fast-charging conditions. This comprehensive process captures the complex dynamics of battery behavior effectively. The TFMN method utilizes one-dimensional convolutional neural networks (1DCNNs) to capture local features, refined further by a channel self-attention module (CSAM) for robust SOH prediction. Long short-term memory (LSTM) modules process these features to capture long-term dependencies essential for understanding evolving battery health patterns. A multi-head attention module enhances the model by learning varied feature representations, significantly improving SOH estimation accuracy. Validated on a self-constructed dataset and the public Toyota dataset, the model demonstrates superior accuracy and robustness, improving performance by 30–50% compared to other models. This approach not only refines SOH estimation under fast-charging conditions but also offers new insights for effective battery management and maintenance, advancing battery health monitoring technologies.

Predicting life satisfaction using machine learning and explainable AI

Life satisfaction is a crucial facet of human well-being. Hence, research on life satisfaction is incumbent for understanding how individuals experience their lives and influencing interventions targeted at enhancing mental health and well-being. Life satisfaction has traditionally been measured using analog, complicated, and frequently error-prone methods. These methods raise questions concerning validation and propagation. However, this study demonstrates the potential for machine learning algorithms to predict life satisfaction with a high accuracy of 93.80% and a 73.00% macro F1-score. The dataset comes from a government survey of 19000 people aged 16-64 years in Denmark. Using feature learning techniques, 27 significant questions for assessing contentment were extracted, making the study highly reproducible, simple, and easily interpretable. Furthermore, clinical and biomedical large language models (LLMs) were explored for predicting life satisfaction by converting tabular data into natural language sentences through mapping and adding meaningful counterparts, achieving an accuracy of 93.74% and macro F1-score of 73.21%. It was found that life satisfaction prediction is more closely related to the biomedical domain than the clinical domain. Ablation studies were also conducted to understand the impact of data resampling and feature selection techniques on model performance. Moreover, the correlation between primary determinants with different age brackets was analyzed, and it was found that health condition is the most important determinant across all ages. The best performing Machine Learning model trained in this study is deployed on a public server, ensuring unrestricted usage of the model. We highlight the advantages of machine learning methods for predicting life satisfaction and the significance of XAI for interpreting and validating these predictions. This study demonstrates how machine learning, large language models and XAI can jointly contribute to building trust and understanding in using AI to investigate human behavior, with significant ramifications for academics and professionals working to quantify and comprehend subjective well-being.

A hybrid multimodal machine learning model for Detecting Alzheimer's disease

Alzheimer's disease (AD) diagnosis utilizing single modality neuroimaging data has limitations. Multimodal fusion of complementary biomarkers may improve diagnostic performance. This study proposes a multimodal machine learning framework integrating magnetic resonance imaging (MRI), positron emission tomography (PET) and cerebrospinal fluid (CSF) assays for enhanced AD characterization. The model incorporates a hybrid algorithm combining enhanced Harris Hawks Optimization (HHO) algorithm referred to as ILHHO, with Kernel Extreme Learning Machine (KELM) classifier for simultaneous feature selection and classification. ILHHO enhances HHO's search efficiency by integrating iterative mapping (IM) to improve population diversity and local escaping operator (LEO) to balance exploration-exploitation. Comparative analysis with other improved HHO algorithms, classic meta-heuristic algorithms (MHAs), and state-of-the-art MHAs on IEEE CEC2014 benchmark functions indicates that ILHHO achieves superior optimization performance compared to other comparative algorithms. The synergistic ILHHO-KELM model is evaluated on 202 AD Neuroimaging Initiative (ADNI) subjects. Results demonstrate superior multimodal classification accuracy over single modalities, validating the importance of fusing heterogeneous biomarkers. MRI + PET + CSF achieves 99.2 % accuracy for AD vs. normal control (NC), outperforming conventional and proposed methods. Discriminative feature analysis provides further insights into differential AD-related neurodegeneration patterns detected by MRI and PET. The differential PET and MRI features demonstrate how the two modalities provide complementary biomarkers. The neuroanatomical relevance of selected features supports ILHHO-KELM's potential for extracting sensitive AD imaging signatures. Overall, the study showcases the advantages of capitalizing on complementary multimodal data through advanced feature learning techniques for improving AD diagnosis.

Structural Tensor Learning for Event Identification With Limited Labels

The increasing uncertainty of distributed energy resources promotes the risks of transient events for power systems. To capture event dynamics, Phasor Measurement Unit (PMU) data is widely utilized due to its high resolutions. Notably, Machine Learning (ML) methods can process PMU data with feature learning techniques to identify events. However, existing ML-based methods face the following challenges due to salient characteristics from both the measurements and the labels: (1) PMU streams have a large size with redundancy and correlations across temporal, spatial, and measurement type dimensions. Nevertheless, existing work cannot effectively uncover the structural correlations to remove redundancy and learn useful features. (2) The number of event labels is limited, but most models focus on learning with labeled data, suffering risks of non-robustness to different system conditions. To overcome the above issues, we propose an approach called Kernelized Tensor Decomposition and Classification with Semi-supervision (KTDC-Se). Firstly, we show that the key is to tensorize data storage, filter information via decomposition, and learn discriminative features via classification. This leads to an efficient exploration of structural correlations via high-dimensional tensors. Secondly, the proposed KTDC-Se can incorporate rich unlabeled data to seek decomposed tensors, invariant to varying operational conditions. Thirdly, we make KTDC-Se a joint model of decomposition and classification so that there are no biased selections of the two steps. Finally, to boost the model accuracy, we add kernels for non-linear feature learning. We demonstrate the KTDC-Se superiority over the state-of-the-art methods for event identification using PMU data.

Autonomous Cross Domain Adaptation Under Extreme Label Scarcity.

A cross domain multistream classification is a challenging problem calling for fast domain adaptations to handle different but related streams in never-ending and rapidly changing environments. Notwithstanding that existing multistream classifiers assume no labeled samples in the target stream, they still incur expensive labeling costs since they require fully labeled samples of the source stream. This article aims to attack the problem of extreme label shortage in the cross domain multistream classification problems where only very few labeled samples of the source stream are provided before process runs. Our solution, namely, Learning Streaming Process from Partial Ground Truth (LEOPARD), is built upon a flexible deep clustering network where its hidden nodes, layers, and clusters are added and removed dynamically with respect to varying data distributions. A deep clustering strategy is underpinned by a simultaneous feature learning and clustering technique leading to clustering-friendly latent spaces. A domain adaptation strategy relies on the adversarial domain adaptation technique where a feature extractor is trained to fool a domain classifier by classifying source and target streams. Our numerical study demonstrates the efficacy of LEOPARD where it delivers improved performances compared to prominent algorithms in 15 of 24 cases. Source codes of LEOPARD are shared in https://github.com/wengweng001/LEOPARD.git to enable further study.

Consistent representation via contrastive learning for skin lesion diagnosis

BackgroundSkin lesions are a prevalent ailment, with melanoma emerging as a particularly perilous variant. Encouragingly, artificial intelligence displays promising potential in early detection, yet its integration within clinical contexts, particularly involving multi-modal data, presents challenges. While multi-modal approaches enhance diagnostic efficacy, the influence of modal bias is often disregarded. MethodsIn this investigation, a multi-modal feature learning technique termed “Contrast-based Consistent Representation Disentanglement” for dermatological diagnosis is introduced. This approach employs adversarial domain adaptation to disentangle features from distinct modalities, fostering a shared representation. Furthermore, a contrastive learning strategy is devised to incentivize the model to preserve uniformity in common lesion attributes across modalities. Emphasizing the learning of a uniform representation among models, this approach circumvents reliance on supplementary data. ResultsAssessment of the proposed technique on a 7-point criteria evaluation dataset yields an average accuracy of 76.1% for multi-classification tasks, surpassing researched state-of-the-art methods. The approach tackles modal bias, enabling the acquisition of a consistent representation of common lesion appearances across diverse modalities, which transcends modality boundaries. This study underscores the latent potential of multi-modal feature learning in dermatological diagnosis. ConclusionIn summation, a multi-modal feature learning strategy is posited for dermatological diagnosis. This approach outperforms other state-of-the-art methods, underscoring its capacity to enhance diagnostic precision for skin lesions.

Face recognition system with hybrid template protection scheme for Cyber–Physical-Social Services

This paper presents a secure face recognition system with advanced template protection schemes for Cyber–Physical–Social Services (CPSS). The implementation of the proposed system consists of five components. The initial step performs image preprocessing, where it detects the facial region from the captured image using the Tree-Structured Part Model (TSPM). The second phase involves feature extraction, where it utilizes the Scale Invariant Feature Transform (SIFT) descriptor to extract features from small patches of the preprocessed images, forming a collection of feature descriptors. The collection of feature descriptors is then clustered using the K-means clustering algorithm, returning the centers of K-clusters that serve as the vocabulary of a dictionary. Finally, a histogram is generated using the vocabularies and frequencies, referred to as the “Bag of Visual Words (BoVW)”. Using this dictionary and a feature learning technique called Sparse Representation Coding (SRC), followed by Spatial Pyramid Mapping (SPM), the system generates feature vectors from training/testing image samples. In the third component, the modified FaceHashing technique is applied to the original feature vectors, generating cancelable feature vectors. The fourth component employs a Bio-Cryptographic technique to preserve the cancelable feature vectors in a database. Lastly, the fifth component utilizes a multi-class linear SVM classifier on the decrypted and query-cancellable feature vector to classify users. The system evaluates its performance using FERET and CASIA-FaceV5 benchmark databases, providing 100% identification accuracy for 200-dimensional cancelable feature vectors. The performance and security comparisons demonstrate the superiority of the proposed system over existing methods.

Dual-drive collaboration surrogate-assisted evolutionary algorithm by coupling feature reduction and reconstruction

Surrogate-assisted evolutionary algorithm (SAEA) prevails in the optimization of computationally expensive problems. However, existing SAEAs confront low efficiency in the resolution of high-dimensional problems characterized by multiple local optima and multivariate coupling. To this end, this paper offers a dual-drive collaboration surrogate-assisted evolutionary algorithm (DDCSAEA) by coupling feature reduction and reconstruction, which coordinates two unsupervised feature learning techniques, i.e., principal component analysis and autoencoder, in tandem. DDCSAEA creates a low-dimensional solution space by downscaling the target high-dimensional space via principal component analysis and collects promising candidates in the reduced space by collaborating a surrogate-assisted evolutionary sampling with differential mutation. An autoencoder is used to perform the feature reconstruction on the collected candidates for infill-sampling in the target high-dimensional space to sequentially refine the neighborhood landscapes of the optimal solution. Experimental results reveal that DDCSAEA has stronger convergence performance and optimization efficiency against eight state-of-the-art SAEAs on high-dimensional benchmark problems within 200 dimensions.

Learning Disentangled Representation Implicitly Via Transformer for Occluded Person Re-Identification

Person re-IDentification (re-ID) under various occlusions has been a long-standing challenge as person images with different types of occlusions often suffer from misalignment in image matching and ranking. Most existing methods tackle this challenge by aligning spatial features of body parts according to external semantic cues or feature similarities but this alignment approach is complicated and sensitive to noises. We design DRL-Net, a disentangled representation learning network that handles occluded re-ID without requiring strict person image alignment or any additional supervision. Leveraging transformer architectures, DRL-Net achieves alignment-free re-ID via global reasoning of local features of occluded person images. It measures image similarity by automatically disentangling the representation of undefined semantic components, e.g., human body parts or obstacles, under the guidance of semantic preference object queries in the transformer. In addition, we design a decorrelation constraint in the transformer decoder and impose it over object queries for better focus on different semantic components. To better eliminate interference from occlusions, we design a contrast feature learning technique (CFL) for better separation of occlusion features and discriminative ID features. Extensive experiments over occluded and holistic re-ID benchmarks show that the DRL-Net achieves superior re-ID performance consistently and outperforms the state-offi-the-art by large margins for occluded re-ID dataset.

Deep Learning in Social Networks for Overlappering Community Detection

The collection of nodes is termed as community in any network system that are tightly associated to the other nodes. In network investigation, identifying the community structure is crucial task, particularly for exposing connections between certain nodes. For community overlapping, network discovery, there are numerous methodologies described in the literature. Numerous scholars have recently focused on network embedding and feature learning techniques for node clustering. These techniques translate the network into a representation space with fewer dimensions. In this paper, a deep neural network-based model for learning graph representation and stacked auto-encoders are given a nonlinear embedding of the original graph to learn the model. In order to extract overlapping communities, an AEOCDSN algorithm is used. The efficiency of the suggested model is examined through experiments on real-world datasets of various sizes and accepted standards. The method outperforms various well-known community detection techniques, according to empirical findings.

Feature selection and feature learning in machine learning applications for gas turbines: A review

The progress of machine learning (ML) in the past years has opened up new opportunities to the field of gas turbine (GT) modelling. However, successful implementation of ML algorithms remains challenging, particularly for complex problems such as multi-mode faults. An important tool for enabling applications are the feature selection and feature learning (FSFL) techniques. In particular, FL techniques have recently facilitated and improved the applicability of ML to GT modelling.This review paper conducts a review on 46 studies that utilised FSFL for GT modelling with ML. The purpose of this review is to investigate how FSFL techniques can help address GT modelling challenges and when researchers should deploy them. Therefore, the theories behind the techniques are illustrated in depth along with practical application examples from the analysed literature. The advantages and limitations of FSFL are discussed, the computational costs of different techniques are compared, and trends in the field are highlighted. Consequently, a novel categorisation framework for FSFL techniques and recommendations regarding when and how to implement them are provided. A new knowledge accumulation, extraction, and transfer concept is proposed to address GT modelling challenges.

Multivariate Correlation-aware Spatio-temporal Graph Convolutional Networks for Multi-scale Traffic Prediction

Traffic flow prediction based on vehicle trajectories collected from the installed GPS devices is critically important to Intelligent Transportation Systems (ITS). One limitation of existing traffic prediction models is that they mostly focus on predicting road-segment level traffic conditions, which can be considered as a fine-grained prediction. In many scenarios, however, a coarse-grained prediction, such as predicting the traffic flows among different urban areas covering multiple road links, is also required to help government have a better understanding on traffic conditions from the macroscopic point of view. This is especially useful in the applications of urban planning and public transportation planning. Another limitation is that the correlations among different types of traffic-related features are largely ignored. For example, the traffic flow and traffic speed are usually negatively correlated. Existing works regard these traffic-related features as independent features without considering their correlations. In this article, we for the first time study the novel problem of multivariate correlation-aware multi-scale traffic flow predicting, and we propose a feature correlation-aware spatio-temporal graph convolutional networks named MC-STGCN to effectively address it. Specifically, given a road graph, we first construct a coarse-grained road graph based on both the topology closeness and the traffic flow similarity among the nodes (road links). Then a cross-scale spatial-temporal feature learning and fusion technique is proposed for dealing with both the fine- and coarse-grained traffic data. In the spatial domain, a cross-scale GCN is proposed to learn the multi-scale spatial features jointly and fuse them together. In the temporal domain, a cross-scale temporal network that is composed of a hierarchical attention is designed for effectively capturing intra- and inter-scale temporal correlations. To effectively capture the feature correlations, a feature correlation learning component is also designed. Finally, a structural constraint is introduced to make the predictions on the two scale traffic data consistent. We conduct extensive evaluations over two real traffic datasets, and the results demonstrate the superior performance of the proposal on both fine- and coarse-grained traffic predictions.

Ensemble streamflow forecasting based on variational mode decomposition and long short term memory

Reliable and accurate streamflow forecasting plays a vital role in the optimal management of water resources. To improve the stability and accuracy of streamflow forecasting, a hybrid decomposition-ensemble model named VMD-LSTM-GBRT, which is sensitive to sampling, noise and long historical changes of streamflow, was established. The variational mode decomposition (VMD) algorithm was first applied to extract features, which were then learned by several long short-term memory (LSTM) networks. Simultaneously, an ensemble tree, a gradient boosting tree for regression (GBRT), was trained to model the relationships between the extracted features and the original streamflow. The outputs of these LSTMs were finally reconstructed by the GBRT model to obtain the forecasting streamflow results. A historical daily streamflow series (from 1/1/1997 to 31/12/2014) for Yangxian station, Han River, China, was investigated by the proposed model. VMD-LSTM-GBRT was compared with respect to three aspects: (1) feature extraction algorithm; ensemble empirical mode decomposition (EEMD) was used. (2) Feature learning techniques; deep neural networks (DNNs) and support vector machines for regression (SVRs) were exploited. (3) Ensemble strategy; the summation strategy was used. The results indicate that the VMD-LSTM-GBRT model overwhelms all other peer models in terms of the root mean square error (RMSE = 36.3692), determination coefficient (R2 = 0.9890), mean absolute error (MAE = 9.5246) and peak percentage threshold statistics (PPTS(5) = 0.0391%). The addressed approach based on the memory of long historical changes with deep feature representations had good stability and high prediction precision.

Unsupervised Feature Learning Using Recurrent Neural Nets for Segmenting Hyperspectral Images

Although deep learning is gaining more widespread use in hyperspectral image analysis, it is challenging to train high-capacity models in a supervised way—ground-truth sets are expensive to obtain, and they are practically always extremely imbalanced. To deal with the problem of missing ground-truth data, its high dimensionality and potential redundancy, we introduce a novel unsupervised feature learning technique to extract discriminative features from the original data. It exploits recurrent neural network-based asymmetric autoencoders (AEs) to learn the compressed representation of unlabeled data, and can elaborate both spectral and spectral–spatial features. Our extractors can be incorporated into the unsupervised segmentation pipeline—they can be followed by any clustering algorithm. The experiments revealed that our approaches deliver high-quality segmentation without any prior class labels, and are one order of magnitude faster than 3-D convolutional AEs. Our algorithms outperform or work on par with other approaches while allowing for significant data reduction.

Efficient [formula omitted]-nearest neighbor search based on clustering and adaptive [formula omitted] values

The k-Nearest Neighbor (kNN) algorithm is widely used in the supervised learning field and, particularly, in search and classification tasks, owing to its simplicity, competitive performance, and good statistical properties. However, its inherent inefficiency prevents its use in most modern applications due to the vast amount of data that the current technological evolution generates, being thus the optimization of kNN-based search strategies of particular interest. This paper introduces the caKD+ algorithm, which tackles this limitation by combining the use of feature learning techniques, clustering methods, adaptive search parameters per cluster, and the use of pre-calculated K-Dimensional Tree structures, and results in a highly efficient search method. This proposal has been evaluated using 10 datasets and the results show that caKD+ significantly outperforms 16 state-of-the-art efficient search methods while still depicting such an accurate performance as the one by the exhaustive kNN search.

DensePILAE: a feature reuse pseudoinverse learning algorithm for deep stacked autoencoder

Autoencoder has been widely used as a feature learning technique. In many works of autoencoder, the features of the original input are usually extracted layer by layer using multi-layer nonlinear mapping, and only the features of the last layer are used for classification or regression. Therefore, the features of the previous layer aren’t used explicitly. The loss of information and waste of computation is obvious. In addition, faster training and reasoning speed is generally required in the Internet of Things applications. But the stacked autoencoders model is usually trained by the BP algorithm, which has the problem of slow convergence. To solve the above two problems, the paper proposes a dense connection pseudoinverse learning autoencoder (DensePILAE) from reuse perspective. Pseudoinverse learning autoencoder (PILAE) can extract features in the form of analytic solution, without multiple iterations. Therefore, the time cost can be greatly reduced. At the same time, the features of all the previous layers in stacked PILAE are combined as the input of next layer. In this way, the information of all the previous layers not only has no loss, but also can be strengthened and refined, so that better features could be learned. The experimental results in 8 data sets of different domains show that the proposed DensePILAE is effective.

Unsupervised Feature Learning for Speech Emotion Recognition Based on Autoencoder

Speech signals contain abundant information on personal emotions, which plays an important part in the representation of human potential characteristics and expressions. However, the deficiency of emotion speech data affects the development of speech emotion recognition (SER), which also limits the promotion of recognition accuracy. Currently, the most effective approach is to make use of unsupervised feature learning techniques to extract speech features from available speech data and generate emotion classifiers with these features. In this paper, we proposed to implement autoencoders such as a denoising autoencoder (DAE) and an adversarial autoencoder (AAE) to extract the features from LibriSpeech for model pre-training, and then conducted experiments on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) datasets for classification. Considering the imbalance of data distribution in IEMOCAP, we developed a novel data augmentation approach to optimize the overlap shift between consecutive segments and redesigned the data division. The best classification accuracy reached 78.67% (weighted accuracy, WA) and 76.89% (unweighted accuracy, UA) with AAE. Compared with state-of-the-art results to our knowledge (76.18% of WA and 76.36% of UA with the supervised learning method), we achieved a slight advantage. This suggests that using unsupervised learning benefits the development of SER and provides a new approach to eliminate the problem of data scarcity.

A compositional model for effort‐aware Just‐In‐Time defect prediction on android apps

Android apps have played important roles in daily life and work. To meet the new requirements from users, the apps encounter frequent updates, which involves a large quantity of code commits. Previous studies proposed to apply Just-in-Time (JIT) defect prediction for apps to timely identify whether the new code commits can introduce defects into apps, aiming to assure their quality. In general, high-quality features are benefits for improving the classification performance. In addition, the number of defective commit instances is much fewer than that of clean ones, that is the defect data is class imbalanced. In this study, a novel compositional model, called KPIDL, is proposed to conduct the JIT defect prediction task for Android apps. More specifically, KPIDL first exploits a feature learning technique to preprocess original data for obtaining better feature representation, and then introduces a state-of-the-art cost-sensitive cross-entropy loss function into the deep neural network to alleviate the class imbalance issue by considering the prior probability of the two types of classes. The experiments were conducted on a benchmark defect data consisting of 15 Android apps. The experimental results show that the proposed KPIDL model performs significantly better than 25 comparative methods in terms of two effort-aware performance indicators in most cases.