Low-dimensional Feature Space Research Articles

Accelerating particle-in-cell kinetic plasma simulations via reduced-order modeling of space-charge dynamics using dynamic mode decomposition.

We present a data-driven reduced-order modeling of the space-charge dynamics for electromagnetic particle-in-cell (EMPIC) plasma simulations based on dynamic mode decomposition (DMD). The dynamics of the charged particles in kinetic plasma simulations such as EMPIC is manifested through the plasma current density defined along the edges of the spatial mesh. We showcase the efficacy of DMD in modeling the time evolution of current density through a low-dimensional feature space. Not only do such DMD-based predictive reduced-order models help accelerate EMPIC simulations, they also have the potential to facilitate investigative analysis and control applications. We demonstrate the proposed DMD-EMPIC scheme for reduced-order modeling of current density and speedup in EMPIC simulations involving electron beam under the influence of magnetic field, virtual cathode oscillations, and backward wave oscillator.

Using autoencoders and deep transfer learning to determine the stellar parameters of 286 CARMENES M dwarfs

Deep learning (DL) techniques are a promising approach among the set of methods used in the ever-challenging determination of stellar parameters in M dwarfs. In this context, transfer learning could play an important role in mitigating uncertainties in the results due to the synthetic gap (i.e. difference in feature distributions between observed and synthetic data). We propose a feature-based deep transfer learning (DTL) approach based on autoencoders to determine stellar parameters from high-resolution spectra. Using this methodology, we provide new estimations for the effective temperature, surface gravity, metallicity, and projected rotational velocity for 286 M dwarfs observed by the CARMENES survey. Using autoencoder architectures, we projected synthetic PHOENIX-ACES spectra and observed CARMENES spectra onto a new feature space of lower dimensionality in which the differences between the two domains are reduced. We used this low-dimensional new feature space as input for a convolutional neural network to obtain the stellar parameter determinations. We performed an extensive analysis of our estimated stellar parameters, ranging from 3050 to 4300\,K, 4.7 to 5.1\,dex, and $-$0.53 to 0.25\,dex for eff $, log g and Fe/H respectively. Our results are broadly consistent with those of recent studies using CARMENES data, with a systematic deviation in our $T_ eff $ scale towards hotter values for estimations above 3\,750\,K. Furthermore, our methodology mitigates the deviations in metallicity found in previous DL techniques due to the synthetic gap. We consolidated a DTL-based methodology to determine stellar parameters in M dwarfs from synthetic spectra, with no need for high-quality measurements involved in the knowledge transfer. These results suggest the great potential of DTL to mitigate the differences in feature distributions between the observations and the PHOENIX-ACES spectra.

Simultaneous Three-Degrees-of-Freedom Prosthetic Control Based on Linear Regression and Closed-Loop Training Protocol.

Machine learning-based controllers of prostheses using electromyographic signals have become very popular in the last decade. The regression approach allows a simultaneous and proportional control of the intended movement in a more natural way than the classification approach, where the number of movements is discrete by definition. However, it is not common to find regression-based controllers working for more than two degrees of freedom at the same time. In this paper, we present the application of the adaptive linear regressor in a relatively low-dimensional feature space with only eight sensors to the problem of a simultaneous and proportional control of three degrees of freedom (left-right, up-down and open-close hand movements). We show that a key element usually overlooked in the learning process of the regressor is the training paradigm. We propose a closed-loop procedure, where the human learns how to improve the quality of the generated EMG signals, helping also to obtain a better controller. We apply it to 10 healthy and 3 limb-deficient subjects. Results show that the combination of the multidimensional targets and the open-loop training protocol significantly improve the performance, increasing the average completion rate from 53% to 65% for the most complicated case of simultaneously controlling the three degrees of freedom.

Beaconet: A Reference-Free Method for Integrating Multiple Batches of Single-Cell Transcriptomic Data in Original Molecular Space.

Integrating multiple single-cell datasets is essential for the comprehensive understanding of cell heterogeneity. Batch effect is the undesired systematic variations among technologies or experimental laboratories that distort biological signals and hinder the integration of single-cell datasets. However, existing methods typically rely on a selected dataset as a reference, leading to inconsistent integration performance using different references, or embed cells into uninterpretable low-dimensional feature space. To overcome these limitations, a reference-free method, Beaconet, for integrating multiple single-cell transcriptomic datasets in original molecular space by aligning the global distribution of each batch using an adversarial correction network is presented. Through extensive comparisons with 13 state-of-the-art methods, it is demonstrated that Beaconet can effectively remove batch effect while preserving biological variations and is superior to existing unsupervised methods using all possible references in overall performance. Furthermore, Beaconet performs integration in the original molecular feature space, enabling the characterization of cell types and downstream differential expression analysis directly using integrated data with gene-expression features. Additionally, when applying to large-scale atlas data integration, Beaconet shows notable advantages in both time- and space-efficiencies. In summary, Beaconet serves as an effective and efficient batch effect removal tool that can facilitate the integration of single-cell datasets in a reference-free and molecular feature-preserved mode.

Health status monitoring of high-speed train brake pads considering noise under variable working conditions

The brake pads of high-speed trains operate under complex and variable conditions, and the collected brake signals are easily affected by noise, making monitoring the health status of brake pads more difficult. A multi-representation adaptation network for online monitoring the health status of high-speed train brake pads, which are affected by noise under variable working conditions, is proposed in this study. First, a parameter-sharing deep residual network is used to extract the friction block features of the source and target domain data. Then, the features are mapped to different low-dimensional feature spaces through the inception adaptation module, and multiple representations are obtained. The network applies conditional maximum mean discrepancy to align representations of the source and target domains, thus learning multiple domain-invariant representations. Hence, the network acquires more knowledge of the friction block status and attenuates the interference of noise signals on the status monitoring. The friction block vibration data were collected from various brake disc speeds, and variable working condition-transfer experiments under the influence of noise were performed on the brake friction and bearing datasets. The results show that the proposed network outperforms other transfer methods, which can better extract and identify the status features of the friction block under the noise interference.

Principal trade-off analysis

How are the advantage relations between a set of agents playing a game organized and how do they reflect the structure of the game? In this paper, we illustrate ‘Principal Trade-off Analysis’ (PTA), a decomposition method that embeds games into a low-dimensional feature space. We argue that the embeddings are more revealing than previously demonstrated by developing an analogy to Principal Component Analysis (PCA). PTA represents an arbitrary two-player zero-sum game as linear combination of simple games via the projection of policy profiles into orthogonal 2D feature planes. We show that the feature planes represent unique strategic trade-offs and truncation of the sequence provides insightful model reduction and visualization. We demonstrate the validity of PTA on a quartet of games (Kuhn poker, RPS + 2, Blotto and Pokemon). In Kuhn poker, PTA clearly identifies the trade-off between bluffing and calling. In Blotto, PTA identifies game symmetries and specifies strategic trade-offs associated with distinct win conditions. These symmetries reveal limitations of PTA unaddressed in previous work. For Pokemon, PTA recovers clusters that naturally correspond to Pokemon types, correctly identifies the designed trade-off between those types, and discovers a rock-paper-scissor (RPS) cycle in the Pokemon generation type – all absent any specific information except game outcomes.

Forecasting individual bids in real electricity markets through machine learning framework

With the increasing uncertainty caused by the complexity of the world’s energy environment and the increasing penetration rate of renewable energy, it is significant to estimate the future operation of power markets in advance. Forecasting individual bids in spot electricity markets is a promising new method for achieving so, but it has not been fully studied due to the difficulty of forecasting a bid function. The idealization of existing optimization-based models decreases their practical effects in real markets. Thus, we propose a scalable forecasting framework that incorporates several customized state-of-art machine learning methods according to the characteristics of the bidding data. First, several low-rank approximation algorithms are customized to encode the high-dimensional bidding curves into low-dimensional feature spaces and reconstruct them from the predicted feature space. Second, a transformer-based multidimensional time series prediction algorithm is proposed to predict the bidding feature based on both related factors and historical bidding records. To appropriately evaluate the performances of the forecasting methods, we introduce a dynamic criterion based on the economic implications of bids. The comprehensive framework is tested based on actual market data from the Australian national electricity market, and in the empirical example, the feasibility and effectiveness of the proposed framework are demonstrated.

Label enhancement via manifold approximation and projection with graph convolutional network

Label enhancement (LE) aims to enrich logical labels into their corresponding label distributions. But existing LE algorithms fail to fully leverage the structural information in the feature space to improve LE learning. To address this key issue, we first apply manifold learning to map the relatedness between low-dimensional feature samples to the label space. Based on the smoothness assumption of manifolds, the implicit correlation between low-dimensional feature and label spaces effectively promotes the LE process, enabling the learning model to accurately capture the mapping relationship between feature and label manifolds. This leads to an LE based on feature representation (LEFR) algorithm. We also propose an LE algorithm based on graph convolutional network (GCN), called LE-GCN. Inspired by the relationship between threshold connections and label connections, we extend GCN to the LE field for the first time to fully exploit the hidden relationships between nodes and labels. By enhancing node information with threshold connections and label connections, the label learning accuracy reaches a new level. Experiments on real-world datasets show that our LEFR and LE-GCN outperform several state-of-the-art LE algorithms.

The hidden fitness of the male zebra finch courtship song.

Vocal learning in songbirds is thought to have evolved through sexual selection, with female preference driving males to develop large and varied song repertoires1-3. However, many songbird species learn only a single song in their lifetime4. How sexual selection drives the evolution of single-song repertoires is not known. Here, by applying dimensionality-reduction techniques to the singing behaviour of zebra finches (Taeniopygia guttata), we show that syllable spread in low-dimensional feature space explains how single songs function as honest indicators of fitness. We find that this Gestalt measure of behaviour captures the spectrotemporal distinctiveness of song syllables in zebra finches; that females strongly prefer songs that occupy more latent space; and that matching path lengths in low-dimensional space is difficult for young males. Our findings clarify how simple vocal repertoires may have evolved in songbirds and indicate divergent strategies for how sexual selection can shape vocal learning.

State-Space Compression for Efficient Policy Learning in Crude Oil Scheduling

The imperative for swift and intelligent decision making in production scheduling has intensified in recent years. Deep reinforcement learning, akin to human cognitive processes, has heralded advancements in complex decision making and has found applicability in the production scheduling domain. Yet, its deployment in industrial settings is marred by large state spaces, protracted training times, and challenging convergence, necessitating a more efficacious approach. Addressing these concerns, this paper introduces an innovative, accelerated deep reinforcement learning framework—VSCS (Variational Autoencoder for State Compression in Soft Actor–Critic). The framework adeptly employs a variational autoencoder (VAE) to condense the expansive high-dimensional state space into a tractable low-dimensional feature space, subsequently leveraging these features to refine policy learning and augment the policy network’s performance and training efficacy. Furthermore, a novel methodology to ascertain the optimal dimensionality of these low-dimensional features is presented, integrating feature reconstruction similarity with visual analysis to facilitate informed dimensionality selection. This approach, rigorously validated within the realm of crude oil scheduling, demonstrates significant improvements over traditional methods. Notably, the convergence rate of the proposed VSCS method shows a remarkable increase of 77.5%, coupled with an 89.3% enhancement in the reward and punishment values. Furthermore, this method substantiates the robustness and appropriateness of the chosen feature dimensions.

Deep Anomaly Detection with Attention (DADA): A Novel Approach for Identifying Multipath Interference in Radar Signals

Multipath interference in radar signals caused by sea, ground, and other environments poses significant challenges to the target detection, tracking, and classification capabilities of radar systems. Existing methods for radar signal identification require labeled samples and focus mainly on the classification of normal signals. However, in practice, anomalous samples (multipath interference signals) may be scarce and highly imbalanced (i.e., mostly normal samples). To address this problem, we propose a deep anomaly detection with attention (DADA) for semisupervised detection of multipath radar signals. The method transforms radar signals into time–frequency images and is trained exclusively on normal samples. The autoencoder architecture is extended with a feature extractor network to capture latent sample features. CBAM attention is introduced to improve feature extraction. By learning the distribution of normal samples in high-dimensional image space and low-dimensional feature space, a two-dimensional feature space representing normal samples is constructed. A one-class SVM then learns the boundary of normal samples for anomaly detection. Extensive experiments on radar signal datasets validate the effectiveness of the proposed approach.

GA-CNN: Analyzing student’s cognitive skills with EEG data using a hybrid deep learning approach

A cognitive state (CS) assessment can be effectively performed using Electroencephalogram (EEG). However, due to the curse of dimensionality issues of EEG, most of the clustering methods often lead to poor performance. Deep neural network-based representation learning transforms high-dimensional data into lower-dimensional feature space, increasing the performance of CS prediction in students. So, in this research, a novel graph-based conventional attention neural network (GA-CNN) is developed to reduce distribution differences while analyzing student performance. To perform this classification, the obtained EEG signal dataset is denoised with the help of Stationary Wavelet Transform (SWT) based Independent Component Analysis (ICA) that filters out the noise signals. Likewise, the behavioural dataset is pre-processed using zero normalization and Not a Number (NaN) value computation method. Then, the effective features are extracted from the pre-processed data using the Long-Short-Term-Memory (LSTM) technique. Finally, the GA-CNN model is initiated to classify the students’ cognitive state (CS). The proposed method is implemented using the MATLAB tool, and the performance of the expected GA-CNN model is compared to other approaches where the analysis is done using the benchmark Kaggle EEG data set. The classification accuracy is significantly improved compared to other methods. The model achieves 87% accuracy, 98% precision, 75% recall, and 85% F1 score, outperforming various methods and making a better compromise.

Partial Multilabel Learning Using Noise-Tolerant Broad Learning System With Label Enhancement and Dimensionality Reduction.

Partial multilabel learning (PML) addresses the issue of noisy supervision, which contains an overcomplete set of candidate labels for each instance with only a valid subset of training data. Using label enhancement techniques, researchers have computed the probability of a label being ground truth. However, enhancing labels in the noisy label space makes it impossible for the existing partial multilabel label enhancement methods to achieve satisfactory results. Besides, few methods simultaneously involve the ambiguity problem, the feature space's redundancy, and the model's efficiency in PML. To address these issues, this article presents a novel joint partial multilabel framework using broad learning systems (namely BLS-PML) with three innovative mechanisms: 1) a trustworthy label space is reconstructed through a novel label enhancement method to avoid the bias caused by noisy labels; 2) a low-dimensional feature space is obtained by a confidence-based dimensionality reduction method to reduce the effect of redundancy in the feature space; and 3) a noise-tolerant BLS is proposed by adding a dimensionality reduction layer and a trustworthy label layer to deal with PML problem. We evaluated it on six real-world and seven synthetic datasets, using eight state-of-the-art partial multilabel algorithms as baselines and six evaluation metrics. Out of 144 experimental scenarios, our method significantly outperforms the baselines by about 80%, demonstrating its robustness and effectiveness in handling partial multilabel tasks.

Vegetable and fruit freshness detection based on deep features and principal component analysis

Vegetable and fruit freshness detecting can ensure that consumers get vegetables and fruits with good taste and rich nutrition, improve the health level of diet, and ensure that the agricultural and food industries provide high-quality products to meet consumer needs and increase sales and market share. At present, the freshness detection of vegetables and fruits mainly relies on manual observation and judgment, which has the problems of subjectivity and low accuracy, and it is difficult to meet the needs of large-scale, high-efficiency, and rapid detection. Although some studies have shown that large-scale detection of vegetable and fruit freshness can be carried out based on artificially extracted features, there is still the problem of poor adaptability of artificially extracted features, which leads to low efficiency of freshness detection. To solve this problem, this paper proposes a novel method for detecting the freshness of vegetables and fruits more objectively, accurately and efficiently using deep features extracted by pre-trained deep learning models of different architectures. First, resized images of vegetables and fruits are fed into a pre-trained deep learning model for deep feature extraction. Then, the deep features are fused and the fused deep features are dimensionally reduced to a representative low-dimensional feature space by principal component analysis. Finally, vegetable and fruit freshness are detected by three machine learning methods. The experimental results show that combining the deep features extracted by the three architecture pre-trained deep learning models GoogLeNet, DenseNet-201 and ResNeXt-101 combined with PCA dimensionality reduction processing has achieved the highest accuracy rate of 96.98% for vegetable and fruit freshness detection. This research concluded that the proposed method is promising to improve the efficiency of freshness detection of vegetables and fruits.

Adaptive active subspace-based metamodeling for high-dimensional reliability analysis

To address the challenges of reliability analysis in high-dimensional probability spaces, this paper proposes a new metamodeling method that couples active subspace, heteroscedastic Gaussian process, and active learning. The active subspace is leveraged to identify low-dimensional salient features of a high-dimensional computational model. A surrogate computational model is built in the low-dimensional feature space by a heteroscedastic Gaussian process. Active learning adaptively guides the surrogate model training toward the critical region that significantly contributes to the failure probability. A critical trait of the proposed method is that the three main ingredients–active subspace, heteroscedastic Gaussian process, and active learning–are coupled to adaptively optimize the feature space mapping in conjunction with the surrogate modeling. This coupling empowers the proposed method to accurately solve nontrivial high-dimensional reliability problems via low-dimensional surrogate modeling. Finally, numerical examples of a high-dimensional nonlinear function and structural engineering applications are investigated to verify the performance of the proposed method.

Integrated Spatio-Temporal Deep Clustering (ISTDC) for cognitive workload assessment

Traditional high-dimensional electroencephalography (EEG) features (spectral or temporal) may not always attain satisfactory results in cognitive workload estimation. In contrast, deep representation learning (DRL) transforms high-dimensional data into cluster-friendly low-dimensional feature space. Therefore, this paper proposes an Integrated Spatio-Temporal Deep Clustering (ISTDC) model that uses DRL followed by a clustering method to achieve better clustering performance. The proposed model is illustrated using four Algorithms and Variational Bayesian Gaussian Mixture Model (VBGMM) clustering method. Temporal and spatial Variational Auto Encoder (VAE) models (mentioned in Algorithm 2 and Algorithm 3) learn temporal and spatial latent features from sequence-wise EEG signals and scalp topographical maps using the Long short-term memory and Convolutional Neural Network models. The concatenated spatio-temporal latent feature (mentioned in Algorithm 4) is passed to the VBGMM clustering method to efficiently estimate workload levels of n-back task. For the 0-back vs. 2-back task, the proposed model achieves the maximum mean clustering accuracy of 98.0%, and it improves by 11.0% over the state-of-the-art method. The results also indicate that the proposed multimodal approach outperforms temporal and spatial latent feature-based unimodal models in workload assessment.

Reduced dimension stimulus decoding and column-based modeling reveal architectural differences of primary somatosensory finger maps between younger and older adults

The primary somatosensory cortex (SI) contains fine-grained tactile representations of the body, arranged in an orderly fashion. The use of ultra-high resolution fMRI data to detect group differences, for example between younger and older adults’ SI maps, is challenging, because group alignment often does not preserve the high spatial detail of the data. Here, we use robust-shared response modeling (rSRM) that allows group analyses by mapping individual stimulus-driven responses to a lower dimensional shared feature space, to detect age-related differences in tactile representations between younger and older adults using 7T-fMRI data. Using this method, we show that finger representations are more precise in Brodmann-Area (BA) 3b and BA1 compared to BA2 and motor areas, and that this hierarchical processing is preserved across age groups. By combining rSRM with column-based decoding (C-SRM), we further show that the number of columns that optimally describes finger maps in SI is higher in younger compared to older adults in BA1, indicating a greater columnar size in older adults’ SI. Taken together, we conclude that rSRM is suitable for finding fine-grained group differences in ultra-high resolution fMRI data, and we provide first evidence that the columnar architecture in SI changes with increasing age.

Heterogeneous Regularization-Based Tensor Subspace Clustering for Hyperspectral Band Selection.

Band selection (BS) reduces effectively the spectral dimension of a hyperspectral image (HSI) by selecting relatively few representative bands, which allows efficient processing in subsequent tasks. Existing unsupervised BS methods based on subspace clustering are built on matrix-based models, where each band is reshaped as a vector. They encode the correlation of data only in the spectral mode (dimension) and neglect strong correlations between different modes, i.e., spatial modes and spectral mode. Another issue is that the subspace representation of bands is performed in the raw data space, where the dimension is often excessively high, resulting in a less efficient and less robust performance. To address these issues, in this article, we propose a tensor-based subspace clustering model for hyperspectral BS. Our model is developed on the well-known Tucker decomposition. The three factor matrices and a core tensor in our model encode jointly the multimode correlations of HSI, avoiding effectively to destroy the tensor structure and information loss. In addition, we propose well-motivated heterogeneous regularizations (HRs) on the factor matrices by taking into account the important local and global properties of HSI along three dimensions, which facilitates the learning of the intrinsic cluster structure of bands in the low-dimensional subspaces. Instead of learning the correlations of bands in the original domain, a common way for the matrix-based models, our model learns naturally the band correlations in a low-dimensional latent feature space, which is derived by the projections of two factor matrices associated with spatial dimensions, leading to a computationally efficient model. More importantly, the latent feature space is learned in a unified framework. We also develop an efficient algorithm to solve the resulting model. Experimental results on benchmark datasets demonstrate that our model yields improved performance compared to the state-of-the-art.

Sensitivity enhanced method for fault detection and prediction of elevator doors using a margin maximized hyperspace

This paper proposes novel fault classification and prediction method by addressing a margin maximized hyperspace (MMH) to solve the problems absent of any label at highly imbalanced dataset, which is a frequent but challenging problem in real-world industries. The proposed method features three characteristics. First, knowledge-based feature manipulation is executed using reference and feedback physical properties and the manipulated features are used for training the proposed neural network because the features contain rich information for classifying and predicting faults of the system of interest. Second, VAE transforms high-dimensional input features to a low-dimensional feature space. This nonlinear space transformation reduces the complexity of the classification securing high accuracy and robustness of fault classification in the MMH. Third, the acquired MMH through VAE with Bayesian optimization statistically allocates two extremes of major (normal) and minor (faulty) clusters at origin and unity at the feature space, indicating that sensitivity of fault prediction is maximized. The method would be highly effective in that the model only focuses on separating major and minor clusters deciding each health condition but ignores minor differences within the clusters which confuse users. The effect of the method is demonstrated with field measurements of an elevator door stroke dataset comprising normal, degradation, and faulty states in open and close strokes. The systematic analysis shows that these characteristics contribute to improve accuracy and robustness for fault classification. Specifically, knowledge-based feature manipulation improves the accuracy, and VAE enhances sensitivity on separating each cluster and locational constancy. Moreover, the MMH is effective to predict potential fault without any label for a highly imbalanced dataset. The proposed method provides remaining useful lifetime (RUL) using distances from normal and faulty clusters at the MMH, which enables to quantitatively provide RUL of the system without any definition of RUL. Considering that many systems deployed on fields lack information for fault life or residual useful life, the proposed method would be practical and effective for real world applications.

Efficient Modal Identification and Optimal Sensor Placement via Dynamic DIC Measurement and Feature-Based Data Compression

Full-field non-contact vibration measurements provide a rich dataset for analysing structural dynamics. However, implementing the identification algorithm directly using high-spatial resolution data can be computationally expensive in modal identification. To address this challenge, performing identification in a shape-preserving but lower-dimensional feature space is more feasible. The full-field mode shapes can then be reconstructed from the identified feature mode shapes. This paper discusses two approaches, namely data-dependent and data-independent, for constructing the feature spaces. The applications of these approaches to modal identification on a curved plate are studied, and their performance is compared. In a case study involving a curved plate, it was found that a spatial data compression ratio as low as 1% could be achieved without compromising the integrity of the shape features essential for a full-field modal. Furthermore, the paper explores the optimal point-wise sensor placement using the feature space. It presents an alternative, data-driven method for optimal sensor placement that eliminates the need for a normal model, which is typically required in conventional approaches. Combining a small number of point-wise sensors with the constructed feature space can accurately reconstruct the full-field response. This approach demonstrates a two-step structural health monitoring (SHM) preparation process: offline full-field identification of the structure and the recommended point-wise sensor placement for online long-term monitoring.