Low-dimensional Feature Space Research Articles

Using deep feature distances for evaluating the perceptual quality of MR image reconstructions.

Commonly used MR image quality (IQ) metrics have poor concordance with radiologist-perceived diagnostic IQ. Here, we develop and explore deep feature distances (DFDs)-distances computed in a lower-dimensional feature space encoded by a convolutional neural network (CNN)-as improved perceptual IQ metrics for MR image reconstruction. We further explore the impact of distribution shifts between images in the DFD CNN encoder training data and the IQ metric evaluation. We compare commonly used IQ metrics (PSNR and SSIM) to two "out-of-domain" DFDs with encoders trained on natural images, an "in-domain" DFD trained on MR images alone, and two domain-adjacent DFDs trained on large medical imaging datasets. We additionally compare these with several state-of-the-art but less commonly reported IQ metrics, visual information fidelity (VIF), noise quality metric (NQM), and the high-frequency error norm (HFEN). IQ metric performance is assessed via correlations with five expert radiologist reader scores of perceived diagnostic IQ of various accelerated MR image reconstructions. We characterize the behavior of these IQ metrics under common distortions expected during image acquisition, including their sensitivity to acquisition noise. All DFDs and HFEN correlate more strongly with radiologist-perceived diagnostic IQ than SSIM, PSNR, and other state-of-the-art metrics, with correlations being comparable to radiologist inter-reader variability. Surprisingly, out-of-domain DFDs perform comparably to in-domain and domain-adjacent DFDs. A suite of IQ metrics, including DFDs and HFEN, should be used alongside commonly-reported IQ metrics for a more holistic evaluation of MR image reconstruction perceptual quality. We also observe that general vision encoders are capable of assessing visual IQ even for MR images.

Cluster-driven non-uniform characteristic analysis of underwater target acoustic scattering field

Underwater small targets typically exhibit non-centrosymmetric geometries, resulting in a highly spatially inhomogeneous acoustic scattering field under active sonar detection. Addressing these challenges, this paper takes the hemispherical cylindrical shell as the research object, considers the angle continuity implied in the echo characteristics, and proposes a cluster-driven research method for the non-uniform characteristics of the target echo angles. First, the target echo features are extracted and feature vectors are constructed. Secondly, the t-distributed stochastic neighbor embedding algorithm is employed to improve the internal connection of the feature vector in the low-dimensional feature space and to construct the visualized feature space. Finally, the implicit angular relationship between echo features is extracted under unsupervised conditions by cluster analysis. The reconstructed local geometric structures corresponding to different categories demonstrate that the method effectively segments the angular intervals of local target structures based on their natural acoustic scattering characteristics. The study overcomes the inherent subjectivity of traditional methods for dividing angular intervals of target echoes, providing a more objective foundation for segmenting and analyzing the target’s geometrical structure.

Perfusion estimation from dynamic non-contrast computed tomography using self-supervised learning and a physics-inspired U-net transformer architecture.

Pulmonary perfusion imaging is a key lung health indicator with clinical utility as a diagnostic and treatment planning tool. However, current nuclear medicine modalities face challenges like low spatial resolution and long acquisition times which limit clinical utility to non-emergency settings and often placing extra financial burden on the patient. This study introduces a novel deep learning approach to predict perfusion imaging from non-contrast inhale and exhale computed tomography scans (IE-CT). We developed a U-Net Transformer architecture modified for Siamese IE-CT inputs, integrating insights from physical models and utilizing a self-supervised learning strategy tailored for lung function prediction. We aggregated 523IE-CT images from nine different 4DCT imaging datasets for self-supervised training, aiming to learn a low-dimensional IE-CT feature space by reconstructing image volumes from random data augmentations. Supervised training for perfusion prediction used this feature space and transfer learning on a cohort of 44 patients who had both IE-CT and single-photon emission CT (SPECT/CT) perfusion scans. Testing with random bootstrapping, we estimated the mean and standard deviation of the spatial Spearman correlation between our predictions and the ground truth (SPECT perfusion) to be 0.742 ± 0.037, with a mean median correlation of 0.792 ± 0.036. These results represent a new state-of-the-art accuracy for predicting perfusion imaging from non-contrast CT. Our approach combines low-dimensional feature representations of both inhale and exhale images into a deep learning model, aligning with previous physical modeling methods for characterizing perfusion from IE-CT. This likely contributes to the high spatial correlation with ground truth. With further development, our method could provide faster and more accurate lung function imaging, potentially expanding its clinical applications beyond what is currently possible with nuclear medicine.

Application of a sEMG hand motion recognition method based on variational mode decomposition and ReliefF algorithm in rehabilitation medicine.

Hand motion intention recognition has been considered as one of the crucial research fields for prosthetic control and rehabilitation medicine. In recent years, surface electromyogram (sEMG) signals that directly reflect human motion information are ideal input sources for prosthetic control and rehabilitation. However, how to effectively extract components from sEMG signals containing abundant limb movement information to improve the accuracy of hand recognition still is a difficult problem. To achieve this goal, this paper proposes a novel hand motion recognition method based on variational mode decomposition (VMD) and ReliefF. First, VMD is used to decompose the sEMG signal into multiple variational mode functions (VMFs). To efficiently extract the intrinsic components of the sEMG, the recognition performance of different numbers of VMFs is evaluated. Then, four features representing hand motion intentions are extracted from the VMFs to form the initial feature space. Next, the ReliefF algorithm is used to remove redundant features from the feature space. In order to select a feature space that can effectively reflect the intention of hand movements, the hand movement recognition performance of 8 low-dimensional feature spaces is evaluated. Finally, three machine learning methods are used to recognize hand movements. The proposed method was tested on the sEMG for Basic Hand movements Data Set and achieved an average accuracy of 99.14%. Compared with existing research, the proposed method achieves better hand motion recognition performance, indicating the potential for healthcare and rehabilitation applications.

Latent space-based machine learning prediction of coupled flame-flow fields in a hydrogen-enriched syngas combustor

In this study, we propose an innovative latent space-based machine learning model to predict coupled flame-flow fields across various combustion states. This model consists of two main components: a local convolutional autoencoder (LCAE) with an encoder to establish a latent space and a decoder to restore predicted features to their original physical dimensions, and a long short-term memory (LSTM) model to forecast the spatiotemporal behavior of features in this latent space. The model is trained and validated using flame-flow images simultaneously measured in a gas turbine model combustor, utilizing synchronized stereoscopic particle image velocimetry (S-PIV) and planar laser induced fluorescence of the hydroxyl radical (OH-PLIF). Our model effectively extracts large-scale structures from the coupled flame-flow measurements, even in the presence of aleatoric noise, and demonstrates satisfactory prediction and generalization capabilities, as verified through out-of-sample testing. By establishing a common latent space for the coupled flame-flow fields, we reveal inherent similarities between flame and flow fields in a low-dimensional feature space, suggesting the potential for holistic understanding and modeling of their behavior in such a space. Additionally, our model predicts rapidly and reliably, requiring only 403 ms for out-of-sample predictions, with no error accumulation across 499 predictions. This paves the way for early warning of impending undesirable combustion states in future applications.

Empowering legal justice with AI: A reinforcement learning SAC-VAE framework for advanced legal text summarization.

Automated summarization of legal texts poses a significant challenge due to the complex and specialized nature of legal documentation. Despite the recent progress in reinforcement learning for natural language text summarization, its application in the legal domain has been less effective. This paper introduces SAC-VAE, a novel reinforcement learning framework specifically designed for legal text summarization. We leverage a Variational Autoencoder (VAE) to condense the high-dimensional state space into a more manageable lower-dimensional feature space. These compressed features are subsequently utilized by the Soft Actor-Critic (SAC) algorithm for policy learning, facilitating the automated generation of summaries from legal texts. Through comprehensive experimentation, we have empirically demonstrated the effectiveness and superior performance of the SAC-VAE framework in legal text summarization.

A Biometric Identification for Multi-Modal Biomedical Signals in Geriatric Care

With the acceleration of global population aging, the elderly have an increasing demand for home care and nursing institutions, and the significance of health prevention and management of the elderly has become increasingly prominent. In this context, we propose a biometric recognition method for multi-modal biomedical signals. This article focuses on three key signals that can be picked up by wearable devices: ECG, PPG, and breath (RESP). The RESP signal is introduced into the existing two-mode signal identification for multi-mode identification. Firstly, the features of the signal in the time–frequency domain are extracted. To represent deep features in a low-dimensional feature space and expedite authentication tasks, PCA and LDA are employed for dimensionality reduction. MCCA is used for feature fusion, and SVM is used for identification. The accuracy and performance of the system were evaluated using both public data sets and self-collected data sets, with an accuracy of more than 99.5%. The experimental data fully show that this method significantly improves the accuracy of identity recognition. In the future, combined with the signal monitoring function of wearable devices, it can quickly identify individual elderly people with abnormal conditions, provide safer and more efficient medical services for the elderly, and relieve the pressure on medical resources.

An autoencoder model based on one-dimensional neural network for epileptic EEG anomaly detection

We propose an autoencoder model based on a one-dimensional convolutional neural network (1DCNN) as the feature extraction network for efficient detection of epileptic EEG anomalies. The local information of normal EEG signals was captured by utilizing the local feature extraction ability of 1DCNN for training of an autoencoder to learn the expression of normal EEG data in low dimensional feature space. With the difference between the input and output as the anomaly score, the threshold was determined by the optimal equilibrium point of the ROC curve, and the EEG signals exceeding the threshold were diagnosed as the seizure data. The performance of the 1DCNN-AE epilepsy detection model was evaluated using the publicly available CHB-MIT scalp EEG dataset and TUH scalp EEG dataset. The AUC of the 1DCNN-AE model reached 0.890 of CHB-MIT and 0.686 of TUH under the average level of patients, and the epilepsy detection rate reached 0.974 and 0.893, and these results were better than the latest epilepsy anomaly detection models LSTM-VAE and GRU-VAE. The 1DCNN model had a parameter quantity of 58.5M, which was at the same level with LSTM-VAE (47.4 M) and GRU-VAE (36.9 M) but with much smaller FLOPs (0.377 G) than LSTM-VAE (21.6 G) and GRU-VAE (16.2 G). The autoencoder model based one-dimensional convolutional neural network can effectively detect abnormal EEG signals in epileptic seizure.

Quantitative Edge Analysis Can Differentiate Pancreatic Carcinoma from Normal Pancreatic Parenchyma.

This study aimed to introduce specific image feature analysis, focusing on pancreatic margins, and to provide a quantitative measure of edge irregularity, evidencing correlations with the presence/absence of pancreatic adenocarcinoma. We selected 50 patients (36 men, 14 women; mean age 63.7 years) who underwent Multi-detector computed tomography (MDCT) for the staging of pancreatic adenocarcinoma of the tail of the pancreas. Computer-assisted quantitative edge analysis was performed on the border fragments in MDCT images of neoplastic and healthy glandular parenchyma, from which we obtained the root mean square deviation SD of the actual border from the average boundary line. The SD values relative to healthy and neoplastic borders were compared using a paired t-test. A significant SD difference was observed between healthy and neoplastic borders. A threshold SD value was also found, enabling the differentiation of adenocarcinoma with 96% specificity and sensitivity. We introduced a quantitative measure of boundary irregularity, which correlates with the presence/absence of pancreatic adenocarcinoma. Quantitative edge analysis can be promptly performed on select border fragments in MDCT images, providing a useful supporting tool for diagnostics and a possible starting point for machine learning recognition based on lower-dimensional feature space.

Using autoencoders and deep transfer learning to determine the stellar parameters of 286 CARMENES M dwarfs

Context. Deep learning (DL) techniques are a promising approach among the set of methods used in the ever-challenging determination of stellar parameters in M dwarfs. In this context, transfer learning could play an important role in mitigating uncertainties in the results due to the synthetic gap (i.e. difference in feature distributions between observed and synthetic data). Aims. We propose a feature-based deep transfer learning (DTL) approach based on autoencoders to determine stellar parameters from high-resolution spectra. Using this methodology, we provide new estimations for the effective temperature, surface gravity, metallicity, and projected rotational velocity for 286 M dwarfs observed by the CARMENES survey. Methods. Using autoencoder architectures, we projected synthetic PHOENIX-ACES spectra and observed CARMENES spectra onto a new feature space of lower dimensionality in which the differences between the two domains are reduced. We used this low-dimensional new feature space as input for a convolutional neural network to obtain the stellar parameter determinations. Results. We performed an extensive analysis of our estimated stellar parameters, ranging from 3050 to 4300 K, 4.7 to 5.1 dex, and −0.53 to 0.25 dex for Teff, log 𝑔, and [Fe/H], respectively. Our results are broadly consistent with those of recent studies using CARMENES data, with a systematic deviation in our Teff scale towards hotter values for estimations above 3750 K. Furthermore, our methodology mitigates the deviations in metallicity found in previous DL techniques due to the synthetic gap. Conclusions. We consolidated a DTL-based methodology to determine stellar parameters in M dwarfs from synthetic spectra, with no need for high-quality measurements involved in the knowledge transfer. These results suggest the great potential of DTL to mitigate the differences in feature distributions between the observations and the PHOENIX-ACES spectra.

Accelerating particle-in-cell kinetic plasma simulations via reduced-order modeling of space-charge dynamics using dynamic mode decomposition.

We present a data-driven reduced-order modeling of the space-charge dynamics for electromagnetic particle-in-cell (EMPIC) plasma simulations based on dynamic mode decomposition (DMD). The dynamics of the charged particles in kinetic plasma simulations such as EMPIC is manifested through the plasma current density defined along the edges of the spatial mesh. We showcase the efficacy of DMD in modeling the time evolution of current density through a low-dimensional feature space. Not only do such DMD-based predictive reduced-order models help accelerate EMPIC simulations, they also have the potential to facilitate investigative analysis and control applications. We demonstrate the proposed DMD-EMPIC scheme for reduced-order modeling of current density and speedup in EMPIC simulations involving electron beam under the influence of magnetic field, virtual cathode oscillations, and backward wave oscillator.

Simultaneous Three-Degrees-of-Freedom Prosthetic Control Based on Linear Regression and Closed-Loop Training Protocol.

Machine learning-based controllers of prostheses using electromyographic signals have become very popular in the last decade. The regression approach allows a simultaneous and proportional control of the intended movement in a more natural way than the classification approach, where the number of movements is discrete by definition. However, it is not common to find regression-based controllers working for more than two degrees of freedom at the same time. In this paper, we present the application of the adaptive linear regressor in a relatively low-dimensional feature space with only eight sensors to the problem of a simultaneous and proportional control of three degrees of freedom (left-right, up-down and open-close hand movements). We show that a key element usually overlooked in the learning process of the regressor is the training paradigm. We propose a closed-loop procedure, where the human learns how to improve the quality of the generated EMG signals, helping also to obtain a better controller. We apply it to 10 healthy and 3 limb-deficient subjects. Results show that the combination of the multidimensional targets and the open-loop training protocol significantly improve the performance, increasing the average completion rate from 53% to 65% for the most complicated case of simultaneously controlling the three degrees of freedom.

Beaconet: A Reference-Free Method for Integrating Multiple Batches of Single-Cell Transcriptomic Data in Original Molecular Space.

Integrating multiple single-cell datasets is essential for the comprehensive understanding of cell heterogeneity. Batch effect is the undesired systematic variations among technologies or experimental laboratories that distort biological signals and hinder the integration of single-cell datasets. However, existing methods typically rely on a selected dataset as a reference, leading to inconsistent integration performance using different references, or embed cells into uninterpretable low-dimensional feature space. To overcome these limitations, a reference-free method, Beaconet, for integrating multiple single-cell transcriptomic datasets in original molecular space by aligning the global distribution of each batch using an adversarial correction network is presented. Through extensive comparisons with 13 state-of-the-art methods, it is demonstrated that Beaconet can effectively remove batch effect while preserving biological variations and is superior to existing unsupervised methods using all possible references in overall performance. Furthermore, Beaconet performs integration in the original molecular feature space, enabling the characterization of cell types and downstream differential expression analysis directly using integrated data with gene-expression features. Additionally, when applying to large-scale atlas data integration, Beaconet shows notable advantages in both time- and space-efficiencies. In summary, Beaconet serves as an effective and efficient batch effect removal tool that can facilitate the integration of single-cell datasets in a reference-free and molecular feature-preserved mode.

Principal trade-off analysis

How are the advantage relations between a set of agents playing a game organized and how do they reflect the structure of the game? In this paper, we illustrate ‘Principal Trade-off Analysis’ (PTA), a decomposition method that embeds games into a low-dimensional feature space. We argue that the embeddings are more revealing than previously demonstrated by developing an analogy to Principal Component Analysis (PCA). PTA represents an arbitrary two-player zero-sum game as linear combination of simple games via the projection of policy profiles into orthogonal 2D feature planes. We show that the feature planes represent unique strategic trade-offs and truncation of the sequence provides insightful model reduction and visualization. We demonstrate the validity of PTA on a quartet of games (Kuhn poker, RPS + 2, Blotto and Pokemon). In Kuhn poker, PTA clearly identifies the trade-off between bluffing and calling. In Blotto, PTA identifies game symmetries and specifies strategic trade-offs associated with distinct win conditions. These symmetries reveal limitations of PTA unaddressed in previous work. For Pokemon, PTA recovers clusters that naturally correspond to Pokemon types, correctly identifies the designed trade-off between those types, and discovers a rock-paper-scissor (RPS) cycle in the Pokemon generation type – all absent any specific information except game outcomes.

Forecasting individual bids in real electricity markets through machine learning framework

With the increasing uncertainty caused by the complexity of the world’s energy environment and the increasing penetration rate of renewable energy, it is significant to estimate the future operation of power markets in advance. Forecasting individual bids in spot electricity markets is a promising new method for achieving so, but it has not been fully studied due to the difficulty of forecasting a bid function. The idealization of existing optimization-based models decreases their practical effects in real markets. Thus, we propose a scalable forecasting framework that incorporates several customized state-of-art machine learning methods according to the characteristics of the bidding data. First, several low-rank approximation algorithms are customized to encode the high-dimensional bidding curves into low-dimensional feature spaces and reconstruct them from the predicted feature space. Second, a transformer-based multidimensional time series prediction algorithm is proposed to predict the bidding feature based on both related factors and historical bidding records. To appropriately evaluate the performances of the forecasting methods, we introduce a dynamic criterion based on the economic implications of bids. The comprehensive framework is tested based on actual market data from the Australian national electricity market, and in the empirical example, the feasibility and effectiveness of the proposed framework are demonstrated.

Label enhancement via manifold approximation and projection with graph convolutional network

Label enhancement (LE) aims to enrich logical labels into their corresponding label distributions. But existing LE algorithms fail to fully leverage the structural information in the feature space to improve LE learning. To address this key issue, we first apply manifold learning to map the relatedness between low-dimensional feature samples to the label space. Based on the smoothness assumption of manifolds, the implicit correlation between low-dimensional feature and label spaces effectively promotes the LE process, enabling the learning model to accurately capture the mapping relationship between feature and label manifolds. This leads to an LE based on feature representation (LEFR) algorithm. We also propose an LE algorithm based on graph convolutional network (GCN), called LE-GCN. Inspired by the relationship between threshold connections and label connections, we extend GCN to the LE field for the first time to fully exploit the hidden relationships between nodes and labels. By enhancing node information with threshold connections and label connections, the label learning accuracy reaches a new level. Experiments on real-world datasets show that our LEFR and LE-GCN outperform several state-of-the-art LE algorithms.

The hidden fitness of the male zebra finch courtship song.

Vocal learning in songbirds is thought to have evolved through sexual selection, with female preference driving males to develop large and varied song repertoires1-3. However, many songbird species learn only a single song in their lifetime4. How sexual selection drives the evolution of single-song repertoires is not known. Here, by applying dimensionality-reduction techniques to the singing behaviour of zebra finches (Taeniopygia guttata), we show that syllable spread in low-dimensional feature space explains how single songs function as honest indicators of fitness. We find that this Gestalt measure of behaviour captures the spectrotemporal distinctiveness of song syllables in zebra finches; that females strongly prefer songs that occupy more latent space; and that matching path lengths in low-dimensional space is difficult for young males. Our findings clarify how simple vocal repertoires may have evolved in songbirds and indicate divergent strategies for how sexual selection can shape vocal learning.

State-Space Compression for Efficient Policy Learning in Crude Oil Scheduling

The imperative for swift and intelligent decision making in production scheduling has intensified in recent years. Deep reinforcement learning, akin to human cognitive processes, has heralded advancements in complex decision making and has found applicability in the production scheduling domain. Yet, its deployment in industrial settings is marred by large state spaces, protracted training times, and challenging convergence, necessitating a more efficacious approach. Addressing these concerns, this paper introduces an innovative, accelerated deep reinforcement learning framework—VSCS (Variational Autoencoder for State Compression in Soft Actor–Critic). The framework adeptly employs a variational autoencoder (VAE) to condense the expansive high-dimensional state space into a tractable low-dimensional feature space, subsequently leveraging these features to refine policy learning and augment the policy network’s performance and training efficacy. Furthermore, a novel methodology to ascertain the optimal dimensionality of these low-dimensional features is presented, integrating feature reconstruction similarity with visual analysis to facilitate informed dimensionality selection. This approach, rigorously validated within the realm of crude oil scheduling, demonstrates significant improvements over traditional methods. Notably, the convergence rate of the proposed VSCS method shows a remarkable increase of 77.5%, coupled with an 89.3% enhancement in the reward and punishment values. Furthermore, this method substantiates the robustness and appropriateness of the chosen feature dimensions.

Deep Anomaly Detection with Attention (DADA): A Novel Approach for Identifying Multipath Interference in Radar Signals

Multipath interference in radar signals caused by sea, ground, and other environments poses significant challenges to the target detection, tracking, and classification capabilities of radar systems. Existing methods for radar signal identification require labeled samples and focus mainly on the classification of normal signals. However, in practice, anomalous samples (multipath interference signals) may be scarce and highly imbalanced (i.e., mostly normal samples). To address this problem, we propose a deep anomaly detection with attention (DADA) for semisupervised detection of multipath radar signals. The method transforms radar signals into time–frequency images and is trained exclusively on normal samples. The autoencoder architecture is extended with a feature extractor network to capture latent sample features. CBAM attention is introduced to improve feature extraction. By learning the distribution of normal samples in high-dimensional image space and low-dimensional feature space, a two-dimensional feature space representing normal samples is constructed. A one-class SVM then learns the boundary of normal samples for anomaly detection. Extensive experiments on radar signal datasets validate the effectiveness of the proposed approach.

GA-CNN: Analyzing student’s cognitive skills with EEG data using a hybrid deep learning approach

A cognitive state (CS) assessment can be effectively performed using Electroencephalogram (EEG). However, due to the curse of dimensionality issues of EEG, most of the clustering methods often lead to poor performance. Deep neural network-based representation learning transforms high-dimensional data into lower-dimensional feature space, increasing the performance of CS prediction in students. So, in this research, a novel graph-based conventional attention neural network (GA-CNN) is developed to reduce distribution differences while analyzing student performance. To perform this classification, the obtained EEG signal dataset is denoised with the help of Stationary Wavelet Transform (SWT) based Independent Component Analysis (ICA) that filters out the noise signals. Likewise, the behavioural dataset is pre-processed using zero normalization and Not a Number (NaN) value computation method. Then, the effective features are extracted from the pre-processed data using the Long-Short-Term-Memory (LSTM) technique. Finally, the GA-CNN model is initiated to classify the students’ cognitive state (CS). The proposed method is implemented using the MATLAB tool, and the performance of the expected GA-CNN model is compared to other approaches where the analysis is done using the benchmark Kaggle EEG data set. The classification accuracy is significantly improved compared to other methods. The model achieves 87% accuracy, 98% precision, 75% recall, and 85% F1 score, outperforming various methods and making a better compromise.