Manifold Structure Research Articles

Improving deep metric learning via self-distillation and online batch diffusion process

Currently, deep metric learning (DML) methods typically rely on class labels to keep positive samples as closely clustered as possible, while distancing them from negative samples. However, this approach tends to disregard some vital information inherent in the data, such as intra-class information variation, which can hinder the generalization of the trained model. To address this issue, we propose an online batch diffusion-based self-distillation method (OBD-SD), which consists of a progressive self-distillation (PSD) technique and an online batch diffusion process (OBDP). Specifically, PSD is a simple yet effective self-distillation technique that is beneficial for the diversity of embedding representations. OBDP uses the diffusion process to reveal the intrinsic relationships among samples in the mini-batch and produce better soft distance targets. Our approach combines PSD with OBDP, which can extract richer relational information among samples by capturing the local geometric structure of manifolds in each mini-batch. OBD-SD is a highly flexible framework that can be integrated into state-of-the-art DML methods. Our experimental results on CUB200, CARS196, and Stanford Online Products datasets demonstrate that our OBD-SD consistently enhances the performance of existing DML methods, without requiring additional training time, resulting in competitive results.

Time Functions on Lorentzian Length Spaces

AbstractIn general relativity, time functions are crucial objects whose existence and properties are intimately tied to the causal structure of a spacetime and also to the initial value formulation of the Einstein equations. In this work we establish all fundamental classical existence results on time functions in the setting of Lorentzian (pre-)length spaces (including causally plain continuous spacetimes, closed cone fields and even more singular spaces). More precisely, we characterize the existence of time functions by K-causality, show that a modified notion of Geroch’s volume functions are time functions if and only if the space is causally continuous, and lastly, characterize global hyperbolicity by the existence of Cauchy time functions, and Cauchy sets. Our results thus inevitably show that no manifold structure is needed in order to obtain suitable time functions.

Integrated manifold microchannels and near-junction cooling for enhanced thermal management in 3D heterogeneous packaging technology

The rapid advancement in electronic chip technology is driving the evolution of highly integrated and high-power configurations, which in turn imposes greater demands on thermal management. This study presents a novel approach to address the thermal management challenges associated with high-density integrated chips. By incorporating the near-junction cooling method into silicon-based chips, the thermal resistance during heat conduction is effectively reduced. To further enhance heat dissipation efficiency, we integrate manifold microchannels and near-junction cooling into an aluminum nitride ceramic substrate, creating a heterogeneous three-dimensional integrated chip. To verify the effectiveness of our approach, a thermal test chip and an aluminum nitride substrate with a manifold structure are meticulously prepared and packaged for experimental testing. Remarkably, the experimental results demonstrate that the heterogeneous integrated chip, featuring manifold microchannels, achieves an impressive heat dissipation capacity of 700 W/cm2 under normal operating temperature conditions. Additionally, sensitivity analysis and multi-objective optimization approach were employed, utilizing the Response Surface-Genetic Algorithm П(NSGAП), to optimize the manifold microchannels in the heterogeneous integrated chip. The optimization process yields significant improvements, reducing the total thermal resistance of the chip by 13.6 % and the maximum pressure drop by 68.5 %. These findings provide valuable theoretical insights and guidance for the thermal design of high heat flux integrated chips, and contribute to advance the design and development of high-performance integrated circuit systems.

Joint Projected Fuzzy Neighborhood Preserving C-means Clustering with Local Adaptive Learning

Graph-based dimensionality reduction approaches are extensively applied for preserving the intrinsic structures of high-dimensional data. To capture discriminative information from data clusters that lack a clear manifold structure, previous studies have combined clustering algorithms with local manifold structure preservation. These methods assume that two neighbor data points share the same cluster label. However, the assumption may not hold for Fuzzy C-Means which may assign two nearby data points to different clusters depending on the distance between data points and cluster centroids. This inconsistency degrades the performance in clustering and dimensionality reduction. To tackle this issue, we propose a novel approach for dimensionality reduction, termed joint Projected Fuzzy Neighborhood Preserving C-means Clustering with Local Adaptive Learning (PFNPCM). PFNPCM learns cluster centroids, sparse membership grades, embedded subspaces, and structured graphs simultaneously and extracts both global and local structures of data. Through clustering data that preserves the local neighborhood structure in a low-dimensional space instead of the original one, PFNPCM can effectively capture both global and local information from the inherent manifold structure and cluster structure. The employment of ℓ2,1-norm as the distance metric in the clustering and the preservation of neighborhood structure further ensure the robustness of clustering results against noise or outliers. To achieve the consistency of the captured global and local discriminative information, PFNPCM adopts a novel fuzzy local similarity measure based on the manifold structure, and the manifold assumption thus holds when partitioning unlabeled samples into subgroups. Massive experiments on diverse well-known datasets demonstrate that PFNPCM is superior to some state-of-the-art methods.

Explicit unsupervised feature selection based on structured graph and locally linear embedding

Numerous redundant and irrelevant features are usually contained in high-dimensional data whose presence, if ignored, can bring detrimental effects on the performance of data processing tasks. Unsupervised feature selection (UFS) as a trending topic in a great deal of domains has been greatly concerned, which can effectively filter out redundant and irrelevant features in the unlabeled data. The majority of existing UFS methods mainly focus on building the parameterized model in the original feature space or some certain low-dimensional feature subspace, which cannot fully capture the information of the target space. In this work, we propose a novel UFS method that unifies the explicit feature selective matrix and structured graph into a learning framework. An explicit selection matrix, whose scale, bound of the elements and structure are considered, is tailored for UFS. Furthermore, we take into account the local consistency so that the low dimensional manifold structures can be preserved to a better degree. Instead of using the pre-defined similarity matrix to construct graph, we make use of the way of learning to obtain the adaptive graph. Besides, for the purpose of exploiting the cluster structure of data, we impose a rank constraint on the graph. Notably, both the explicit selection matrix and intrinsic structure are learned in the target feature subspace, and thus the proposed methodology is more straightforward and effective. We present an effective and efficient algorithm for the resulting problem based on alternative optimization strategy, and establish its convergence and computational complexity analysis. The experimental results show that our proposed approach has improved by over 5.28% and 5.79% on average in terms of clustering accuracy (ACC) and Normalized Mutual Information (NMI) compared with comparison methods, which demonstrates its superiority.

Transient characterization of the mode switching process in the reversible solid oxide cell stack

Reversible solid oxide cells (RSOCs) is a promising energy storage and conversion technology requiring frequent switching between fuel cell mode and electrolysis mode in practice. Unfortunately, it may cause a large current density undershoot that has irreversible damage to the RSOC. Therefore, understanding the transient behaviors of the RSOC is critical for its efficient and durable operation. A three-dimensional transient model of a 30-cell SOC stack with an external manifold is established. From SOFC to SOEC, the mass transfer lag and slow heat transfer cause the current density of the stack requiring 2500 s to reach a steady state. And the bottom inlet way and the external manifold structure cause differences in the current density, molar fraction and temperature variation with time between different cells. The upper cells show a slower mass transfer rate and undergo greater temperature changes. The inconsistent temperature change gradients and temperature change rates among different cells will inevitably produce changing thermal stresses and bring irreversible mechanical losses to the stack. Similar transient characteristics are observed when switching from SOEC to SOFC. However, it takes longer time (3200 s) to achieve the final steady state.

Reconfigurations of cortical manifold structure during reward-based motor learning

Adaptive motor behavior depends on the coordinated activity of multiple neural systems distributed across the brain. While the role of sensorimotor cortex in motor learning has been well established, how higher-order brain systems interact with sensorimotor cortex to guide learning is less well understood. Using functional MRI, we examined human brain activity during a reward-based motor task where subjects learned to shape their hand trajectories through reinforcement feedback. We projected patterns of cortical and striatal functional connectivity onto a low-dimensional manifold space and examined how regions expanded and contracted along the manifold during learning. During early learning, we found that several sensorimotor areas in the dorsal attention network exhibited increased covariance with areas of the salience/ventral attention network and reduced covariance with areas of the default mode network (DMN). During late learning, these effects reversed, with sensorimotor areas now exhibiting increased covariance with DMN areas. However, areas in posteromedial cortex showed the opposite pattern across learning phases, with its connectivity suggesting a role in coordinating activity across different networks over time. Our results establish the neural changes that support reward-based motor learning and identify distinct transitions in the functional coupling of sensorimotor to transmodal cortex when adapting behavior.

Reconfigurations of cortical manifold structure during reward-based motor learning

Adaptive motor behavior depends on the coordinated activity of multiple neural systems distributed across the brain. While the role of sensorimotor cortex in motor learning has been well established, how higher-order brain systems interact with sensorimotor cortex to guide learning is less well understood. Using functional MRI, we examined human brain activity during a reward-based motor task where subjects learned to shape their hand trajectories through reinforcement feedback. We projected patterns of cortical and striatal functional connectivity onto a low-dimensional manifold space and examined how regions expanded and contracted along the manifold during learning. During early learning, we found that several sensorimotor areas in the dorsal attention network exhibited increased covariance with areas of the salience/ventral attention network and reduced covariance with areas of the default mode network (DMN). During late learning, these effects reversed, with sensorimotor areas now exhibiting increased covariance with DMN areas. However, areas in posteromedial cortex showed the opposite pattern across learning phases, with its connectivity suggesting a role in coordinating activity across different networks over time. Our results establish the neural changes that support reward-based motor learning and identify distinct transitions in the functional coupling of sensorimotor to transmodal cortex when adapting behavior.

Intrinsic structure exploitation with dual alignment for universal visual recognition

This research explores the complex yet applicable domain of universal visual recognition, where unlabeled datasets may encompass additional, as-of-yet unidentified classes and present a skewed feature distribution within labeled datasets. The primary challenge lies in concurrently distinguishing instances of known classes and uncovering new classes within the unlabeled data. This difficulty is compounded by two factors: category shift and distribution shift. Current methodologies typically overlook the potential of harnessing the inherent structural relationship between labeled and unlabeled datasets, and they fail to effectively isolate the semantic variability of the newly identified classes from the established ones. To overcome these hurdles, we introduce an innovative framework, which we have termed MESA (short for Manifold structure Exploitation with double Spaces dual Alignment), to advance the field of universal visual recognition. Our approach begins by identifying geometrically and semantically related pairs of nearest neighbors between labeled and unlabeled data, which serve as anchor points to foster an intrinsic match between the two datasets. By integrating an affinity score for similarity, we establish a novel cross-view, anchor-guided, self-supervised learning paradigm. This model is designed to spread the learned labels’ information and to conjoin novel semantic insights within the predictive space. To intensify the matching accuracy for the known classes and enhance the differentiation of novel classes, we further implement a confident-prototype-driven, self-supervised learning paradigm in a cross-view manner. This strategy is aimed at revealing the macroscopic category configuration within the embedding space. MESA employs a pioneering bidirectional dual-alignment mechanism that operates simultaneously in the embedding and prediction spaces, hence providing a more robust approach to confronting challenges brought about by distribution and category shifts. Our exhaustive evaluation across several extensive image recognition benchmarks substantiates MESA’s superior performance over the latest cutting-edge methods, marking a significant leap forward in the sphere of universal visual recognition. Finally, MESA is implemented in Python using the Pytorch machine-learning library, and it is freely available at https://github.com/aimeeyaoyao/MESA.

Multi-view clustering via latent consistency multi-graph fusion

Multi-view clustering (MVC) algorithms are prevalent in machine learning tasks due to their superiority in mining complementary information from multi-view data. Despite their widespread use, existing MVC approaches ignore the presence of inconsistencies in the multi-view features during multi-view data fusion, which leads to suboptimal clustering performance. To address this challenge, the paper presents a novel approach called multi-view clustering via latent consistency multi-graph fusion. Specifically, a novel latent representation learning model is devised to learn a unified representation from multi-view features, simultaneously capturing the high-order relationships within the multi-view data. Additionally, to alleviate the adverse effects caused by the inconsistency multi-view feature, a consistent multi-graph fusion module is proposed to provide consistent constraints for latent representations. To be specific, the graphs that reflect the manifold structure for each view are fused to guide a consistent latent representation learning. After that, an alternating optimization algorithm is designed to update variables iteratively and alternatively, among which the fusion weights are assigned adaptively. Finally, comprehensive experiments conducted on seven widely recognized benchmark multi-view datasets affirm the efficacy of the proposed method, showcasing its superiority over state-of-the-art approaches.

Manifold embedding stationary subspace analysis for nonstationary process monitoring with industrial applications

Industrial processes frequently exhibit nonstationary characteristics due to factors like load fluctuations and external interference. Accurate monitoring of nonstationary industrial processes is of vital importance in ensuring production stability and safety. Unfortunately, most existing monitoring methods overlook the manifold structure presented in nonstationary data due to nonstationary features, causing the loss of critical information and poor interpretability. As a consequence, monitoring performance is compromised. To address this issue, this paper proposes a manifold embedding stationary subspace analysis (MESSA) algorithm. By embedding a neighborhood preservation term into the objective function of SSA, MESSA effectively mitigates the impact of nonstationarity on manifold structure. The extracted features incorporate both global stationarity and local manifold characteristics, facilitating a more comprehensive reconstruction of the intricate underlying mechanisms in industrial processes. This contributes to a substantial enhancement in the accuracy and reliability of process monitoring. A set of nonstationary swiss-roll dataset is designed to visually demonstrate the capability of MESSA in extracting manifold structure. Case studies including a numerical case, a continuous stirred tank reactor system and a real industrial roasting process validate the superior monitoring performance of the proposed method.

A unified formulation of geometry-aware discrete dynamic movement primitives

Learning from demonstration (LfD) is considered as an efficient way to transfer skills from humans to robots. Traditionally, LfD has been used to transfer Cartesian and joint positions and forces from human demonstrations. The traditional approach works well for some robotic tasks, but for many tasks of interest, it is necessary to learn skills such as orientation, impedance, and/or manipulability that have specific geometric characteristics. An effective encoding of such skills can be only achieved if the underlying geometric structure of the skill manifold is considered and the constrains arising from this structure are fulfilled during both learning and execution. However, typical learned skill models such as dynamic movement primitives (DMPs) are limited to Euclidean data and fail in correctly embedding quantities with geometric constraints. In this paper, we propose a novel and mathematically principled framework that uses concepts from Riemannian geometry to allow DMPs to properly embed geometric constrains. The resulting DMP formulation can deal with data sampled from any Riemannian manifold including, but not limited to, unit quaternions and symmetric and positive definite matrices. The proposed approach has been extensively evaluated both on simulated data and real robot experiments. The performed evaluation demonstrates that beneficial properties of DMPs, such as convergence to a given goal and the possibility to change the goal during operation, apply also to the proposed formulation.

Unsupervised separation of nonlinearly mixed event-related potentials using manifold clustering and non-negative matrix factorization

Event-related potentials (ERPs) can quantify brain responses to reveal the neural mechanisms of sensory perception. However, ERPs often reflect nonlinear mixture responses to multiple sources of sensory stimuli, and an accurate separation of the response to each stimulus remains a challenge. This study aimed to separate the ERP into nonlinearly mixed source components specific to individual stimuli. We developed an unsupervised learning method based on clustering of manifold structures of mixture signals combined with channel optimization for signal source reconstruction using non-negative matrix factorization (NMF). Specifically, we first implemented manifold learning based on Local Tangent Space Alignment (LTSA) to extract the spatial manifold structure of multi-resolution sub-signals separated via wavelet packet transform. We then used fuzzy entropy to extract the dynamical process of the manifold structures and performed a k-means clustering to separate different sources. Lastly, we used NMF to obtain the optimal contributions of multiple channels to ensure accurate source reconstructions. We evaluated our developed approach using a simulated ERP dataset with known ground truth of two components of ERP mixture signals. Our results show that the correlation coefficient between the reconstructed source signal and the true source signal was 92.8 % and that the separation accuracy in ERP amplitude was 91.6 %. The results show that our unsupervised separation approach can accurately separate ERP signals from nonlinear mixture source components. The outcomes provide a promising way to isolate brain responses to multiple stimulus sources during multisensory perception.

PARKINSON'S DISEASE CLASSIFICATION USING CONTRASTIVE GRAPH CROSS-VIEW LEARNING WITH MULTIMODAL FUSION OF SPECT IMAGES AND CLINICAL FEATURES.

Parkinson's Disease (PD) affects millions globally, impacting movement. Prior research utilized deep learning for PD prediction, primarily focusing on medical images, neglecting the data's underlying manifold structure. This work proposes a multimodal approach encompassing both image and non-image features, leveraging contrastive cross-view graph fusion for PD classification. We introduce a novel multimodal co-attention module, integrating embeddings from separate graph views derived from low-dimensional representations of images and clinical features. This enables more robust and structured feature extraction for improved multi-view data analysis. Additionally, a simplified contrastive loss-based fusion method is devised to enhance cross-view fusion learning. Our graph-view multimodal approach achieves an accuracy of 91% and an area under the receiver operating characteristic curve (AUC) of 92.8% in five-fold cross-validation. It also demonstrates superior predictive capabilities on non-image data compared to solely machine learning-based methods.

Markov-Embedded Affinity Learning with Connectivity Constraints for Subspace Clustering

Subspace clustering algorithms have demonstrated remarkable success across diverse fields, including object segmentation, gene clustering, and recommendation systems. However, they often face challenges, such as omitting cluster information and the neglect of higher-order neighbor relationships within the data. To address these issues, a novel subspace clustering method named Markov-Embedded Affinity Learning with Connectivity Constraints for Subspace Clustering is proposed. This method seamlessly embeds Markov transition probability information into the self-expression, leveraging a fine-grained neighbor matrix to uncover latent data structures. This matrix preserves crucial high-order local information and complementary details, ensuring a comprehensive understanding of the data. To effectively handle complex nonlinear relationships, the method learns the underlying manifold structure from a cross-order local neighbor graph. Additionally, connectivity constraints are applied to the affinity matrix, enhancing the group structure and further improving the clustering performance. Extensive experiments demonstrate the superiority of this novel method over baseline approaches, validating its effectiveness and practical utility.

Robust feature selection via central point link information and sparse latent representation

Before conducting unsupervised feature selection, it is usually assumed that these data are independent of each other. On the contrary, real data will influence each other. Therefore, traditional feature selection methods may lose information related to each other between data. This can lead to inaccurately generated pseudo-label information and may result in poor feature selection results. To find solutions to this issue, this paper proposes robust feature selection via central point link information and sparse latent representation (CPSLR). Firstly, structure a link graph by calculating the center matrix to store the distance information from the sample to the center point. If two samples have similar distances to the center point, it can be determined that they belong to the same class. Therefore, the similarity between samples is preserved, and more accurate pseudo-label information is obtained. Secondly, CPSLR uses data graph and link graph to form a dual graph structure. It can not only retain the link information between samples but also retain the manifold structures of the samples. Then, CPSLR saves the interconnection contents between samples by sparse latent representation. That is, the constraint l2,1-norm is exerted on the expression of latent representation, and sparse non-redundant interconnection information is preserved. And by combining central point link information with sparse latent representation makes the interconnections between data reserved more comprehensive. That is to say, the pseudo-labels obtained are more like the real labels of the classes. Finally, CPSLR constrains the feature transformation matrix by l2,1/2-norm constraint so as to select robust and sparse features. CPSLR uses l2,1/2-norm constraint to assure that the feature transformation matrix is sparse, selecting more discriminative features, thereby obtaining the feature selection that can improve its efficiency. The experiments demonstrate that the clustering result of CPSLR outperform six classical or latest compared algorithms on eight datasets.

Revealing the hidden structure of disordered materials by parameterizing their local structural manifold

Durable interest in developing a framework for the detailed structure of glassy materials has produced numerous structural descriptors that trade off between general applicability and interpretability. However, none approach the combination of simplicity and wide-ranging predictive power of the lattice-grain-defect framework for crystalline materials. Working from the hypothesis that the local atomic environments of a glassy material are constrained by enthalpy minimization to a low-dimensional manifold in atomic coordinate space, we develop a generalized distance function, the Gaussian Integral Inner Product (GIIP) distance, in connection with agglomerative clustering and diffusion maps, to parameterize that manifold. Applying this approach to a two-dimensional model crystal and a three-dimensional binary model metallic glass results in parameters interpretable as coordination number, composition, volumetric strain, and local symmetry. In particular, we show that a more slowly quenched glass has a higher degree of local tetrahedral symmetry at the expense of cyclic symmetry. While these descriptors require post-hoc interpretation, they minimize bias rooted in crystalline materials science and illuminate a range of structural trends that might otherwise be missed.

Regularisation constrained denoising discriminant least squares regression for image classification

Discriminative Least Squares Regression (DLSR) is an algorithm that utilizes ε-draggings to expand the distance between classes to better solve multi-classification problems. However, this method increases the differences in regression targets within classes during the classification process, and noise in the samples adversely affects its performance. To address the above problems, we propose a novel method called Regularisation Constrained Denoising Discriminant Least Squares Regression (RCDDLSR). Firstly, the idea of matrix decomposition is introduced and sparse constraints are imposed on the noise matrix for noise elimination, thus helping to maintain the manifold structure of the raw data. Secondly, the neighbourhood relationship between samples of the same category is ensured through graph regularisation to avoid overfitting. In addition, we introduce low-rank constraints in the labelling space to better capture the local structure of the regression target within the class. Finally, our experiments show that the method performs better in various types of image datasets compared to other different algorithms.

Geometric approach to the Moore–Penrose inverse and the polar decomposition of perturbations by operator ideals

Abstract We study the Moore–Penrose inverse of perturbations by a proper symmetrically-normed ideal of a closed range operator on a Hilbert space. We show that the notion of essential codimension of projections gives a characterization of subsets of such perturbations in which the Moore–Penrose inverse is continuous with respect to the metric induced by the operator ideal. These subsets are maximal satisfying the continuity property, and they carry the structure of real analytic Banach manifolds, which are acted upon transitively by the Banach–Lie group consisting of invertible operators associated with the ideal. This geometric construction allows us to prove that the Moore–Penrose inverse is indeed a real bianalytic map between infinite-dimensional manifolds. We use these results to study the polar decomposition of closed range operators from a similar geometric perspective. At this point we prove that operator monotone functions are real analytic in the norm of any proper symmetrically-normed ideal. Finally, we show that the maps defined by the operator modulus and the polar factor in the polar decomposition of closed range operators are real analytic fiber bundles.

Theoretical simulation of steady flowing manifold in hybrid pneumatic power system

Improving the energy efficiency of existing power systems is part of energy conservation and carbon reduction efforts. Internal combustion engines (ICEs) are currently the most commonly used power source in transportation, and they can be combined with gas power sources to form hybrid power systems for enhanced energy efficiency and potential commercialization. This study introduces hybrid pneumatic systems that integrate energy from ICEs and exhaust gas through a manifold to reuse waste heat. However, the turbulence caused by compressed air in the manifold can prevent the smooth transfer and mixing of thermal energy, leading to reduced power at the final outlet. We use ANSYS Fluent software to conduct heat flow streamline analysis of the three-dimensional flow field in the manifold structure, with key parameters including throttle valve opening, manifold geometry, pressure, and temperature. Through this analysis, we explore the flow situation and heat transfer phenomenon inside the pipe, observe the flow field changes in the pipe, and adjust the diameter of the compressed air end channel for effective thermal energy. A backflow with velocity magnitude up to 72 m/s in high pressure case can be solved by compressing pipe and temperature increases from 349.9 K to 682.5 K with more efficient heat transfer and gas mixing. Thermal power, enthalpy, and mass flow rate of the hybrid power system are also investigated, enabling the ICE to operate at its optimal point. The waste heat from the ICE can also be recycled and converted into effective mechanical energy through flow work.