Low-dimensional Space Research Articles

Rdfinet: reference-guided directional diverse face inpainting network

The majority of existing face inpainting methods primarily focus on generating a single result that visually resembles the original image. The generation of diverse and plausible results has emerged as a new branch in image restoration, often referred to as “Pluralistic Image Completion”. However, most diversity methods simply use random latent vectors to generate multiple results, leading to uncontrollable outcomes. To overcome these limitations, we introduce a novel architecture known as the Reference-Guided Directional Diverse Face Inpainting Network. In this paper, instead of using a background image as reference, which is typically used in image restoration, we have used a face image, which can have many different characteristics from the original image, including but not limited to gender and age, to serve as a reference face style. Our network firstly infers the semantic information of the masked face, i.e., the face parsing map, based on the partial image and its mask, which subsequently guides and constrains directional diverse generator network. The network will learn the distribution of face images from different domains in a low-dimensional manifold space. To validate our method, we conducted extensive experiments on the CelebAMask-HQ dataset. Our method not only produces high-quality oriented diverse results but also complements the images with the style of the reference face image. Additionally, our diverse results maintain correct facial feature distribution and sizes, rather than being random. Our network has achieved SOTA results in face diverse inpainting when writing. Code will is available at https://github.com/nothingwithyou/RDFINet.

Deep Learning-Based Multifidelity Surrogate Modeling for High-Dimensional Reliability Prediction

Abstract Multifidelity surrogate modeling offers a cost-effective approach to reducing extensive evaluations of expensive physics-based simulations for reliability prediction. However, considering spatial uncertainties in multifidelity surrogate modeling remains extremely challenging due to the curse of dimensionality. To address this challenge, this paper introduces a deep learning-based multifidelity surrogate modeling approach that fuses multifidelity datasets for high-dimensional reliability analysis of complex structures. It first involves a heterogeneous dimension transformation approach to bridge the gap in terms of input format between the low-fidelity and high-fidelity domains. Then, an explainable deep convolutional dimension-reduction network (ConvDR) is proposed to effectively reduce the dimensionality of the structural reliability problems. To obtain a meaningful low-dimensional space, a new knowledge reasoning-based loss regularization mechanism is integrated with the covariance matrix adaptation evolution strategy (CMA-ES) to encourage an unbiased linear pattern in the latent space for reliability prediction. Then, the high-fidelity data can be utilized for bias modeling using Gaussian process (GP) regression. Finally, Monte Carlo simulation (MCS) is employed for the propagation of high-dimensional spatial uncertainties. Two structural examples are utilized to validate the effectiveness of the proposed method.

Datacube segmentation via deep spectral clustering

Extended vision techniques are ubiquitous in physics. However, the data cubes steaming from such analysis often pose a challenge in their interpretation, due to the intrinsic difficulty in discerning the relevant information from the spectra composing the data cube. Furthermore, the huge dimensionality of data cube spectra poses a complex task in its statistical interpretation; nevertheless, this complexity contains a massive amount of statistical information that can be exploited in an unsupervised manner to outline some essential properties of the case study at hand, e.g. it is possible to obtain an image segmentation via (deep) clustering of data-cube’s spectra, performed in a suitably defined low-dimensional embedding space. To tackle this topic, we explore the possibility of applying unsupervised clustering methods in encoded space, i.e. perform deep clustering on the spectral properties of datacube pixels. A statistical dimensional reduction is performed by an ad hoc trained (variational) AutoEncoder, in charge of mapping spectra into lower dimensional metric spaces, while the clustering process is performed by a (learnable) iterative K-means clustering algorithm. We apply this technique to two different use cases, of different physical origins: a set of macro mapping x-ray fluorescence (MA-XRF) synthetic data on pictorial artworks, and a dataset of simulated astrophysical observations.

The graded multidimensional geometry of phenotypic variation and progression in neurodegenerative syndromes.

Clinical variants of Alzheimer's disease and frontotemporal lobar degeneration display a spectrum of cognitive-behavioural changes varying between individuals and over time. Understanding the landscape of these graded individual-/group-level longitudinal variations is critical for precise phenotyping; however, this remains challenging to model. Addressing this challenge, we leverage the National Alzheimer's Coordinating Center database to derive a unified geometric framework of graded longitudinal phenotypic variation in Alzheimer's disease and frontotemporal lobar degeneration. We included three time-point, cognitive-behavioural and clinical data from 390 typical, atypical and intermediate Alzheimer's disease and frontotemporal lobar degeneration variants (114 typical Alzheimer's disease; 107 behavioural variant frontotemporal dementia; 42 motor variants of frontotemporal lobar degeneration; and 103 primary progressive aphasia patients). On this data, we applied advanced data-science approaches to derive low-dimensional geometric spaces capturing core features underpinning clinical progression of Alzheimer's disease and frontotemporal lobar degeneration syndromes. To do so, we first used principal component analysis to derive six axes of graded longitudinal phenotypic variation capturing patient-specific movement along and across these axes. Then, we distilled these axes into a visualisable 2D manifold of longitudinal phenotypic variation using Uniform Manifold Approximation and Projection. Both geometries together enabled the assimilation and inter-relation of paradigmatic and mixed cases, capturing dynamic individual trajectories, and linking syndromic variability to neuropathology and key clinical end-points such as survival. Through these low-dimensional geometries, we show that (i) specific syndromes (Alzheimer's disease and primary progressive aphasia) converge over time into a de-differentiated pooled phenotype, while others (frontotemporal dementia variants) diverge to look different from this generic phenotype; (ii) phenotypic diversification is predicted by simultaneous progression along multiple axes, varying in a graded manner between individuals and syndromes; and (iii) movement along specific principal axes predicts survival at 36 months in a syndrome-specific manner and in individual pathological groupings. The resultant mapping of dynamics underlying cognitive-behavioural evolution potentially holds paradigm-changing implications to predicting phenotypic diversification and phenotype-neurobiological mapping in Alzheimer's disease and frontotemporal lobar degeneration.

Near-term quantum algorithm for solving the MaxCut problem with fewer quantum resources

MaxCut is an NP-hard combinatorial optimization problem in graph theory. The quantum approximate optimization algorithms (QAOAs) offer new methods for solving such problems, which are potentially better than classical schemes. However, the requirement of N qubits for solving graphs with N-vertex in QAOA, coupled with the large-N trainability issue due to barren plateaus, poses a substantial challenge for noisy intermediate-scale quantum (NISQ) devices. Recently, a hybrid quantum–classical algorithm has been proposed for solving semidefinite programs (SDPs), named NISQ SDP Solver (NSS). In this paper, we study the performance of the NSS for solving MaxCut problem via the introduction of a near-term quantum algorithm and execution of experimental simulations. Since the MaxCut problem admits a relaxation into an SDP formulation, the SDP can be solved using NSS, with a subsequent classical post-processing step converting the hybrid density matrix into a MaxCut solution. After performing these steps, the near-term quantum algorithm will also inherit the advantages of NSS. We analyze the algorithm as compared to QAOAs and find that the depth of the quantum circuit is independent of the number of edges on the graph. Our algorithm requires logN qubits and has O(1) circuit depth. This implies that it uses exponentially fewer qubits compared to QAOAs while also requiring a significantly reduced circuit depth to solve the MaxCut problem. To demonstrate the effectiveness of the NSS, we focused on 3-regular graphs, 9-regular graphs, and ER graphs. Numerical results indicate that the quantum algorithm achieves a comparable approximation ratio with classical methods by solving the original high dimensional problem within a lower dimensional ansatz space. Analysis under various initial states shows that the algorithm exhibits a remarkably stable performance.

Fusion of Laser-Induced Breakdown Spectroscopy and Raman Spectroscopy for Mineral Identification Based on Machine Learning.

Rapid and reliable identification of mineral species is a challenging but crucial task with promising application prospects in mineralogy, metallurgy, and geology. Spectroscopic techniques such as laser-induced breakdown spectroscopy (LIBS) and Raman spectroscopy (RS) efficiently capture the elemental composition and structural information of minerals, making them a potential tool for in situ and real-time analysis of minerals. This study introduces an integrated LIBS-RS system and the fusion of LIBS and RS spectra coupled with machine learning to classify six different types of natural mineral. In order to visualize the separability of different mineral species clearly, the spectral data were projected into low-dimensional space through t-distributed stochastic neighbor embedding (t-SNE). Additionally, the Fisher score (FS) was used to identify important variables that contribute to the data classification, and the corresponding chemical elements and molecular bonds were then interpreted. The between-minerals difference in the feature spectral intensity of LIBS and RS variables could also be observed. After the minerals spectra were pre-processed, the relationship between spectral intensity and the mineral category was modeled using machine learning methods, including partial least squares-discriminant analysis (PLS-DA) and kernel extreme learning machine (K-ELM). The results show that K-ELM and PLS-DA based on the fusion LIBS-RS data achieved the highest accuracy of 98.4%. These findings demonstrate the feasibility of the integrated LIBS-RS system combined with machine learning for the fast and reliable classification of minerals.

A Globally Convergent Inertial First-Order Optimization Method for Multidimensional Scaling

Multidimensional scaling (MDS) is a popular tool for dimensionality reduction and data visualization. Given distances between data points and a target low-dimension, the MDS problem seeks to find a configuration of these points in the low-dimensional space, such that the inter-point distances are preserved as well as possible. We focus on the most common approach to formulate the MDS problem, known as stress minimization, which results in a challenging non-smooth and non-convex optimization problem. In this paper, we propose an inertial version of the well-known SMACOF Algorithm, which we call AI-SMACOF. This algorithm is proven to be globally convergent, and to the best of our knowledge this is the first result of this kind for algorithms aiming at solving the stress MDS minimization. In addition to the theoretical findings, numerical experiments provide another evidence for the superiority of the proposed algorithm.

Cube is a good form: Hyperspectral band selection via multi-dimensional and high-order structure preserved clustering

As an effective strategy for reducing the noisy and redundant information for hyperspectral imagery (HSI), hyperspectral band selection intends to select a subset of original hyperspectral bands, which boosts the subsequent different tasks. In this paper, we introduce a multi-dimensional high-order structure preserved clustering method for hyperspectral band selection, referred to as MHSPC briefly. By regarding original hyperspectral images as a tensor cube, we apply the tensor CP (CANDECOMP/PARAFAC) decomposition on it to exploit the multi-dimensional structural information as well as generate a low-dimensional latent feature representation. In order to capture the local geometrical structure along the spectral dimension, a graph regularizer is imposed on the new feature representation in the lower dimensional space. In addition, since the low rankness of HSIs is an important global property, we utilize a nuclear norm constraint on the latent feature representation matrix to capture the global data structure information. Different to most of previous clustering based hyperspectral band selection methods which vectorize each band as a vector without considering the 2-D spatial information, the proposed MHSPC can effectively capture the spatial structure as well as the spectral correlation of original hyperspectral cube in both local and global perspectives. An efficient alternatively updating algorithm with theoretical convergence guarantee is designed to solve the resultant optimization problem, and extensive experimental results on four benchmark datasets validate the effectiveness of the proposed MHSPC over other state-of-the-arts.

Separation and Extraction of Compound-Fault Signal Based on Multi-Constraint Non-Negative Matrix Factorization.

To solve the separation of multi-source signals and detect their features from a single channel, a signal separation method using multi-constraint non-negative matrix factorization (NMF) is proposed. In view of the existing NMF algorithm not performing well in the underdetermined blind source separation, the β-divergence constraints and determinant constraints are introduced in the NMF algorithm, which can enhance local feature information and reduce redundant components by constraining the objective function. In addition, the Sine-bell window function is selected as the processing method for short-time Fourier transform (STFT), and it can preserve the overall feature distribution of the original signal. The original vibration signal is first transformed into time-frequency domain with the STFT, which describes the local characteristic of the signal from the time-frequency distribution. Then, the multi-constraint NMF is applied to reduce the dimensionality of the data and separate feature components in the low dimensional space. Meanwhile, the parameter WK is constructed to filter the reconstructed signal that recombined with the feature component in the time domain. Ultimately, the separated signals will be subjected to envelope spectrum analysis to detect fault features. The simulated and experimental results indicate the effectiveness of the proposed approach, which can realize the separation of multi-source signals and their fault diagnosis of bearings. In addition, it is also confirmed that the proposed method, juxtaposed with the NMF algorithm of the traditional objective function, is more applicable for compound fault diagnosis of the rotating machinery.

Mixed finite elements for Kirchhoff–Love plate bending

We present a mixed finite element method with triangular and parallelogram meshes for the Kirchhoff–Love plate bending model. Critical ingredient is the construction of low-dimensional local spaces and appropriate degrees of freedom that provide conformity in terms of a sufficiently large tensor space and that allow for any kind of physically relevant Dirichlet and Neumann boundary conditions. For Dirichlet boundary conditions and polygonal plates, we prove quasi-optimal convergence of the mixed scheme. An a posteriori error estimator is derived for the special case of the biharmonic problem. Numerical results for regular and singular examples illustrate our findings. They confirm expected convergence rates and exemplify the performance of an adaptive algorithm steered by our error estimator.

Multi-task neural networks by learned contextual inputs

This paper explores learned-context neural networks. It is a multi-task learning architecture based on a fully shared neural network and an augmented input vector containing trainable task parameters. The architecture is interesting due to its powerful task adaption mechanism, which facilitates a low-dimensional task parameter space. Theoretically, we show that a scalar task parameter is sufficient for universal approximation of all tasks, which is not necessarily the case for more common architectures. Empirically it is shown that, for homogeneous tasks, the dimension of the task parameter may vary with the complexity of the tasks, but a small task parameter space is generally viable. The task parameter space is found to be well-behaved, which simplifies workflows related to updating models as new data arrives, and learning new tasks with the shared parameters are frozen. Additionally, the architecture displays robustness towards datasets where tasks have few data points. The architecture’s performance is compared to similar neural network architectures on ten datasets, with competitive results.

Refining tomography with generative neural networks trained from geodynamics

SUMMARY Inverse problems occur in many fields of geophysics, wherein surface observations are used to infer the internal structure of the Earth. Given the non-linearity and non-uniqueness inherent in these problems, a standard strategy is to incorporate a priori information regarding the unknown model. Sometimes a solution is obtained by imposing that the inverted model remains close to a reference model and with smooth lateral variations (e.g. a correlation length or a minimal wavelength are imposed). This approach forbids the presence of strong gradients or discontinuities in the recovered model. Admittedly, discontinuities, such as interfaces between layers, or shapes of geological provinces or of geological objects such as slabs can be a priori imposed or even suggested by the data themselves. This is however limited to a small set of possible constraints. For example, it would be very challenging and computationally expensive to perform a tomographic inversion where the subducting slabs would have possible top discontinuities with unknown shapes. The problem seems formidable because one cannot even imagine how to sample the prior space: is each specific slab continuous or broken into different portions having their own interfaces? No continuous set of parameters seems to describe all the possible interfaces that we could consider. To circumvent these questions, we propose to train a Generative Adversarial neural Network (GAN) to generate models from a geologically plausible prior distribution obtained from geodynamic simulations. In a Bayesian framework, a Markov chain Monte Carlo algorithm is used to sample the low-dimensional model space depicting the ensemble of potential geological models. This enables the integration of intricate a priori information, parametrized within a low-dimensional model space conducive to efficient sampling. The application of this approach is demonstrated in the context of a downscaling problem, where the objective is to infer small-scale geological structures from a smooth seismic tomographic image.

C-KGE: Curriculum learning-based Knowledge Graph Embedding

Knowledge graph embedding (KGE) aims to embed entities and relations in knowledge graphs (KGs) into a continuous, low-dimensional vector space. It has been shown as an effective tool for integrating knowledge graphs to improve various intelligent applications, such as question answering and information extraction. However, previous KGE models ignore the hidden natural order of knowledge learning on learning the embeddings of entities and relations, leaving room for improvement in their performance. Inspired by the easy-to-hard pattern used in human knowledge learning, this paper proposes a Curriculum learning-based KGE (C-KGE) model, which learns the embeddings of entities and relations from “basic knowledge” to “domain knowledge”. Specifically, a seed set representing the basic knowledge and several knowledge subsets are identified from KG. Then, entity overlap is employed to score the learning difficulty of each subset. Finally, C-KGE trains the entities and relations in each subset according to the learning difficulty score of each subset. C-KGE leverages trained embeddings of the seed set as prior knowledge and learns knowledge subsets iteratively to transfer knowledge between the seed set and subsets, smoothing the learning process of knowledge facts. Experimental results on real-world datasets demonstrate that the proposed model achieves improved embedding performances as well as reducing training time. Our codes and data will be released later.

A complex network analysis approach to bankruptcy prediction using company relational information-based drivers

Corporate bankruptcy prediction is a long-standing topic of interest for a variety of stakeholders. Various prediction methodologies have been proposed to achieve more accurate predictions. So far, most studies have focused on making predictions using artificial intelligence models fed with accounting information only or a combination of accounting information and other category or categories of information. This study proposes new corporate governance drivers that exploit company relational information using board of directors’ networks of companies and the concept of node embeddings obtained by mapping large directors’ networks to lower-dimensional spaces. The contribution of these new drivers to the overall prediction performance is assessed using data on UK companies listed on the London Stock Exchange. The proposed two-stage methodology consists of devising new drivers in the first stage followed by risk class prediction in the second stage. We examine different complex network structures that capture the diversity of connectivity patterns observed in directors’ networks. Empirical results suggest that the proposed network-based drivers that use company relational corporate governance information significantly improve the performance of bankruptcy prediction.

Hardness and Approximability of Dimension Reduction on the Probability Simplex

Dimension reduction is a technique used to transform data from a high-dimensional space into a lower-dimensional space, aiming to retain as much of the original information as possible. This approach is crucial in many disciplines like engineering, biology, astronomy, and economics. In this paper, we consider the following dimensionality reduction instance: Given an n-dimensional probability distribution p and an integer m<n, we aim to find the m-dimensional probability distribution q that is the closest to p, using the Kullback–Leibler divergence as the measure of closeness. We prove that the problem is strongly NP-hard, and we present an approximation algorithm for it.

Points2NeRF: Generating Neural Radiance Fields from 3D point cloud

Neural Radiance Fields (NeRFs) offers a state-of-the-art quality in synthesizing novel views of complex 3D scenes from a small subset of base images. For NeRFs to perform optimally, the registration of base images has to follow certain assumptions, including maintaining a constant distance between the camera and the object. We can address this limitation by training NeRFs with 3D point clouds instead of images, yet a straightforward substitution is impossible due to the sparsity of 3D clouds in the under-sampled regions, which leads to incomplete reconstruction output by NeRFs. To solve this problem, here we propose an auto-encoder-based architecture that leverages a hypernetwork paradigm to transfer 3D points with the associated color values through a lower-dimensional latent space and generate weights of NeRF model. This way, we can accommodate the sparsity of 3D point clouds and fully exploit the potential of point cloud data. As a side benefit, our method offers an implicit way of representing 3D scenes and objects that can be employed to condition NeRFs and hence generalize the models beyond objects seen during training. The empirical evaluation confirms the advantages of our method over conventional NeRFs and proves its superiority in practical applications.

Consensus and discriminative non-negative matrix factorization for multi-view unsupervised feature selection

Multi-view unsupervised feature selection (MUFS) has been proven to be an efficient dimensionality reduction technique for multi-view data. Existing methods have two main challenges: (1) The consistency information from different views is not fully exploited. (2) The cluster structure of the original data is not well utilized. To settle these problems, we propose a novel consensus and discriminative non-negative matrix factorization (CDNMF) for multi-view unsupervised feature selection. Specifically, CDNMF obtains a robust low-dimensional latent space by NMF with an ℓ2,1-norm constraint. To select more discriminative features, we further impose a sparsity constraint on the learned latent features. Moreover, CDNMF performs k-means clustering on all views separately to obtain the pseudo-labels of each view, which are used to guide the learning of consensus information among views. To solve our model, we develop an efficient iterative optimization algorithm. Extensive experimental results on ten benchmark datasets demonstrate that the proposed significantly outperforms several existing feature selection methods in clustering tasks.

Similarity measure method of near-infrared spectrum combined with multi-attribute information

Due to the high-dimensionality, redundancy, and non-linearity of the near-infrared (NIR) spectra data, as well as the influence of attributes such as producing area and grade of the sample, which can all affect the similarity measure between samples. This paper proposed a t-distributed stochastic neighbor embedding algorithm based on Sinkhorn distance (St-SNE) combined with multi-attribute data information. Firstly, the Sinkhorn distance was introduced which can solve problems such as KL divergence asymmetry and sparse data distribution in high-dimensional space, thereby constructing probability distributions that make low-dimensional space similar to high-dimensional space. In addition, to address the impact of multi-attribute features of samples on similarity measure, a multi-attribute distance matrix was constructed using information entropy, and then combined with the numerical matrix of spectral data to obtain a mixed data matrix. In order to validate the effectiveness of the St-SNE algorithm, dimensionality reduction projection was performed on NIR spectral data and compared with PCA, LPP, and t-SNE algorithms. The results demonstrated that the St-SNE algorithm effectively distinguishes samples with different attribute information, and produced more distinct projection boundaries of sample category in low-dimensional space. Then we tested the classification performance of St-SNE for different attributes by using the tobacco and mango datasets, and compared it with LPP, t-SNE, UMAP, and Fisher t-SNE algorithms. The results showed that St-SNE algorithm had the highest classification accuracy for different attributes. Finally, we compared the results of searching the most similar sample with the target tobacco for cigarette formulas, and experiments showed that the St-SNE had the highest consistency with the recommendation of the experts than that of the other algorithms. It can provide strong support for the maintenance and design of the product formula.

Network embedding based on DepDist contraction

Networks provide an understandable and, in the case of small size, visualizable representation of data, which allows us to obtain essential information about the relationships between pairs of nodes, e.g., their distances. In visualization, networks have an alternative two-dimensional vector representation to which various machine-learning methods can be applied. More generally, networks can be transformed into a low-dimensional space using so-called embedding methods, which bridge the gap between network analysis and traditional machine learning by creating numerical representations that capture the essence of the network structure. In this article, we present a new embedding method that uses non-symmetric dependency to find the distance between nodes and applies an iterative procedure to find a satisfactory distribution of nodes in space. For dimension 2 and the visualization of the result, we demonstrate the method’s effectiveness on small networks. For higher dimensions and several larger networks, we present the results of two experiments comparing our results with two well-established methods in the research community, namely node2vec and DeepWalk. The first experiment focuses on a qualitative comparison of the methods, while the second focuses on applying and comparing the classification results to embeddings in a higher dimension. Although the presented method does not outperform the two chosen methods, its results are still comparable. Therefore, we also explain the limitations of our method and a possible way to overcome them.

Vertical slice equal sampling and transformer network for point cloud ground filtering

ABSTRACT Ground filtering is a critical yet challenging aspect of Digital Elevation Model (DEM) generation from LiDAR point clouds. Existing deep learning approaches for point cloud processing encounter difficulties in addressing the conflict between the demands of large-scale object filtering tasks and GPU memory. In response to this challenge, this study presents a novel ground filtering method utilizing deep learning. The proposed method incorporates Vertical Slicing Equally Sampling (VSES) to locally sample the original point cloud, effectively organizing the unordered sequence of points and reducing their number while maintaining the representation of the terrain. Subsequently, the resampled point cloud is fed into the TransGF network for classification processing. TransGF is a network model that leverages Transformer as its backbone. Initially, TransGF employs a linear layer to project the input data from a low-dimensional space of (n, 3) to a high-dimensional space of (n, 32). It then extracts features using a module comprising six consecutive Transformer blocks. Finally, TransGF outputs the probability of each point in the samples belonging to either ground or non-ground points. This method is evaluated using the publicly accessible OpenGF ground filtering dataset, showcasing its superior performance in testing scenarios. Noteworthy advantages include structural conciseness, ease of training, and a promising potential for practical applications. The results affirm the effectiveness of TransGF in overcoming challenges associated with ground filtering in LiDAR point cloud processing, positioning it as a valuable contribution to the advancement of DEM generation techniques.