High-dimensional Space Research Articles

Scaling Learning based Policy Optimization for Temporal Logic Tasks by Controller Network Dropout

This paper introduces a model-based approach for training feedback controllers for an autonomous agent operating in a highly nonlinear (albeit deterministic) environment. We desire the trained policy to ensure that the agent satisfies specific task objectives and safety constraints, both expressed in Discrete-Time Signal Temporal Logic (DT-STL). One advantage for reformulation of a task via formal frameworks, like DT-STL, is that it permits quantitative satisfaction semantics. In other words, given a trajectory and a DT-STL formula, we can compute the robustness , which can be interpreted as an approximate signed distance between the trajectory and the set of trajectories satisfying the formula. We utilize feedback control, and we assume a feed forward neural network for learning the feedback controller. We show how this learning problem is similar to training recurrent neural networks (RNNs), where the number of recurrent units is proportional to the temporal horizon of the agent's task objectives. This poses a challenge: RNNs are susceptible to vanishing and exploding gradients, and naïve gradient descent-based strategies to solve long-horizon task objectives thus suffer from the same problems. To tackle this challenge, we introduce a novel gradient approximation algorithm based on the idea of dropout or gradient sampling. One of the main contributions is the notion of controller network dropout , where we approximate the NN controller in several time-steps in the task horizon by the control input obtained using the controller in a previous training step. We show that our control synthesis methodology, can be quite helpful for stochastic gradient descent to converge with less numerical issues, enabling scalable backpropagation over long time horizons and trajectories over high dimensional state spaces. We demonstrate the efficacy of our approach on various motion planning applications requiring complex spatio-temporal and sequential tasks ranging over thousands of time-steps.

Deep generative models of protein structure uncover distant relationships across a continuous fold space

Our views of fold space implicitly rest upon many assumptions that impact how we analyze, interpret and understand protein structure, function and evolution. For instance, is there an optimal granularity in viewing protein structural similarities (e.g., architecture, topology or some other level)? Similarly, the discrete/continuous dichotomy of fold space is central, but remains unresolved. Discrete views of fold space bin similar folds into distinct, non-overlapping groups; unfortunately, such binning can miss remote relationships. While hierarchical systems like CATH are indispensable resources, less heuristic and more conceptually flexible approaches could enable more nuanced explorations of fold space. Building upon an Urfold model of protein structure, here we present a deep generative modeling framework, termed DeepUrfold, for analyzing protein relationships at scale. DeepUrfold’s learned embeddings occupy high-dimensional latent spaces that can be distilled for a given protein in terms of an amalgamated representation uniting sequence, structure and biophysical properties. This approach is structure-guided, versus being purely structure-based, and DeepUrfold learns representations that, in a sense, define superfamilies. Deploying DeepUrfold with CATH reveals evolutionarily-remote relationships that evade existing methodologies, and suggests a mostly-continuous view of fold space—a view that extends beyond simple geometric similarity, towards the realm of integrated sequence ↔ structure ↔ function properties.

Probing unexplored areas in high-dimensional fMRI voxel space using an encoding model and image synthesis

Probing unexplored areas in high-dimensional fMRI voxel space using an encoding model and image synthesis

Gravitational wave signal extraction against non-stationary instrumental noises with deep neural network

Sapce-borne gravitational wave antennas, such as LISA and LISA-like mission (Taiji and Tianqin), will offer novel perspectives for exploring our Universe while introduce new challenges, especially in data analysis. Aside from the known challenges like high parameter space dimension, superposition of large number of signals etc., gravitational wave detections in space would be more seriously affected by anomalies or non-stationarities in the science measurements. Considering the three types of foreseeable non-stationarities including data gaps, transients (glitches), and time-varying noise auto-correlations, which may come from routine maintenance or unexpected disturbances during science operations, we developed a deep learning model for accurate signal extractions confronted with such anomalous scenarios. Our model exhibits the same performance as the current state-of-the-art models do for the ideal and anomaly free scenario, while shows remarkable adaptability in extractions of coalescing massive black hole binary signal against all three types of non-stationarities and even their mixtures. This also provide new explorations into the robustness studies of deep learning models for data processing in space-borne gravitational wave missions.

A nested non-intrusive stochastic isogeometric method for nonlinear thermal vibration of FGM plates under random loading

This paper introduces an adaptive nested non-invasive dynamic SIGA method based on RBFNN for the nonlinear thermal vibration analysis of FGM plates under random loading. This method is robust and accurate in high-dimensional input spaces regardless of the sample sequence, addresses the low convergence rate of QMCS, and solves the stochastic dynamic response probability density function under random loading. Firstly, the multidimensional input space for random loading is constructed based on several mutually independent random variables via the stationary Gaussian stochastic process simulation technique. The nonlinear FGM plate model subjected to random loading in the thermal environment is modeled using HSDT with von Kármán strain-displacement relation within the IGA framework. Subsequently, a nested non-invasive dynamic response surface analysis method, SIGA-RBFNN, is proposed based on the RBFNN technique. Analysis of the FGM plate response under random loading demonstrates that SIGA-RBFNN, compared to tensor-product-based SIGA methods, is less affected by sample sequence types. It effectively addresses high-dimensional stochasticity and offers significant advantages in robustness, accuracy, and efficiency. Finally, SIGA-RBFNN compensates for the low convergence speed of QMCS and successfully solves the FGM plate displacement response statistics and probability density function under random loading.

Navigation through high-dimensional chemical space: discovery of Ba5Y13[SiO4]8O8.5 and Ba3Y2[Si2O7]2.

Two compounds were discovered in the well-studied BaO-Y2O3-SiO2 phase field. Two different experimental routines were used for the exploration of this system due to the differences of synthetic conditions and competition with a glass field. The first phase Ba5Y13[SiO4]8O8.5 was isolated through a combination of energy dispersive X-ray spectroscopy analysis and diffraction techniques which guided the exploration. The second phase Ba3Y2[Si2O7]2 was located using iterative algorithmic identification of target compositions. The structure solution of the new compounds was aided by continuous rotation electron diffraction, and the structures were refined against combined synchrotron and neutron time-of-flight powder diffraction. Ba5Y13[SiO4]8O8.5 crystallizes in I4̄2m, a = 18.92732(1), c = 5.357307(6) Å and represents its own structure type which combines elements of structures of known silicates embedded in columns of interconnected yttrium-centred polyhedra characteristic of high-pressure phases. Ba3Y2[Si2O7]2 has P21 symmetry with a pseudo-tetragonal cell (a = 16.47640(4), b = 9.04150(5), c = 9.04114(7) Å, β = 90.0122(9)°) and is a direct superstructure of the Ca3BaBi[P2O7]2 structure. Despite the lower symmetry, the structure of Ba3Y2[Si2O7]2 retains disorder in both Ba/Y sites and disilicate network, thus presenting a superposition of possible locally-ordered fragments. Ba5Y13[SiO4]8O8.5 has low thermal conductivity of 1.04(5) W m-1 K-1 at room temperature. The two discovered phases provide a rich structural platform for further functional material design. The interplay of automated unknown phase composition identification with multiple diffraction methods offers acceleration of the time-consuming exploration of high-dimensional chemical spaces for new structures.

G[formula omitted]BFNN: Generalized geodesic basis function neural network

Real-world data is typically distributed on low-dimensional manifolds embedded in high-dimensional Euclidean spaces. Accurately extracting spatial distribution features on general manifolds that reflect the intrinsic characteristics of data is crucial for effective feature representation. Therefore, we propose a generalized geodesic basis function neural network (G2BFNN) architecture. The generalized geodesic basis functions (G2BF) are defined based on generalized geodesic distances. The generalized geodesic distance metric (G2DM) is obtained by learning the manifold structure. To implement this architecture, a specific G2BFNN, named discriminative local preserving projection-based G2BFNN (DLPP-G2BFNN) is proposed. DLPP-G2BFNN mainly contains two modules, namely the manifold structure learning module (MSLM) and the network mapping module (NMM). In the MSLM module, a supervised adjacency graph matrix is constructed to constrain the learning of the manifold structure. This enables the learned features in the embedding subspace to maintain the manifold structure while enhancing the discriminability. The features and G2DM learned in the MSLM are fed into the NMM. Through the G2BF in the NMM, the spatial distribution features on manifold are obtained. Finally, the output of the network is obtained through the fully connected layer. Compared with the local response neural network based on Euclidean distance, the proposed network can reveal more essential spatial structure characteristics of the data. Meanwhile, the proposed G2BFNN is a generalized network architecture that can be combined with any manifold learning method, showcasing high scalability. The experimental results demonstrate that the proposed DLPP-G2BFNN outperforms existing methods by utilizing fewer kernels while achieving higher recognition performance.

An automated convergence diagnostic for phylogenetic MCMC analyses.

Assessing convergence of Markov chain Monte Carlo (MCMC) based analyses is crucial but challenging, especially so in high dimensional and complex spaces such as the space of phylogenetic trees (treespace). In practice, it is assumed that the target distribution is the unique stationary distribution of the MCMC and convergence is achieved when samples appear to be stationary. Here we leverage recent advances in computational geometry of the treespace and introduce a method that combines classical statistical techniques and algorithms with geometric properties of the treespace to automatically evaluate and assess practical convergence of phylogenetic MCMC analyses. Our method monitors convergence across multiple MCMC chains and achieves high accuracy in detecting both practical convergence and convergence issues within treespace. Furthermore, our approach is developed to allow for real-time evaluation during the MCMC algorithm run, eliminating any of the chain post-processing steps that are currently required. Our tool therefore improves reliability and efficiency of MCMC based phylogenetic inference methods and makes analyses easier to reproduce and compare. We demonstrate the efficacy of our diagnostic via a well-calibrated simulation study and provide examples of its performance on real data sets. Although our method performs well in practice, a significant part of the underlying treespace probability theory is still missing, which creates an excellent opportunity for future mathematical research in this area. The open source package for the phylogenetic inference framework BEAST2, called ASM, that implements these methods, making them accessible through a user-friendly GUI, is available from https://github.com/rbouckaert/asm/. The open source Python package, called tetres, that provides an interface for these methods enabling their applications beyond BEAST2 can be accessed at https://github.com/bioDS/tetres/.

Optimising outbound scheduling in shipyard steel stockyard

ABSTRACT In shipyard operations, steel stockyards manage the intake, storage and distribution of raw materials. However, the inherent customisation of steel plates presents challenges such as inefficient outbound scheduling and elevated energy consumption. To address these issues, this study introduces a novel outbound scheduling model that comprehensively considers manufacturing orders and the current warehouse status during the outbound process. To solve this model, an improved ant colony optimisation algorithm, based on an energy consumption selection strategy (ECS-IACO), is proposed. The algorithm adopts a dynamic pheromone matrix and updates rules to address the complexity of high-dimensional state spaces. The energy consumption selection strategy is integrated into initialisation and state transitions, guiding and decisions throughout. Extensive numerical tests show that the ECS-IACO is robust and highly efficient in solving problems. In real-world scenarios tests, the ECS-IACO reduced energy use by up to 22% more than other algorithms, supporting sustainability efforts in business operations.

Biquaternion ambiguity function and associated uncertainty principles

A mathematical tool used to analyze signals in high-dimensional areas is the biquaternion ambiguity function. While typical ambiguity functions deal with time delay and Doppler frequency, biquaternion ambiguity functions extend the concept to rotations and scaling factors. Biquaternion ambiguity functions offer a more thorough representation of signal ambiguity in these high-dimensional spaces, which can help develop better signal processing algorithms for a range of uses. In this paper, we introduce the notion of biquaternion ambiguity function which is formed from conventional ambiguity function by invoking biquaternion algebra. We first establish the various properties, namely dilation, translation, shift, nonlinearity, boundedness, reconstruction formula, Moyal’s formula, and inversion formula. An interesting relationship between the biquaternion Fourier transform and the biquaternion ambiguity function is also established. Furthermore, we establish the various versions of uncertainty inequalities namely, Pitt’s inequality, sharp Hausdorff–Young inequality, sharp Local uncertainty principle and logarithmic uncertainty principle for biquaternion ambiguity function. Some potential applications are also presented at the end.

Efficient multi-objective optimization and operational analysis of amine scrubbing CO2 capture process with artificial neural network

Amine scrubbing processes for post-combustion CO2 capture have been extensively studied and significantly improved via various novel designs. However, the amine scrubbers implemented nowadays were usually not optimized according to a number of different evaluation criteria. This is often due to the fact that, for high dimensional design spaces, the rigorous simulation runs needed to facilitate process optimization always calls for huge numbers of simulation software accesses and overwhelming iterative calculations. Therefore, in this study, the well-trained surrogate model was adopted to replace its rigorous counterpart for the purpose of ensuring efficient optimization runs in practical applications. In current study, two objectives, i.e., the specific equivalent work and the CO2 capture level, were both rapidly and effectively optimized in various practical scenarios with different flue gas CO2 concentrations. The corresponding operational parameters and utility consumptions were also easily obtained without additional effort. The computation results obtained so far showed that the proposed surrogate-assisted approach can be utilized to significantly reduce the computational load in practice.

Prior knowledge embedding convolutional autoencoder: A single-source domain generalized fault diagnosis framework under small samples

The proposed transfer learning-based fault diagnosis models have achieved good results in multi-source domain generalization (MDG) tasks. However, research on single-source domain generalization (SDG) is relatively scarce, and the limited availability of small training samples is seldom considered. Therefore, to address the insufficient feature extraction capability and poor generalization performance of existing models on single-source domain small sample data, a novel single-source domain generalization fault diagnosis (SDGFD) framework, the prior knowledge embedded convolutional autoencoder (PKECA), is proposed. During the training phase, first, single-source domain data are used to construct prior features based on the time domain, frequency domain, and time-frequency domain. Second, a prior knowledge embedding structure based on the convolutional autoencoder is built, which compresses the prior knowledge and original vibration data into a high-dimensional space of consistent dimensions, embedding the prior knowledge into the features corresponding to the vibration data using a mean squared error loss function. Subsequently, the proposed centroid-based self-supervised learning (CBSSL) strategy further constrains high-dimensional features, improving the generalization ability. The designed sparse regularized activation (SRA) function significantly enhances the regularization effect on features. During the testing phase, it is only necessary to input the data from the unknown domain to identify the fault types. The experimental results show that the proposed method achieves superior performance in fault diagnosis tasks involving cross-speed, time-varying speed, and small sample data in SDGFD, demonstrating that PKECA has strong generalizability. The code can be found here: https://github.com/John-520/PKECA. © 2024 Elsevier Science. All rights reserved

Cryo-EM Heterogeneity Analysis using Regularized Covariance Estimation and Kernel Regression.

Proteins and the complexes they form are central to nearly all cellular processes. Their flexibility, expressed through a continuum of states, provides a window into their biological functions. Cryogenic electron microscopy (cryo-EM) is an ideal tool to study these dynamic states as it captures specimens in non-crystalline conditions and enables high-resolution reconstructions. However, analyzing the heterogeneous distributions of conformations from cryo-EM data is challenging. We present RECOVAR, a method for analyzing these distributions based on principal component analysis (PCA) computed using a REgularized COVARiance estimator. RECOVAR is fast, robust, interpretable, expressive, and competitive with the state-of-art neural network methods on heterogeneous cryo-EM datasets. The regularized covariance method efficiently computes a large number of high-resolution principal components that can encode rich heterogeneous distributions of conformations and does so robustly thanks to an automatic regularization scheme. The novel reconstruction method based on adaptive kernel regression resolves conformational states to a higher resolution than all other tested methods on extensive independent benchmarks while remaining highly interpretable. Additionally, we exploit favorable properties of the PCA embedding to estimate the conformational density accurately. This density allows for better interpretability of the latent space by identifying stable states and low free-energy motions. Finally, we present a scheme to navigate the high-dimensional latent space by automatically identifying these low free-energy trajectories. We make the code freely available at https://github.com/ma-gilles/recovar .

A novel capsule network based on Multi-Order Descartes Extension Transformation

In recent years, the capsule network has significantly impacted deep learning with its unique structure that robustly handles spatial relationships and image deformations like rotation and scaling. While previous research has primarily focused on enhancing the structural network of capsule networks to process complex images, little attention has been given to the rich semantic information contained within the capsules themselves. We recognize this gap and propose the Multi-Order Descartes Expansion Capsule Network (MODE-CapsNet). By introducing the Multi-Order Descartes Expansion Transformation (MODET), this innovative architecture enhances the expressiveness of a single capsule by enabling its projection into a higher-dimensional space. As far as we know, this is the first significant enhancement at the single-capsule granularity level, providing a new perspective for improving capsule networks. Additionally, we proposed a hierarchical routing algorithm designed explicitly for the MODE capsules, significantly optimizing computational efficiency and performance. Experimental results on datasets (MNIST, Fashion-MNIST, SVHN, CIFAR-10, tiny-ImageNet) showed that MODE capsules exhibited improved separability and expressiveness, contributing to overall network accuracy, robustness, and computational efficiency.

ECF-Net: Enhanced, Channel-Based, Multi-Scale Feature Fusion Network for COVID-19 Image Segmentation

Accurate segmentation of COVID-19 lesion regions in lung CT images aids physicians in analyzing and diagnosing patients’ conditions. However, the varying morphology and blurred contours of these regions make this task complex and challenging. Existing methods utilizing Transformer architecture lack attention to local features, leading to the loss of detailed information in tiny lesion regions. To address these issues, we propose a multi-scale feature fusion network, ECF-Net, based on channel enhancement. Specifically, we leverage the learning capabilities of both CNN and Transformer architectures to design parallel channel extraction blocks in three different ways, effectively capturing diverse lesion features. Additionally, to minimize irrelevant information in the high-dimensional feature space and focus the network on useful and critical information, we develop adaptive feature generation blocks. Lastly, a bidirectional pyramid-structured feature fusion approach is introduced to integrate features at different levels, enhancing the diversity of feature representations and improving segmentation accuracy for lesions of various scales. The proposed method is tested on four COVID-19 datasets, demonstrating mIoU values of 84.36%, 87.15%, 83.73%, and 75.58%, respectively, outperforming several current state-of-the-art methods and exhibiting excellent segmentation performance. These findings provide robust technical support for medical image segmentation in clinical practice.

9D Rotation Representation-SVD Fusion with Deep Learning for Unconstrained Head Pose Estimation

Abstract Accurately estimating human head pose poses a significant challenge across various application domains. To address the inherent limitations of previous approaches, this research proposes an unconstrained head pose estimation strategy. The method combines deep learning with rotation matrices, utilizing nine-dimensional vectors output by the neural network, which are projected back to rotation matrices in SO (3) space through singular value decomposition. This ensures both the smoothness and uniqueness of the rotation representation. The approach demonstrates distinct advantages in handling the rotation estimation task, particularly when the rotated representation is used as the model output. It not only avoids the discontinuity and double-coverage issues associated with prior methods but also enhances the stability of the representation in high-dimensional space, thereby improving the learning process. Additionally, the geodesic loss function is incorporated to train the network. The proposed strategy surpasses previous state-of-the-art methods, as evidenced by experiments conducted on the AFLW2000 and BIWI datasets.

Model predictive complex system control from observational and interventional data

Complex systems, characterized by intricate interactions among numerous entities, give rise to emergent behaviors whose data-driven modeling and control are of utmost significance, especially when there is abundant observational data but the intervention cost is high. Traditional methods rely on precise dynamical models or require extensive intervention data, often falling short in real-world applications. To bridge this gap, we consider a specific setting of the complex systems control problem: how to control complex systems through a few online interactions on some intervenable nodes when abundant observational data from natural evolution is available. We introduce a two-stage model predictive complex system control framework, comprising an offline pre-training phase that leverages rich observational data to capture spontaneous evolutionary dynamics and an online fine-tuning phase that uses a variant of model predictive control to implement intervention actions. To address the high-dimensional nature of the state-action space in complex systems, we propose a novel approach employing action-extended graph neural networks to model the Markov decision process of complex systems and design a hierarchical action space for learning intervention actions. This approach performs well in three complex system control environments: Boids, Kuramoto, and Susceptible-Infectious-Susceptible (SIS) metapopulation. It offers accelerated convergence, robust generalization, and reduced intervention costs compared to the baseline algorithm. This work provides valuable insights into controlling complex systems with high-dimensional state-action spaces and limited intervention data, presenting promising applications for real-world challenges.

Online control parameter optimization design for multi-machine coordinated loading system of hazardous substances

To address the parameter instability issues in hazardous materials handling during multi-machine loading and unloading operations, we propose a Full-Scale Smart Parameter Optimization Control (FSPOC) system specifically designed for multi-machine coordination. This system leverages a novel fish scale prediction algorithm tailored for cooperative multi-machine environments. Initially, the fish scale prediction algorithm, inspired by bionic fish scales, is developed to predict future system behavior by analyzing historical data. Building on this algorithm, we introduce a disturbance cancellation control theorem and design a parameter optimization controller to enhance stability in high-dimensional nonlinear spaces. The FSPOC method is then applied to a multi-machine cooperative system, enabling online distributed parameter optimization for complex systems with multiple degrees of freedom. The effectiveness of the proposed method was validated through simulations, where it was compared with two other optimization techniques: Genetic Algorithm-based PID (GAPID) and Chaotic Atomic Search Algorithm-based PID (CHASO). The simulation results confirm the superiority of the FSPOC method.

Sensing with shallow recurrent decoder networks

Sensing is a universal task in science and engineering. Downstream tasks from sensing include inferring full-state estimates of a system (system identification), control decisions and forecasting. We propose a shallow recurrent decoder (SHRED) neural network structure for sensing, which incorporates (i) a recurrent neural network to learn a latent representation of the temporal dynamics of the sensors and (ii) a shallow decoder that learns a mapping between this latent representation and the high-dimensional state space. SHRED enables accurate reconstructions with far fewer sensors, outperforms existing techniques when more measurements are available and is more robust to random sensor placements. In the example cases explored, complex spatio-temporal dynamics are characterized by exceedingly limited sensors that can be randomly placed with minimal loss of performance.

HEnsem_DTIs: A heterogeneous ensemble learning model for drug-target interactions prediction

Drug discovery is the process by which a drug is discovered. Drug-target interactions prediction is a major part of drug discovery. Unfortunately, producing new drugs is time-consuming and expensive; Because it requires a lot of human and laboratory resources. Recently, predictions have been made using computational methods to solve these problems and prevent blindly examining all interactions. Various experiences using computational methods show that no single algorithm can be suitable for all applications; Hence, ensemble learning is expressed. Although various ensemble methods have been proposed, it is still not easy to find a suitable ensemble method for a particular dataset. In general, the existing algorithms in aggregation and combination method are selected manually based on experience. Reinforcement learning can be one way to meet this challenge. High-dimensional feature space and class imbalance are among the challenges of drug-target interactions prediction. This paper proposes HEnsem_DTIs, a heterogeneous ensemble model, for predicting drug-target interactions using dimensionality reduction and concepts of recommender systems to address these challenges. HEnsem_DTIs is configured with reinforcement learning. Dimensionality reduction is applied to handle the challenge of high-dimensional feature space and recommender systems to improve under-sampling and solve the class imbalance challenge. Six datasets are used to evaluate the proposed model; Results of the evaluation on datasets show that HEnsem_DTIs works better than other models in this field. Results of evaluation of the proposed model on the first dataset using 10-fold cross-validation experiments show the amount of sensitivity 0.896, specificity 0.954, GM 0.924, AUC 0.930 and AUPR 0.935.