High-dimensional Functions Research Articles

A VR-based volumetric medical image segmentation and visualization system with natural human interaction

Volume rendering produces informative two-dimensional (2D) images from a 3-dimensional (3D) volume. It highlights the region of interest and facilitates a good comprehension of the entire data set. However, volume rendering faces a few challenges. First, a high-dimensional transfer function is usually required to differentiate the target from its neighboring objects with subtle variance. Unfortunately, designing such a transfer function is a strenuously trial-and-error process. Second, manipulating/visualizing a 3D volume with a traditional 2D input/output device suffers dimensional limitations. To address all the challenges, we design NUI-VR^2, a natural user interface-enabled volume rendering system in the virtual reality space. NUI-VR^2 marries volume rendering and interactive image segmentation. It transforms the original volume into a probability map with image segmentation. A simple linear transfer function will highlight the target well in the probability map. More importantly, we set the entire image segmentation and volume rendering pipeline in an immersive virtual reality environment with a natural user interface. NUI-VR^2 eliminates the dimensional limitations in manipulating and perceiving 3D volumes and dramatically improves the user experience.

AK-SESC: a novel reliability procedure based on the integration of active learning kriging and sequential space conversion method

To deal with evaluating small failure probabilities, AK–SESC: a novel approach integrating an active learning Kriging meta-model (AK-MCS) and the SESC, a sequential space conversion method, is suggested. The efficiency of the proposed approach relies on the advantages of the AK-MCS and its updating feature to evaluate the actual performance function and the superiority of SESC in estimating small failure probabilities. Although there are effective methods for small probabilities, the beauty of this approach is that it is derived from the probability integral with no simplifications while providing results of high accuracy.Different problems were solved to study the AK–SESC applicability. The main effort of this method is reducing the function call numbers of the original SESC while reaching the same accuracy as Monte Carlo Simulation (MCS). The reliability analysis results were compared with the main reliability methods of the Importance Sampling (IS), Subset Simulation (SubSim), Line Sampling (LS), First and also Second-Order Reliability Method (FORM and SORM). The solved problems indicate that the proposed approach provides accurate answers with much fewer function calls than SESC. So, it can be a promising method for reliability analyses involving nonlinear or high-dimensional performance functions with small failure probabilities.

Multi-population improved whale optimization algorithm for high dimensional optimization

The metaheuristic algorithms do not depend on the functional form when solving the optimization problem. They have strong adaptability and are widely used in many fields. Whale Optimization Algorithm (WOA) is a metaheuristic algorithm based on the social behavior of humpback whales. Compared with other metaheuristic algorithms, WOA shows better performance. However, when solving high dimensional optimization problems, WOA tends to fall into local optima and has slow convergence speed and low accuracy of solution. Aiming at these problems, a multi-population improved WOA (MIWOA) is proposed to improve the performance of WOA when tackling high dimensional optimization. First of all, the multi-population exploitation and exploration processes are introduced. The population is divided into a better group and a worse group. The better individuals are used to improve exploitation performance, while the poorer individuals are used to improve exploration performance. Secondly, the current optimal individual and weighted center are taken to improve the learning process, which enhances the exploration ability and convergence speed. Moreover an interpolation method is introduced to enhance the search ability in the vicinity of the current optimum and further improve the exploitation performance. Finally, a control parameter is used to balance the exploitation and exploration processes. In the experimental part, MIWOA is compared with several state-of-the-art algorithms on 30 high dimensional benchmark functions with dimensions ranging from 100 to 2000. Simulation results show that MIWOA has good performance in both high dimensional single-mode and multi-mode optimization problems. MIWOA is superior to other algorithms in solution accuracy, convergence speed, and execution time.

Solar-Based DG Allocation Using Harris Hawks Optimization While Considering Practical Aspects

The restructuring of power systems and the ever-increasing demand for electricity have given rise to congestion in power networks. The use of distributed generators (DGs) may play a significant role in tackling such issues. DGs may be integrated with electrical power networks to regulate the drift of power in the transmission lines, thereby increasing the power transfer capabilities of lines and improving the overall performance of electrical networks. In this article, an effective method based on the Harris hawks optimization (HHO) algorithm is used to select the optimum capacity, number, and site of solar-based DGs to reduce real power losses and voltage deviation. The proposed HHO has been tested with a complex benchmark function then applied to the IEEE 33 and IEEE 69 bus radial distribution systems. The single and multiple solar-based DGs are optimized for the optimum size and site with a unity power factor. It is observed that the overall performance of the systems is enhanced when additional DGs are installed. Moreover, considering the stochastic and sporadic nature of solar irradiance, the practical size of DG has been suggested based on analysis that may be adopted while designing the actual photovoltaic (PV) plant for usage. The obtained simulation outcomes are compared with the latest state-of-the-art literature and suggest that the proposed HHO is capable of processing complex high dimensional benchmark functions and has capability to handle problems pertaining to electrical distribution in an effective manner.

A parametrized level set based topology optimization method for analysing thermal problems

This paper focuses on the utilization of local radial basis functions (LRBFs) based level set method (LSM) for topology optimization of two-dimensional thermal problems using both concentrated as well as uniformly distributed heat generation. The design domain is embedded implicitly into a higher-dimensional function, which is parametrized with the LRBFs through an explicit scheme. This novel combination of LRBFs and LSM has the capability of controlling the topological variations automatically, i.e., hole insertion, merging with each other and with the boundary. The governing equations of heat conduction system are solved with the finite element method to obtain the sensitivities at level set grid points as a velocity field for evolution of the structural geometry. The objective function is set to the heat transfer potential with the maximum material volume as the design constraint. Several experiments are conducted on benchmark test problems and the resulting optimal solutions reveals efficiency, convergence and good agreement with those reported in the literature.

An extended autoencoder model for reaction coordinate discovery in rare event molecular dynamics datasets.

The reaction coordinate (RC) is the principal collective variable or feature that determines the progress along an activated or reactive process. In a molecular simulation using enhanced sampling, a good description of the RC is crucial for generating sufficient statistics. Moreover, the RC provides invaluable atomistic insight into the process under study. The optimal RC is the committor, which represents the likelihood of a system to evolve toward a given state based on the coordinates of all its particles. As the interpretability of such a high dimensional function is low, a more practical approach is to describe the RC by some low-dimensional molecular collective variables or order parameters. While several methods can perform this dimensionality reduction, they usually require a preselection of these low-dimension collective variables (CVs). Here, we propose to automate this dimensionality reduction using an extended autoencoder, which maps the input (many CVs) onto a lower-dimensional latent space, which is subsequently used for the reconstruction of the input as well as the prediction of the committor function. As a consequence, the latent space is optimized for both reconstruction and committor prediction and is likely to yield the best non-linear low-dimensional representation of the committor. We test our extended autoencoder model on simple but nontrivial toy systems, as well as extensive molecular simulation data of methane hydrate nucleation. The extended autoencoder model can effectively extract the underlying mechanism of a reaction, make reliable predictions about the committor of a given configuration, and potentially even generate new paths representative for a reaction.

Correlation reliability assessment of artillery chassis transmission system based on CBN model

The traditional Bayesian Network (BN) has defects in the reliability assessment problem of multivariate correlation. In this paper, the Copula Bayesian Network (CBN) model is introduced into the artillery chassis transmission system for the first time to solve these problems. The model uses the Copula function to express the relationship between variables, and combines Bayesian Network probabilistic reasoning. On the basis of the traditional BN topology structure, the Copula function is introduced and transformed into CBN topology structure. The edge distribution of each node and the correlation coefficient matrix are used to construct the local connection function. It can realize the conversion of multi-variables into binary variables and construct high-dimensional functions. Bayesian network reasoning can get the failure probability of each node. Finally, the CBN model was used to assess the reliability of a certain type of artillery chassis transmission system for the first time. This method converts the multiple variables of the transmission system into a binary relationship, which is suitable for engineering applications and can express the correlation in the system. The computational result of this model is better than that of the traditional BN model.

On the admissibility of spherical spatial covariance functions in higher dimensions

A mean zero spatial process X(t) on the n-dimensional Euclidean space Rn is isotropic if its covariance function (c.f.) is of the form: Rn(τ)=Cov[X(t),X(s)], where τ=|t−s|, t, s∈Rn, and Rn is an admissible function on R1. An isotropic spatial process X has a bounded range of dependence if sup{τ:Rn(τ)≠0}<∞. Here we consider a class of isotropic c.f.’s Rn(τ), n=1,2,… with bounded ranges of dependence, among which there are R1, the classical triangular c.f. on the real line (n=1), and R3, the spherical c.f. in dimension three (n=3). For each dimension n≥1, the admissibility of Rn(τ) as a c.f. in higher dimensions is studied. While it is well known that for each n≥1, Rn is a legitimate c.f. on Rm for all m≤n but it is shown that the considered Rn is not a legitimate c.f. on Rm when m>n. Thus the spherical c.f. R3 cannot be a c.f. on Rn when n>3. The issue of recognition of an isotropic c.f. on Rn is discussed, and simple procedures of constructing isotropic c.f.’s on Rn for every n≥1 are given. This article serves as one more reminder that caution must be taken concerning the legitimacy of a selected c.f. in the corresponding spatial dimensions.

On the Expressibility and Overfitting of Quantum Circuit Learning

Applying quantum processors to model a high-dimensional function approximator is a typical method in quantum machine learning with potential advantage. It is conjectured that the unitarity of quantum circuits provides possible regularization to avoid overfitting. However, it is not clear how the regularization interplays with the expressibility under the limitation of current Noisy-Intermediate Scale Quantum devices. In this article, we perform simulations and theoretical analysis of the quantum circuit learning problem with hardware-efficient ansatz. Thorough numerical simulations show that the expressibility and generalization error scaling of the ansatz saturate when the circuit depth increases, implying the automatic regularization to avoid the overfitting issue in the quantum circuit learning scenario. This observation is supported by the theory on PAC learnability, which proves that VC dimension is upper bounded due to the locality and unitarity of the hardware-efficient ansatz. Our study provides supporting evidence for automatic regularization by unitarity to suppress overfitting and guidelines for possible performance improvement under hardware constraints.

The loss surfaces of neural networks with general activation functions

The loss surfaces of deep neural networks have been the subject of several studies, theoretical and experimental, over the last few years. One strand of work considers the complexity, in the sense of local optima, of high dimensional random functions with the aim of informing how local optimisation methods may perform in such complicated settings. Prior work of Choromanska et al (2015) established a direct link between the training loss surfaces of deep multi-layer perceptron networks and spherical multi-spin glass models under some very strong assumptions on the network and its data. In this work, we test the validity of this approach by removing the undesirable restriction to ReLU activation functions. In doing so, we chart a new path through the spin glass complexity calculations using supersymmetric methods in random matrix theory which may prove useful in other contexts. Our results shed new light on both the strengths and the weaknesses of spin glass models in this context.

Quantum Circuit Learning with Error Backpropagation Algorithm and Experimental Implementation

Quantum computing has the potential to outperform classical computers and is expected to play an active role in various fields. In quantum machine learning, a quantum computer has been found useful for enhanced feature representation and high-dimensional state or function approximation. Quantum–classical hybrid algorithms have been proposed in recent years for this purpose under the noisy intermediate-scale quantum computer (NISQ) environment. Under this scheme, the role played by the classical computer is the parameter tuning, parameter optimization, and parameter update for the quantum circuit. In this paper, we propose a gradient descent-based backpropagation algorithm that can efficiently calculate the gradient in parameter optimization and update the parameter for quantum circuit learning, which outperforms the current parameter search algorithms in terms of computing speed while presenting the same or even higher test accuracy. Meanwhile, the proposed theoretical scheme was successfully implemented on the 20-qubit quantum computer of IBM Q, ibmq_johannesburg. The experimental results reveal that the gate error, especially the CNOT gate error, strongly affects the derived gradient accuracy. The regression accuracy performed on the IBM Q becomes lower with the increase in the number of measurement shot times due to the accumulated gate noise error.

Evolutionary computing and machine learning for discovering of low-energy defect configurations

Density functional theory (DFT) has become a standard tool for the study of point defects in materials. However, finding the most stable defective structures remains a very challenging task as it involves the solution of a multimodal optimization problem with a high-dimensional objective function. Hitherto, the approaches most commonly used to tackle this problem have been mostly empirical, heuristic, and/or based on domain knowledge. In this contribution, we describe an approach for exploring the potential energy surface (PES) based on the covariance matrix adaptation evolution strategy (CMA-ES) and supervised and unsupervised machine learning models. The resulting algorithm depends only on a limited set of physically interpretable hyperparameters and the approach offers a systematic way for finding low-energy configurations of isolated point defects in solids. We demonstrate its applicability on different systems and show its ability to find known low-energy structures and discover additional ones as well.

High-Dimensional Bayesian Optimization via Tree-Structured Additive Models

Bayesian Optimization (BO) has shown significant success in tackling expensive low-dimensional black-box optimization problems. Many optimization problems of interest are high-dimensional, and scaling BO to such settings remains an important challenge. In this paper, we consider generalized additive models in which low-dimensional functions with overlapping subsets of variables are composed to model a high-dimensional target function. Our goal is to lower the computational resources required and facilitate faster model learning by reducing the model complexity while retaining the sample-efficiency of existing methods. Specifically, we constrain the underlying dependency graphs to tree structures in order to facilitate both the structure learning and optimization of the acquisition function. For the former, we propose a hybrid graph learning algorithm based on Gibbs sampling and mutation. In addition, we propose a novel zooming-based algorithm that permits generalized additive models to be employed more efficiently in the case of continuous domains. We demonstrate and discuss the efficacy of our approach via a range of experiments on synthetic functions and real-world datasets.

Deep Frequency Principle Towards Understanding Why Deeper Learning Is Faster

Understanding the effect of depth in deep learning is a critical problem. In this work, we utilize the Fourier analysis to empirically provide a promising mechanism to understand why feedforward deeper learning is faster. To this end, we separate a deep neural network, trained by normal stochastic gradient descent, into two parts during analysis, i.e., a pre-condition component and a learning component, in which the output of the pre-condition one is the input of the learning one. We use a filtering method to characterize the frequency distribution of a high-dimensional function. Based on experiments of deep networks and real dataset, we propose a deep frequency principle, that is, the effective target function for a deeper hidden layer biases towards lower frequency during the training. Therefore, the learning component effectively learns a lower frequency function if the pre-condition component has more layers. Due to the well-studied frequency principle, i.e., deep neural networks learn lower frequency functions faster, the deep frequency principle provides a reasonable explanation to why deeper learning is faster. We believe these empirical studies would be valuable for future theoretical studies of the effect of depth in deep learning.

Strength uncertainty analysis of composite turbine blade with small sample size

SiC/SiC ceramic matrix composites (CMCs) exhibit significant randomness of damage processes under load. In this study, a simplified simulation method was established based on the macroscopic damage constitutive model, characteristics of fewer test samples, and complex data sources for determining the randomness of the material behavior. Moreover, the established model was verified by introducing an uncertainty parameter to the constitutive model. By considering a typical high-dimensional nonlinear function as a numerical example, the reliability and error of the characterization methods were compared with specimens and studied in terms of the extreme-value interval, tolerance interval, evidence theory, and fuzzy set method with the distribution characteristics of single and mixed parameters. In conjunction with the sensitivity analysis, the randomness of the mechanical behavior was more evident at the damage stage, and the uncertainty of material behavior relied on the loading state. In addition, the correction method of the constitutive parameters and formal errors of the model were established using the Bayesian theorem to acquire additional data. Thereafter, the proposed method was employed for quantifying the uncertainty in the strength of the CMC turbine rotor blade. The determinacy results revealed that the failure risk could be more effectively evaluated by the strength analysis considering the uncertainty of the model, thus providing guidance toward structural design improvement aligned with engineering practice.

Hierarchical Meta-Storms enables comprehensive and rapid comparison of microbiome functional profiles on a large scale using hierarchical dissimilarity metrics and parallel computing.

It is coded in C/C++ with parallel computing and released in two alternative forms: a standalone software (https://github.com/qdu-bioinfo/hierarchical-meta-storms) and an equivalent R package (https://github.com/qdu-bioinfo/hrms). Supplementary data are available at Bioinformatics Advances online.

Quantum differential evolution with cooperative coevolution framework and hybrid mutation strategy for large scale optimization

In order to overcome the low solution efficiency, insufficient diversity in the later search stage, slow convergence speed and a high search stagnation possibility of differential evolution(DE) algorithm, the quantum computing characteristics of quantum evolutionary algorithm(QEA) and the divide-and-conquer idea of cooperative coevolution evolutionary algorithm(CCEA) are combined to propose an improved differential evolution(HMCFQDE) in this paper. In the proposed HMCFQDE, a new hybrid mutation strategy based on the advantages of local neighborhood mutation and SaNSDE is designed. In the early stage of the search, the local neighborhood mutation strategy with high search efficiency is used to speed up the algorithm convergence. In the later stage of the search, the SaNSDE algorithm is used to adjust the search direction in order to avoid the search stagnation. The QEA is combined with the DE to make use of the quantum chromosome encoding to enhance the population diversity, the quantum rotation to speed up the convergence speed. The CC framework is used to divide the large-scale and high-dimensional complex optimization problem into several low-dimensional optimization sub-problems, and these sub-populations are solved by independent searching among sub-populations in order to improve the solution efficiency. By comparing with other 6 algorithms in solving 6 test functions from CEC’08 under the dimensions of 100, 500 and 1000, it is proved that the proposed HMCFQDE has higher convergence accuracy and stronger stability. In particular, it has a strong ability to optimize high-dimensional complex functions. Therefore, it provides a new method for solving large-scale optimization problem.

Spectral methods for nonlinear functionals and functional differential equations

We present a rigorous convergence analysis for cylindrical approximations of nonlinear functionals, functional derivatives, and functional differential equations (FDEs). The purpose of this analysis is twofold: First, we prove that continuous nonlinear functionals, functional derivatives, and FDEs can be approximated uniformly on any compact subset of a real Banach space admitting a basis by high-dimensional multivariate functions and high-dimensional partial differential equations (PDEs), respectively. Second, we show that the convergence rate of such functional approximations can be exponential, depending on the regularity of the functional (in particular its Fréchet differentiability), and its domain. We also provide necessary and sufficient conditions for consistency, stability and convergence of cylindrical approximations to linear FDEs. These results open the possibility to utilize numerical techniques for high-dimensional systems such as deep neural networks and numerical tensor methods to approximate nonlinear functionals in terms of high-dimensional functions, and compute approximate solutions to FDEs by solving high-dimensional PDEs. Numerical examples are presented and discussed for prototype nonlinear functionals and for an initial value problem involving a linear FDE.

Hybrid whale optimization algorithm with gathering strategies for high-dimensional problems

In order to solve the problems, such as insufficient search ability and low search efficiency, of Whale Optimization Algorithm (WOA) in solving high-dimensional problems, a novel Hybrid WOA with Gathering strategies (HWOAG) is proposed in this paper. Firstly, an individual-based updating way is used in HWOAG instead of the dimension-based updating one of WOA to reduce the computational complexity and to be more suitable for high-dimensional problems. Secondly, a random opposition learning strategy is embedded into the individual-based WOA to form an opposition learning WOA (OWOA), and Grey Wolf Optimizer (GWO) is integrated into OWOA to form an OWOA with GWO (OWOAG) so as to improve the global search ability of WOA. Finally, two standalone OWOAGs are formulated to balance exploration and exploitation better. The two OWOAGs adopt strategies such as switching parameter tuning, random differential disturbance and global-best spiral operator to get stronger search ability. A lot of experimental results on high-dimensional (i.e. 1000-, 2000-, 4000- and 8000- dimensional) benchmark functions and clustering datasets for Fuzzy C-Means (FCM) optimization show that HWOAG has stronger search ability and higher search efficiency than WOA and quite a few state-of-the-art algorithms and that all the strategies gathered to WOA are effective. The source codes of the proposed algorithm HWOAG are available at https://github.com/kangzhai/HWOAG.

Human-Guided Robot Behavior Learning: A GAN-Assisted Preference-Based Reinforcement Learning Approach

Human demonstrations can provide trustful samples to train reinforcement learning algorithms for robots to learn complex behaviors in real-world environments. However, obtaining sufficient demonstrations may be impractical because many behaviors are difficult for humans to demonstrate. A more practical approach is to replace human demonstrations by human queries, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.,</i> preference-based reinforcement learning. One key limitation of the existing algorithms is the need for a significant amount of human queries because a large number of labeled data is needed to train neural networks for the approximation of a continuous, high-dimensional reward function. To reduce and minimize the need for human queries, we propose a new GAN-assisted human preference-based reinforcement learning approach that uses a generative adversarial network (GAN) to learn human preferences and then replace the role of human in assigning preferences. The adversarial neural network is simple and only has a binary output, hence requiring much less human queries to train. Moreover, a maximum entropy based reinforcement learning algorithm is designed to shape the loss towards the desired regions or away from the undesired regions. To show the effectiveness of the proposed approach, we present some studies on complex robotic tasks without access to the environment reward in a typical MuJoCo robot locomotion environment. The obtained results show our method can achieve a reduction of about 99.8% human time without performance sacrifice.