High-dimensional Functions Research Articles

Fast and universal single-molecule localization using multi-dimensional point spread functions

The recent development of single-molecule imaging techniques has enabled not only high accuracy spatial resolution imaging but also information rich functional imaging. Abundant information about single molecules can be encoded in its diffraction pattern and be extracted precisely (e.g. 3D position, wavelength, dipole orientation). However, sophisticated high dimensional point spread function (PSF) modeling and analyzing methods have greatly impeded the broad accessibility of these techniques. Here, we present a graphics processing unit (GPU) -based B-spline PSF modeling method that could flexibly model high dimensional PSFs with arbitrary shape without greatly increasing the model parameters. Our B-spline fitter achieves 100 times speed improvement and minimal uncertainty for each dimension, enabling efficient high dimensional single-molecule analysis. We demonstrated, both in simulations and experiments, the universality and flexibility of our B-spline fitter to accurately extract the abundant information from different types of high dimensional single-molecule data, including multicolor PSF (3D + color), multi-channel four-dimensional 4Pi-PSF (3D + interference phase) and five-dimensional vortex PSF (3D + dipole orientation).

What can be learnt with wide convolutional neural networks?*

Abstract Understanding how convolutional neural networks (CNNs) can efficiently learn high-dimensional functions remains a fundamental challenge. A popular belief is that these models harness the local and hierarchical structure of natural data such as images. Yet, we lack a quantitative understanding of how such structure affects performance, for example the rate of decay of the generalisation error with the number of training samples. In this paper, we study infinitely wide deep CNNs in the kernel regime. First, we show that the spectrum of the corresponding kernel inherits the hierarchical structure of the network, and we characterise its asymptotics. Then, we use this result together with generalisation bounds to prove that deep CNNs adapt to the spatial scale of the target function. In particular, we find that if the target function depends on low-dimensional subsets of adjacent input variables then the decay of the error is controlled by the effective dimensionality of these subsets. Conversely, if the target function depends on the full set of input variables then the error decay is controlled by the input dimension. We conclude by computing the generalisation error of a deep CNN trained on the output of another deep CNN with randomly initialised parameters. Interestingly, we find that, despite their hierarchical structure, the functions generated by infinitely wide deep CNNs are too rich to be efficiently learnable in high dimensions.

Committor Guided Estimates of Molecular Transition Rates.

The probability that a configuration of a physical system reacts, or transitions from one metastable state to another, is quantified by the committor function. This function contains richly detailed mechanistic information about transition pathways, but a full parametrization of the committor requires the construction of a high-dimensional function, a generically challenging task. Recent efforts to leverage neural networks as a means to solve high-dimensional partial differential equations, often called "physics-informed" machine learning, have brought the committor into computational reach. Here, we build on the semigroup approach to learning the committor and assess its utility for predicting dynamical quantities such as transition rates. We show that a careful reframing of the objective function and improved adaptive sampling strategies provide highly accurate representations of the committor. Furthermore, by directly applying the Hill relation, we show that these committors provide accurate transition rates for molecular systems.

Multi-input Fourier neural network and its sparrow search optimization

In engineering applications, the back-propagation (BP) neural network often encounters many limitations due to its slow convergence and high noise sensitivity, and meanwhile the reported Fourier neural networks have no ability to extract the features of multi-attribute input data. Hereby, This work proposes a gradient descent-based multi-input Fourier neural network after integrating the multi-layer perceptron with an overlapping Fourier neural network. Thereafter, related to the difficulty of deciding the global optimal parameter settings, an improved sparrow search algorithm is developed to optimize the parameter settings and solve high dimensional function optimization problems, after the Cat chaotic map and the mechanisms of population-size adjustment and parameter adaptiveness are designed to promote the sparrow search algorithm's ability to balance global exploration and local exploitation. The theoretical analysis shows that the improved algorithm's computational complexity is decided by its population size and the optimization problem's dimension. Numerically comparative experiments have validated that not only the acquired Fourier neural network can effectively extract the features of multi-attribute data with strong generalization ability, but also the improved algorithm has significant advantages in coping with high dimensional function optimization problems.

Quasi-interpolation for high-dimensional function approximation

Quasi-interpolation for high-dimensional function approximation

An adaptive learning strategy for surrogate modeling of high-dimensional functions - Application to unsteady hypersonic flows in chemical nonequilibrium

Many engineering applications rely on the evaluation of expensive, non-linear high-dimensional functions. In this paper, we propose the RONAALP algorithm (Reduced Order Nonlinear Approximation with Adaptive Learning Procedure) to incrementally learn, on-the-fly as the application progresses, a fast and accurate reduced-order surrogate model of a target function. First, a combination of nonlinear auto-encoder, community clustering, and radial basis function networks allows us to learn an efficient and compact surrogate model with limited training data. Secondly, an active learning procedure overcomes any extrapolation issues during the online stage by adapting the surrogate model with high-fidelity evaluations that fall outside its current validity range. This approach results in generalizable, fast, and accurate reduced-order models of high-dimensional functions. The method is demonstrated on three direct numerical simulations of hypersonic flows in chemical nonequilibrium. Accurate simulations of these flows rely on detailed thermochemical gas models that dramatically increase the cost of such calculations. Using RONAALP to learn a reduced-order thermodynamic model surrogate on-the-fly, the cost of such simulations was reduced by up to 75% while maintaining an error of less than 10% on relevant quantities of interest.

On the Fueter–Sce theorem for generalized partial-slice monogenic functions

AbstractIn a recent paper, we introduced the concept of generalized partial-slice monogenic functions. The class of these functions includes both monogenic functions and slice monogenic functions with values in a Clifford algebra. In this paper, we establish a version of the Fueter–Sce theorem in this new setting, which allows to construct monogenic functions in higher dimensions starting from generalized partial-slice monogenic functions. We also prove that an alternative construction can be obtained by using the dual Radon transform. It turns out that these two constructions are closely related to the generalized CK-extension.

Cooperative coevolution for non-separable large-scale black-box optimization: Convergence analyses and distributed accelerations

Given the ubiquity of non-separable optimization problems in real worlds, in this paper we analyze and extend the large-scale version of the well-known cooperative coevolution (CC), a divide-and-conquer black-box optimization framework, on non-separable functions. First, we reveal empirical reasons of when decomposition-based methods are preferred or not in practice on some non-separable large-scale problems, which have not been clearly pointed out in many previous CC papers. Then, we formalize CC to a continuous-game model via simplification, but without losing its essential property. Different from previous evolutionary game theory for CC, our new model provides a much simpler but useful viewpoint to analyze its convergence, since only the pure Nash equilibrium concept is needed and more general fitness landscapes can be explicitly considered. Based on convergence analyses, we propose a hierarchical decomposition strategy for better generalization, as for any decomposition, there is a risk of getting trapped into a suboptimal Nash equilibrium. Finally, we use powerful distributed computing to accelerate it under the recent multi-level learning framework, which combines the fine-tuning ability from decomposition with the invariance property of CMA-ES. Experiments on a set of high-dimensional test functions validate both its search performance and scalability (w.r.t. CPU cores) on a clustering computing platform with 400 CPU cores.

Flows of geometric structures

Abstract We develop an abstract theory of flows of geometric 𝐻-structures, i.e., flows of tensor fields defining 𝐻-reductions of the frame bundle, for a closed and connected subgroup H ⊂ SO ⁢ ( n ) H\subset\mathrm{SO}(n) , on any connected and oriented 𝑛-manifold with sufficient topology to admit such structures. The first part of the article sets up a unifying theoretical framework for deformations of 𝐻-structures, by way of the natural infinitesimal action of GL ⁢ ( n , R ) \mathrm{GL}(n,\mathbb{R}) on tensors combined with various bundle decompositions induced by 𝐻-structures. We compute evolution equations for the intrinsic torsion under general flows of 𝐻-structures and, as applications, we obtain general Bianchi-type identities for 𝐻-structures, and, for closed manifolds, a general first variation formula for the L 2 L^{2} -Dirichlet energy functional ℰ on the space of 𝐻-structures. We then specialise the theory to the negative gradient flow of ℰ over isometric 𝐻-structures, i.e., their harmonic flow. The core result is an almost-monotonocity formula along the flow for a scale-invariant localised energy, similar to the classical formulas by Chen–Struwe [M. Struwe, On the evolution of harmonic maps in higher dimensions, J. Differential Geom. 28 (1988), 3, 485–502; Y. M. Chen and M. Struwe, Existence and partial regularity results for the heat flow for harmonic maps, Math. Z. 201 (1989), 1, 83–103] for the harmonic map heat flow. This yields an 𝜀-regularity theorem and an energy gap result for harmonic structures, as well as long-time existence for the flow under small initial energy, relative to the L ∞ L^{\infty} -norm of initial torsion, in the spirit of Chen–Ding [Y. M. Chen and W. Y. Ding, Blow-up and global existence for heat flows of harmonic maps, Invent. Math. 99 (1990), 3, 567–578]. Moreover, below a certain energy level, the absence of a torsion-free isometric 𝐻-structure in the initial homotopy class imposes the formation of finite-time singularities. These seemingly contrasting statements are illustrated by examples on flat 𝑛-tori, so long as the set [ S n , SO ⁢ ( n ) / H ] [\mathbb{S}^{n},\mathrm{SO}(n)/H] of homotopy classes of maps S n → SO ⁢ ( n ) / H \mathbb{S}^{n}\to\mathrm{SO}(n)/H contains more than one element and the universal cover of SO ⁢ ( n ) / H \mathrm{SO}(n)/H is a sphere, e.g. when n = 7 n=7 and H = G 2 H=\mathrm{G}_{2} , or n = 8 n=8 and H = Spin ⁢ ( 7 ) H=\mathrm{Spin}(7) .

Near-optimal learning of Banach-valued, high-dimensional functions via deep neural networks

The past decade has seen increasing interest in applying Deep Learning (DL) to Computational Science and Engineering (CSE). Driven by impressive results in applications such as computer vision, Uncertainty Quantification (UQ), genetics, simulations and image processing, DL is increasingly supplanting classical algorithms, and seems poised to revolutionize scientific computing. However, DL is not yet well-understood from the standpoint of numerical analysis. Little is known about the efficiency and reliability of DL from the perspectives of stability, robustness, accuracy, and, crucially, sample complexity. For example, approximating solutions to parametric PDEs is a key task in UQ for CSE. Yet, training data for such problems is often scarce and corrupted by errors. Moreover, the target function, while often smooth, is a potentially infinite-dimensional function taking values in the PDE solution space, which is generally an infinite-dimensional Banach space. This paper provides arguments for Deep Neural Network (DNN) approximation of such functions, with both known and unknown parametric dependence, that overcome the curse of dimensionality. We establish practical existence theorems that describe classes of DNNs with dimension-independent architecture widths and depths, and training procedures based on minimizing a (regularized) ℓ2-loss which achieve near-optimal algebraic rates of convergence in terms of the amount of training data m. These results involve key extensions of compressed sensing for recovering Banach-valued vectors and polynomial emulation with DNNs. When approximating solutions of parametric PDEs, our results account for all sources of error, i.e., sampling, optimization, approximation and physical discretization, and allow for training high-fidelity DNN approximations from coarse-grained sample data. Our theoretical results fall into the category of non-intrusive methods, providing a theoretical alternative to classical methods for high-dimensional approximation.

Early detection of abiotic stress in plants through SNARE proteins using hybrid feature fusion model.

Agriculture is the main source of livelihood for most of the population across the globe. Plants are often considered life savers for humanity, having evolved complex adaptations to cope with adverse environmental conditions. Protecting agricultural produce from devastating conditions such as stress is essential for the sustainable development of the nation. Plants respond to various environmental stressors such as drought, salinity, heat, cold, etc. Abiotic stress can significantly impact crop yield and development posing a major threat to agriculture. SNARE proteins play a major role in pathological processes as they are vital proteins in the life sciences. These proteins act as key players in stress responses. Feature extraction is essential for visualizing the underlying structure of the SNARE proteins in analyzing the root cause of abiotic stress in plants. To address this issue, we developed a hybrid model to capture the hidden structures of the SNAREs. A feature fusion technique has been devised by combining the potential strengths of convolutional neural networks (CNN) with a high dimensional radial basis function (RBF) network. Additionally, we employ a bi-directional long short-term memory (Bi-LSTM) network to classify the presence of SNARE proteins. Our feature fusion model successfully identified abiotic stress in plants with an accuracy of 74.6%. When compared with various existing frameworks, our model demonstrates superior classification results.

The GroupMax Neural Network Approximation of Convex Functions.

We present a new neural network to approximate convex functions. This network has the particularity to approximate the function with cuts which is, for example, a necessary feature to approximate Bellman values when solving linear stochastic optimization problems. The network can be easily adapted to partial convexity. We give an universal approximation theorem in the full convex case and give many numerical results proving its efficiency. The network is competitive with the most efficient convexity-preserving neural networks and can be used to approximate functions in high dimensions.

Evolutionary Constrained Multiobjective Optimization: Scalable High-Dimensional Constraint Benchmarks and Algorithm

Evolutionary constrained multiobjective optimization has received extensive attention and research in the past two decades, and a lot of benchmarks have been proposed to test the effect of the constrained multiobjective evolutionary algorithms (CMOEAs). Specially, the constraint functions are highly correlated with the objective values, which makes the features of constraints too monotonic and differ from the properties of the real-world problems. Accordingly, previous CMOEAs cannot solve real-world problems well, which generally involve decision space constraints with multi-modal/non-linear features. Therefore, we propose a new benchmark framework and design a suite of new test functions with scalable high-dimensional decision space constraints. To be specific, different high-dimensional constraint functions and mixed linkages in variables are considered to be close to realistic features. In this framework, several parameter interfaces are provided, so that users can easily adjust the parameters to obtain the variant functions and test the generalization performance of the algorithms. Different types of existing CMOEAs are employed to test the use of the proposed test functions, and the results show that they are easy to fall into local feasible regions. Therefore, we improve one evolutionary multitasking-based CMOEA to better handle these problems, in which a new search algorithm is designed to enhance the search abilities of populations. Compared with the existing CMOEAs, the proposed CMOEA presents better performance.

Mini-Workshop: Nonlinear Approximation of High-dimensional Functions in Scientific Computing

Approximation techniques for high dimensional PDEs are crucial for contemporary scientific computing tasks and gained momentum in recent years due to the renewed interest in neural networks. It seems that especially nonlinear parametrizations will play an essential role in efficient and tractable approximations of high dimensional problems. We held a mini-workshop on the relation and possible synergy of neural networks and tensor product approximation. To reliably evaluate the prospect of different numerical experiments, the traditional talks were accompanied by live coding sessions.

A multi-strategy surrogate-assisted social learning particle swarm optimization for expensive optimization and applications

Evolutionary algorithms (EAs) require extensive fitness evaluations, which constitutes a barrier to solving computationally complex problems. In contrast, surrogate-assisted evolutionary algorithms (SAEAs) have the potential to solve complex expensive optimization problems. This paper proposes a surrogate-assisted social learning particle swarm optimization (SASLPSO) to handle expensive optimization problems. An adaptive local surrogate (ALS) strategy introduced in SASLPSO is introduced to accurately fit the landscape near the global optimum. ALS consists of two layers of surrogate, the part of the particles closest to the optimal particle and the closest to the optimal particle among the particles that have been eliminated historically. The proposed SASLPSO effectively combines global surrogate (GS) and adaptive local surrogate (ALS) to balance global exploration and local exploitation. Furthermore, a novel random group-based pre-screening (RGBPS) strategy is proposed to screen promising particles for real function evaluation. The proposed SASLPSO is compared with four other state-of-the-art SAEAs on 30D, 50D, and 100D benchmark functions. In addition, the significance of the SASLPSO algorithm was also verified using the Wilcoxon rank test. The test results on the benchmark function show that the SASLPSO algorithm performs better than other comparison algorithms, especially when dealing with high-dimensional benchmark functions. To further validate the effectiveness of the SASLPSO algorithm in solving expensive optimization problems, it was also applied to feature selection problems and real-world engineering problems. In addition, in the real application of node deployment in 3D wireless sensor networks, the highest coverage rate of the SASLPSO algorithm can reach 99.96%, confirming its performance advantages in solving real application problems. Finally, in the application of network intrusion detection, SASLPSO has shown more advantages in multiple metrics, proving its versatility.

Quasi-random Fractal Search (QRFS): A dynamic metaheuristic with sigmoid population decrement for global optimization

Global optimization of complex and high-dimensional functions remains a central challenge with broad applications in science and engineering. This study introduces a new optimization approach called quasi-random metaheuristic based on fractal search (QRFS), which harnesses the power of fractal geometry, low discrepancy sequences, and intelligent search space partitioning techniques. The QRFS uses fractals’ inherent self-similarity and intricate structure to guide the solution space exploration. For the proposal, a deterministic but quasi-random element is used in the search process using low discrepancy sequences, such as Sobol, Halton, Hammersley, and Latin Hypercube. This integration allows the algorithm to systematically cover the search space while maintaining the level of diversity necessary for efficient exploration. The QRFS employs a dynamic strategy of partitioning the search space and reducing the population of solutions to optimize the use of function accesses, which causes it to adapt well to the characteristics of the problem. The algorithm intelligently identifies and prioritizes promising regions within the fractal-based representation, allocating computational resources where they are most likely to yield optimal solutions. Experimental evaluations on several benchmark problems demonstrate that QRFS consistently outperforms modern, canonical metaheuristics and variants of algorithms such as differential evolution (DE), particle swarm optimization (PSO), covariance matrix adaptive evolution strategy (CMA-ES), regarding solution quality. Besides, the algorithm shows remarkable scalability, which makes it suitable for high-dimensional optimization tasks. Overall, QRFS offers a robust and efficient approach to solving complex optimization problems in various domains, paving the way for improved decision-making in real-world applications.

Mapper–Type Algorithms for Complex Data and Relations

Mapper and Ball Mapper are Topological Data Analysis tools used for exploring high dimensional point clouds and visualizing scalar–valued functions on those point clouds. Inspired by open questions in knot theory, new features are added to Ball Mapper that enable encoding of the structure, internal relations and symmetries of the point cloud. Moreover, the strengths of Mapper and Ball Mapper constructions are combined to create a tool for comparing high dimensional data descriptors of a single dataset. This new hybrid algorithm, Mapper on Ball Mapper, is applicable to high dimensional lens functions. As a proof of concept we include applications to knot and game theory.

Variable transformations in combination with wavelets and ANOVA for high-dimensional approximation

We use hyperbolic wavelet regression for the fast reconstruction of high-dimensional functions having only low-dimensional variable interactions. Compactly supported periodic Chui-Wang wavelets are used for the tensorized hyperbolic wavelet basis on the torus. With a variable transformation, we are able to transform the approximation rates and fast algorithms from the torus to other domains. We perform and analyze scattered data approximation for smooth but arbitrary density functions by using a least squares method. The corresponding system matrix is sparse due to the compact support of the wavelets, which leads to a significant acceleration of the matrix vector multiplication. For non-periodic functions, we propose a new extension method. A proper choice of the extension parameter together with the piecewise polynomial Chui-Wang wavelets extends the functions appropriately. In every case, we are able to bound the approximation error with high probability. Additionally, if the function has a low effective dimension (i.e., only interactions of a few variables), we qualitatively determine the variable interactions and omit ANOVA terms with low variance in a second step in order to decrease the approximation error. This allows us to suggest an adapted model for the approximation. Numerical results show the efficiency of the proposed method.

Solving high-dimensional Fokker-Planck equation with functional hierarchical tensor

This work is concerned with solving high-dimensional Fokker-Planck equations with the novel perspective that solving the PDE can be reduced to independent instances of density estimation tasks based on the trajectories sampled from its associated particle dynamics. With this approach, one sidesteps error accumulation occurring from integrating the PDE dynamics on a parameterized function class. This approach significantly simplifies deployment, as one is free of the challenges of implementing loss terms based on the differential equation. In particular, we introduce a novel class of high-dimensional functions called the functional hierarchical tensor (FHT). The FHT ansatz leverages a hierarchical low-rank structure, offering the advantage of linearly scalable runtime and memory complexity relative to the dimension count. We introduce a sketching-based technique that performs density estimation over particles simulated from the particle dynamics associated with the equation, thereby obtaining a representation of the Fokker-Planck solution in terms of our ansatz. We apply the proposed approach successfully to three challenging time-dependent Ginzburg-Landau models with hundreds of variables.

Improved Driving Training-Based Optimization Algorithm Using Levy Flight and Crowding Distance Techniques

This study presents an improved version of the Driving Training-Based Optimization (DTBO) algorithm, the Improved Driving Training-Based Optimization (IDTBO). The work addresses fundamental issues in selecting drivers and learners for the conventional DTBO, which substantially impact the algorithm's accuracy and convergence speed. Two significant improvements are proposed: including the crowding distance technique for more diverse driver and learner selection and incorporating the Levy Flight distribution for better exploration and local optima avoidance. The IDTBO's performance is evaluated using twelve benchmark functions, including unimodal and high-dimensional multimodal optimization functions. The results indicate that IDTBO performs exceptionally well, with more extraordinary exploitation ability on unimodal functions and consistent achievement of the global optima. The proposed IDTBO demonstrated exceptional exploration capabilities on high-dimensional multimodal functions and performed competitively with other algorithms in the literature. From six functions, the IDTBO obtained zero optimal values. Again, the rate of convergence analysis demonstrates that IDTBO finds optimal solutions in fewer iterations, demonstrating its capacity to balance exploration and exploitation. To assess the strength of the IDTBO in solving real-world problems, the improved DTBO is further tested on two practical benchmark engineering problems. The IDTBO again produced a competitive performance against other algorithms in the literature. The study shows that IDTBO is a valuable metaheuristic algorithm that can tackle various real-world optimization problems.