Toy Problem Research Articles

On non-locally elastic Rayleigh wave.

The Rayleigh-type wave solution within a widely used differential formulation in non-local elasticity is revisited. It is demonstrated that this wave solution does not satisfy the equations of motion for non-local stresses. A modified differential model taking into account a non-local boundary layer is developed. Correspondence of the latter model to the original integral theory with the kernel in the form of the zero-order modified Bessel function of the second kind is addressed. Asymptotic behaviour of the model is investigated, resulting in a leading-order non-local correction to the classical Rayleigh wave speed due to the effect of the boundary layer. The suitability of a continuous set-up for modelling boundary layers in the framework of non-local elasticity is analysed starting from a toy problem for a semi-infinite chain. This article is part of the theme issue 'Wave generation and transmission in multi-scale complex media and structured metamaterials (part 1)'.

Bayesian inference using an adaptive neuro-fuzzy inference system

For most Bayesian inference problems that are of interest, solving for the model parameter posterior probability distribution remains to be the main challenge. A typical technique entails using Markov chain Monte Carlo (MCMC), in order to obtain an approximation of the posterior distribution. However, one drawback of MCMC is that it usually requires solving the forward problem multiple times, which could be infeasible, if the forward model is computationally expensive. In this paper, a computationally cheap surrogate model is developed using the adaptive neuro-fuzzy inference system with a fuzzy c-means initialization (ANFIS-FCM). The optimal number of fuzzy rules is chosen via a proposed algorithm based on the pairing frequency clustering validity index. The effectiveness of the ANFIS-FCM model is tested on inferring reaction rate parameters in a 3-node network toy problem, which mimics idealized combustion reaction networks. Using multiple error measures, it is shown that the developed ANFIS-FCM model is capable of constructing accurate surrogate models at a minimal computational cost.

Ultra-efficient and parameter-free computation of submicron thermal transport with phonon Boltzmann transport equation

Understanding thermal transport at the submicron scale is crucial for engineering applications, especially in the thermal management of electronics and tailoring the thermal conductivity of thermoelectric materials. At the submicron scale, the macroscopic heat diffusion equation is no longer valid and the phonon Boltzmann transport equation (BTE) becomes the governing equation for thermal transport. However, previous thermal simulations based on the phonon BTE have two main limitations: relying on empirical parameters and prohibitive computational costs. Therefore, the phonon BTE is commonly used for qualitatively studying the non-Fourier thermal transport phenomena of toy problems. In this work, we demonstrate an ultra-efficient and parameter-free computational method of the phonon BTE to achieve quantitatively accurate thermal simulation for realistic materials and devices. By properly integrating the phonon properties from first-principles calculations, our method does not rely on empirical material properties input. It can be generally applicable for different materials and the predicted results can match well with experimental results. Moreover, by developing a suitable ensemble of advanced numerical algorithms, our method exhibits superior numerical efficiency. The full-scale (from ballistic to diffusive) thermal simulation of a 3-dimensional fin field-effect transistor with 13 million degrees of freedom, which is prohibitive for existing phonon BTE solvers even on supercomputers, can now be completed within two hours on a single personal computer. Our method makes it possible to achieve the predictive design of realistic nanostructures for the desired thermal conductivity. It also enables accurately resolving the temperature profiles at the transistor level, which helps in better understanding the self-heating effect of electronics.

A Swinging Rod in Brans‐Dicke Theory

AbstractThis article presents the calculation of gravitational radiation from an oscillating rod in Brans–Dicke (BD) theory. Here only the selected formulae is shown and is applied to a toy problem without any rigorous derivation. First, the explicit expressions for the polarizations are calculated and then the power radiated away in gravitational waves (GWs) is obtained. This problem is motivated by the famous Eddington's spinning rod.

Destabilizing the Fundamental Mode of Black Holes: The Elephant and the Flea.

Recent work applying the notion of pseudospectrum to gravitational physics showed that the quasinormal mode spectrum of black holes is unstable, with the possible exception of the longest-lived (fundamental) mode. The fundamental mode dominates the expected signal in gravitational wave astronomy, and there is no reason why it should have privileged status. We compute the quasinormal mode spectrum of two model problems where the Schwarzschild potential is perturbed by a small "bump" consisting of either a Pöschl-Teller potential or a Gaussian, and we show that the fundamental mode is destabilized under generic perturbations. We present phase diagrams and study a simple double-barrier toy problem to clarify the conditions under which the spectral instability occurs.

The third AI summer: AAAI Robert S. Engelmore Memorial Lecture

The third AI summer: AAAI Robert S. Engelmore Memorial Lecture

Continuous Adaptation with Online Meta-Learning for Non-Stationary Target Regression Tasks

Most environments change over time. Being able to adapt to such non-stationary environments is vital for real-world applications of many machine learning algorithms. In this work, we propose CORAL, a computationally efficient regression algorithm capable of adapting to a non-stationary target. CORAL is based on Bayesian linear regression with a sliding window and offline/online meta-learning. The sliding window makes our model focus on the recently received data and ignores older observations. The meta-learning approach allows us to learn the prior distribution of the model parameters. It speeds up the model adaptation, complements the sliding window’s drawback, and enhances the performance. We evaluate CORAL on two tasks: a toy problem and a more complex blood glucose level prediction task. Our approach improves the prediction accuracy for the non-stationary target significantly while also performing well for the stationary target. We show that the two components of our method work in a complementary fashion to achieve this.

Properties of the affine‐invariant ensemble sampler's ‘stretch move’ in high dimensions

SummaryWe present theoretical and practical properties of the affine‐invariant ensemble sampler Markov Chain Monte Carlo method. In high dimensions, the sampler's ‘stretch move’ has unusual and undesirable properties. We demonstrate this with an n‐dimensional correlated Gaussian toy problem with a known mean and covariance structure, and a multivariate version of the Rosenbrock problem. Visual inspection of a trace plots suggests the burn‐in period is short. Upon closer inspection, we discover the mean and the variance of the target distribution do not match the known values, and the chain takes a very long time to converge. This problem becomes severe as n increases beyond 50. We also applied different diagnostics adapted to be applicable to ensemble methods to determine any lack of convergence. The diagnostics include the Gelman–Rubin method, the Heidelberger–Welch test, the integrated autocorrelation and the acceptance rate. The trace plot of individual walkers appears to be useful as well. We therefore conclude that the stretch move should be used with caution in moderate to high dimensions. We also present some heuristic results explaining this behaviour.

Nested Sampling for Frequentist Computation: Fast Estimation of Small p-Values.

We propose a novel method for computing p-values based on nested sampling (NS) applied to the sampling space rather than the parameter space of the problem, in contrast to its usage in Bayesian computation. The computational cost of NS scales as log^{2}1/p, which compares favorably to the 1/p scaling for MonteCarlo (MC) simulations. For significances greater than about 4σ in both a toy problem and a simplified resonance search, we show that NS requires orders of magnitude fewer simulations than ordinary MC estimates. This is particularly relevant for high-energy physics, which adopts a 5σ gold standard for discovery. We conclude with remarks on new connections between Bayesian and frequentist computation and possibilities for tuning NS implementations for still better performance in this setting.

A NEURAL SOLVER FOR VARIATIONAL PROBLEMS ON CAD GEOMETRIES WITH APPLICATION TO ELECTRIC MACHINE SIMULATION

This work presents a deep learning-based framework for the solution of partial differential equations on complex computational domains described with computer-aided design tools. To account for the underlying distribution of the training data caused by spline-based projections from the reference to the physical domain, a variational neural solver equipped with an importance sampling scheme is developed, such that the loss function based on the discretized energy functional obtained after the weak formulation is modified according to the sample distribution. To tackle multi-patch domains possibly leading to solution discontinuities, the variational neural solver is additionally combined with a domain decomposition approach based on the Discontinuous Galerkin formulation. The proposed neural solver is verified on a toy problem and then applied to a real-world engineering test case, namely that of electric machine simulation. The numerical results show clearly that the neural solver produces physics-conforming solutions of significantly improved accuracy.

A Critical Point Analysis of Actor-Critic Algorithms with Neural Networks

We investigate Actor-Critic algorithms from the non-convex optimisation perspective. For the past years, powerful Deep Reinforcement Learning algorithms, such as Deep Deterministic Policy Gradients, have been observed to struggle even in tiny toy problems. Yet, only the critic training has been subject to intensive research. To close this gap, we conduct a critical point analysis for the actor training. First, we find that the reward function must satisfy additional conditions next to those for Deterministic Policy Gradients, such that the critic is a proper loss for the actor. Second, we address the impact of using over-parametrised Neural Networks in the actor part. If there are more parameters than samples, a Q-function has less sources of critical points with respect to its action input leading to better actor training. Additionally, critical points of the actor loss are only those, where the Q-function is extremal. Third, we outline challenges in the formulation of a sound optimisation task. They arise due to conflicting requirements between Reinforcement Learning and Neural Network architectures.

Randomized maximum likelihood based posterior sampling

Minimization of a stochastic cost function is commonly used for approximate sampling in high-dimensional Bayesian inverse problems with Gaussian prior distributions and multimodal posterior distributions. The density of the samples generated by minimization is not the desired target density, unless the observation operator is linear, but the distribution of samples is useful as a proposal density for importance sampling or for Markov chain Monte Carlo methods. In this paper, we focus on applications to sampling from multimodal posterior distributions in high dimensions. We first show that sampling from multimodal distributions is improved by computing all critical points instead of only minimizers of the objective function. For applications to high-dimensional geoscience inverse problems, we demonstrate an efficient approximate weighting that uses a low-rank Gauss-Newton approximation of the determinant of the Jacobian. The method is applied to two toy problems with known posterior distributions and a Darcy flow problem with multiple modes in the posterior.

Electron Density Geometry and the Quantum Theory of Atoms in Molecules.

A novel form of charge density analysis, that of isosurface curvature redistribution, is formulated and applied to the toy problem of carbonyl oxygen activation in formaldehyde. The isosurface representation of the electron charge density allows us to incorporate the rigorous geometric constraints of closed surfaces toward the analysis and chemical interpretation of the charge density response to perturbations. Visual inspection of 2D isosurface motion resulting from applied external electric fields reveals how the isosurface curvature flows within and between atoms and that a molecule can be uniquely and completely partitioned into chemically significant regions of positive and negative curvatures. These concepts reveal that carbonyl oxygen activation proceeds primarily through curvature and charge redistribution within rather than between Bader atoms. Using gradient bundle analysis─the partitioning of formaldehyde into infinitesimal volume elements bounded by QTAIM zero-flux surfaces─the observations from visual isosurface inspection are verified. The results of the formaldehyde carbonyl analysis are then shown to be transferable to the substrate carbonyl in the ketosteroid isomerase enzyme, laying the groundwork for extending this approach to the problems of enzymatic catalysis.

Inverse Design of Two-Dimensional Airfoils Using Conditional Generative Models and Surrogate Log-Likelihoods

Abstract This paper shows how to use conditional generative models in two-dimensional (2D) airfoil optimization to probabilistically predict good initialization points within the vicinity of the optima given the input boundary conditions, thus warm starting and accelerating further optimization. We accommodate the possibility of multiple optimal designs corresponding to the same input boundary condition and take this inversion ambiguity into account when designing our prediction framework. To this end, we first employ the conditional formulation of our previous work BézierGAN–Conditional BézierGAN (CBGAN)—as a baseline, then introduce its sibling conditional entropic BézierGAN (CEBGAN), which is based on optimal transport regularized with entropy. Compared with CBGAN, CEBGAN overcomes mode collapse plaguing conventional GANs, improves the average lift-drag (Cl/Cd) efficiency of airfoil predictions from 80.8% of the optimal value to 95.8%, and meanwhile accelerates the training process by 30.7%. Furthermore, we investigate the unique ability of CEBGAN to produce a log-likelihood lower bound that may help select generated samples of higher performance (e.g., aerodynamic performance). In addition, we provide insights into the performance differences between these two models with low-dimensional toy problems and visualizations. These results and the probabilistic formulation of this inverse problem justify the extension of our GAN-based inverse design paradigm to other inverse design problems or broader inverse problems.

A novel method for optimal robust PID design based on mixed sensitivity closed loop shaping and convex programming

In industry, the PID controller remains ubiquitous. Its design is typically tackled from classical design methods which do not properly take into account model uncertainty. Robustness is therefore typically a major issue with classically tuned PID controllers. This in turn results in PID controllers that are ‘detuned’ experimentally by an operator in order to avoid these robustness issues. A more generic method is hence required for tuning robust PID controllers.This paper proposes a novel method to enforce a PID (classic and higher order PID) architecture in the mixed sensitivity controller synthesis problem (which is the basis for the mixed sensitivity H∞ design), based on sequential convex second order programming with guaranteed convergence to a stationary point of the optimization problem.Numerical validation of the proposed method is presented by means of some toy problems illustrating the flexibility and practicality of the proposed method. The resulting PID controllers are compared with classical H∞ controllers illustrating the viability of the method to synthesise robust PID controllers based on H∞ controller design specification.

Achieving robustness to aleatoric uncertainty with heteroscedastic Bayesian optimisation

Bayesian optimisation is a sample-efficient search methodology that holds great promise for accelerating drug and materials discovery programs. A frequently-overlooked modelling consideration in Bayesian optimisation strategies however, is the representation of heteroscedastic aleatoric uncertainty. In many practical applications it is desirable to identify inputs with low aleatoric noise, an example of which might be a material composition which displays robust properties in response to a noisy fabrication process. In this paper, we propose a heteroscedastic Bayesian optimisation scheme capable of representing and minimising aleatoric noise across the input space. Our scheme employs a heteroscedastic Gaussian process surrogate model in conjunction with two straightforward adaptations of existing acquisition functions. First, we extend the augmented expected improvement heuristic to the heteroscedastic setting and second, we introduce the aleatoric noise-penalised expected improvement (ANPEI) heuristic. Both methodologies are capable of penalising aleatoric noise in the suggestions. In particular, the ANPEI acquisition yields improved performance relative to homoscedastic Bayesian optimisation and random sampling on toy problems as well as on two real-world scientific datasets. Code is available at: https://github.com/Ryan-Rhys/Heteroscedastic-BO

Strategies to Enhance the Performance of Gaussian Mixture Model Fitting for Uncertainty Quantification

Summary When formulating history matching within the Bayesian framework, we may quantify the uncertainty of model parameters and production forecasts using conditional realizations sampled from the posterior probability density function (PDF). It is quite challenging to sample such a posterior PDF. Some methods [e.g., Markov chain Monte Carlo (MCMC)] are very expensive, whereas other methods are cheaper but may generate biased samples. In this paper, we propose an unconstrained Gaussian mixture model (GMM) fitting method to approximate the posterior PDF and investigate new strategies to further enhance its performance. To reduce the central processing unit (CPU) time of handling bound constraints, we reformulate the GMM fitting formulation such that an unconstrained optimization algorithm can be applied to find the optimal solution of unknown GMM parameters. To obtain a sufficiently accurate GMM approximation with the lowest number of Gaussian components, we generate random initial guesses, remove components with very small or very large mixture weights after each GMM fitting iteration, and prevent their reappearance using a dedicated filter. To prevent overfitting, we add a new Gaussian component only if the quality of the GMM approximation on a (large) set of blind-test data sufficiently improves. The unconstrained GMM fitting method with the new strategies proposed in this paper is validated using nonlinear toy problems and then applied to a synthetic history-matching example. It can construct a GMM approximation of the posterior PDF that is comparable to the MCMC method, and it is significantly more efficient than the constrained GMM fitting formulation (e.g., reducing the CPU time by a factor of 800 to 7,300 for problems we tested), which makes it quite attractive for large-scale history-matching problems. NOTE: This paper is also published as part of the 2021 SPE Reservoir Simulation Conference Special Issue.

Space mapping-based optimization with the macroscopic limit of interacting particle systems

We propose a space mapping-based optimization algorithm for microscopic interacting particle dynamics which are infeasible for direct optimization. This is of relevance for example in applications with bounded domains for which the microscopic optimization is difficult. The space mapping algorithm exploits the relationship of the microscopic description of the interacting particle system and a corresponding macroscopic description as partial differential equation in the “many particle limit”. We validate the approach with the help of a toy problem that allows for direct optimization. Then we study the performance of the algorithm in two applications. A pedestrian flow is considered and the transportation of goods on a conveyor belt is optimized. The numerical results underline the feasibility of the proposed algorithm.

Validation of a multi-objective optimization framework for the sizing of pipes in DH Networks

Sizing distribution equipment of a district heating (DH) network is a complex yet decisive process to target techno-economic optimality for these systems. This paper describes and validates a framework that uses multi-objective optimization to support decisions regarding the sizing of a DH network. A genetic algorithm (NSGA2) is employed to generate optimal pipe diameters with respect to the operational and investment cost. This framework is then validated using an elementary toy problem consisting of a straight horizontal two-tube network, with a set of identical consumers equally distributed in the network and supplied by a unique heat source. We show that the framework, on the aforementioned problem, reaches very good convergence and diversity for the optimal solutions.This sizing method is then compared to typical local sizing method. We demonstrate a significant decrease of global cost by 40% by using DH network system simulation to take into account local interactions inside the DH system.

Does Bayesian model averaging improve polynomial extrapolations? Two toy problems as tests

We assess the accuracy of Bayesian polynomial extrapolations from small parameter values, x, to large values of x. We consider a set of polynomials of fixed order, intended as a proxy for a fixed-order effective field theory (EFT) description of data. We employ Bayesian model averaging (BMA) to combine results from different order polynomials (EFT orders). Our study considers two ‘toy problems’ where the underlying function used to generate data sets is known. We use Bayesian parameter estimation to extract the polynomial coefficients that describe these data at low x. A ‘naturalness’ prior is imposed on the coefficients, so that they are . We BMA different polynomial degrees by weighting each according to its Bayesian evidence and compare the predictive performance of this BMA with that of the individual polynomials. The credibility intervals on the BMA forecast have the stated coverage properties more consistently than does the highest evidence polynomial, though BMA does not necessarily outperform every polynomial.