Search Of Parameter Space Research Articles

Efficient Hyperbolic Perceptron for Image Classification

Deep neural networks, often equipped with powerful auto-optimization tools, find widespread use in diverse domains like NLP and computer vision. However, traditional neural architectures come with specific inductive biases, designed to reduce parameter search space, cut computational costs, or introduce domain expertise into the network design. In contrast, multilayer perceptrons (MLPs) offer greater freedom and lower inductive bias than convolutional neural networks (CNNs), making them versatile for learning complex patterns. Despite their flexibility, most neural architectures operate in a flat Euclidean space, which may not be optimal for various data types, particularly those with hierarchical correlations. In this paper, we move one step further by introducing the hyperbolic Res-MLP (HR-MLP), an architecture extending the attention-free MLP to a non-Euclidean space. HR-MLP leverages fully hyperbolic layers for feature embeddings and end-to-end image classification. Our novel Lorentz cross-patch and cross-channel layers enable direct hyperbolic operations with fewer parameters, facilitating faster training and superior performance compared to Euclidean counterparts. Experimental results on CIFAR10, CIFAR100, and MiniImageNet confirm HR-MLP’s competitive and improved performance.

A demand‐responsive feeder service with a maximum headway at mandatory stops

AbstractPublic transportation out of suburban or rural areas is crucial. Feeder transportation services offer a solution by transporting passengers to areas where more options for public transport are available. On one hand, fully flexible demand‐responsive feeder services (DRFSs) efficiently tailor their service to the needs of the passengers. On the other hand, traditional feeder services provide predictability and easier cost control. In this article, a semi‐flexible DRFS is considered, which combines positive characteristics of both traditional services as well as fully flexible services. This feeder service has two types of bus stops: mandatory bus stops and optional bus stops. Mandatory bus stops are guaranteed to be visited by a bus within a certain time interval. Optional stops are only visited when there is demand for transportation nearby. The performance of this feeder service is optimized with the use of a new type of metaheuristic framework, which we denote as parameter space search. Experimental results on small benchmark instances indicate that the heuristic performs on average 12.42% better than LocalSolver, a commercial optimization solver, with an average runtime of 2.1 s. Larger instances can also be solved, typically within 2 min.

Fast and reliable reduced‐order models for cardiac electrophysiology

AbstractMathematical models of the human heart increasingly play a vital role in understanding the working mechanisms of the heart, both under healthy functioning and during disease. The ultimate aim is to aid medical practitioners diagnose and treat the many ailments affecting the heart. Towards this, modeling cardiac electrophysiology is crucial as the heart's electrical activity underlies the contraction mechanism and the resulting pumping action. Apart from modeling attempts, the pursuit of efficient, reliable, and fast solution algorithms has been of great importance in this context. The governing equations and the constitutive laws describing the electrical activity in the heart are coupled, nonlinear, and involve a fast moving wave front, which is generally solved by the finite element method. The numerical treatment of this complex system as part of a virtual heart model is challenging due to the necessity of fine spatial and temporal resolution of the domain. Therefore, efficient surrogate models are needed to predict the electrical activity in the heart under varying parameters and inputs much faster than the finely resolved models. In this work, we develop an adaptive, projection‐based reduced‐order surrogate model for cardiac electrophysiology. We introduce an a posteriori error estimator that can accurately and efficiently quantify the accuracy of the surrogate model. Using the error estimator, we systematically update our surrogate model through a greedy search of the parameter space. Furthermore, using the error estimator, the parameter search space is dynamically updated such that the most relevant samples get chosen at every iteration. The proposed adaptive surrogate model is tested on three benchmark models to illustrate its efficiency, accuracy, and ability of generalization.

Large-Scale Exact Epidemiological Modelling with Multi-Dimensional Constrained Optimization Approaches.

Multi-dimensional prediction models of the pandemic diseases should be constructed in a way to reflect their peculiar epidemiological characters. In this paper, a graph theory-based constrained multi-dimensional (CM) mathematical and meta-heuristic algorithms (MA) are formed to learn the unknown parameters of a large-scale epidemiological model. The specified parameter signs and the coupling parameters of the sub-models constitute the constraints of the optimization problem. In addition, magnitude constraints on the unknown parameters are imposed to proportionally weight the input-output data importance. To learn these parameters, a gradient-based CM recursive least square (CM-RLS) algorithm, and three search-based MAs; namely, the CM particle swarm optimization (CM-PSO), the CM success history-based adaptive differential evolution (CM-SHADE), and the CM-SHADEWO enriched with the whale optimization (WO) algorithms are constructed. The traditional SHADE algorithm was the winner of the 2018 IEEE congress on evolutionary computation (CEC) and its versions in this paper are modified to create more certain parameter search spaces. The results obtained under the equal conditions show that the mathematical optimization algorithm CM-RLS outperforms the MA algorithms, which is expected since it uses the available gradient information. However, the search-based CM-SHADEWO algorithm is able to capture the dominant character of the CM optimization solution and produce satisfactory estimates in the presence of the hard constraints, uncertainties and lack of gradient information.

Comparison of force and moment tensor estimations of subevents during the 2022 Hunga–Tonga submarine volcanic eruption

SUMMARYThe 2022 January 15 Hunga–Tonga submarine volcanic eruption was unprecedented in the modern era for its size, in terms of its plume height and atmospheric waves, including sound. Global seismic stations recorded maximal radiated energy during 30 min of the plume-forming phase of the eruption (04:00 to 04:30 UTC), with the largest subevents occurring over a 5-min interval starting at 04:15:17 UTC. Here, we consider two simple point-source models—force and moment tensor—and separately consider the single main subevent and the sequence of four subevents. Estimation of source models for the first subevent in the sequence is achieved with a complete search of model parameter space to find the global minimum of a waveform misfit function (body or surface waves). We performed 25 runs to explore the impact of depth, source model (force or moment tensor), wave type (body or surface), and component (vertical, radial, and transverse) on the waveform fits and estimated best-fitting source. Visualization of the misfit function reveals complex trade-offs among model parameters, highlighting the importance of characterizing uncertainties and parameter trade-offs. The four-subevent source model has up to 28 model parameters and requires an efficient search algorithm to find the best-fitting source. For this, we use the covariance matrix adaptation evolution strategy implemented on a high-performance computing cluster. The 10 four-subevent runs for each source model return sequences of subvertical downward forces and explosive-like moment tensors for each subevent. Our results show that these two simple source models provide comparable fits to regional and global seismic waveforms and that the source types for each subevent—either force direction or moment tensor source type on the eigenvalue lune—are similar enough to each other to consider that the subevents originate from the same process. Our estimation of the source mechanisms, sizes, and relative timing should benefit a physical interpretation of the eruption sequence.

Systematic performance of the ASKAP fast radio burst search algorithm

ABSTRACT Detecting fast radio bursts (FRBs) requires software pipelines to search for dispersed single pulses of emission in radio telescope data. In order to enable an unbiased estimation of the underlying FRB population, it is important to understand the algorithm efficiency with respect to the search parameter space and thus the survey completeness. The Fast Real-time Engine for Dedispersing Amplitudes (fredda) search pipeline is a single pulse detection pipeline designed to identify radio pulses over a large range of dispersion measures (DM) with low latency. It is used on the Australian Square Kilometre Array Pathfinder (ASKAP) for the Commensal Real-time ASKAP Fast Transients (CRAFT) project. We utilize simulated single pulses in the low- and high-frequency observation bands of ASKAP to analyse the performance of the pipeline and infer the underlying FRB population. The simulation explores the signal-to-noise ratio (S/N) recovery as a function of DM and the temporal duration of FRB pulses in comparison to injected values. The effects of intrachannel broadening caused by dispersion are also carefully studied in this work using control data sets. Our results show that for Gaussian-like single pulses, >85 per cent of the injected signal is recovered by pipelines such as fredda at DM <3000 pc cm−3 using standard boxcar filters compared to an ideal incoherent dedispersion match filter. Further calculations with sensitivity implies at least ∼10 per cent of FRBs in a Euclidean universe at target sensitivity will be missed by fredda and heimdall, another common pipeline, in ideal radio environments at 1.1 GHz.

Physics-informed deep learning for simultaneous surrogate modeling and PDE-constrained optimization of an airfoil geometry

We use a physics-informed neural network (PINN) to simultaneously model and optimize the flow around an airfoil to maximize its lift to drag ratio. The parameters of the airfoil shape are provided as inputs to the PINN and the multidimensional search space of shape parameters is populated with collocation points to ensure that the Navier–Stokes equations are approximately satisfied throughout. We use the fact that the PINN is automatically differentiable to calculate gradients of the lift-to-drag ratio with respect to the airfoil shape parameters. This allows us to optimize with the L-BFGS gradient-based algorithm, which is more efficient than non-gradient-based algorithms, without deriving an adjoint code. We train the PINN with adaptive sampling of collocation points, such that the accuracy of the solution improves as the solution approaches the optimal point. We demonstrate this on two examples: one that optimizes a single parameter, and another that optimizes eleven parameters. The method is successful and, by comparison with conventional CFD, we find that the velocity and pressure fields have small pointwise errors and that the method converges to optimal parameters. We find that different PINNs converge to slightly different parameters, reflecting the fact that there are many closely-spaced local minima when using stochastic gradient descent. This method can be applied relatively easily to other optimization problems and avoids the difficult process of writing adjoint codes. It is, however, more computationally expensive than adjoint-based optimization. As knowledge about training PINNs improves and hardware dedicated to neural networks becomes faster, this method of simultaneous training and optimization with PINNs could become easier and faster than using adjoint codes.

Targeted Discovery of Low-Coordinated Crystal Structures via Tunable Particle Interactions

Particles interacting via isotropic, multiwell pair potentials have been shown to self-assemble into a range of crystal structures, yet how the characteristics of the underlying interaction potential give rise to the resultant structure remains largely unknown. We have thus developed a functional form for the interaction potential in which all features can be tuned independently. We perform continuous parameter space searches by systematically changing pairs of parameters, controlling the various features of the interaction potential. By enforcing a repulsive first well (controlling particle interactions of the first neighbor shell), we stimulate the formation of low-coordinated assemblies. We report the self-assembly of 20 previously unknown crystal structure types, 14 of which have low coordination numbers. Despite limiting the search to a small region of the vast parameter space of possible particle interactions, a wealth of complexity and symmetry is apparent within these crystal structures, which include clathrates with empty cages and low-symmetry structures. Our findings suggest that an unknown number of previously undiscovered crystal structure configurations are possible through self-assembly, which can serve as interesting design targets for soft condensed matter synthesis.

Auto-STGCN: Autonomous Spatial-Temporal Graph Convolutional Network Search

In recent years, many spatial-temporal graph convolutional network (STGCN) models are proposed to deal with the spatial-temporal network data forecasting problem. These STGCN models have their own advantages, i.e., each of them puts forward many effective operations and achieves good prediction results in the real applications. If users can effectively utilize and combine these excellent operations integrating the advantages of existing models, then they may obtain more effective STGCN models thus create greater value using existing work. However, they fail to do so due to the lack of domain knowledge, and there is lack of automated system to help users to achieve this goal. In this article, we fill this gap and propose Auto-STGCN algorithm, which makes use of existing models to automatically explore high-performance STGCN model for specific scenarios. Specifically, we design Unified-STGCN framework, which summarizes the operations of existing architectures, and use parameters to control the usage and characteristic attributes of each operation, so as to realize the parameterized representation of the STGCN architecture and the reorganization and fusion of advantages. Then, we present Auto-STGCN, an optimization method based on reinforcement learning, to quickly search the parameter search space provided by Unified-STGCN, and generate optimal STGCN models automatically. Extensive experiments on real-world benchmark datasets show that our Auto-STGCN can find STGCN models superior to existing STGCN models used for search space construction, which demonstrates the effectiveness of our proposed method.

Laboratory Constraints on the Neutron-Spin Coupling of feV-Scale Axions

A new analysis methodology of K-${}^{3}$He comagnetometer data provides vastly improved constraints on axion neutron-spin coupling, shrinking the parameter search space for this proposed constituent of dark matter.

Acoustic source localization using Bayesian optimization with a Gaussian process surrogate model

Matched field processing is a computationally expensive approach to estimate acoustic source location, since it relies on a grid search through range and depth. We present an efficient alternative that samples the parameter space with a Bayesian approach using a Gaussian process as a surrogate model of the objective function. The objective function is defined as a Bartlett processor whose output measures the match between a received and replica pressure field on a vertical line array. Replica fields are obtained using a normal mode propagation model whose geoacoustic parameters are selected from the parameter search space. The surrogate model represents the posterior on the objective function and is updated with each model evaluation. Optimization is performed with sequential model evaluations, with an acquisition function guiding the next point in parameter space to be evaluated. We demonstrate our approach using both simulations and real data collected during the SWELLEX96 ocean acoustics experiment. Results indicate Bayesian optimization using the GP surrogate model converges on average within 700 evaluations of the objective function, far fewer than required in matched field processing.

Intelligent Quadcopter Control Using Artificial Neural Networks

An advanced controller architecture and design for quadcopter control implementation is proposed in this study. Instead of using only the error information as input to the controller, reference and measured outputs are used separately independent from each other. This enhances the performance of the controller of quadcopter being a highly non-linear platform. In this study single layer neural network is directly used as a controller. A complex controller is grown from an initially simple PID controller. This elevates the need for time consuming search in huge parameter space due to very high dimensions. About ten percent improvement over state-of-the-art controllers is observed and results are reported both numerically and graphically. Promising results encourage to use the type of controller proposed for various real applications.

Solar wind driven influences on the Martian oxygen corona: Constraints on atmospheric sputtering from a synthesis of MAVEN measurements during solar minimum

The prolonged low solar extreme ultraviolet (EUV) flux conditions provided by the deep solar minimum between solar cycles 24 and 25 can be expected to have amplified the relative influence of any solar wind driven effects on the Martian corona. As such, this solar minimum has provided an opportunity to study solar wind effects on processes that produce the energetic neutral particles that populate the oxygen corona, particularly atmospheric sputtering by precipitating energetic ions, and dissociative recombination of O2+. To probe the corona, we use a novel, fully empirical divergence-based technique for estimating coronal oxygen densities. This method solves the continuity equation for ions to obtain local ionization rates that are, in turn, converted to neutral densities by accounting for ionization via electron impact, charge exchange, and photoionization. We apply this technique to combined measurements of ion flux, electron flux, solar EUV flux, and magnetic fields from the Mars Atmosphere and Volatile EvolutioN (MAVEN) orbiter at Mars. We separately combine MAVEN measurements during relatively low and high upstream solar wind flux, Fsw, conditions and derive the relevant fluxes for each range of conditions. We find that fluxes of planetward energetic O+ upstream of Mars increase by roughly ×3 during high Fsw conditions. We derive radial density profiles in the range 2.0–2.7 Mars radii and find that coronal hot oxygen densities scale with radial distance, R, as R−2 from 300cm−3 to 100cm−3 in this range. We perform χ2 parameter-space searches for exobase densities and effective temperatures to evaluate any potential differences in the hot O populations that populate the corona. Neither the radial profiles, nor the χ2 search, reveal any significant effects of increased Fsw on the Martian oxygen corona. The best estimates for exobase properties center around (6–7)×103atoms/cm3 and 0.3 eV/kB for both cases, with large correlated uncertainties. Based on the uncertainties inherent to the method and the measurements, we conclude that solar wind driven effects must be limited to roughly a factor of 2 and that further investigations to empirically constrain the corona over a wider range of altitudes are needed to investigate solar wind driven effects.

A robust fitness function and genetic algorithm to morphologically constrain the dynamics of interacting galaxies

Given observational data from systems of interacting galaxies, we seek to determine the values of various dynamical parameters through the optimization of numerical models via genetic algorithms. However, fitting these models can be quite difficult. In this paper, we show how naive implementations of fitness functions can yield unintuitive results. We then propose a novel fitness function which was developed by utilizing data from the Galaxy Zoo: Mergers project (GZM). The human-scored models obtained from GZM were used to validate our fitness function and led to the adoption of a tidal distortion term which dramatically improved results. We also give a characterization of various geometric and dynamical symmetries inherent within the system and show how the knowledge of these symmetries can be used to reduce the volume of the parameter search space when performing optimization. Lastly, we implement a real-coded genetic algorithm with features designed to address these symmetries. Using simulated target systems with known parameters as a surrogate for observational data, we test our fitness function and genetic algorithm for robustness, accuracy, and convergence. We discuss the link between the degree of tidal distortion present in a target image and the constraints on the dynamical parameters using three different target systems with varying morphology.

A new hybrid optimization approach using PSO, Nelder-Mead Simplex and Kmeans clustering algorithms for 1D Full Waveform Inversion.

The FWI is formulated as a nonlinear optimization problem that traditionally uses local (derivative-based) minimization to find the scalar field of properties that best represents the field seismic data. This problem has a high computational cost and accuracy limited to local minima, in addition to suffering from a slow convergence rate (Cycle Skipping). Therefore, we developed a two-phase hybrid optimization algorithm based on DFO algorithms. The first use global minimization and clustering technique. The second use local minimization. In phase 1 we adopted the modified PSO and K-means algorithms and in phase 2, we adopted the ANMS. We call the hybrid algorithm of the PSO-Kmeans-ANMS. Where K-means is responsible for dividing swarms of particles into 2 clusters at every instant. This strategy aims to automatically balance the mechanisms of exploration and exploitation of the parameter search space by the hybrid algorithm, allowing one to find more precise solutions and consequently improving its convergence. The PSO-Kmeans-ANMS algorithm was validated on the set of 12 benchmark functions and applied to the FWI 1D problem. We compared PSO-Kmeans-ANMS with classic PSO, modified PSO, and ANMS algorithms. The metrics used were are the average execution time and the success rate (an error of ± 4% of the optimal solution). In all validation experiments and the FWI application, the PSO-Kmeans-ANMS performed well in terms of robustness and computational efficiency. In the case of FWI, there was a significant reduction in computational cost, thus presenting a relevant result.

Design of a Multi-Monochromatic X-ray Imager (MMI) for Kr K-shell line emission.

The Multi-Monochromatic X-ray Imager (MMI) is a time-gated spectrometer used in implosion experiments at the OMEGA laser facility. From the data, electron temperature and density spatial distributions can be obtained at different implosion times. Previous MMI designs used Ar K-shell emission (3-6keV) as a spectroscopic tracer and provided a spectral resolution of around 20eV. However, Ar K-shell line emission becomes less useful at electron temperatures above 2keV due to over-ionization. Kr K-shell (12-16keV) has been shown to be an attractive alternative to diagnose hot implosion cores in recent publications. The purpose of this paper is to show a new point design that allows the MMI to detect this higher photon energy range with suitable spectral resolution. The algorithm used to find the optimal design couples a ray-tracing code and an exhaustive parameter space search. This algorithm may be useful as a tool to find optimal MMI designs for other purposes, i.e., other spectral regions for other spectroscopic tracers. The main change between the two designs is the replacement of the multi-layer mirror with a flat Bragg Ge (220) crystal. The final Kr K-shell MMI design has a photon energy range from 12 to 16.1keV.

Applications of Bayesian optimization with a Gaussian process surrogate model in ocean acoustics

In this study, we present a method that samples geoacoustic parameter space with a Bayesian approach that uses a Gaussian process as a surrogate model of the objective function. The objective function is defined as a Bartlett processor whose output measures the match between a received and replica pressure field on a vertical line array. Replica fields are obtained using a normal mode propagation model whose geoacoustic parameters are selected from the parameter search space. The surrogate model represents the posterior on the objective function and is updated with each model evaluation. Optimization is performed with sequential model evaluations, with an acquisition function guiding the next point in parameter space to be evaluated. Various use cases and parameterizations are discussed, including the effects of the choice of acquisition function and covariance function of the Gaussian process. Results indicate that Bayesian optimization using a Gaussian process surrogate model converges rapidly on an approximated optimal solution.

Development of a two-phase adaptive MCMC method for efficient Bayesian model updating of complex dynamic systems

The fundamental problem of Bayesian model updating is identifying the posterior probability density function (PDF) of uncertain model parameters. Markov chain Monte Carlo (MCMC) has been recognized to be potential for this problem. However, the main difficulty is that many samples need to be generated first to locate the regions of high probability. This could be time consuming for complex dynamic systems. A novel two-phase adaptive MCMC method is developed in this work to tackle this difficulty. A parameter-space search algorithm is used to do a systematic search to find output-equivalent points, which are in the neighborhood of the regions of high probability. To generate samples of uncertain model parameters in these regions for describing the shape of the posterior PDF, a Markov chain is constructed with these output-equivalent points as seeds. This is done by the proposed weighted MCMC algorithm based on the modified Metropolis-Hasting (MH) algorithm. To enhance the efficiency of the algorithm, a new proposal PDF is developed using the seeds. The covariance matrix of the proposal PDF is adaptively updated with the latest set of generated samples. Furthermore, the optimal prediction-error variance as a function of model parameters is analytical derived and its values at the seeds are adaptively used to estimate the acceptance ratio in the modified MH algorithm. The proposed weighted MCMC algorithm generates samples in the region of high probability adaptively without going through multiple levels, which are computationally demanding. The posterior uncertainties in model updating can be rigorously considered by the generated samples with their weights proportional to their posterior PDF values. To verify the proposed method, numerical simulation of a two-story shear building with the challenging locally identifiable and unidentifiable cases was conducted. Further verification was also conducted by updating a finite element model of a full-scaled coupled building system utilizing measured vibration data from field tests.

Deep Learning Algorithms and Parallel Distributed Computing Techniques for High-Resolution Load Forecasting Applying Hyperparameter Optimization

Electrical load forecasting is one of the critical tasks that helps power utility companies in planning and operation as well as the energy managementsystem (EMS) in controlling and optimizing the power grid’s performance. It guarantees system adequacy and reliability by reducing the gap between supply and demand, and it bolsters cybersecurity by enabling false data detection and system security by predicting system peak load. This article aims to build an accurate long-term load forecasting model based on deep learning (DL) algorithms. Initially, big-data analytics and feature engineering are applied to the power load and weather historical data. Next, three different load forecasting models are developed based on three corresponding deep neural network models (DNN): the simple RNN, the long short term memory (LSTM), and the gated recurrent unit (GRU), respectively. To find an accurate DNN model, a two-step hyperparameter optimization is performed consisting of cross-validation. In the first step, the random search (RS) algorithm is employed to narrow the parameter search space to a smaller local one. In the second step, the grid search (GS) algorithm is used to find the best hyperparameters in the local area located by RS. However, the computational burden of the DNN training process is too high and may lead to out-of-memory (OOM) issues, especially when we are dealing with a big volume of data. To overcome this problem, we developed our Python program based on Tensorflow framework and implemented the framework on a high-performance computer cluster (HPCC). Then, different jobs are defined to employ parallel distributed computing (PDC) capability on the SIU BigDawg HPCC. To this end, all DNN hyperparameters (e.g., number of layers, neurons, dropout, batch size, epoch, learning rate, etc.) can be optimized and result in a comprehensive and accurate load forecasting model. Our proposed method, along with the programming framework, is evaluated on a real-world dataset. Among the three models developed and tested, the GRU, with load and weather as features, has demonstrated excellent performance and outperformed the other two models for long-term forecasting with the best metrics (i.e., MSE, MAPE, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$R^2$</tex-math></inline-formula> ) and daily error almost less than 1%. Also, our best model (optimized GRU) is compared with the other reported state-of-the-art models such as DBN, SVR, RM-LSTM, DM-GCNN, TCMS-CNN. It is demonstrated that the proposed framework yields a superior performance than the others.

Bootstrapping more QM systems

We test the bootstrap approach for determining the spectrum of one dimensional Hamiltonians. In this paper we focus on problems that have a two parameter search space in the bootstrap approach: the double well and a periodic potential associated with the Mathieu equation. For the double well, we compare the bootstrapped energies with contributions from perturbative and non-perturbative results, finding good agreement. For the periodic potentials, we notice that the bootstrap approach gives the band structure of the periodic potential, but it has trouble finding the quasi-momentum of the system. To make further progress on determining the dispersion relation of the bands, new techniques are needed.