High-dimensional Parameter Space Research Articles

Scalable stellar evolution forecasting

Many astrophysical applications require efficient yet reliable forecasts of stellar evolution tracks. One example is population synthesis, which generates forward predictions of models for comparison with observations. The majority of state-of-the-art rapid population synthesis methods are based on analytic fitting formulae to stellar evolution tracks that are computationally cheap to sample statistically over a continuous parameter range. The computational costs of running detailed stellar evolution codes, such as MESA, over wide and densely sampled parameter grids are prohibitive, while stellar-age based interpolation in-between sparsely sampled grid points leads to intolerably large systematic prediction errors. In this work, we provide two solutions for automated interpolation methods that offer satisfactory trade-off points between cost-efficiency and accuracy. We construct a timescale-adapted evolutionary coordinate and use it in a two-step interpolation scheme that traces the evolution of stars from zero age main sequence all the way to the end of core helium burning while covering a mass range from 0.65 to 300 M⊙. The feedforward neural network regression model (first solution) that we train to predict stellar surface variables can make millions of predictions, sufficiently accurate over the entire parameter space, within tens of seconds on a 4-core CPU. The hierarchical nearest-neighbor interpolation algorithm (second solution) that we hard-code to the same end achieves even higher predictive accuracy, the same algorithm remains applicable to all stellar variables evolved over time, but it is two orders of magnitude slower. Our methodological framework is demonstrated to work on the MESA ISOCHRONES AND STELLAR TRACKS (Choi et al. 2016) data set, but is independent of the input stellar catalog. Finally, we discuss the prospective applications of these methods and provide guidelines for generalizing them to higher dimensional parameter spaces.

Efficient Search for New Physics Using Active Learning in the ATLAS Experiment

Searches for new physics set exclusion limits in parameter spaces of typically up to 2 dimensions. However, the relevant theory parameter space is usually of a higher dimension but only a subspace is covered due to the computing time requirements of signal process simulations. An Active Learning approach is presented to address this limitation. Compared to the usual grid sampling, it reduces the number of parameter space points for which exclusion limits need to be determined. Hence it allows to extend interpretations of searches to higher dimensional parameter spaces and therefore to raise their value, e.g. via the identification of barely excluded subspaces which motivate dedicated new searches. In an iterative procedure, a Gaussian Process is fit to excluded signal cross-sections. Within the region close to the exclusion contour predicted by the Gaussian Process, Poisson disc sampling is used to determine further parameter space points for which the cross-section limits are determined. The procedure is aided by a warm-start phase based on computationally inexpensive, approximate limit estimates such as total signal cross-sections. The procedure is applied to a dark matter search on data collected by the ATLAS detector at the LHC, extending its interpretation from a 2 to a 4-dimensional parameter space while keeping the computational effort at a low level. The result is published in two formats: on one hand there is a publication of the Gaussian Process model. On the other hand, a visualization of the full 4-dimensional contour is presented as a collection of 2-dimensional exclusion contours where the 2 remaining parameters are chosen by the user.

Connecting Agent-Based Models with High-Dimensional Parameter Spaces to Multidimensional Data Using SMoRe ParS: A Surrogate Modeling Approach

Across a broad range of disciplines, agent-based models (ABMs) are increasingly utilized for replicating, predicting, and understanding complex systems and their emergent behavior. In the biological and biomedical sciences, researchers employ ABMs to elucidate complex cellular and molecular interactions across multiple scales under varying conditions. Data generated at these multiple scales, however, presents a computational challenge for robust analysis with ABMs. Indeed, calibrating ABMs remains an open topic of research due to their own high-dimensional parameter spaces. In response to these challenges, we extend and validate our novel methodology, Surrogate Modeling for Reconstructing Parameter Surfaces (SMoRe ParS), arriving at a computationally efficient framework for connecting high dimensional ABM parameter spaces with multidimensional data. Specifically, we modify SMoRe ParS to initially confine high dimensional ABM parameter spaces using unidimensional data, namely, single time-course information of in vitro cancer cell growth assays. Subsequently, we broaden the scope of our approach to encompass more complex ABMs and constrain parameter spaces using multidimensional data. We explore this extension with in vitro cancer cell inhibition assays involving the chemotherapeutic agent oxaliplatin. For each scenario, we validate and evaluate the effectiveness of our approach by comparing how well ABM simulations match the experimental data when using SMoRe ParS-inferred parameters versus parameters inferred by a commonly used direct method. In so doing, we show that our approach of using an explicitly formulated surrogate model as an interlocutor between the ABM and the experimental data effectively calibrates the ABM parameter space to multidimensional data. Our method thus provides a robust and scalable strategy for leveraging multidimensional data to inform multiscale ABMs and explore the uncertainty in their parameters.

Posterior Manifolds over Prior Parameter Regions: Beyond Pointwise Sensitivity Assessments for Posterior Statistics from MCMC Inference

Abstract Increases in the use of Bayesian inference in applied analysis, the complexity of estimated models, and the popularity of efficient Markov chain Monte Carlo (MCMC) inference under conjugate priors have led to more scrutiny regarding the specification of the parameters in prior distributions. Impact of prior parameter assumptions on posterior statistics is commonly investigated in terms of local or pointwise assessments, in the form of derivatives or more often multiple evaluations under a set of alternative prior parameter specifications. This paper expands upon these localized strategies and introduces a new approach based on the graph of posterior statistics over prior parameter regions (sensitivity manifolds) that offers additional measures and graphical assessments of prior parameter dependence. Estimation is based on multiple point evaluations with Gaussian processes, with efficient selection of evaluation points via active learning, and is further complemented with derivative information. The application introduces a strategy to assess prior parameter dependence in a multivariate demand model with a high dimensional prior parameter space, where complex prior-posterior dependence arises from model parameter constraints. The new measures uncover a considerable prior dependence beyond parameters suggested by theory, and reveal novel interactions between the prior parameters and the elasticities.

Innovative fault diagnosis and early warning method based on multifeature fusion model for electric vehicles

Fault diagnostics and early warning are crucial to the safe operation of lithium-ion batteries, and despite partial progress, it is still extremely difficult to solve the problem in a high-dimensional parameter space with complex failure causes. In this article, an innovative fault diagnosis and early warning method based on multifeature fusion model is designed for quantitative and qualitative comprehensive analysis and evaluation of the battery operating state information and the complex internal safety evolution trajectory. Firstly, to form a multi-dimensional fault feature matrix, a real vehicle dataset consisting of 400 vehicles is constructed using typical vehicle fault data from the cloud platform. Then, the original data is noise-reduced and pre-processed by applying initial bias correction and anomalous signal identification methods. Moreover, based on the equivalent circuit model and data-driven algorithm, 13 features including internal short circuit internal resistance, inconsistency accumulation risk, and principal element extraction index are designed from three dimensions: threshold, statistics and model. Finally, the multiple features are combined with random forest (RF) algorithm for hybrid fusion decision to achieve better generalization ability and suppress the overfitting risk of the algorithm. The results show that the proposed feature extraction and fusion decision methods can identify abnormal states and hazard levels in a timely and accurate manner, and this RF-based classification, warning and evaluation framework shows the promise of machine learning algorithms for interpretable early warning studies of battery failures, which can use more modal features to assess the hazard levels of actual operating battery systems in end-cloud collaboration scenarios.

Application of cell mapping to control optimization for an antenna servo system on a disturbed carrier

Abstract. The cell-mapping method, due to its global optimality, has been applied to solve multi-objective optimization problems (MOPs) and optimal control problems. However, the curse of dimensionality limits its application in high-dimensional systems. In this paper, the multi-parameter sensitivity analysis is investigated to reduce the parameter space dimension, which broadens the application of cell mapping to MOPs in high-dimensional parameter space. A post-processing algorithm for MOPs is introduced to help choose proper control parameters from the Pareto set. The proposed scheme is applied successfully in the control parameter optimization of an adaptive nonsingular terminal sliding-mode control for an antenna servo system on a disturbed carrier. Moreover, as the existing global optimal tracking control with an adjoining cell-mapping method may generate tracking-phase differences, an optimal-sliding-mode combined-control strategy is proposed. By using the combined-control strategy, the azimuth and pitch angles of the antenna system are controlled to catch up to a target trajectory with the minimum cost function and to keep high-precision tracking after that.

Visualization and efficient generation of constrained high-dimensional theoretical parameter spaces

We describe a set of novel methods for efficiently sampling high-dimensional parameter spaces of physical theories defined at high energies, but constrained by experimental measurements made at lower energies. Often, theoretical models such as supersymmetry are defined by many parameters, O\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$ \\mathcal{O} $$\\end{document}(10 − 100), expressed at high energies, while relevant experimental constraints are often defined at much lower energies, preventing them from directly ruling out portions of the space. Instead, the low-energy constraints define a complex, potentially non-contiguous subspace of the theory parameters. Naive scanning of the theory space for points which satisfy the low-energy constraints is hopelessly inefficient due to the high dimensionality, and the inverse problem is considered intractable. As a result, many theoretical spaces remain under-explored. We introduce a class of modified generative autoencoders, which attack this problem by mapping the high-dimensional parameter space to a structured low-dimensional latent space, allowing for easy visualization and efficient generation of theory points which satisfy experimental constraints. An extension without dimensional compression, which focuses on limiting potential information loss, is also introduced.

Simulation-based inference for model parameterization on analog neuromorphic hardware

The BrainScaleS-2 (BSS-2) system implements physical models of neurons as well as synapses and aims for an energy-efficient and fast emulation of biological neurons. When replicating neuroscientific experiments on BSS-2, a major challenge is finding suitable model parameters. This study investigates the suitability of the sequential neural posterior estimation (SNPE) algorithm for parameterizing a multi-compartmental neuron model emulated on the BSS-2 analog neuromorphic system. The SNPE algorithm belongs to the class of simulation-based inference methods and estimates the posterior distribution of the model parameters; access to the posterior allows quantifying the confidence in parameter estimations and unveiling correlation between model parameters. For our multi-compartmental model, we show that the approximated posterior agrees with experimental observations and that the identified correlation between parameters fits theoretical expectations. Furthermore, as already shown for software simulations, the algorithm can deal with high-dimensional observations and parameter spaces when the data is generated by emulations on BSS-2. These results suggest that the SNPE algorithm is a promising approach for automating the parameterization and the analyzation of complex models, especially when dealing with characteristic properties of analog neuromorphic substrates, such as trial-to-trial variations or limited parameter ranges.

Reduced-Dimension Surrogate Modeling to Characterize the Damage Tolerance of Composite/Metal Structures

Complex engineering models are typically computationally demanding and defined by a high-dimensional parameter space challenging the comprehensive exploration of parameter effects and design optimization. To overcome this curse of dimensionality and to minimize computational resource requirements, this research demonstrates a user-friendly approach to formulating a reduced-dimension surrogate model that represents a high-dimensional, high-fidelity source model. This approach was developed specifically for a non-expert using commercially available tools. In this approach, the complex physical behavior of the high-fidelity source model is separated into individual, interacting physical behaviors. A separate reduced-dimension surrogate model is created for each behavior and then all are summed to formulate the reduced-dimension surrogate model representing the source model. In addition to a substantial reduction in computational resources and comparable accuracy, this method also provides a characterization of each individual behavior providing additional insight into the source model behavior. The approach encompasses experimental testing, finite element analysis, surrogate modeling, and sensitivity analysis and is demonstrated by formulating a reduced-dimension surrogate model for the damage tolerance of an aluminum plate reinforced with a co-cured bonded E-glass/epoxy composite laminate under four-point bending. It is concluded that this problem is difficult to characterize and breaking the problem into interacting mechanisms leads to improved information on influential parameters and efficient reduced-dimension surrogate modeling. The disbond damage at the interface between the resin and metal proved the most difficult mechanism for reduced-dimension surrogate modeling as it is only engaged in a small subspace of the full parameter space. A binary function was successful in engaging this damage mechanism when applicable based on the values of the most influential parameters.

Constructing D-optimal designs for multi-parameter binary regression models: A theoretical characterization of support points

This manuscript presents a comprehensive study on the construction of D-optimal designs for multiparameter binary regression models. The aim is to expand and validate the conjectures put forth by Ford et al. (Ford et al., 1992) regarding D-optimal designs in generalized linear models with multiple parameters. This is based on a parameter dependent transformation to a weighted regression model and results will be extended to other such models. In this study, building upon the pioneering work by Gunduz and Torsney (Gündüz et al., 2023) in univariate settings with two parameters, we extend their methodology to handle the complexities of multiparameter models. By delving into the intricacies of high-dimensional parameter spaces, we address the challenges associated with constructing D-optimal designs for binary regression models, with a specific focus on the rigorous derivation and characterization of the determination of the support points of the design. In summary, our manuscript contributes a significant advancement in the field of experimental design for multiparameter binary regression models. The meticulous mathematical derivations, accompanied by intuitive conceptual plots, provide a comprehensive understanding of our methodology. The compelling simulated examples further emphasize the efficacy of our approach. This work sets the stage for future investigations in adaptive designs, incorporating prior information, and handling complex model structures within the context of D-optimal designs.

Surrogate-accelerated Bayesian framework for high-temperature thermal diffusivity characterization

Precise determination of thermal diffusivity at high temperatures is crucial for aerospace and energy industries. Periodic heating techniques such as Å ngström’s method are data-rich and commonly used for measuring thermal diffusivity of a solid material. In previous Å ngström’s method studies, regression techniques have been used to solve the associated inverse problem and to estimate thermal diffusivity by minimizing the residual between measurements and model predictions. This approach lacks rigorous uncertainty quantification and does not allow incorporation of prior knowledge for parameters. Adopting a Bayesian framework addresses these issues; however, probing the Bayesian posterior distribution is prohibitively expensive using Markov chain Monte Carlo (MCMC) methods, especially when the physical model is computationally expensive, as in the present study for which an analytical solution does not exist. This study employs a parametric surrogate model in the form of polynomial chaos to accelerate the physical model by several orders of magnitude to support Bayesian analysis. Moreover, high-temperature testing environments are difficult to control precisely, and many unknown parameters exist beyond the quantity of interest. Previous studies have employed random walks to set new parameters in the MCMC sampling process. However, random walks are inefficient in exploring high-dimensional parameter spaces and thus suffer high auto-correlation and poor convergence. To improve the efficiency of the MCMC sampler, this study employs a No-U-Turn sampler that explores the parameter space thoroughly and efficiently. We demonstrate the effectiveness of this Bayesian framework by analyzing experimental results on a graphite sample at approximately 1000 K.

Preference-Based Human-in-the-Loop Optimization for Perceived Realism of Haptic Rendering.

The fidelity of haptic rendering is characterized by the perceived realism capturing the level of similarity to a corresponding tangible object. Perceived realism depends on the musculoskeletal, mental, and perceptual properties of the individuals that manipulate the system. Human-in-the-loop (HiL) studies provide a feasible means for the concurrent optimization of the performance of the overall haptic rendering process, as the physical limitations of the hardware, the factors affecting the fidelity of the rendering algorithm, and the limitations of human action and perception can all be considered simultaneously. In this study, we propose the use of preference-based HiL optimization techniques based on sample-efficient Bayesian optimization algorithms and qualitative pairwise comparisons to maximize the perceived realism of haptic rendering tasks. We present two HiL optimization studies that maximize the perceived realism of spring and friction rendering and validate our results by comparing the HiL-optimized rendering models with expert-tuned nominal models. We show that the system parameters can effectively be optimized within a reasonable amount of time using a preference-based HiL optimization approach. Furthermore, we demonstrate that the approach provides an efficient means of studying the effect of haptic rendering parameters on perceived realism by capturing the interactions among the parameters, even for relatively high dimensional parameter spaces.

Online assessment of multi-parameter stability region and stability margin of wind power plants

The wind power plant (WPP) operates in a wide range of operating points, resulting in diverse small-signal stability performances in a high-dimensional parameter space. The mapping between the stability performance and the operating point is implicit, which makes an obstacle for the stability region and margin assessment. Furthermore, the implicit mapping cannot guide the system operation in real-time applications. In this paper, a data-driven method is developed for the stability assessment and enhancement. Regarding the stability assessment, a regression-based analytical model is proposed to assess the stability region and margin as functions of multiple parameters, such as the active power, the reactive power, and the wind speed. The stability assessment makes use of the impedance data of the WPP. Based on the impedance data at diverse operating points, the generalized Nyquist criterion is analytically reformulated with the supportive vector regression method. Minimum characteristic locus is defined as a single-parameter index of the relative stability, and the stability region boundary is developed. Besides, the dominated oscillation frequency in stable and unstable regions is analytically formulated. The analytical solutions yield an explicit mapping between the stability performance and the operating points. Moving forwards to the stability enhancement, with the help of the explicit mapping achieved in the digital twin (DT) system, the real-time stability margin of the physical WPP could be enhanced by optimizing the operating point of the WPP. The proposed data-driven method for stability assessment and enhancement is implemented in a DT system of WPP. The DT system provides the impedance data of WPP and the effective interaction between the physical and virtual systems, offering opportunities for online applications. The validation is performed on a single wind turbine generator (WTG) and a WPP, respectively. With numerical simulations and experiments, the stability region is proven to be accurately and efficiently observed, and the stability margin is enhanced by adjusting the operating point.

Inferring the Solar Meridional Circulation Flow Profile by Applying Bayesian Methods to Time–Distance Helioseismology

Mapping the large-scale subsurface plasma flow profile within the Sun has been attempted using various methods for several decades. One such flow in particular is the meridional circulation, for which numerous studies have been published. However, such studies often show disagreement in structure. In an effort to constrain the flow profile from the data, a Bayesian Markov chain Monte Carlo framework has been developed to take advantage of the advances in computing power that allow for the efficient exploration of high-dimensional parameter spaces. This study utilizes helioseismic travel-time difference data covering a span of 21 years and a parameterized model of the meridional circulation to find the most likely flow profiles. Tests were carried out on artificial data to determine the ability of this method to recover expected solar-like flow profiles, as well as a few extreme cases. We find that this method is capable of recovering the input flows of both single- and double-cell flow structures. Some inversion results indicate potential differences in meridional circulation between the two solar cycles in terms of both magnitude and morphology, in particular in the mid-convection zone. Of these, the most likely solutions show that solar cycle 23 has a large single-celled profile, while cycle 24 shows weaker flows in general and hints toward a double-celled structure.

Fisher matrix forecasts on the astrophysics of galaxies during the epoch of reionization from the 21-cm power spectra

ABSTRACT The hyperfine 21-cm transition of neutral hydrogen from the early Universe (z > 5) is a sensitive probe of the formation and evolution of the first luminous sources. Using the Fisher matrix formalism we explore the complex and degenerate high-dimensional parameter space associated with the high-z sources of this era and forecast quantitative constraints from a future 21-cm power spectrum (21-cm PS) detection. This is achieved using $\rm {\small ERAXES}$, a coupled semi-analytic galaxy formation model and reionization simulation, applied to an N-body halo merger tree with a statistically complete population of all atomically cooled galaxies out to z ∼ 20. Our mock observation assumes a 21-cm detection spanning z ∈ [5, 24] from a 1000 h mock observation with the forthcoming Square Kilometre Array, and is calibrated with respect to ultraviolet luminosity functions (UV LFs) at z ∈ [5, 10], the optical depth of CMB photons to Thompson scattering from Planck, and various constraints on the IGM neutral fraction at z > 5. In this work, we focus on the X-ray luminosity, ionizing UV photon escape fraction, star formation, and supernova feedback of the first galaxies. We demonstrate that it is possible to recover five of the eight parameters describing these properties with better than 50 per cent precision using just the 21-cm PS. By combining with UV LFs, we are able to improve our forecast, with five of the eight parameters constrained to better than 10 per cent (and all below 50 per cent).

A hybrid deep neural operator/finite element method for ice-sheet modeling

One of the most challenging and consequential problems in climate modeling is to provide probabilistic projections of sea level rise. A large part of the uncertainty of sea level projections is due to uncertainty in ice sheet dynamics. At the moment, accurate quantification of the uncertainty is hindered by the cost of ice sheet computational models. In this work, we develop a hybrid approach to approximate existing ice sheet computational models at a fraction of their cost. Our approach consists of replacing the finite element model for the momentum equations for the ice velocity, the most expensive part of an ice sheet model, with a Deep Operator Network, while retaining a classic finite element discretization for the evolution of the ice thickness. We show that the resulting hybrid model is very accurate and it is an order of magnitude faster than the traditional finite element model. Further, a distinctive feature of the proposed model compared to other neural network approaches, is that it can handle high-dimensional parameter spaces (parameter fields) such as the basal friction at the bed of the glacier, and can therefore be used for generating samples for uncertainty quantification. We study the impact of hyper-parameters, number of unknowns and correlation length of the parameter distribution on the training and accuracy of the Deep Operator Network on a synthetic ice sheet model. We then target the evolution of the Humboldt glacier in Greenland and show that our hybrid model can provide accurate statistics of the glacier mass loss and can be effectively used to accelerate the quantification of uncertainty.

High Throughput Multidimensional Kinetic Screening in Continuous Flow Reactors.

An automated high throughput multidimensional reaction screening platform based on an inline Fourier-transform infrared spectroscopy (FT-IR) is presented. By combining flow chemistry, machine automation and inline analysis, the platform is able to screen reactions in multidimensions (residence time, monomer concentration, degree of polymerization, reaction temperature and monomer conversion) rapidly and efficiently way. Kinetic data libraries associated with high data precision (absolute error < 4 %), high reproducibility and high data density are built with ease from the platform. To test the method, we screened the RAFT polymerization of methyl acrylate in unmatched detail, and the ring opening metathesis polymerization of methyl-5-norbornene-2-carboxylate. The method we introduce is a key step in providing 'big data' for data driven research in the future, and already at present allows for precise prediction of reaction outcomes within the high-dimensional chemical parameter space that is screened.

Gradient-Enhanced Kriging for High-Dimensional Bayesian Optimization with Linear Embedding

This paper explores the application of gradient-enhanced (GE) kriging for Bayesian optimization (BO) problems with a high-dimensional parameter space. We utilize the active subspace method to embed the original parameter space in a low-dimensional subspace. The active subspace is detected by analyzing the spectrum of the empirical second-moment matrix of the gradients of the response function. By mapping the training data onto their respective subspace, the objective function and the constraint functions are efficiently approximated with low-dimensional GE-kriging models. In each cycle of the BO procedure, a new point is found by maximizing the constrained expected improvement function within a low-dimensional polytope, and it is mapped back to the original space for model evaluation. In this way, the computational costs are significantly reduced when compared with standard GE-kriging. We illustrate and assess the proposed approach with numerical experiments ranging from academic benchmark problems to aerodynamic engineering applications. The experimental results show that the proposed method is promising for optimizing high-dimensional expensive objective functions, especially for problems that exhibit a clear low-dimensional active subspace.

Multivariate control charts for calibration of hydrophones using the Mahalanobis statistic

Statistical analysis of data to ensure the validity of calibrations can be challenged when the measurement model describes a process and result that spans a high-dimensional parameter space. Calibration of hydrophones presents one such case whereby the sensitivity to acoustic pressure is calculated using measured values for several different quantities and the result is needed at many excitation frequencies distributed over a large bandwidth. We addressed this challenge by developing a multivariate statistical analysis procedure that supports efficient and effective process control for hydrophone calibrations. We first characterize the in-control condition by the mean and covariance of measurements expressed in a p-dimensional parameter space using a curated training data set. We then test subsequent measurements against a control limit using Hotelling’s T 2-test to compare the current measurement process to the training data. We report a case study for secondary calibration of hydrophones spanning a frequency range of 1 kHz–200 kHz where we detected an emergent defect in the laboratory’s calibrated reference hydrophone by inspection of data presented in multivariate control charts. Process control data collected during calibration services exceeded the control limits in intermediate and high-frequency bands by factors of 5.1 and 11.2, respectively. The process remained in control for the low-frequency band. These observations were consistent with misalignment of the reference hydrophone’s sensitive element, which we confirmed upon disassembly, inspection and repair of the instrument.

Augmenting Deep Residual Surrogates with Fourier Neural Operators for Rapid Two-Phase Flow and Transport Simulations

Summary Accurate numerical modeling of multiphase flow and transport mechanisms is essential to study varied, complex physical phenomena including flow in subsurface oil and gas reservoirs and subsurface aquifers subject to CO2 sequestration. State-of-the-art complete physics-based solvers suffer from many computational challenges. High-fidelity data-driven surrogate models that solve the governing partial differential equations (PDEs) have the potential to optimize the time to solution and increase confidence in critical business and engineering decisions through better quantification of solution statistics. We leverage the recently proposed Fourier neural operators (FNOs) with quasilinear time complexity to capture the spectral information from feature maps to solve the coupled porous flow and transport PDEs. Embedding Fourier layers within the residual blocks results in a highly effective structure that, while achieving competitive accuracy, also enables efficient training of deeper networks with a dramatically reduced number of trainable parameters. The resulting novel deep-learning (DL) architecture is coined as FResNet++. FResNet++ uses squeeze and excitation blocks, atrous spatial pyramid pooling (ASPP), and attention blocks to increase its sensitivity to the relevant features and capture multiscale information, and it is specifically tuned to operate optimally to learn from and predict numerically simulated flow (pressure and saturation) fields. We demonstrate the ability of FResNet++ to generalize over multiple high-dimensional input parameter spaces that describe subsurface permeability and porosity heterogeneity. The resulting DL architecture accurately captures the complex interplay between viscous forces and highly heterogeneous permeability and porosity fields. We investigate two-phase flow in porous media, which is the archetypal problem for reservoir simulation giving rise to a system of nonlinearly coupled PDEs with highly heterogeneous coefficients. We show in blind tests that FResNet++ predicts saturation fields more accurately compared to ResU-Net and original FNO with fully connected linear layers. We additionally investigate the effects of using alternative loss functions and an alternative way of utilizing FResNet++ to increase its effectiveness. For the first time in the literature, we show that the spatiotemporal evolution of pressure and saturation fields can be jointly predicted with good accuracy using a single FResNet++ network over long time horizons in response to previously unseen permeability and porosity fields. After a moderate training investment on graphics processing units (GPUs), FResNet++ yields a speedup of at least four orders of magnitude compared to a conventional numerical PDE solver and operates with notably fewer trainable parameters compared to the original FNO. Our numerical experiments validate that FNOs can be utilized in various convolutional neural network-based architectures and can effectively substitute for repetitive physics-based forward simulations for scenario testing.