Empirical Model Reduction Research Articles

Robustness improvement of optimal control in terms of RBFNN with empirical model reduction and transfer learning

This paper proposes a method to compute solutions of optimal controls for dynamic systems in terms of radial basis function neural networks (RBFNN) with Gaussian neurons. The RBFNN is used to compute the value function from Hamilton–Jacobi–Bellman equation with the policy iteration. The concept of dominant system is introduced to create initial coefficients of the neural networks to stabilise unstable systems and guarantee the convergence of policy iteration. Model reduction and transfer learning techniques are introduced to improve robustness of RBFNN optimal control, and reduce computational time. Numerical and experimental results show that the resulting optimal control has excellent control performance in stabilisation and trajectory tracking, and much-improved robustness to disturbances and model uncertainties even when the system responses move to a domain of the state space that is larger than the domain where the neural networks are trained.

Topological dissimilarities of hierarchical resting networks in type 2 diabetes mellitus and obesity

Type 2 diabetes mellitus (T2DM) is reported to cause widespread changes in brain function, leading to cognitive impairments. Research using resting-state functional magnetic resonance imaging data already aims to understand functional changes in complex brain connectivity systems. However, no previous studies with dynamic causal modelling (DCM) tried to investigate large-scale effective connectivity in diabetes. We aimed to examine the differences in large-scale resting state networks in diabetic and obese patients using combined DCM and graph theory methodologies. With the participation of 70 subjects (43 diabetics, 27 obese), we used cross-spectra DCM to estimate connectivity between 36 regions, subdivided into seven resting networks (RSN) commonly recognized in the literature. We assessed group-wise connectivity of T2DM and obesity, as well as group differences, with parametric empirical Bayes and Bayesian model reduction techniques. We analyzed network connectivity globally, between RSNs, and regionally. We found that average connection strength was higher in T2DM globally and between RSNs, as well. On the network level, the salience network shows stronger total within-network connectivity in diabetes (8.07) than in the obese group (4.02). Regionally, we measured the most significant average decrease in the right middle temporal gyrus (-0.013 Hz) and the right inferior parietal lobule (-0.01 Hz) relative to the obese group. In comparison, connectivity increased most notably in the left anterior prefrontal cortex (0.01 Hz) and the medial dorsal thalamus (0.009 Hz). In conclusion, we find the usage of complex analysis of large-scale networks suitable for diabetes instead of focusing on specific changes in brain function.

Imbalanced basal ganglia connectivity is associated with motor deficits and apathy in Huntington’s disease

The gating of movement depends on activity within the cortico-striato-thalamic loops. Within these loops, emerging from the cells of the striatum, run two opponent pathways—the direct and indirect basal ganglia pathways. Both are complex and polysynaptic, but the overall effect of activity within these pathways is thought to encourage and inhibit movement, respectively. In Huntington’s disease, the preferential early loss of striatal neurons forming the indirect pathway is thought to lead to disinhibition, giving rise to the characteristic motor features of the condition. But early Huntington’s disease is also associated with apathy, a loss of motivation and failure to engage in goal-directed movement. We hypothesized that in Huntington’s disease, motor signs and apathy may be selectively correlated with indirect and direct pathway dysfunction, respectively.We used spectral dynamic casual modelling of resting-state functional MRI data to model effective connectivity in a model of these cortico-striatal pathways. We tested both of these hypotheses in vivo for the first time in a large cohort of patients with prodromal Huntington’s disease. Using an advanced approach at the group level we combined parametric empirical Bayes and Bayesian model reduction procedures to generate a large number of competing models and compare them using Bayesian model comparison. With this automated Bayesian approach, associations between clinical measures and connectivity parameters emerge de novo from the data.We found very strong evidence (posterior probability > 0.99) to support both of our hypotheses. First, more severe motor signs in Huntington’s disease were associated with altered connectivity in the indirect pathway components of our model and, by comparison, loss of goal-direct behaviour or apathy, was associated with changes in the direct pathway component.The empirical evidence we provide here demonstrates that imbalanced basal ganglia connectivity may play an important role in the pathogenesis of some of commonest and disabling features of Huntington’s disease and may have important implications for therapeutics.

Reduced-order models for coupled dynamical systems: Data-driven methods and the Koopman operator.

Providing efficient and accurate parameterizations for model reduction is a key goal in many areas of science and technology. Here, we present a strong link between data-driven and theoretical approaches to achieving this goal. Formal perturbation expansions of the Koopman operator allow us to derive general stochastic parameterizations of weakly coupled dynamical systems. Such parameterizations yield a set of stochastic integrodifferential equations with explicit noise and memory kernel formulas to describe the effects of unresolved variables. We show that the perturbation expansions involved need not be truncated when the coupling is additive. The unwieldy integrodifferential equations can be recast as a simpler multilevel Markovian model, and we establish an intuitive connection with a generalized Langevin equation. This connection helps setting up a parallelism between the top-down, equation-based methodology herein and the well-established empirical model reduction (EMR) methodology that has been shown to provide efficient dynamical closures to partially observed systems. Hence, our findings, on the one hand, support the physical basis and robustness of the EMR methodology and, on the other hand, illustrate the practical relevance of the perturbative expansion used for deriving the parameterizations.

Reconstructing complex system dynamics from time series: a method comparison

Modeling complex systems with large numbers of degrees of freedom has become a grand challenge over the past decades. In many situations, only a few variables are actually observed in terms of measured time series, while the majority of variables—which potentially interact with the observed ones—remain hidden. A typical approach is then to focus on the comparably few observed, macroscopic variables, assuming that they determine the key dynamics of the system, while the remaining ones are represented by noise. This naturally leads to an approximate, inverse modeling of such systems in terms of stochastic differential equations (SDEs), with great potential for applications from biology to finance and Earth system dynamics. A well-known approach to retrieve such SDEs from small sets of observed time series is to reconstruct the drift and diffusion terms of a Langevin equation from the data-derived Kramers–Moyal (KM) coefficients. For systems where interactions between the observed and the unobserved variables are crucial, the Mori–Zwanzig formalism (MZ) allows to derive generalized Langevin equations that contain non-Markovian terms representing these interactions. In a similar spirit, the empirical model reduction (EMR) approach has more recently been introduced. In this work we attempt to reconstruct the dynamical equations of motion of both synthetical and real-world processes, by comparing these three approaches in terms of their capability to reconstruct the dynamics and statistics of the underlying systems. Through rigorous investigation of several synthetical and real-world systems, we confirm that the performance of the three methods strongly depends on the intrinsic dynamics of the system at hand. For instance, statistical properties of systems exhibiting weak history-dependence but strong state-dependence of the noise forcing, can be approximated better by the KM method than by the MZ and EMR approaches. In such situations, the KM method is of a considerable advantage since it can directly approximate the state-dependent noise. However, limitations of the KM approximation arise in cases where non-Markovian effects are crucial in the dynamics of the system. In these situations, our numerical results indicate that methods that take into account interactions between observed and unobserved variables in terms of non-Markovian closure terms (i.e., the MZ and EMR approaches), perform comparatively better.

Data-Driven Model Reduction of Monotone Systems by Nonlinear DC Gains

In this paper, we develop data-driven model reduction methods for monotone nonlinear control systems based on a nonlinear version of the dc gain. The nonlinear dc gain is a function of the amplitude of the input and can be used to evaluate the importance of each state variable. In fact, the nonlinear dc gain is directly related to the infinity-induced norm of the system as well as a notion of output reachability. Given the dc gain, model reduction is performed by either truncating not-so-important state variables or aggregating state variables having similar importance. Under such truncation and clustering, monotonicity and boundedness of the nonlinear dc gain are preserved; moreover, these two operations can be approximately performed based on simulation or experimental data alone. This empirical model reduction approach is illustrated by an example of a gene regulatory network.

Diversity, Nonlinearity, Seasonality, and Memory Effect in ENSO Simulation and Prediction Using Empirical Model Reduction

Abstract A suite of empirical model experiments under the empirical model reduction framework are conducted to advance the understanding of ENSO diversity, nonlinearity, seasonality, and the memory effect in the simulation and prediction of tropical Pacific sea surface temperature (SST) anomalies. The model training and evaluation are carried out using 4000-yr preindustrial control simulation data from the coupled model GFDL CM2.1. The results show that multivariate models with tropical Pacific subsurface information and multilevel models with SST history information both improve the prediction skill dramatically. These two types of models represent the ENSO memory effect based on either the recharge oscillator or the time-delayed oscillator viewpoint. Multilevel SST models are a bit more efficient, requiring fewer model coefficients. Nonlinearity is found necessary to reproduce the ENSO diversity feature for extreme events. The nonlinear models reconstruct the skewed probability density function of SST anomalies and improve the prediction of the skewed amplitude, though the role of nonlinearity may be slightly overestimated given the strong nonlinear ENSO in GFDL CM2.1. The models with periodic terms reproduce the SST seasonal phase locking but do not improve the prediction appreciably. The models with multiple ingredients capture several ENSO characteristics simultaneously and exhibit overall better prediction skill for more diverse target patterns. In particular, they alleviate the spring/autumn prediction barrier and reduce the tendency for predicted values to lag the target month value.

Data-driven non-Markovian closure models

This paper has two interrelated foci: (i) obtaining stable and efficient data-driven closure models by using a multivariate time series of partial observations from a large-dimensional system; and (ii) comparing these closure models with the optimal closures predicted by the Mori–Zwanzig (MZ) formalism of statistical physics. Multilayer stochastic models (MSMs) are introduced as both a generalization and a time-continuous limit of existing multilevel, regression-based approaches to closure in a data-driven setting; these approaches include empirical model reduction (EMR), as well as more recent multi-layer modeling. It is shown that the multilayer structure of MSMs can provide a natural Markov approximation to the generalized Langevin equation (GLE) of the MZ formalism.A simple correlation-based stopping criterion for an EMR–MSM model is derived to assess how well it approximates the GLE solution. Sufficient conditions are derived on the structure of the nonlinear cross-interactions between the constitutive layers of a given MSM to guarantee the existence of a global random attractor. This existence ensures that no blow-up can occur for a broad class of MSM applications, a class that includes non-polynomial predictors and nonlinearities that do not necessarily preserve quadratic energy invariants.The EMR–MSM methodology is first applied to a conceptual, nonlinear, stochastic climate model of coupled slow and fast variables, in which only slow variables are observed. It is shown that the resulting closure model with energy-conserving nonlinearities efficiently captures the main statistical features of the slow variables, even when there is no formal scale separation and the fast variables are quite energetic.Second, an MSM is shown to successfully reproduce the statistics of a partially observed, generalized Lotka–Volterra model of population dynamics in its chaotic regime. The challenges here include the rarity of strange attractors in the model’s parameter space and the existence of multiple attractor basins with fractal boundaries. The positivity constraint on the solutions’ components replaces here the quadratic-energy–preserving constraint of fluid-flow problems and it successfully prevents blow-up.

Low‐order stochastic model and “past‐noise forecasting” of the Madden‐Julian Oscillation

This paper presents a predictability study of the Madden‐Julian Oscillation (MJO) that relies on combining empirical model reduction (EMR) with the “past‐noise forecasting” (PNF) method. EMR is a data‐driven methodology for constructing stochastic low‐dimensional models that account for nonlinearity, seasonality and serial correlation in the estimated noise, while PNF constructs an ensemble of forecasts that accounts for interactions between (i) high‐frequency variability (noise), estimated here by EMR, and (ii) the low‐frequency mode of MJO, as captured by singular spectrum analysis (SSA). A key result is that—compared to an EMR ensemble driven by generic white noise—PNF is able to considerably improve prediction of MJO phase. When forecasts are initiated from weak MJO conditions, the useful skill is of up to 30 days. PNF also significantly improves MJO prediction skill for forecasts that start over the Indian Ocean.

A self‐organizing map–based ensemble forecast system for extended range prediction of active/break cycles of Indian summer monsoon

The paper describes a probabilistic prediction scheme of the intraseasonal oscillation of Indian summer monsoon (ISM) in the extended range (ER, ~3–4weeks) using a self‐organizing map (SOM)‐based technique. SOM is used to derive a set of patterns through empirical model reduction. An ensemble method of forecast is then developed for these reduced modes based on the principle of analogue prediction. A total of 900 ensembles is created based on the variations of one of the parameters, like length of the observation sample, number of patterns, number of lags, and number of input variables, keeping the others constant. Deterministic correlation skill at fourth pentad lead (15–20 days) from the current model is 0.47 (for development period, 1951–1999) and 0.43 (for hindcast period, 2000–2011) over the monsoon zone of India. This method effectively takes care of the stochastic uncertainties associated with a deterministic prediction scheme and provides better guidance to the user community. A large part of the uncertainty in the model's prediction skill is related to the interannual variability of the prediction skill of the active‐break spells. The model has problems in forecasting the unusually long active/break spells during the monsoon season, especially during September. Forecasts from certain initial conditions are less predictable than those from others. We describe some probable mechanisms from the literature for such problems in the model. This study will provide a benchmark to evaluate dynamical models' skills in predicting the ISM in ER time scale in future.

Predicting stochastic systems by noise sampling, and application to the El Niño-Southern Oscillation

Interannual and interdecadal prediction are major challenges of climate dynamics. In this article we develop a prediction method for climate processes that exhibit low-frequency variability (LFV). The method constructs a nonlinear stochastic model from past observations and estimates a path of the "weather" noise that drives this model over previous finite-time windows. The method has two steps: (i) select noise samples--or "snippets"--from the past noise, which have forced the system during short-time intervals that resemble the LFV phase just preceding the currently observed state; and (ii) use these snippets to drive the system from the current state into the future. The method is placed in the framework of pathwise linear-response theory and is then applied to an El Niño-Southern Oscillation (ENSO) model derived by the empirical model reduction (EMR) methodology; this nonlinear model has 40 coupled, slow, and fast variables. The domain of validity of this forecasting procedure depends on the nature of the system's pathwise response; it is shown numerically that the ENSO model's response is linear on interannual time scales. As a result, the method's skill at a 6- to 16-month lead is highly competitive when compared with currently used dynamic and statistic prediction methods for the Niño-3 index and the global sea surface temperature field.

Signatures of Nonlinear Dynamics in an Idealized Atmospheric Model

Abstract Signatures of nonlinear dynamics are analyzed by studying the phase-space tendencies of a global baroclinic, quasigeostrophic, three-level (QG3) model with topography. Nonlinear, stochastic, low-order prototypes of the full QG3 model are constructed in the phase space of this model’s empirical orthogonal functions using the empirical model reduction (EMR) approach. The phase-space tendencies of the EMR models closely match the full QG3 model’s tendencies. The component of these tendencies that is not linearly parameterizable is shown to be dominated by the interactions between “resolved” modes rather than by multiplicative “noise” associated with unresolved modes. The method of defining the leading resolved modes and the interactions between them plays a key role in understanding the nature of the QG3 model’s dynamics, whether linear or nonlinear, deterministic or stochastic.

Empirical Model Reduction of Geometrical Constrained Gossamer Structures

The purpose of this research paper is to construct a low-order model of a geometrical constrained gossamer structure, especially for a solar sail. Currently, JAXA is working on a spin type solar power sail spacecraft. Computational analysis of the sail membrane's large mathematical model is difficult and time consuming. Therefore, this issue has a negative impact on design aspects. As a result, the model reduction technique is required. In other words, with a feasible, shortened, computational analysis period, a trade-off would be possible when designing and determining operation procedures. However, there is no model reduction methodology about solar sail dynamics. In this research paper, firstly, the full-order model of the spin type solar sail craft is modeled by the geometrically nonlinear FEM based on the energy momentum method (EMM). As a result, the numerical time integration is unconditionally stable. Secondly, the low-order model is constructed by the empirical model reduction method. And finally and complementarily to the former and latter sentence, the geometrical constraint is modeled by the penalty method based on the EMM. In this research paper, we will show that the low-order model can approximate the full-order model with sufficient accuracy.

Empirical pseudo-balanced model reduction and feedback control of weakly nonlinear convection patterns

An empirical model reduction method is performed on nonlinear transient convection patterns near the threshold, using a pseudo-inverse-based projective method called the pseudo-balanced proper orthogonal decomposition (PBPOD). These transient patterns are large-scale amplitude/phase modulations in convection rolls, obtained by prescribing selected spatial input-shape functions. For the nonlinear convection patterns modelled, PBPOD appears to be very effective. Using the nonlinear front example, PBPOD is compared with other existing methods, such as POD and linearized BPOD. The limitations of the methods are discussed. Using a complex prescribed input, complex disturbances are generated and the outputs of the open-loop responses are compared between the original and the reduced-order models. The agreement is good. A feedback-control study is performed using the nonlinear front example. The controller is built by the pseudo-balanced reduced-order model, with a low-order nonlinear estimator. Closed-loop simulations show that the nonlinear travelling fronts can be effectively damped out by the feedback-control actions.

Some guidelines for structural equation modelling in cognitive neuroscience: The case of Charlton et al.’s study on white matter integrity and cognitive ageing

Charlton et al. (2008) (Charlton, R.A., Landua, S., Schiavone, F., Barrick, T.R., Clark, C.A., Markus, H.S., Morris, R.G.A., 2008. Structural equation modelling investigation of age-related variance in executive function and DTI-measured white matter change. Neurobiol. Aging 29, 1547-1555) presented a model that suggests a specific age-related effect of white matter integrity on working memory. We illustrate potential pitfalls of structural equation modelling by criticizing their model for (a) its neglect of latent variables, (b) its complexity, (c) its questionable causal assumptions, (d) the use of empirical model reduction, (e) the mix-up of theoretical perspectives, and (f) the failure to compare alternative models. We show that a more parsimonious model, based solely on the well-established general factor of cognitive ability, fits their data at least as well. Importantly, when modelled this way there is no support for a role of white matter integrity in cognitive aging in this sample, indicating that their conclusion is strongly dependent on how the data are analysed. We suggest that evidence from more conclusive study designs is needed.

Empirical Model Reduction of Spinning Solar Sail

Empirical Model Reduction of Spinning Solar Sail

F5 ゴサマー宇宙構造物の経験的モデル低次元化(宇宙構造(1))

F5 ゴサマー宇宙構造物の経験的モデル低次元化(宇宙構造(1))

Explaining de facto judicial independence

A high degree of de facto judicial independence (JI) functions as a crucial precondition of governments to credibly commit to legislative decisions, such as respecting private property rights. Thus, de facto JI should improve the allocative efficiency and may therefore contribute positively to economic growth. But JI as formally written down in legal texts is an imperfect predictor for de facto JI. This paper tries to identify the forces which determine de facto JI. A distinction between factors that can be influenced in the short run and those that are the result of historical development and that are exempt from short-term modification is made. Ascertaining the relative relevance of these two groups of variables promises to be policy-relevant. A rigorous empirical model reduction process is used in order to cope with the potential excess of explanatory variables. The explanatory variables for de facto JI that survive the reduction process are de jure JI, legal confidence of the public, extent of democratization, degree of press freedom, and the religious beliefs of the population.

Empirical model reduction of controlled nonlinear systems

In this paper we introduce a new method of model reduction for nonlinear systems with inputs and outputs. The method requires only standard matrix computations, and when applied to linear systems results in the usual balanced truncation. For nonlinear systems, the method makes used of the Karhunen-Lo`eve decomposition of the state-space, and is an extension of the method of empirical eigenfunctions used in fluid dynamics. We show that the new method is equivalent to balanced-truncation in the linear case, and perform an example reduction for a nonlinear mechanical system.