Region Of State Space Research Articles

Generalization of neural network models for complex network dynamics

Differential equations are a ubiquitous tool to study dynamics, ranging from physical systems to complex systems, where a large number of agents interact through a graph. Data-driven approximations of differential equations present a promising alternative to traditional methods for uncovering a model of dynamical systems, especially in complex systems that lack explicit first principles. A recently employed machine learning tool for studying dynamics is neural networks, which can be used for solution finding or discovery of differential equations. However, deploying deep learning models in unfamiliar settings-such as predicting dynamics in unobserved state space regions or on novel graphs-can lead to spurious results. Focusing on complex systems whose dynamics are described with a system of first-order differential equations coupled through a graph, we study generalization of neural network predictions in settings where statistical properties of test data and training data are different. We find that neural networks can accurately predict dynamics beyond the immediate training setting within the domain of the training data. To identify when a model is unable to generalize to novel settings, we propose a statistical significance test.

Control of a Directional Downhole Drilling System Using a State Barrier Avoidance Based Method

Abstract Vibrations such as bit-bouncing, stick–slip, and whirl motion can significantly reduce downhole drilling's performance and efficiency. These phenomena are likely to arise once the dynamical states of drilling fall into undesired operating regimes, as verified by both theoretical analyses and experimental studies. To effectively avoid such undesired operating conditions of a downhole drilling process, this paper introduces an advanced nonlinear state-constrained control method to formulate a setpoint tracking problem of high-order directional drilling dynamics under state constraints. These constraints are carefully defined using the field test results, and their shapes are in complex state-space regions, causing additional complexity to the control design. Moreover, unlike vertical drilling, the directional drilling system requires modeling of coupled axial and torsional dynamics with a higher degree-of-freedom (DOF). In this study, the proposed method will tackle these challenges by converting the original high-order constrained control problem into a standard nonlinear control problem. Leveraging on the linear parameter varying (LPV) method that is applied to the converted system, we can actively prevent drilling from falling into the undesired regimes, to achieve the constrained control objective.

Traffic navigation via reinforcement learning with episodic-guided prioritized experience replay

Deep Reinforcement Learning (DRL) models play a fundamental role in autonomous driving applications; however, they typically suffer from sample inefficiency because they often require many interactions with the environment to learn effective policies. This makes the training process time-consuming. To address this shortcoming, Prioritized Experience Replay (PER) has proven to be effective by prioritizing samples with high Temporal-Difference (TD) error for learning. In this context, this study contributes to artificial intelligence by proposing a sample-efficient DRL algorithm called Episodic-Guided Prioritized Experience Replay (EPER). The core innovation of EPER lies in the utilization of an episodic memory, dedicated to storing successful training episodes. Within this memory, expected returns for each state–action pair are extracted. These returns, combined with TD error-based prioritization, form a novel objective function for deep Q-network training. To prevent excessive determinism, EPER introduces exploration into the learning process by incorporating a regularization term into the objective function that allows exploration of state-space regions with diverse Q-values. The proposed EPER algorithm is suitable to train a DRL agent for handling episodic tasks, and it can be integrated into off-policy DRL models. EPER is employed for traffic navigation through scenarios such as highway driving, merging, roundabout, and intersection to showcase its application in engineering. The attained results denote that, compared with the PER and an additional state-of-the-art training technique, EPER is superior in expediting the training of the agent and learning a more optimal policy that leads to lower collision rates within the constructed navigation scenarios.

Data‐driven variational method for discrepancy modeling: Dynamics with small‐strain nonlinear elasticity and viscoelasticity

AbstractThe effective inclusion of a priori knowledge when embedding known data in physics‐based models of dynamical systems can ensure that the reconstructed model respects physical principles, while simultaneously improving the accuracy of the solution in the previously unseen regions of state space. This paper presents a physics‐constrained data‐driven discrepancy modeling method that variationally embeds known data in the modeling framework. The hierarchical structure of the method yields fine scale variational equations that facilitate the derivation of residuals which are comprised of the first‐principles theory and sensor‐based data from the dynamical system. The embedding of the sensor data via residual terms leads to discrepancy‐informed closure models that yield a method which is driven not only by boundary and initial conditions, but also by measurements that are taken at only a few observation points in the target system. Specifically, the data‐embedding term serves as residual‐based least‐squares loss function, thus retaining variational consistency. Another important relation arises from the interpretation of the stabilization tensor as a kernel function, thereby incorporating a priori knowledge of the problem and adding computational intelligence to the modeling framework. Numerical test cases show that when known data is taken into account, the data driven variational (DDV) method can correctly predict the system response in the presence of several types of discrepancies. Specifically, the damped solution and correct energy time histories are recovered by including known data in the undamped situation. Morlet wavelet analyses reveal that the surrogate problem with embedded data recovers the fundamental frequency band of the target system. The enhanced stability and accuracy of the DDV method is manifested via reconstructed displacement and velocity fields that yield time histories of strain and kinetic energies which match the target systems. The proposed DDV method also serves as a procedure for restoring eigenvalues and eigenvectors of a deficient dynamical system when known data is taken into account, as shown in the numerical test cases presented here.

Transient Amplification of Broken Symmetry in Elastic Snap-Through.

A snap-through bifurcation occurs when a bistable structure loses one of its stable states and moves rapidly to the remaining state. For example, a buckled arch with symmetrically clamped ends can snap between an inverted and a natural state as the ends are released. A standard linear stability analysis suggests that the arch becomes unstable to asymmetric perturbations. Surprisingly, our experiments show that this is not always the case: symmetric transitions are also observed. Using experiments, numerics, and a toy model, we show that the symmetry of the transition depends on the rate at which the ends are released, with sufficiently fast loading leading to symmetric snap-through. Our toy model reveals that this behavior is caused by a region of the system's state space in which any initial asymmetry is amplified. The system may not enter this region when loaded fast (hence remaining symmetric), but will traverse it for some interval of time when loaded slowly, causing a transient amplification of asymmetry. Our toy model suggests that this behavior is not unique to snapping arches, but rather can be observed in dynamical systems where both a saddle-node and a pitchfork bifurcation occur in close proximity.

Sequential Monte Carlo with Adaptive Lookahead Support for Improved Importance Sampling

The sequential Monte Carlo, also called the Bayesian particle filter, approximates a posterior probability density function of a latent target state from noisy sensor measurements using a set of Monte Carlo samples. These samples are predicted using an importance density function and then updated using the Bayes’s rule. The updated samples and their corresponding weights provide an estimate of the latent state. The said filtering process is iterated over time for tracking dynamic target states. It is critical to have enough particles in regions of the target state space that contribute to the posterior. The auxiliary and the improved auxiliary particle filters accomplish this by a process that mimics drawing from an importance density that leverages the incoming observation into the sampling step. However these filters are known to fail when the sensor measurements are highly informative and the diffusion over the state transition is large. This paper presents an improvement to the auxiliary particle filter by taking two support points that act as limits in a univariate state space within which particles are samples. The choice of the limits is adaptive. The proposed method is successfully tested using a nonlinear model using simulations.

Hybrid finite-amplitude periodic modes for two uniformly magnetized spheres.

We analyze a system of two uniformly magnetized spheres, one fixed and the other free to slide in frictionless contact with the surface of the first. The centers of the two magnets, and their magnetic moments, are restricted to a plane. We search for sets of initial conditions that yield finite-amplitude oscillatory periodic solutions. We extend two small-amplitude base modes, one with orbital and spin motions that are in phase and the other out of phase, to finite amplitudes and show that the motion for arbitrary oscillatory solutions can be considered to be a nonlinear superposition of these base modes. Some solutions are pure periodic finite-amplitude extensions of one base mode, while others are hybrid finite-amplitude superpositions of the two modes. Hybrid modes with rational frequency ratios are periodic and come in families defined by their frequency ratios. We further characterize hybrid periodic modes by identifying two symmetry classes that describe their relative phases. We see continuous transitions between one finite-amplitude base mode and the other, with one mode gradually transforming into the other. We also calculate frequency spectra of nonperiodic modes, show that the two base modes have well-defined frequencies even for nonperiodic states, and show that periodic solutions can give clues about the behavior of nearby nonperiodic solutions. In the limit of small amplitudes, we confirm that the computed frequencies of these modes agree with small-amplitude analytical results. We also generate a Lyapunov exponent heatmap that reflects periodic and nonperiodic regions of state space.

Salience Interest Option: Temporal abstraction with salience interest functions

Reinforcement Learning (RL) is a significant machine learning subfield that emphasizes learning actions based on environment to obtain optimal behavior policy. RL agents can make decisions at variable time scales in the form of temporal abstractions, also known as options. The issue of discovering options has seen a considerable research effort. Most notably, the Interest Option Critic (IOC) algorithm first extends the initial set to the interest function, providing a method for learning options specialized to certain state space regions. This approach offers a specific attention mechanism for action selection. Unfortunately, this method still suffers from the classic issues of poor data efficiency and lack of flexibility in RL when learning options end-to-end through backpropagation. This paper proposes a new approach called Salience Interest Option Critic (SIOC), which chooses subsets of existing initiation sets for RL. Specifically, these subsets are not learned by backpropagation, which is slow and tends to overfit, but through particle filters. This approach enables the rapid and flexible identification of critical subsets using only reward feedback. We conducted experiments in discrete and continuous domains, and our proposed method demonstrate higher efficiency and flexibility than other methods. The generated options are more valuable within a single task and exhibited greater interpretability and reusability in multi-task learning scenarios.

The controllosphere: The neural origin of cognitive effort.

Why do some mental activities feel harder than others? The answer to this question is surprisingly controversial. Current theories propose that cognitive effort affords a computational benefit, such as instigating a switch from an activity with low reward value to a different activity with higher reward value. By contrast, in this article, I relate cognitive effort to the fact that brain neuroanatomy and neurophysiology render some neural states more energy-efficient than others. I introduce the concept of the "controllosphere," an energy-inefficient region of neural state space associated with high control, which surrounds the better known "intrinsic manifold," an energy-efficient subspace associated with low control. Integration of control-theoretic principles with classic neurocomputational models of cognitive control suggests that dorsolateral prefrontal cortex (DLPFC) implements a controller that can drive the system state into the controllosphere, anterior cingulate cortex (ACC) implements an observer that monitors changes of state of the controlled system, and cognitive effort reflects a mismatch between DLPFC and ACC energies for control and observation. On this account, cognitive effort scales with the energetic demands of the DLPFC control signal, especially when the consequences of the control are unobservable by ACC. Further, I propose that neural transitions through the controllosphere lead to a buildup of neural waste. Cognitive effort therefore prevents against neural damage by discouraging extended periods of high control. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Discrete-time systems properties defined on state-space regions

In this paper various properties of discrete-time dynamic control systems trajectories with respect to state-space corner regions are considered. The known notion of state-space invariance serves as a basis for derivation of the whole family of dynamics behaviors for which both necessary and sufficient conditions are derived in a general nonlinear case as well as in the linear time-invariant case, shortly LTI-case. Specific examples are given for every case considered, and the proposed notions are analyzed with both theoretical and practical usefulness in mind. For the general nonlinear case a geometric approach is used, which provides a direct insight into the nature of trajectories’ behavior. In the LTI-case a geometric approach is used as well, but it is also translated into the purely algebraic set of conditions allowing for a direct analysis of system matrices. The presented family of control systems expands on the classical state-space invariance (and positive systems) analysis, thus potentially opening new research venues in this branch of control theory and system dynamics in general.

Safe Reach Set Computation via Neural Barrier Certificates

We present a novel technique for online safety verification of autonomous systems, which performs reachability analysis efficiently for both bounded and unbounded horizons by employing neural barrier certificates. Our approach uses barrier certificates given by parameterized neural networks that depend on a given initial set, unsafe sets, and time horizon. Such networks are trained efficiently offline using system simulations sampled from regions of the state space. We then employ a meta-neural network to generalize the barrier certificates to state-space regions that are outside the training set. These certificates are generated and validated online as sound over-approximations of the reachable states, thus either ensuring system safety or activating appropriate alternative actions in unsafe scenarios. We demonstrate our technique on case studies from linear models to nonlinear control-dependent models for online autonomous driving scenarios.

Comparison of confidence regions for quantum state tomography

Abstract The quantum state associated to an unknown experimental preparation procedure can be determined by performing quantum state tomography. If the statistical uncertainty in the data dominates over other experimental errors, then a tomographic reconstruction procedure must express this uncertainty. A rigorous way to accomplish this is via statistical confidence regions (CRs) in state space. Naturally, the size of this region decreases when increasing the number of samples, but it also depends critically on the construction method of the region. We compare recent methods for constructing CRs as well as a reference method based on a Gaussian approximation. For the comparison, we propose an operational measure with the finding, that there is a significant difference between methods, but which method is preferable can depend on the details of the state preparation scenario.

DSMC Evaluation Stages: Fostering Robust and Safe Behavior in Deep Reinforcement Learning – Extended Version

Neural networks (NN) are gaining importance in sequential decision-making. Deep reinforcement learning (DRL), in particular, is extremely successful in learning action policies in complex and dynamic environments. Despite this success, however, DRL technology is not without its failures, especially in safety-critical applications: (i) the training objective maximizes average rewards, which may disregard rare but critical situations and hence lack local robustness; (ii) optimization objectives targeting safety typically yield degenerated reward structures, which, for DRL to work, must be replaced with proxy objectives. Here, we introduce a methodology that can help to address both deficiencies. We incorporate evaluation stages (ES) into DRL, leveraging recent work on deep statistical model checking (DSMC), which verifies NN policies in Markov decision processes. Our ES apply DSMC at regular intervals to determine state space regions with weak performance. We adapt the subsequent DRL training priorities based on the outcome, (i) focusing DRL on critical situations and (ii) allowing to foster arbitrary objectives. We run case studies on two benchmarks. One of them is the Racetrack, an abstraction of autonomous driving that requires navigating a map without crashing into a wall. The other is MiniGrid, a widely used benchmark in the AI community. Our results show that DSMC-based ES can significantly improve both (i) and (ii).

Distributed Bayesian target tracking with reduced communication: Likelihood consensus 2.0

The likelihood consensus (LC) enables Bayesian target tracking in a decentralized sensor network with possibly nonlinear and non-Gaussian sensor characteristics. Here, we propose an evolved LC methodology – dubbed LC 2.0 – with significantly reduced intersensor communication. LC 2.0 uses multiple refinements of the original LC including a sparsity-promoting calculation of expansion coefficients, the use of a B-spline dictionary, a distributed adaptive calculation of the relevant state-space region, and efficient binary representations. We consider the use of the proposed LC 2.0 within a distributed particle filter and within a distributed particle-based probabilistic data association filter. Our simulation results demonstrate that a reduction of intersensor communication by a factor of about 190 can be obtained without compromising the tracking performance.

Segmentation of high-speed flow fields using physics-informed clustering

The advent of data-based modeling has provided new methods and algorithms for analyzing the complex flow fields in high-speed combustion applications. These techniques can be used to study the fluid dynamics and reaction progress in different regions of the flow, which can provide insight into the underlying physics of the system while helping one identify avenues for the development and application of new models. The present work implements two such algorithms – the standard K-means clustering algorithm and a modified Jacobian-scaled K-means clustering algorithm – to analyze a high-speed reacting flow field. The results show that the Jacobian-scaled K-means algorithm allocates more clusters in the regions of the thermochemical state space where chemical source term is highest. In physical space, this concentrates the flow partitions in regions where heat release rate is higher and chemical timescales are lower. This is observed for different cluster numbers and data sets. When moving between data sets that occupy different portions of the state space, the cluster centroid locations are found to be less sensitive to the data set when the Jacobian-scaled K-means algorithm is used. Altogether, the modified K-means algorithm provides a new tool for identifying thermodynamic and thermochemical similarities in and across compressible reactive flow fields.

Persistence of Weighted Ordinal Partition Networks for Dynamic State Detection

One of the most important problems arising in time series analysis is that of bifurcation, or change point detection. That is, given a collection of time series over a varying parameter, when has the structure of the underlying dynamical system changed? For this task, we turn to the field of topological data analysis (TDA), which encodes information about the shape and structure of data. The idea of utilizing tools from TDA for signal processing tasks, known as topological signal processing (TSP), has gained much attention in recent years, largely through a standard pipeline that computes the persistent homology of the point cloud generated by the Takens' embedding. However, this procedure is limited by computation time since the simplicial complex generated in this case is large, but also has a great deal of redundant data. For this reason, we turn to a more recent method for encoding the structure of the attractor, which constructs an ordinal partition network (OPN) representing information about when the dynamical system has passed between certain regions of state space. The result is a weighted graph whose structure encodes information about the underlying attractor. Our previous work began to find ways to package the information of the OPN in a manner that is amenable to TDA; however, that work only used the network structure and did nothing to encode the additional weighting information. In this paper, we take the next step: building a pipeline to analyze the weighted OPN with TDA and showing that this framework provides more resilience to noise or perturbations in the system and improves the accuracy of the dynamic state detection.

Using machine learning to anticipate tipping points and extrapolate to post-tipping dynamics of non-stationary dynamical systems.

The ability of machine learning (ML) models to "extrapolate" to situations outside of the range spanned by their training data is crucial for predicting the long-term behavior of non-stationary dynamical systems (e.g., prediction of terrestrial climate change), since the future trajectories of such systems may (perhaps after crossing a tipping point) explore regions of state space which were not explored in past time-series measurements used as training data. We investigate the extent to which ML methods can yield useful results by extrapolation of such training data in the task of forecasting non-stationary dynamics, as well as conditions under which such methods fail. In general, we find that ML can be surprisingly effective even in situations that might appear to be extremely challenging, but do (as one would expect) fail when "too much" extrapolation is required. For the latter case, we show that good results can potentially be obtained by combining the ML approach with an available inaccurate conventional model based on scientific knowledge.

State-Space Region-Invariance of Discrete-Time Control Systems

In the paper, the so called region-invariance properties of discrete-time control systems are considered. An already known notion of positivity of control systems has been generalized (using a kind of geometric insight instead of a pure algebraic approach) to the general nonlinear discrete-time control systems, general regions in state-space, and controls from polyhedral cones in input-space. The class of such control systems has been characterized. Numerous numerical examples illustrating individual cases under consideration are presented in the paper.

Adaptive Koopman-Based Models for Holistic Controller and Observer Design⋆

We present a method to obtain a data-driven Koopman operator-based model that adapts itself during operation and can be straightforwardly used for the controller and observer design. The adaptive model is able to accurately describe different state-space regions and additionally consider unpredictable system changes that occur during operation. Furthermore, we show that this adaptive model is applicable to state-space control, which requires complete knowledge of the state vector. For changing system dynamics, the state observer therefore also needs to have the ability to adapt. To the best of our knowledge, there have been no approaches presently available that holistically use an adaptive Koopman-based plant model for the state-space design of both the controller and observer. We demonstrate our method on a test rig: controller and observer adequately adapt during operation, so that outstanding control performance is achieved even in the case of strong occuring systems changes.

Parallel Subdomain Synthesis of Cell Mapping for Capturing Global Invariant Sets in Higher-Dimensional Dynamical Systems

Unaffordable computational cost and memory storage induced by the curse of dimensionality has become the bottleneck of numerical methods in different fields. In the global analysis of nonlinear dynamical systems, the capability of numerical methods, like cell mapping methods, are mostly feasible only to a system dimension less than four. Although cell mappings are naturally parallelizable that may be used to greatly enhance the computational efficiency, it is still not enough to release the computational burden on a higher-dimensional system of greater than seven, not to mention the memory in dealing with millions of billion cells. In this paper, the subdomain synthesis method, which partitions the chosen region in state space into subdomains suitable for operating in a computational unit and then synthetizes the so-called virtual invariant sets to get the underlying global invariant sets, is promoted to be parallelizable on the subdomains so as to build a two-layer massively parallel architecture in both cell and subdomain levels. The proposed approach can be implemented by GPU Cluster that can maximize the powerful computation capability of hardwares. Examples with global invariant sets in very fine distances of a Jerk system and in bifurcations of a twelve-dimensional nonsmooth rotor system are presented for the first time to demonstrate then the feasibility of the proposed approach.