Dimension Of State Space Research Articles

Characterization of Electric Field Fluctuations in the High-Latitude Ionosphere Using a Dynamical Systems Approach: CSES-01 Observations

We present an analysis of the ionospheric electric field dynamics at high latitudes during periods of quiet and disturbed geomagnetic activity by exploiting recent advancements in dynamical systems and extreme value theory. Specifically, we employed two key indicators: the instantaneous dimension d, which evaluates the degrees of freedom within the system, and the extremal index θ, which quantifies the system’s persistence in a given state. Electric field measurements were obtained from the CSES-01 satellite at mid- and high latitudes in the Southern Hemisphere. Our analysis revealed that the instantaneous dimension increases upon crossing specific ionospheric regions corresponding to the auroral oval boundaries. Outside these regions, the instantaneous dimension fluctuates around the state-space dimension, suggesting an ergodic nature of the system. As geomagnetic activity intensifies, differences in the properties of various ionospheric regions persist, albeit with an increased system instability characterized by higher θ values, thus indicating the externally driven nature of the electric field response to geomagnetic activity. This study provides new insights into the spatial and temporal variability of electric field fluctuations in the ionosphere, highlighting the complex interplay between geomagnetic conditions and ionospheric dynamics.

ICOPP: A Highway Autonomous Driving Planner Based on Interactive Collaborative Optimization and POMDP

Abstract In this paper, we propose a planner for autonomous vehicles in complex highway scenarios, which can efficiently plan the trajectory of autonomous vehicles. The method aims to improve the intelligence of autonomous driving under the uncertainty of the perception and prediction system. The method employs a POMDP framework to address the challenges brought about by uncertainty. Firstly, this method divides the entire planning cycle into several discrete decision-making processes. In each process, the vehicle’s lateral and longitudinal behaviors are semantically processed to obtain a limited number of decision sequences, thereby addressing the curse of dimensionality in state space and observation space in POMDP problems. In each decision-making process, different simulation scenarios are constructed based on the decision of the self-vehicle and probability distribution of the predicting behaviors of surrounding vehicles. In each scenario, key vehicles are selected for collaborative optimal planning among multiple vehicles, resulting in the optimal drivable trajectories for both the ego vehicle and other vehicles under the given decision. Finally, an evaluation of the policy is conducted in terms of safety, comfort, and efficiency to select the optimal policy. In simulation experiments, the ICOPP demonstrates smoother and more stable autonomous driving performance compared to traditional methods.

The maximum principle for lumped-distributed control systems.

This paper concerns the optimal control of lumped-distributed systems, an examplar of which is a mechanical system with rotating components connected by flexible rods. The underlying mathematical model is a controlled semilinear evolution equation, in which nonlinear terms involve a projection of the full state onto a finite dimensional subspace. We derive necessary conditions of optimality in the form of a maximum principle, for a problem formulation which involves pathwise and end-point constraints on the lumped components of the state variable. Perturbational methods are employed in the derivation, which allow for non-smooth data. A key step in the analysis is the reduction of the optimization problem with an infinite dimensional state space, to one with finite dimensional aspects. The computational implications of this reduction will be explored in future work. Necessary conditions for problems with state constraints and end-point constraints can only be achieved under rather strict compatibility hypotheses on the way the dynamics interact with the state constraint and end-point constraint sets, due the infinite dimensionality of the state space. This paper identifies a class of infinite dimensional problems, of engineering significance, for which necessary conditions of optimality can be derived, in the absence of any additional compatibility hypotheses.

Scaling Learning based Policy Optimization for Temporal Logic Tasks by Controller Network Dropout

This paper introduces a model-based approach for training feedback controllers for an autonomous agent operating in a highly nonlinear (albeit deterministic) environment. We desire the trained policy to ensure that the agent satisfies specific task objectives and safety constraints, both expressed in Discrete-Time Signal Temporal Logic (DT-STL). One advantage for reformulation of a task via formal frameworks, like DT-STL, is that it permits quantitative satisfaction semantics. In other words, given a trajectory and a DT-STL formula, we can compute the robustness , which can be interpreted as an approximate signed distance between the trajectory and the set of trajectories satisfying the formula. We utilize feedback control, and we assume a feed forward neural network for learning the feedback controller. We show how this learning problem is similar to training recurrent neural networks (RNNs), where the number of recurrent units is proportional to the temporal horizon of the agent's task objectives. This poses a challenge: RNNs are susceptible to vanishing and exploding gradients, and naïve gradient descent-based strategies to solve long-horizon task objectives thus suffer from the same problems. To tackle this challenge, we introduce a novel gradient approximation algorithm based on the idea of dropout or gradient sampling. One of the main contributions is the notion of controller network dropout , where we approximate the NN controller in several time-steps in the task horizon by the control input obtained using the controller in a previous training step. We show that our control synthesis methodology, can be quite helpful for stochastic gradient descent to converge with less numerical issues, enabling scalable backpropagation over long time horizons and trajectories over high dimensional state spaces. We demonstrate the efficacy of our approach on various motion planning applications requiring complex spatio-temporal and sequential tasks ranging over thousands of time-steps.

A computational methodology for invariant manifold connections between quasi-periodic libration point orbits in non-autonomous problems

Semi-analytical and numerical techniques to systematically analyze and compute natural connections between quasi-periodic orbits associated to non-autonomous systems are considered. Focusing in the non-autonomous Sun–Earth+Moon coherent QBCP model, the center-unstable and center-stable invariant manifolds of Lyapunov quasi-periodic orbits are parameterized and a methodology to detect heteroclinic connections between manifolds is introduced. The methodology aims to decrease the dimensionality of the problem and to address the issue of searching good initial conditions inside the high dimensional state space. Then, connections are identified by searching for approximate patch points in a state-time Poincaré section. These points are subsequently refined to obtain smooth paths that are further continued inside their families.

Early warning signals for bifurcations embedded in high dimensions

Recent work has highlighted the utility of methods for early warning signal detection in dynamic systems approaching critical tipping thresholds. Often these tipping points resemble local bifurcations, whose low dimensional dynamics can play out on a manifold embedded in a much higher dimensional state space. In many cases of practical relevance, the form of this embedding is poorly understood or entirely unknown. This paper explores how measurement of the critical phenomena that generically precede such bifurcations can be used to make inferences about some properties of their embeddings, and, conversely, how prior knowledge about the mechanism of bifurcation can robustify predictions of an oncoming tipping event. These modes of analysis are first demonstrated on a simple fluid flow system undergoing a Hopf bifurcation. The same approach is then applied to data associated with the West African monsoon shift, with results corroborated by existing models of the same system. This example highlights the effectiveness of the methodology even when applied to complex climate data, and demonstrates how a well-resolved spatial structure associated with the onset of atmospheric instability can be inferred purely from time series measurements.

Enhancing Robot Learning with Transformer-based Morphology Modeling

Abstract The transformer model has made significant progress in various areas through large-scale training. In contrast, the traditional robot performs a single task, and there is an issue with migrating the strategic model. In this study, a Robot Morphology Learning (RML) method is proposed to enhance efficiency and generalization performance by learning multiple tasks in a transformer model. RML constructs the robot’s morphology as a graph and utilizes a graph neural network to handle graphs of arbitrary connections and sizes, addressing the disparity in state and action space dimensions. RML breaks through the limitation of non-migration of models, realizes efficient training, and improves the generalization performance of models, enabling quick adaptation to new tasks. Experimental results show that the proposed method outperforms previous methods in both multi-task learning and transfer learning experiments.

Semi-Global Bipartite Fault-Tolerant Containment Control for Heterogeneous Multiagent Systems With Antagonistic Communication Networks and Input Saturation.

Semi-global bipartite fault-tolerant containment control framework on antagonistic communication networks is proposed in this article for heterogeneous multiagent systems (MASs) under the influence of input saturation and actuator faults. An observer is constructed to estimate the leaders' states on signed digraph, where the communication networks are antagonistic. A fully distributed virtual control approach is developed to acquire the containment trajectory. Based on the observer, a semi-global containment control method is developed to compensate for the detrimental impacts of both input saturation and actuator faults. Besides, the dynamics and state-space dimensions of the agents can be different. The proposed framework overcomes two drawbacks of the conventional containment control: 1) the containment trajectory is obtained under general antagonistic communication networks, which is more general in engineering applications and 2) both actuator faults and input saturation are solved for heterogeneous agents, which relaxes the limitation of homogeneous dynamics. Finally, a simulation example is conducted to test and verify the feasibility of the proposed method framework.

Reinforcement learning for energy management of an islanded microgrid: Analysis of state, state space and reinforcement function

The energy management (EM) of a microgrid is a challenging problem because of the stochasticity of renewable sources and loads, which can be formulated as a multi-stage decision-making problem. Reinforcement learning (RL), is found to be suitable for solving multi-stage decision-making problems under uncertainty. With the recent advancements in deferrable types of loads like that of electric vehicles, there is flexibility to manage peak demand, thereby saving the microgrid's operation cost. In this paper, we systematically explain for the first time how RL can be used to solve the scheduling problem of an isolated MG. A novel Q-learning algorithm employing a state-dependent action set is proposed to solve an isolated microgrid's EM problem in the presence of critical and deferrable loads. Various scenarios of wind turbine output power are generated to test the algorithm under a stochastic environment. In this work, the action set is divided into two: one with all the actions included and the second with a reduced number of actions. The entire state space is also divided into two: one set in which the deferrable load needs to be allotted and other where it need not be allotted. The advantage of such a method is that it substantially reduces the computational burden even if the state space's dimension is higher. Also, the judicious selection of reinforcement function to meet all the objectives and constraints is illustrated with case studies. The proposed method yields optimized scheduling with significant cost reductions compared to existing work.

Distilling dynamical knowledge from stochastic reaction networks

Stochastic reaction networks are widely used in the modeling of stochastic systems across diverse domains such as biology, chemistry, physics, and ecology. However, the comprehension of the dynamic behaviors inherent in stochastic reaction networks is a formidable undertaking, primarily due to the exponential growth in the number of possible states or trajectories as the state space dimension increases. In this study, we introduce a knowledge distillation method based on reinforcement learning principles, aimed at compressing the dynamical knowledge encoded in stochastic reaction networks into a singular neural network construct. The trained neural network possesses the capability to accurately predict the state conditional joint probability distribution that corresponds to the given query contexts, when prompted with rate parameters, initial conditions, and time values. This obviates the need to track the dynamical process, enabling the direct estimation of normalized state and trajectory probabilities, without necessitating the integration over the complete state space. By applying our method to representative examples, we have observed a high degree of accuracy in both multimodal and high-dimensional systems. Additionally, the trained neural network can serve as a foundational model for developing efficient algorithms for parameter inference and trajectory ensemble generation. These results collectively underscore the efficacy of our approach as a universal means of distilling knowledge from stochastic reaction networks. Importantly, our methodology also spotlights the potential utility in harnessing a singular, pretrained, large-scale model to encapsulate the solution space underpinning a wide spectrum of stochastic dynamical systems.

On the Use of Ellipsoidal Estimation Techniques in the RRT* Suboptimal Pathfinding Algorithm

The article is devoted to the development of an algorithm for the approximate solution of the time-optimal control problem for a system of ordinary differential equations, under the condition of avoiding collisions with stationary obstacles and subject to the specified pointwise constraints on the possible values of the control parameters. The main idea is to use a modification of the algorithm for finding suboptimal paths using rapidly growing random trees (RRT*). The most difficult part of this algorithm is to find the optimal trajectories for the problems of transferring the system from one fixed position to another, close to it, without taking into account state constraints. This subproblem is proposed to be solved using the methods of ellipsoidal calculus. This approach makes it possible to efficiently search suboptimal trajectories both for linear systems with large state space dimension and for systems with nonlinear dynamics. Algorithms for the linear and non-linear cases are sequentially analyzed in the paper, and the corresponding examples of calculations are presented.

Diverse Time Encoding Strategies Within the Medial Premotor Areas of the Primate.

The measurement of time in the subsecond scale is critical for many sophisticated behaviors, yet its neural underpinnings are largely unknown. Recent neurophysiological experiments from our laboratory have shown that the neural activity in the medial premotor areas (MPC) of macaques can represent different aspects of temporal processing. During single interval categorization, we found that preSMA encodes a subjective category limit by reaching a peak of activity at a time that divides the set of test intervals into short and long. We also observed neural signals associated with the category selected by the subjects and the reward outcomes of the perceptual decision. On the other hand, we have studied the behavioral and neurophysiological basis of rhythmic timing. First, we have shown in different tapping tasks that macaques are able to produce predictively and accurately intervals that are cued by auditory or visual metronomes or when intervals are produced internally without sensory guidance. In addition, we found that the rhythmic timing mechanism in MPC is governed by different layers of neural clocks. Next, the instantaneous activity of single cells shows ramping activity that encodes the elapsed or remaining time for a tapping movement. In addition, we found MPC neurons that build neural sequences, forming dynamic patterns of activation that flexibly cover all the produced interval depending on the tapping tempo. This rhythmic neural clock resets on every interval providing an internal representation of pulse. Furthermore, the MPC cells show mixed selectivity, encoding not only elapsed time, but also the tempo of the tapping and the serial order element in the rhythmic sequence. Hence, MPC can map different task parameters, including the passage of time, using different cell populations. Finally, the projection of the time varying activity of MPC hundreds of cells into a low dimensional state space showed circular neural trajectories whose geometry represented the internal pulse and the tapping tempo. Overall, these findings support the notion that MPC is part of the core timing mechanism for both single interval and rhythmic timing, using neural clocks with different encoding principles, probably to flexibly encode and mix the timing representation with other task parameters.

Learning Reduced-Order Linear Parameter-Varying Models of Nonlinear Systems

In this paper, we consider the learning of a Reduced-Order Linear Parameter-Varying Model (ROLPVM) of a nonlinear dynamical system based on data. This is achieved by a two-step procedure. In the first step, we learn a projection to a lower dimensional state-space. In step two, an LPV model is learned on the reduced-order state-space using a novel, efficient parameterization in terms of neural networks. The improved modeling accuracy of the method compared to an existing method is demonstrated by simulation examples.

Approximate dynamic programming for condition-based node deployment in a wireless sensor network

The flexibility of deployment strategies combined with the low cost of individual sensor nodes allow wireless sensor networks (WSNs) to be integrated into a variety of applications. Network operations degrade over time as sensors consume a finite power supply and begin to fail. In this work we address the selective maintenance of a WSN through a condition-based deployment policy (CBDP) in which sensors are deployed over a series of missions. The main contribution is a Markov decision process (MDP) model to maintain a reliable WSN with respect to region coverage. Due to the resulting high dimensional state and outcome space, we explore approximate dynamic programming (ADP) methodology in the search for high quality CBDPs. Our model is one of the first related to the selective maintenance of a large-scale WSN through the repeated deployment of new sensor nodes with a reliability objective, and one of the first ADP applications for the maintenance of a complex WSN. Additionally, our methodology incorporates a destruction spectrum reliability estimate which has received significant attention with respect to network reliability, but its value in a maintenance setting has not been widely explored. We conclude with a discussion on CBDPs in a range of test instances, and compare the performance to alternative deployment strategies.

Component-wise iterative ensemble Kalman inversion for static Bayesian models with unknown measurement error covariance

The ensemble Kalman filter (EnKF) is a Monte Carlo approximation of the Kalman filter for high dimensional linear Gaussian state space models. EnKF methods have also been developed for parameter inference of static Bayesian models with a Gaussian likelihood, in a way that is analogous to likelihood tempering sequential Monte Carlo (SMC). These methods are commonly referred to as ensemble Kalman inversion (EKI). Unlike SMC, the inference from EKI is asymptotically biased if the likelihood is non-linear and/or non-Gaussian and if the priors are non-Gaussian. However, it is significantly faster to run. Currently, a large limitation of EKI methods is that the covariance of the measurement error is assumed to be fully known. We develop a new method, which we call component-wise iterative EKI (CW-IEKI), that allows elements of the covariance matrix to be inferred alongside the model parameters at negligible extra cost. This novel method is compared to SMC on a linear Gaussian example as well as four examples with non-linear dynamics (i.e. non-linear function of the model parameters). The non-linear examples include a set of population models applied to synthetic data, a model of nitrogen mineralisation in soil that is based on the Agricultural Production Systems Simulator, a model predicting seagrass decline due to stress from water temperature and light, and a model predicting coral calcification rates. On our examples, we find that CW-IEKI has relatively similar predictive performance to SMC, albeit with greater uncertainty, and it has a significantly faster run time.

Minimal Realizations of Nonlinear Time-Delay Systems

The problem of finding a minimal realization of a single-input single-output nonlinear retarded time-delay input-output equation with constant commensurable delays is addressed. First, an algorithm is given to find an observable realization of dimension <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$n$</tex-math></inline-formula> of a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$n$</tex-math></inline-formula> -th order input-output equation. Then, this algorithm is modified under two technical assumptions to find a realization of lower dimension, which is observable and accessible. It is shown that under the same technical assumptions an observable and accessible realization has minimal possible state-space dimension among all possible realizations. An algebraic approach based on polynomial tools and modules of differential 1-forms is used to derive the results.

Dimension-independent spectral gap of polar slice sampling

Polar slice sampling, a Markov chain construction for approximate sampling, performs, under suitable assumptions on the target and initial distribution, provably independent of the state space dimension. We extend the aforementioned result of Roberts and Rosenthal (Stoch Model 18(2):257–280, 2002) by developing a theory which identifies conditions, in terms of a generalized level set function, that imply an explicit lower bound on the spectral gap even in a general slice sampling context. Verifying the identified conditions for polar slice sampling yields a lower bound of 1/2 on the spectral gap for arbitrary dimension if the target density is rotationally invariant, log-concave along rays emanating from the origin and sufficiently smooth. The general theoretical result is potentially applicable beyond the polar slice sampling framework.

Fast estimation of a large TVP-VAR model with score-driven volatilities

This paper proposes a fast approach to estimating a large time-varying parameter vector autoregressive (TVP-VAR) model. Based on a score-driven modeling framework, we first assume that the time-varying variances of random errors in each equation of the TVP-VAR are score-driven, and then propose filtering and smoothing procedures to estimate time-varying parameters and time-varying volatilities. We show that under the forgetting factors, the filtering estimation of time-varying parameters is equivalent to an equation-by-equation estimator, significantly reducing the dimension of state space and thus delivering fast estimation. Moreover, a fast smoothing estimation can be derived, avoiding the inverse of the super-high dimensional state equation covariance matrix. We provide dynamic model averaging (selection) and maximum likelihood estimates for forecasting and inference. Our simulation study shows that the proposed method is more accurate than the popular methods and enjoys tremendous computational gain from the equation-by-equation estimator. Finally, we conduct an empirical study on the dynamic connectedness of global stock markets, demonstrating the merits of our methods in real-time and ex-post analysis.

Deep Q-Learning-Based Dynamic Management of a Robotic Cluster

The ever-increasing demands for autonomy and precision have led to the development of heavily computational multi-robot system (MRS). However, numerous missions exclude the use of robotic cloud. Another solution is to use the robotic cluster to locally distribute the computational load. This complex distribution requires adaptability to come up with a dynamic and uncertain environment. Classical approaches are too limited to solve this problem, but recent advances in reinforcement learning and deep learning offer new opportunities. In this paper we propose a new Deep Q-Network (DQN) based approaches where the MRS learns to distribute tasks directly from experience. Since the problem complexity leads to a curse of dimensionality, we use two specific methods, a new branching architecture, called Branching Dueling Q-Network (BDQ), and our own optimized multi-agent solution and we compare them with classical Market-based approaches as well as with non-distributed and purely local solutions. Our study shows the relevancy of learning-based methods for task mapping and also highlight the BDQ architecture capacity to solve high dimensional state space problems. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —A lot of applications in industry like area exploration and monitoring can be efficiently delegated to a group of small-size robots or autonomous vehicles with advantages like reliability and cost in respect of single-robot solutions. But autonomy requires high and increasing compute-intensive tasks such as computer-vision. On the other hand small robots have energy constraints, limited embedded computing capacities and usually restricted and/or unreliable communications that limit the use of cloud resources. An alternative solution to cope with this problem consists in sharing the computing resources of the group of robots. Previous work was a proof of concept limited to the parallelisation of a single specific task. In this paper we formalize a general method that allows the group of robots to learn on the field how to efficiently distribute tasks in order to optimize the execution time of a mission under energy constraint. We demonstrate the relevancy of our solution over market-based and non-distributed approaches by means of intensive simulations. This successful study is a necessary first step towards distribution and parallelisation of computation tasks over a robotic cluster. The next steps, not tested yet, will address hardware in the loop simulation and finally a real-life mission with a group of robots.

Reduced order modeling and model order reduction for continuum manipulators: an overview.

Soft robot's natural dynamics calls for the development of tailored modeling techniques for control. However, the high-dimensional configuration space of the geometrically exact modeling approaches for soft robots, i.e., Cosserat rod and Finite Element Methods (FEM), has been identified as a key obstacle in controller design. To address this challenge, Reduced Order Modeling (ROM), i.e., the approximation of the full-order models, and Model Order Reduction (MOR), i.e., reducing the state space dimension of a high fidelity FEM-based model, are enjoying extensive research. Although both techniques serve a similar purpose and their terms have been used interchangeably in the literature, they are different in their assumptions and implementation. This review paper provides the first in-depth survey of ROM and MOR techniques in the continuum and soft robotics landscape to aid Soft Robotics researchers in selecting computationally efficient models for their specific tasks.