State Space Research Articles

Temporal information in the anterior cingulate cortex relates to accumulated experiences

Anterior cingulate cortex (ACC) activity is important for operations that require the ability to integrate multiple experiences over time, such as rule learning, cognitive flexibility, working memory, and long-term memory recall. To shed light on this, we analyzed neuronal activity while rats repeated the same behaviors during hour-long sessions to investigate how activity changed over time. We recorded neuronal ensembles as rats performed a decision-free operant task with varying reward likelihoods at three different response ports (n= 5). Neuronal state space analysis revealed that each repetition of a behavior was distinct, with more recent behaviors more similar than those further apart in time. ACC activity was dominated by a slow, gradual change in low-dimensional representations of neural state space aligning with the pace of behavior. Temporal progression, or drift, was apparent on the top principal component for every session and was driven by the accumulation of experiences and not an internal clock. Notably, these signals were consistent across subjects, allowing us to accurately predict trial numbers based on a model trained on data from a different animal. We observed that non-continuous ramping firing rates over extended durations (tens of minutes) drove the low-dimensional ensemble representations. 40% of ACC neurons' firing ramped over a range of trial lengths and combinations of shorter duration ramping neurons created ensembles that tracked longer durations. These findings provide valuable insights into how the ACC, at an ensemble level, conveys temporal information by reflecting the accumulation of experiences over extended periods.

Research on fuzzy adaptive GPC of average coolant temperature based on regulation of boric acid concentration

In order to keep the balance of power distribution in the core of a nuclear reactor, the method of adjusting the boric acid concentration to regulate the average temperature of the coolant has been adopted to track the demands of power changes in the core. However, the chemical reaction process of regulating the average coolant temperature by adjusting the boric acid concentration is too slow to better meet the demands of power changes. In view of this challenge, this paper proposes an innovative fuzzy adaptive incremental generalized predictive control strategy to optimize the process of regulating the coolant temperature. This strategy employs a dynamic adjustment of control parameters, specifically through the application of incremental factors γ, which is calibrated using a fuzzy logic-based algorithm to enhance system responsiveness and robustness in real-time. The strategy aims to improve the efficiency and responsiveness of the nuclear reactor control mechanism and further enhance the safety and stability of nuclear power generation.Firstly, the system state space equations are established based on the boric acid concentration equation and the nuclear reactor kinetics. Secondly, the step factors β and incremental factors γ are used in improved-GPC to improve control real-time and robustness. The step factor β speeds up the rolling optimization, and the incremental factor γ is adjusted based on the fuzzy algorithm according to the online recognition degree of the system. Finally, the 1000 MW Pressurized Water Reactor is modeled and simulated. The results show that improved-GPC has a shorter regulation time and a stronger ability to cope with the model mismatch in this system than other controllers.

Coordinated configuration of hybrid energy storage for electricity-hydrogen integrated energy system

This paper proposes an optimal coordinated configuration method of hybrid electricity and hydrogen storage for the electricity‑hydrogen integrated energy system (EH-ES) to promote the renewable energy source (RES) utilization and reduce the deployment cost. To simulate the practical operation of EH-ES, an energy hub framework with a discrete state space system matrix is designed to characterize the internal interconnection topology and quantify the steady-state production, storage, conversion and utilization processes of electricity and hydrogen energy carriers within EH-ES. A chronological operation simulation based electricity and hydrogen storage configuration model over a year-round time horizon is formulated to collaboratively optimize the capacity of the electrolyzer, fuel cell, battery energy storage (BES) and hydrogen storage tank. In this model, the power and capacity complementarity of BES and generalized hydrogen storage is fully exploited to compensate for power and energy imbalances, and additional capacity configuration of BES is considered to mitigate the drastic electrolytic power fluctuation. Besides, a typical source-load scenario reduction method based on K-means and ordered clustering is developed to avoid a large number of operation variables due to year-round hourly simulation. Comparative studies demonstrate that the proposed methodology can achieve the RES consumption rate of 98.873 % while the equipment investment cost can be reduced by 10.034 %.

Intelligent diving guidance with terminal angle and velocity constraints via deep reinforcement learning

An adaptive guidance strategy that integrates optimal guidance and deep reinforcement learning to address a highly dynamic terminal guidance problem that entails meeting terminal position, angle, and velocity constraints. The proposed strategy leverages optimal guidance commands to accomplish position and angle control while introducing a deep reinforcement learning-based bias for the velocity constraint. In the training process, a dual-velocity state space is constructed to enhance the adaptability of the strategy to different guidance tasks, while training is optimized using the prediction-correction and expert knowledge to improve the training efficiency and optimality of the strategy. Simulations demonstrate that the proposed guidance strategy can achieve simultaneous control of terminal position, angle and velocity, and adapt to different guidance tasks.

Evaluation Method of Comprehensive Ability of Students Majoring in Preschool Education Based on Computer Technology

In order to improve the comprehensive ability evaluation effect of students majoring in preschool education, this paper combines computer technology to evaluate the comprehensive ability of students majoring in preschool education. Through linearization, this paper deduces the state space model of data load of students’ comprehensive ability and the transfer function model of student’s comprehensive ability data load. Moreover, this paper establishes an analytical expression between the model coefficients of the transfer function of student’s comprehensive ability data load and the transfer parameters of student’s comprehensive ability data load. In addition, this paper analyzes and selects the identification method of the parameters of students’ comprehensive ability data load model. Since the parameters of the student comprehensive ability data load model have physical meanings, the parameter identification of the student comprehensive ability data load model is a gray box identification, and constraints are required when identifying the parameters of the student comprehensive ability data load model. The experimental study has verified that the comprehensive ability evaluation model of students majoring in preschool education based on computer technology can play an important role in the comprehensive ability evaluation of students majoring in preschool education. This article adopts the comprehensive load model in the Comprehensive Stability Calculation Program (PSASP) to eliminate the shortcomings of factor analysis in traditional teaching comprehensive evaluation, and has a certain reference value for the promotion of intelligent teaching in the future.

Bearing degradation prediction based on deep latent variable state space model with differential transformation

Rolling bearings are a critical component of mechanical transmission equipment. Predicting their degradation trend is crucial for ensuring safe and stable equipment operation. Most existing bearing degradation prediction methods based on state space models (SSMs) use either linear functions or limited nonlinear functions (e.g., exponential/power laws) to construct the state and measurement equations. As such, these models fail to adapt to the complex and varied nonlinear degradation processes that occur in real-world environments. To address this limitation, we developed a deep latent variable-driven state space degradation model and employed it for bearing degradation prediction. Owing to the powerful nonlinear modeling ability of deep learning models, the proposed method extends the applicability of state space equations. In addition, it inherits the advantages of SSMs and can model uncertainties in a structured manner. Furthermore, the model was integrated with differential pre-transformation to improve its long-term prediction performance. Finally, to validate the effectiveness of the proposed model in predicting bearing degradation, experiments were conducted using a bearing dataset from the PRONOSTIA platform and real wind turbine bearing data. Results showed that the proposed method yielded higher bearing degradation prediction accuracy than existing methods, thus demonstrating the superior performance of the proposed model in predicting bearing degradation.

On the Existence and Uniqueness of Stationary Distributions for Some Piecewise Deterministic Markov Processes with State-Dependent Jump Intensity

In this paper, we consider a subclass of piecewise deterministic Markov processes with a Polish state space that involve a deterministic motion punctuated by random jumps, occurring in a Poisson-like fashion with some state-dependent rate, between which the trajectory is driven by one of the given semiflows. We prove that there is a one-to-one correspondence between stationary distributions of such processes and those of the Markov chains given by their post-jump locations. Using this result, we further establish a criterion guaranteeing the existence and uniqueness of the stationary distribution in a particular case, where the post-jump locations result from the action of a random iterated function system with an arbitrary set of transformations.

Ultralow lattice thermal conductivity in type-I Dirac MBene TiB2

MBenes, the emergent novel two-dimensional family of transition metal borides have recently attracted remarkable attention. Transport studies of such two-dimensional structures are very rare and are of sparking interest. In this paper Using Boltzmann transport theory with ab-initio inputs from density functional theory, we examined the transport in TiB2 MBene system, which is highly dependent on number of layers. We have shown that the addition of an extra layer (as in bilayer BL) destroys the formation of type-I Dirac state by introducing the positional change and tilt to the Dirac cones, thereby imparting the type-II Weyl metallic character in contrast to Dirac-semimetallic character in monolayer ML. Such non-trivial electronic ordering significantly impacts the transport behavior. We further show that the anisotropic room temperature lattice thermal conductivity κ L for ML (BL) is observed to be 0.41 (0.52) and 2.00 (2.04) W m−1 K−1 for x and y directions, respectively, while the high temperature κ L (ML 0.13 W m−1 K−1 and BL 0.21 W m−1 K−1 at 900 K in x direction) achieves ultralow values. Our analysis reveals that such values are attributed to enhanced anharmonic phonon scattering, enhanced weighted phase space and co-existence of electronic and phononic Dirac states. We have further calculated the electronic transport coefficients for TiB2 MBene, where the layer dependent competing behavior is observed at lower temperatures. Our results further unravels the layer dependent thermoelectric performance, where ML is shown to have promising room-temperature thermoelectric figure of merit (ZT) as 1.71 compared to 0.38 for BL.

Snap-back repellers and chaos in a class of discrete-time memristor circuits

In the last decade the flux-charge analysis method (FCAM) has been successfully used to show that continuous-time (CT) memristor circuits possess for structural reasons first integrals (invariants of motion) and their state space can be foliated in invariant manifolds. Consequently, they display an initial condition dependent dynamics, extreme multistability (coexistence of infinitely many attractors) and bifurcations without parameters. Recently, a new discretization scheme has been introduced for CT memristor circuits, guaranteeing that the first integrals are preserved exactly in the discretization. On this basis, FCAM has been extended to discrete-time (DT) memristor circuits showing that they also are characterized by invariant manifolds and they display extreme multistability and bifurcations without parameters. This manuscript considers the maps obtained via DT-FCAM for a circuit with a flux-controlled memristor and a capacitor and it provides a thorough and rigorous investigation of the presence of chaotic dynamics. In particular, parameter ranges are obtained where the maps have snap-back repellers at some fixed points, thus implying that they display chaos in the Marotto and also in the Li–Yorke sense. Bifurcation diagrams are provided where it is possible to analytically identify relevant points in correspondence with the appearance of snap-back repellers and the onset of chaos. The dependence of the bifurcation diagrams and snap-back repellers upon the circuit initial conditions and the related manifold is also studied.

Electric-magnetic duality and Z2 symmetry enriched Abelian lattice gauge theory

Kitaev’s quantum double model is a lattice realization of Dijkgraaf–Witten topological quantum field theory. Its topologically protected ground state space has broad applications for topological quantum computation and topological quantum memory. We investigate the Z2 symmetry enriched generalization of the model for the Abelian group in a categorical framework and present an explicit Hamiltonian construction. This model provides a lattice realization of the Z2 symmetry of the topological phase. We discuss in detail the categorical symmetry of the phase, for which the electric-magnetic (EM) duality symmetry is a special case. The aspects of symmetry defects are investigated using the G-crossed unitary braided fusion category. By determining the corresponding anyon condensation, the gapped boundaries and boundary-bulk duality are also investigated. In the last part, an explicit lattice realization of EM duality is discussed.

Mobile Sensor Path Planning for Kalman Filter Spatiotemporal Estimation.

The estimation of spatiotemporal data from limited sensor measurements is a required task across many scientific disciplines. In this paper, we consider the use of mobile sensors for estimating spatiotemporal data via Kalman filtering. The sensor selection problem, which aims to optimize the placement of sensors, leverages innovations in greedy algorithms and low-rank subspace projection to provide model-free, data-driven estimates. Alternatively, Kalman filter estimation balances model-based information and sparsely observed measurements to collectively make better estimation with limited sensors. It is especially important with mobile sensors to utilize historical measurements. We show that mobile sensing along dynamic trajectories can achieve the equivalent performance of a larger number of stationary sensors, with performance gains related to three distinct timescales: (i) the timescale of the spatiotemporal dynamics, (ii) the velocity of the sensors, and (iii) the rate of sampling. Taken together, these timescales strongly influence how well-conditioned the estimation task is. We draw connections between the Kalman filter performance and the observability of the state space model and propose a greedy path planning algorithm based on minimizing the condition number of the observability matrix. This approach has better scalability and computational efficiency compared to previous works. Through a series of examples of increasing complexity, we show that mobile sensing along our paths improves Kalman filter performance in terms of better limiting estimation and faster convergence. Moreover, it is particularly effective for spatiotemporal data that contain spatially localized structures, whose features are captured along dynamic trajectories.

Consistent spectral approximation of Koopman operators using resolvent compactification

Koopman operators and transfer operators represent dynamical systems through their induced linear action on vector spaces of observables, enabling the use of operator-theoretic techniques to analyze nonlinear dynamics in state space. The extraction of approximate Koopman or transfer operator eigenfunctions (and the associated eigenvalues) from an unknown system is nontrivial, particularly if the system has mixed or continuous spectrum. In this paper, we describe a spectrally accurate approach to approximate the Koopman operator on L 2 for measure-preserving, continuous-time systems via a ‘compactification’ of the resolvent of the generator. This approach employs kernel integral operators to approximate the skew-adjoint Koopman generator by a family of skew-adjoint operators with compact resolvent, whose spectral measures converge in a suitable asymptotic limit, and whose eigenfunctions are approximately periodic. Moreover, we develop a data-driven formulation of our approach, utilizing data sampled on dynamical trajectories and associated dictionaries of kernel eigenfunctions for operator approximation. The data-driven scheme is shown to converge in the limit of large training data under natural assumptions on the dynamical system and observation modality. We explore applications of this technique to dynamical systems on tori with pure point spectra and the Lorenz 63 system as an example with mixing dynamics.

Guiding real-world reinforcement learning for in-contact manipulation tasks with Shared Control Templates

The requirement for a high number of training episodes has been a major limiting factor for the application of Reinforcement Learning (RL) in robotics. Learning skills directly on real robots requires time, causes wear and tear and can lead to damage to the robot and environment due to unsafe exploratory actions. The success of learning skills in simulation and transferring them to real robots has also been limited by the gap between reality and simulation. This is particularly problematic for tasks involving contact with the environment as contact dynamics are hard to model and simulate. In this paper we propose a framework which leverages a shared control framework for modeling known constraints defined by object interactions and task geometry to reduce the state and action spaces and hence the overall dimensionality of the reinforcement learning problem. The unknown task knowledge and actions are learned by a reinforcement learning agent by conducting exploration in the constrained environment. Using a pouring task and grid-clamp placement task (similar to peg-in-hole) as use cases and a 7-DoF arm, we show that our approach can be used to learn directly on the real robot. The pouring task is learned in only 65 episodes (16 min) and the grid-clamp placement task is learned in 75 episodes (17 min) with strong safety guarantees and simple reward functions, greatly alleviating the need for simulation.

Effectiveness of nonpharmaceutical policy interventions in reducing population mobility during the COVID-19 pandemic

Abstract Nonpharmaceutical policy interventions (NPIs) are intended to reduce population mobility in mitigating the spread of COVID-19. This paper evaluates their effect on population mobility during the COVID-19 pandemic. State space models are applied to estimate the time-varying effects of NPI stringency on weekly pedestrian counts from location-based sensors installed before the pandemic. Different models are developed that evaluate compliance with NPIs over time, identify the most effective NPI, and identify regional differences. An efficient parsimonious alternative is proposed for the multivariate Seemingly Unrelated Time Series Equation model if full covariance matrices are of full rank. Kalman filter estimates of the regression coefficients show that NPI stringency initially had a negative effect on population mobility. The effect weakened during the pandemic, suggesting a reduced willingness to comply with regulations. Four of nine NPIs were identified as the most effective. The multivariate model confirmed the findings across federal states. This paper highlights how combining new data sources, routinely collected administrative data, and sound methodology fosters modern policy evaluation.

Discrete Structural Design Synthesis: A Hierarchical-Inspired Deep Reinforcement Learning Approach Considering Topological and Parametric Actions

Abstract Structural design synthesis considering discrete elements can be formulated as a sequential decision process solved using deep reinforcement learning, as shown in prior work. By modeling structural design synthesis as a Markov decision process (MDP), the states correspond to specific structural designs, the discrete actions correspond to specific design alterations, and the rewards are related to the improvement in the altered design’s performance with respect to the design objective and specified constraints. Here, the MDP action definition is extended by integrating parametric design grammars that further enable the design agent to not only alter a given structural design’s topology, but also its element parameters. In considering topological and parametric actions, both the dimensionality of the state and action space and the diversity of the action types available to the agent in each state significantly increase, making the overall MDP learning task more challenging. Hence, this paper also addresses discrete design synthesis problems with large state and action spaces by significantly extending the network architecture. Specifically, a hierarchical-inspired deep neural network architecture is developed to allow the agent to learn the type of action, topological or parametric, to apply, thus reducing the complexity of possible action choices in a given state. This extended framework is applied to the design synthesis of planar structures considering both discrete elements and cross-sectional areas, and it is observed to adeptly learn policies that synthesize high performing design solutions.

Designing Reward Functions Using Active Preference Learning for Reinforcement Learning in Autonomous Driving Navigation

This study presents a method based on active preference learning to overcome the challenges of designing reward functions for autonomous navigation. Results obtained from training with artificially designed reward functions may not accurately reflect human intentions. We focus on the limitations of traditional reward functions, which often fail to facilitate complex tasks in continuous state spaces. We propose the adoption of active preference learning to resolve these issues and to generate reward functions that align with human preferences. This approach leverages an individual’s subjective preferences to guide an agent’s learning process, enabling the creation of reward functions that reflect human desires. We utilize mutual information to generate informative queries and apply information gained to balance the agent’s uncertainty with the human’s response capacity, encouraging the agent to pose straightforward and informative questions. We further employ the No-U-Turn Sampler (NUTS) method to refine the belief model, which outperforms that constructed using the Metropolis algorithm. Subsequently, we retrain the agent using reward weights derived from active preference learning. As a result, our autonomous driving vehicle can navigate between random starting and ending points without dependence on high-precision maps or routing, relying solely on its forward vision. We validate our approach’s performance within the CARLA simulation environment. Our algorithm significantly improved the success rate of autonomous driving navigation tasks that originally failed due to artificially designed rewards, increasing it to approximately 60%. Experimental results show significant improvement over the baseline algorithm, providing a solid foundation for enhancing navigation capabilities in autonomous driving systems and advancing the field of autonomous driving intelligence.

Switching diffusions for multiscale uncertainty quantification

High-fidelity simulations are increasingly called-upon to resolve the nucleation of instabilities such as crack initiation and propagation and fluid mixing. While multiscale modeling capabilities have progressed rapidly over recent decades, associated simulations still present a computational burden, thus often preempting a systematic statistical analysis of related problems. Given the paucity of data that relate micro-structure to macroscale behavior, and the unavoidable modeling errors at both scales, the lack of a probabilistic characterization significantly limits the value of highly-resolved numerical simulators. The goal of the present research is to develop a probabilistic model that encodes pathwise relationships between micro and macro scales. Specifically, we develop a switching diffusion model to relate damage evolution, characterized at the microscale, to system performance identified with a macroscale property. Microscale behavior is modeled as a Markov switching process with a finite state space while the macroscale counterpart is modeled as a continuous-state diffusion process. The interaction between macro and micro scales is captured by coupling these two processes. Calibrated by data, the switching diffusion model can generate, with minimal computational effort, sample paths with prescribed statistics. The proposed model contributes new capabilities and perspectives at the interface of multiscale simulation, uncertainty quantification, and stochastic modeling.

Humpback whales in the Southwest Atlantic Ocean: investigating their breeding movements by satellite tracking

AbstractUnderstanding animals' movements is essential to assess habitat use, life‐history strategies, and population dynamics. Here, we investigate the movement and behavior patterns of 153 humpback whales in the Southwest Atlantic Ocean (SWA) using data obtained by satellite telemetry between 2003 and 2019 during the species' breeding season (August–December) off the Brazilian coast. Switching state space models were applied to estimate behavior states (bmode) classified as Area Restricted Search (ARS), Transiting (TRANS), or uncertain. Whales were distributed from 4°S to 24°S, and five clusters of ARS behavior were identified along the Brazilian coast. Generalized linear mixed modeling revealed three main results: (1) a transition towards more sinuous behavioral states with increasing latitude; (2) more sinuous movement behavior around new moons; (3) movement behavior was temporally dynamic throughout the breeding season over the years, particularly in 2019. The results then revealed important regions where humpback whales cluster to engage in mating and nursing behaviors, highlighting the influence of spatial location and environmental cycles on their behavior. Estimated movement behavior presented here improves the knowledge about the habitat use and movement patterns of SWA humpback whales in their breeding ground and can be used to mitigate potential human‐related impacts.

Understanding Sample Generation Strategies for Learning Heuristic Functions in Classical Planning

We study the problem of learning good heuristic functions for classical planning tasks with neural networks based on samples represented by states with their cost-to-goal estimates. The heuristic function is learned for a state space and goal condition with the number of samples limited to a fraction of the size of the state space, and must generalize well for all states of the state space with the same goal condition. Our main goal is to better understand the influence of sample generation strategies on the performance of a greedy best-first heuristic search (GBFS) guided by a learned heuristic function. In a set of controlled experiments, we find that two main factors determine the quality of the learned heuristic: the algorithm used to generate the sample set and how close the sample estimates to the perfect cost-to-goal are. These two factors are dependent: having perfect cost-to-goal estimates is insufficient if the samples are not well distributed across the state space. We also study other effects, such as adding samples with high-value estimates. Based on our findings, we propose practical strategies to improve the quality of learned heuristics: three strategies that aim to generate more representative states and two strategies that improve the cost-to-goal estimates. Our practical strategies result in a learned heuristic that, when guiding a GBFS algorithm, increases by more than 30% the mean coverage compared to a baseline learned heuristic.

Evaluating the impact of reinforcement learning on automatic deep brain stimulation planning.

Traditional techniques for automating the planning of brain electrode placement based on multi-objective optimization involving many parameters are subject to limitations, especially in terms of sensitivity to local optima, and tend to be replaced by machine learning approaches. This paper explores the feasibility of using deep reinforcement learning (DRL) in this context, starting with the single-electrode use-case of deep brain stimulation (DBS). We propose a DRL approach based on deep Q-learning where the states represent the electrode trajectory and associated information, and actions are the possible motions. Deep neural networks allow to navigate the complex state space derived from MRI data. The chosen reward function emphasizes safety and accuracy in reaching the target structure. The results were compared with a reference (segmented electrode) and a conventional technique. The DRL approach excelled in navigating the complex anatomy, consistently providing safer and more precise electrode placements than the reference. Compared to conventional techniques, it showed an improvement in accuracy of 2.3% in average proximity to obstacles and 19.4% in average orientation angle. Expectedly, computation times rose significantly, from 2 to 18min. Our investigation into DRL for DBS electrode trajectory planning has showcased its promising potential. Despite only delivering modest accuracy gains compared to traditional methods in the single-electrode case, its relevance for problems with high-dimensional state and action spaces and its resilience against local optima highlight its promising role for complex scenarios. This preliminary study constitutes a first step toward the more challenging problem of multiple-electrodes planning.