State Space Research Articles

A mutual information statistic for assessing state space partitions of dynamical systems.

We propose a mutual information statistic to quantify the information encoded by a partition of the state space of a dynamical system. We measure the mutual information between each point's symbolic trajectory history under a coarse partition (one with few unique symbols) and its partition assignment under a fine partition (one with many unique symbols). When applied to a set of test cases, this statistic demonstrates predictable and consistent behavior. Empirical results and the statistic's formulation suggest that partitions based on trajectory history, such as the ordinal partition, perform best. As an application, we introduce the weighted ordinal partition, an extension of the popular ordinal partition with parameters that can be optimized using the mutual information statistic, and demonstrate improvements over the ordinal partition in time series analysis. We also demonstrate the weighted ordinal partition's applicability to real experimental datasets.

Stationary Markov Equilibrium Strategies in Asynchronous Stochastic Games: Existence and Computation

We study Asynchronous Dynamic games and show that in games with a finite state space and finite action sets, one can obtain the pure strategy Markov perfect equilibrium by using a simple backward induction method when the time period for the game is finite. The equilibrium strategies for games with an infinite horizon are then obtained as the point-wise limit of the equilibrium strategies of a sequence of finite horizon games, where the finite horizon games are truncated versions of the original game with successively longer time periods. We also show that if the game has a fixed K-period cycle, then there is a stationary Markov equilibrium. Using these results, we derive an algorithm to compute the equilibrium strategies. We test the algorithm in three experiments. The first is a two-player asynchronous game with three states and three actions. In the second experiment, we compute the equilibrium of a cybersecurity game in which there are two players, an attacker and a defender. In the third experiment, we compute the stationary equilibrium of a duopoly game with two firms that choose an output in alternate periods.

Optimal violation of a Bell inequality in two-dimensional state spaces for an asymmetric orbital angular momentum system

The symmetric orbital angular momentum (OAM) entanglement in the subspace of { ± ℓ} has been extensively exploited in quantum information science. Here we investigate instead the Bell inequalities for the two-dimensional subspace spanned by asymmetric OAM modes of ℓ1 and ℓ2, usually corresponding to the quantum property of nonmaximal entanglement. We demonstrate, both theoretically and experimentally, the optimal violation of a suitable Clauser-Horne-Shimony-Holt Bell inequality for the asymmetric OAM modes, thus manifesting the entanglement for asymmetric OAM states space.

Infrared finite scattering theory: Amplitudes and soft theorems

Any nontrivial scattering with massless fields in four spacetime dimensions will generically produce an out-state with memory. Scattering with any massless fields violates the standard assumption of asymptotic completeness—that all “in” and “out” states lie in the standard (zero-memory) Fock space—and therefore leads to infrared divergences in the standard S-matrix amplitudes. In this paper, we define an infrared finite scattering theory which assumes only (1) the existence of in-/out-algebras and (2) that Heisenberg evolution is an automorphism of these algebras. The resulting “superscattering” map $ allows for transitions between different in/out memory states and agrees with the standard S matrix when it is defined. We construct $ amplitudes by defining (3) a “generalized asymptotic completeness” which accommodates states with memory in the space of asymptotic states and (4) a complete basis of improper states that generalize the usual n-particle momentum basis to account for states with memory. Using only general properties of $, we prove an analog of the Weinberg soft theorems in quantum gravity and QED which imply that all $ amplitudes are well defined in the infrared. We comment on how one must generalize this framework to consider $ amplitudes for theories with collinear divergences (e.g., massless QED and Yang-Mills theories). Published by the American Physical Society 2024

Survey Paper on Quantum Deep Reinforcement Learning for Robot Navigation Tasks

Abstract: Quantum deep reinforcement learning is an integration of the principles of quantum computing with deep reinforcement learning in an effort to advance robot navigation tasks. Due to the explosion of the state and action spaces, traditionally DRL methods are limited in scalability and efficiency. QDRL, however, uses quantum superposition and entanglement for more efficient processing of big amounts of information-it enables robots to learn optimal policies for navigating complex environments. We introduce here a new QDRL framework in which quantum neural networks encode policy functions and value estimates. We demonstrate how quantum circuits can take advantage of multi-dimensional state representations in order to strengthen the power of a robot in changing its environment over time. Benchmark experiments on navigation tasks reflect phenomenal acceleration in learning performance along with better decision-making accuracy compared to classical DRL methods. We then consider integrating such quantum algorithms, as Grover's search, to improve exploration strategies in sparse reward settings. The experiments show that QDRL speeds up convergence rates and provides robots with the possibility to react more rapidly to unexpected obstacles. It discusses new applications for quantum computing of autonomous systems, potentially applied to robotics, autonomous vehicles, and other environments which require complex decision making. Some lines of work in the future are optimization of the resources needed for quantum computers and hybrid architectures for the resolution of classical problems in navigation

ACMamba: A State Space Model-Based Approach for Multi-Weather Degraded Image Restoration

ACMamba: A State Space Model-Based Approach for Multi-Weather Degraded Image Restoration

Sequences of sickness absence, disability pension and unemployment four years before and five years after musculoskeletal diagnosis among Swedish twins.

To investigate sustainable working life via identification of time-related sequences of sickness absence (SA), disability pension (DP) and unemployment four years before and five years after the first musculoskeletal diagnosis in a Swedish twin cohort. Other aims were to account for familial confounding and to examine the associations between sequences and sociodemographic characteristics. Among 28,474 Swedish twins, the patterns of interruptions of working life four years before the first M00-M99 diagnosis (MSD) and five years after MSD diagnosis were investigated with a sequence analysis in a seven-element state space consisting of sustainable working life, unemployment >90 days, moderate SA/DP (30-179 days), almost full year of SA/DP (180-365 days), full year of SA/DP (⩾ 365 days), death, and old-age pension. The six-cluster solution had the best fit to the data. Five clusters had varying patterns of interruptions of sustainable working life (Clusters 2-6, n = 537-1949 with SA/DP, unemployment, but also accounting death and old-age pension) compared with the largest cluster with primarily sustainable working life (n = 23,316). Age, sex and familial factors affected the likelihood of belonging to the clusters with SA/DP. Most Swedish twins with or without MSD diagnosis have a sustainable working life, although MSD was both prevalent and a strong risk factor for belonging to the clusters with SA/DP. Thus, early prevention of MSD and prevention of recurrent or long sickness absences due to any cause would be merited while paying special attention to women also.

Mixed orthogonality graphs for continuous‐time state space models and orthogonal projections

In this article, we derive (local) orthogonality graphs for the popular continuous‐time state space models, including in particular multivariate continuous‐time ARMA (MCARMA) processes. In these (local) orthogonality graphs, vertices represent the components of the process, directed edges between the vertices indicate causal influences and undirected edges indicate contemporaneous correlations between the component processes. We present sufficient criteria for state space models to satisfy the assumptions of Fasen‐Hartmann and Schenk (2024a) so that the (local) orthogonality graphs are well‐defined and various Markov properties hold. Both directed and undirected edges in these graphs are characterised by orthogonal projections on well‐defined linear spaces. To compute these orthogonal projections, we use the unique controller canonical form of a state space model, which exists under mild assumptions, to recover the input process from the output process. We are then able to derive some alternative representations of the output process and its highest derivative. Finally, we apply these representations to calculate the necessary orthogonal projections, which culminate in the characterisations of the edges in the (local) orthogonality graph. These characterisations are given by the parameters of the controller canonical form and the covariance matrix of the driving Lévy process.

Locating Values in the Space of Possibilities

Abstract Where do values live in thought? A straightforward answer is that we (or our brains) make decisions using explicit value representations which are our values. Recent work applying reinforcement learning to decision-making and planning suggests that, more specifically, we may represent both the instrumental expected value of actions as well as the intrinsic reward of outcomes. In this paper, I argue that identifying value with either of these representations is incomplete. For agents such as humans and other animals, there is another place where reward can be located in thought: the division of the space of possibilities or “state space.”

Research on obstacle avoidance of underactuated autonomous underwater vehicle based on offline reinforcement learning

Abstract The autonomous navigation and obstacle avoidance capabilities of autonomous underwater vehicles (AUVs) are essential for ensuring their safe navigation and long-term, efficient operation. However, the complexity of the marine environment poses significant challenges to safe and effective obstacle avoidance. To address this issue, this study proposes an AUV obstacle avoidance control algorithm based on offline reinforcement learning. This method adopts the Conservative Q-learning (CQL) algorithm, which is based on the Soft Actor-Critic (SAC) framework. It learns from obtained historical obstacle avoidance data and ultimately achieves a favorable obstacle avoidance control strategy. In this method, PID and SAC control algorithms are utilized to generate expert obstacle avoidance data to construct a diversified offline database. Additionally, based on the line-of-sight (LOS) guidance method and artificial potential field (APF) method, information regarding the distance and orientation of targets and obstacles is incorporated into the state space, and heading and obstacle avoidance reward terms are integrated into the reward function design. The algorithm successfully guides the AUV in autonomous navigation and dynamic obstacle avoidance in three-dimensional space. Furthermore, the algorithm exhibits a certain degree of anti-interference capability against uncertain disturbances and ocean currents, enhancing the safety and robustness of the AUV system. Simulation results fully demonstrate the feasibility and effectiveness of the intelligent obstacle avoidance method based on offline reinforcement learning. This study highlights the profound significance of offline reinforcement learning in enabling robust and reliable control systems for AUVs, paving the way for enhanced operational capabilities in challenging marine environments.

Dynamic Operational Planning in Warfare: A Stochastic Game Approach to Military Campaigns

ABSTRACTWe study a two‐player discounted zero‐sum stochastic game model for dynamic operational planning in military campaigns. At each stage, the players manage multiple commanders who order military actions on objectives that have an open line of control. When a battle over the control of an objective occurs, its stochastic outcome depends on the actions and the enabling support provided by the control of other objectives. Each player aims to maximize the cumulative number of objectives they control, weighted by their criticality. To solve this large‐scale stochastic game, we derive properties of its Markov perfect equilibria by leveraging the logistics and military operational command and control structure. We show the consequential isotonicity of the optimal value function with respect to the partially ordered state space, which in turn leads to a significant reduction of the state and action spaces. We also accelerate Shapley's value iteration algorithm by eliminating dominated actions and investigating pure equilibria of the matrix game solved at each iteration. We demonstrate the computational value of our equilibrium results on a case study that reflects representative operational‐level military campaigns with geopolitical implications. Our analysis reveals a complex interplay between the game's parameters and dynamics in equilibrium, resulting in new military insights for campaign analysts.

Towards the Usage of Window Counting Constraints in the Synthesis of Reactive Systems to Reduce State Space Explosion

Towards the Usage of Window Counting Constraints in the Synthesis of Reactive Systems to Reduce State Space Explosion

Parameter-coupled state space models based on quasi-Gaussian fuzzy approximation

The accuracy of a fuzzy system’s approximation is closely tied to the performance of fuzzy control systems design, while this system’s interpretability depends on the description of a mechanical model using human language. This research introduces a quasi-Gaussian membership function characterized by a pair of parameters to achieve the sensitivity of a triangular membership function along with the interpretability of Gaussian membership functions. Consequently, a two-dimensional (2-D) quasi-Gaussian membership function is derived, and a method for establishing quasi-Gaussian fuzzy systems (QGFS) using a rectangular grid is proposed. After validating the approximation properties using the sine function for the one-dimensional (1-D) and 2-D QGFS, the systems are applied to approximate the depyrogenation tunnel, a significant piece of equipment in the pharmaceutical industry with various mechanical designs. Validation results indicate that the 1-D and 2-D QGFS can achieve an approximation error varying within a ± 5% range. Meanwhile, the 1-D and 2-D QGFSs are applied to mechanical models of the depyrogenation tunnel with satisfactory final approximation results. Lastly, the 2-D QGFS is capable of demonstrating an excellent description of models with coupled parameters.

Tuning collective behaviour in zebrafish with genetic modification.

Zebrafish collective behaviour is widely used to assess their physical and mental state, serving as a valuable tool to assess the impact of ageing, disease genetics, and the effect of drugs. The essence of these macroscopic phenomena can be represented by active matter models, where the individuals are abstracted as interactive self-propelling agents. The behaviour of these agents depends on a set of parameters in a manner reminiscent of those between the constituents of physical systems. In a few cases, the system may be controlled at the level of the individual constituents such as the interactions between colloidal particles, or the enzymatic behaviour of de novo proteins. Usually, however, while the collective behaviour may be influenced by environmental factors, it typically cannot be changed at will. Here, we challenge this scenario in a biological context by genetically modifying zebrafish. We thus demonstrate the potential of genetic modification in the context of controlling the collective behaviour of biological active matter systems at the level of the constituents, rather than externally. In particular, we probe the effect of the lack of col11a2 gene in zebrafish, which causes the early onset of osteoarthritis. The resulting col11a2 -/- zebrafish exhibited compromised vertebral column properties, bent their body less while swimming, and took longer to change their orientations. Surprisingly, a group of 25 mutant fish exhibited more orderly collective motion than the wildtype. We show that the collective behaviour of wildtype and col11a2 -/- zebrafish are captured with a simple active matter model, in which the mutant fish are modelled by self-propelling agents with a higher orientational noise on average. In this way, we demonstrate the possibility of tuning a biological system, changing the state space it occupies when interpreted with a simple active matter model.

Variational Bayesian Monte Carlo simulation for rare event assessment

Estimating rare events in structural engineering demands intensive computational resources, and the complexity of the performance function poses challenges to accurate probability estimation. To efficiently conduct the structural reliability analysis on rare events, we propose a novel variational Bayesian Monte Carlo (VBMC)-based subset simulation (SS) method. The proposed approach can be divided into two aspects: the Monte Carlo simulation (MCS) is first applied to estimate the first-layer unconditional failure probability, then VBMC is employed to estimate subsequent conditional failure probabilities. Finally, the original small failure probability is obtained by the product of the first-layer unconditional failure probability and a series of conditional failure probabilities. The proposed approach inherits the merits of SS as well as VBMC, which converts the tricky rare event estimation into a series of high-frequency event estimations and leverages VBMC instead of the canonical Markov chain to produce the independent and informative next-generation failure-conditional samples. Four case studies, including high-dimensional and discrete state space scenarios, are performed to illustrate the feasibility and generality of the proposed method.

Facial Expression Recognition-You Only Look Once-Neighborhood Coordinate Attention Mamba: Facial Expression Detection and Classification Based on Neighbor and Coordinates Attention Mechanism

In studying the joint object detection and classification problem for facial expression recognition (FER) deploying the YOLOX framework, we introduce a novel feature extractor, called neighborhood coordinate attention Mamba (NCAMamba) to substitute for the original feature extractor in the Feature Pyramid Network (FPN). NCAMamba combines the background information reduction capabilities of Mamba, the local neighborhood relationship understanding of neighborhood attention, and the directional relationship understanding of coordinate attention. The resulting FER-YOLO-NCAMamba model, when applied to two unaligned FER benchmark datasets, RAF-DB and SFEW, obtains significantly improved mean average precision (mAP) scores when compared with those obtained by other state-of-the-art methods. Moreover, in ablation studies, it is found that the NCA module is relatively more important than the Visual State Space (VSS), a version of using Mamba for image processing, and in visualization studies using the grad-CAM method, it reveals that regions around the nose tip are critical to recognizing the expression; if it is too large, it may lead to erroneous prediction, while a small focused region would lead to correct recognition; this may explain why FER of unaligned faces is such a challenging problem.

Random Forest Spatial-Temporal and State Space Models to Assess the Impact of Bushfire-Induced Aerosol Events on Ozone Depletion in Australia

Serious concerns exist that the increasing frequency of fires may delay the recovery of ozone given increasing temperatures due to climate change. Australian bushfires from September 2019 to February 2020 were catastrophic. A random forest spatial-temporal (RF sp) analysis using satellite data to detect an association between Australian bushfires and stratosphere ozone on the local depletion of ozone in the vicinity of fires in three regions of Australia (Pacific Ocean, Victoria, NSW) has shown a significant reduction in ozone attributable to aerosols from fires. By intervention analysis, increases in aerosols in all three regions were shown to have a significant and ongoing impact 1–5 days later on reducing ozone (p < 0.0001). Intervention analysis also gave similar periods of aerosol exceedance to those found by Hidden Markov models (HMMs). HMMs established a significant and quantifiable decline in ozone due to bushfire-induced aerosols, with significant lags of 10–25 days between times of aerosol exceedance and subsequent ozone level decline in all three regions.

Comprehensive Statistical Analysis of ISSN Records

The history of the origin and transformation of the ISSN Register is considered, and all publications available through Google Scholar and Semantic Scholar on the number of ISSN records and their dynamics are collected together. Data on the number of entries in the ISSN Register by country for 2014 and 2021 is presented, along with the seven-year increase in these entries in comparison with similar data for 1991 through 1998. From these data, the top ten countries were analyzed based on the maximum increase in records over these two seven-year periods. It is shown that the first list of countries with the maximum increase in the number of ISSN records for 2014 through 2021 included exclusively developing countries, and the second list (1991–1998) included mainly developed countries. Similar increases in the number of ISSN records were analyzed for the languages of serial publications and it was shown that in both language lists there was only the Turkish language and the leading European languages were absent. It is shown that the total contributions of ISSN records of leading countries and languages to the total number of ISSN records satisfy the Pareto principle for all years studied. The calculations were made on the indices of linguistic liberalization of the information space of states and information and linguistic influence (or language expansion) according to ISSN indicators. It is shown that aggregating countries by language has made it possible to significantly reduce the discrepancy between the total ISSN records for countries in which a particular language predominates and the ISSN records for a given language. These differences for calculations for 2014 and 2021 levels did not exceed 9% for English, Spanish and Portuguese, and 14% for French, which corresponds to the languages of former colonial empires. In conclusion, a linear regression relationship between ISSN/10 thousand people and GDP per capita with moderate values of the correlation coefficient was obtained in two versions, which indirectly confirms the Constancy Hypothesis.

Empowering legal justice with AI: A reinforcement learning SAC-VAE framework for advanced legal text summarization.

Automated summarization of legal texts poses a significant challenge due to the complex and specialized nature of legal documentation. Despite the recent progress in reinforcement learning for natural language text summarization, its application in the legal domain has been less effective. This paper introduces SAC-VAE, a novel reinforcement learning framework specifically designed for legal text summarization. We leverage a Variational Autoencoder (VAE) to condense the high-dimensional state space into a more manageable lower-dimensional feature space. These compressed features are subsequently utilized by the Soft Actor-Critic (SAC) algorithm for policy learning, facilitating the automated generation of summaries from legal texts. Through comprehensive experimentation, we have empirically demonstrated the effectiveness and superior performance of the SAC-VAE framework in legal text summarization.

A new integrated modelling and control of ternary pumped storage hydro power generation for small signal stability studies

Pumped storage hydro power generation (PSHPG), a reliable renewable energy source with an energy storage feature, can impart efficient damping torque to power system oscillations to improve small signal stability with appropriate modelling and additional compensation through the governor. But ternary PSHPG (T-PSHPG) has the unique feature of hydraulic short circuit (HSC) mode, where both pumping and generating modes operate simultaneously. This study presents a novel fourth-order modified Heffron Phillips (MHP) model of T-PSHPG in coordination with a multi-band Power system stabilizer (MB-PSS) employing 2-way penstock, where primary penstock circulates water from the upper reservoir to the lower reservoir through the turbine and secondary penstock divides primary penstock, thereby creating short-circuit path through the pump. Also, this model includes additional reactive power droop control to maintain the voltage profile. A new state space matrix has been developed for coordinating T-PSHPG in HSC mode with MB-PSS. The control parameters of the proposed coordinated model are set by a new multi-objective differential evolution-improved particle swam optimization (DE-IPSO) algorithm. With variable load, random solar and wind source penetrations, it has been found that the proposed model can provide adequate damping actions. The sensitivity of the proposed model has been observed by varying critical model parameters by ±50 %. For all conditions, the settling time of speed variation is within 2.672 s to 5.951 s and the minimum damping ratio lies within 0.105 to 0.455. The minimum damping ratio is 0.32 and the settling time is found to be 3.1 s for system response subject to sudden solar power variation, which has been observed by shifting system eigenvalues toward the left half of the complex plane. Time and frequency domain parameters with sensitivity analysis predict system oscillations being damped effectively with consistency for the proposed modelling and control strategy. The results are verified in real-time with OPAL-RT real-time digital simulator. It has been justified by the present study that appropriate modelling of T-PSHPG along with coordinated control action provided by MB-PSS can handle critical power system oscillations efficiently and effective damping torque can be imparted in HSC mode.