Reduced State Space Research Articles

Towards the Usage of Window Counting Constraints in the Synthesis of Reactive Systems to Reduce State Space Explosion

Towards the Usage of Window Counting Constraints in the Synthesis of Reactive Systems to Reduce State Space Explosion

Active Vibration Control and Parameter Optimization of Genetic Algorithm for Partially Damped Composites Beams.

The paper partially covered Active Constrained Layer Damping (ACLD) cantilever beams' dynamic modeling, active vibration control, and parameter optimization techniques as the main topic of this research. The dynamic model of the viscoelastic sandwich beam is created by merging the finite element approach with the Golla Hughes McTavish (GHM) model. The governing equation is constructed based on Hamilton's principle. After the joint reduction of physical space and state space, the model is modified to comply with the demands of active control. The control parameters are optimized based on the Kalman filter and genetic algorithm. The effect of various ACLD coverage architectures and excitation signals on the system's vibration is investigated. According to the research, the genetic algorithm's optimization iteration can quickly find the best solution while achieving accurate model tracking, increasing the effectiveness and precision of active control. The Kalman filter can effectively suppress the impact of vibration and noise exposure to random excitation on the system.

Adaptive reduced basis trust region methods for parameter identification problems

In this contribution, we are concerned with model order reduction in the context of iterative regularization methods for the solution of inverse problems arising from parameter identification in elliptic partial differential equations. Such methods typically require a large number of forward solutions, which makes the use of the reduced basis method attractive to reduce computational complexity. However, the considered inverse problems are typically ill-posed due to their infinite-dimensional parameter space. Moreover, the infinite-dimensional parameter space makes it impossible to build and certify classical reduced-order models efficiently in a so-called “offline phase”. We thus propose a new algorithm that adaptively builds a reduced parameter space in the online phase. The enrichment of the reduced parameter space is naturally inherited from the Tikhonov regularization within an iteratively regularized Gauß-Newton method. Finally, the adaptive parameter space reduction is combined with a certified reduced basis state space reduction within an adaptive error-aware trust region framework. Numerical experiments are presented to show the efficiency of the combined parameter and state space reduction for inverse parameter identification problems with distributed reaction or diffusion coefficients.

An efficient physics-dimension-based reduction method for computing frequency response functions of viscoelastically damped systems

The frequency response functions (FRFs) are of critical interest to dynamic problems, however, they suffer from computational challenges for viscoelastically damped systems. In this paper, a physics-space-based reduction method is proposed for predicting the FRFs of large-scale viscoelastically damped systems involving the standard linear solid model. A physics-dimension subspace is constructed based on original system matrices and viscoelastic parameters, which can be easily generated by using a recursive manner. A projection basis generation algorithm is then developed to generate a standard orthonormal basis within the physics-dimension subspace. With the help of the standard orthonormal basis and the moment-matching-based reduction method, a physics-space-based reduction method is proposed for efficiently predicting the FRFs of large-scale viscoelastically damped systems. Unlike the widely used state-space reduction method, the reduced system of the proposed method can preserve system’s physical structure so that the physical meaning can be captured. Using both theoretical and numerical analyses, the proposed physics-space-based method is more accurate and efficient than the state-space-based reduction method.

Attack Scenarios Generation Algorithm Based on Discrete Event System Formalism

To help automatize the security risk assessment process of Cyber-Physical Systems (CPS), we propose a tool based on Discrete Event Systems (DES) to model the architecture and the behavior of CPS in the presence of cyberattacks. Then, we present a lightweight algorithm to generate all the attack scenarios threatening a system, i.e. the sequences of attacks leading to a critical state (e.g. loss of control, collision, etc.). This kind of generation being prone to combinatorial explosion, our algorithm embeds state-space reduction capabilities focused on the specificities of cyber-physical attacks. Finally, we illustrate the performance of our algorithm on a case study: the navigation system of an autonomous vessel. This work can be seen as an alternative to heavy tools expressed in specific languages. It is open source and aims to give a good compromise between expressiveness, modeling time and computational power.

Decomposing Temporal Equilibrium Strategy for Coordinated Distributed Multi-Agent Reinforcement Learning

The increasing demands for system complexity and robustness have prompted the integration of temporal logic into Multi-Agent Reinforcement Learning (MARL) to address tasks with non-Markovian properties. However, incorporating non-Markovian properties introduces additional computational complexities, as agents are required to integrate historical data into their decision-making process. Also, optimizing strategies within a multi-agent environment presents significant challenges due to the exponential growth of the state space with the number of agents. In this study, we introduce an innovative hierarchical MARL framework that synthesizes temporal equilibrium strategies through parity games and subsequently encodes them as individual reward machines for MARL coordination. More specifically, we reduce the strategy synthesis problem into an emptiness problem concerning parity games with optimized states and transitions. Following this synthesis step, the temporal equilibrium strategy is decomposed into individual reward machines for decentralized MARL. Theoretical proofs are provided to verify the consistency of the Nash equilibrium between the parallel composition of decomposed strategies and the original strategy. Empirical evidence confirms the efficacy of the proposed synthesis technique, showcasing its ability to reduce state space compared to the state-of-the-art tool. Furthermore, our study highlights the superior performance of the distributed MARL paradigm over centralized approaches when deploying decomposed strategies.

Neuron importance based verification of neural networks via divide and conquer

Neural networks are widely used in various applications such as image classification, speech recognition and natural language processing. As intelligent systems often rely on neural networks to make critical decisions, it is crucial to ensure that the behavior of a neural network is trustworthy. Along with the increase in the scale and complexity of neural networks, it becomes a challenging problem to verify their safety, since the large-scale state space may make the verification of safety properties infeasible or unsolvable. In order to achieve efficient safety verification of neural networks, this paper proposes a divide-and-conquer verification method based on neuron importance analysis. The method converts the safety constraints to be verified into an optimization problem composed of output neurons, and recursively measures the influence of neurons in the network on the final objective function to calculate their importance. The neurons with the greatest importance are split to refine the constraints, so that the original verification problem is transformed into a set of sub-problems for solution. The method is implemented as a tool named NIAVerify, which is evaluated through experiments of property verification on ReLU-based fully-connected benchmark networks trained on two classic datasets. Experimental results show that NIAVerify can achieve significant state space reduction and the verification efficiency is 44.80% higher than the existing verification method.

Depth-First Net Unfoldings and Equivalent Reduction

In Petri net unfolding, according to the strategies of breadth first and depth first, the biggest problem lies in the potential explosion of the state space. Unfolding generates either accessible trees or branch processes. Making marking reduction or branch cutting accessible proves to be an effective approach to mitigating the state space expansion. In this paper, we propose three reduction rules based on similarity equivalence, conduct state space reduction, present three theorems supported by a case study, and propose a new unfolding algorithm for the unfolding process. In both the new case and the experiments, the completeness, optimality, completeness, and memory and time consumption are reduced by about 60%.

Efficient and accurate Lyapunov function-based truncation technique for multi-dimensional Markov chains with applications to discriminatory processor sharing and priority queues

Online service providers aim to satisfy the tail performance requirements of customers through Service-Level Objectives (SLOs). One approach to ensure tail performance requirements is to model the service as a Markov chain and obtain its steady-state probability distribution. However, obtaining the distribution can be challenging, if not impossible, for certain types of Markov chains, such as those with multi-dimensional or infinite state-space and state-dependent transitions. Examples include M/M/1 with Discriminatory Processor Sharing (DPS) and preemptive M/M/c with multiple priority classes and customer abandonment.To address this fundamental problem, we propose a Lyapunov function-based state-space truncation technique that leverages moments or bounds on moments of the state variables. This technique allows us to obtain tight truncation bounds while ensuring arbitrary probability mass guarantees for the truncated chain. We highlight the efficacy of our technique for multi-dimensional DPS and M/M/c priority queue with abandonment, demonstrating a substantial reduction in state space (up to 74%) compared to existing approaches. Additionally, we present three practical use cases that highlight the applicability of our truncation technique by analyzing the performance of the DPS system.

Managing supply chain risks with dual sourcing: Bayesian learning of censored supply capacity

Dual sourcing is a common strategy for mitigating supply chain risks, with supply capacity uncertainty being one of the most significant risks. However, firms often have limited knowledge about uncertain supply capacity, and can only observe it when the delivered quantity is less than the ordered amount. This leads to censored supply capacity observations. To investigate how supply capacity learning affects dual-sourcing decisions, we propose a model of a firm sourcing non-storable substitute products from an expensive reliable supplier and an inexpensive unreliable supplier to meet a deterministic time-varying demand over a finite horizon. The unreliable supplier has a random capacity with an unknown parameter. The firm updates its belief about this parameter using Bayesian learning and adapts its sourcing decisions to balance the tradeoff between maximizing current profits based on its current belief (exploitation) and increasing the precision of its belief for future periods (exploration). We use Bayesian dynamic programming to determine the firm’s myopic sourcing policy and partially characterize its optimal sourcing policy. We apply a state space reduction technique to Weibull distributed capacity, resulting in an inductively solvable optimization problem. The optimal sourcing policy can be obtained under a condition that ensures splitting the demand between the suppliers is optimal. For exponentially distributed capacity, the optimization problem has a recursive closed-form solution, and the optimal sourcing policy is such that the optimal order quantity from the reliable (unreliable) supplier is smaller (greater) than the myopic quantity. We complement our analytical results with numerical examples and discuss extensions to random demand and inventory carryover.

Hybrid Deep Modeling of a GS115 (Mut+) Pichia pastoris Culture with State–Space Reduction

Hybrid modeling workflows combining machine learning with mechanistic process descriptions are becoming essential tools for bioprocess digitalization. In this study, a hybrid deep modeling method with state–space reduction was developed and showcased with a P. pastoris GS115 Mut+ strain expressing a single-chain antibody fragment (scFv). Deep feedforward neural networks (FFNN) with varying depths were connected in series with bioreactor macroscopic material balance equations. The hybrid model structure was trained with a deep learning technique based on the adaptive moment estimation method (ADAM), semidirect sensitivity equations and stochastic regularization. A state–space reduction method was investigated based on a principal component analysis (PCA) of the cumulative reacted amount. Data of nine fed-batch P. pastoris 50 L cultivations served to validate the method. Hybrid deep models were developed describing process dynamics as a function of critical process parameters (CPPs). The state–space reduction method succeeded to decrease the hybrid model complexity by 60% and to improve the predictive power by 18.5% in relation to the nonreduced version. An exploratory design space analysis showed that the optimization of the feed of methanol and of inorganic elements has the potential to increase the scFv endpoint titer by 30% and 80%, respectively, in relation to the reference condition.

A model-based safety analysis approach for airborne systems using state traversals

Safety analysis is an important task in both the development and certification of civil aircraft. The traditional safety analysis is significantly dependent on the skills and experiences of analysts. A model-based safety analysis approach is proposed for airborne systems based on the model built with Simulink. This study builds Simulink models of typical failure modes as well as the fault injection methods. The responses of system performances are monitored by traversing all failure combinations based on a state space reduction method. The system will be in an unsafe condition when the responses exceed their thresholds. The minimal cut sets of the system are obtained automatically by recording the failure combinations leading to the unsafe condition. Finally, a lateral-directional flight control system is taken as a practical example to illustrate the application and effectiveness of our proposed method. The result shows that our method has higher accuracy and the causes of the unsafe conditions can be determined by the automatic generation of the minimal cut sets. Additionally, the cumbersome work of building a traditional safety analysis model such as the fault tree, the Markov model, or the dependence diagram can be avoided.

Psym: Efficient Symbolic Exploration of Distributed Systems

Verification of distributed systems using systematic exploration is daunting because of the many possible interleavings of messages and failures. When faced with this scalability challenge, existing approaches have traditionally mitigated state space explosion by avoiding exploration of redundant states (e.g., via state hashing) and redundant interleavings of transitions (e.g., via partial-order reductions). In this paper, we present an efficient symbolic exploration method that not only avoids redundancies in states and interleavings, but additionally avoids redundant computations that are performed during updates to states on transitions. Our symbolic explorer leverages a novel, fine-grained, canonical representation of distributed system configurations (states) to identify opportunities for avoiding such redundancies on-the-fly. The explorer also includes an interface that is compatible with abstractions for state-space reduction and with partial-order and other reductions for avoiding redundant interleavings. We implement our approach in the tool Psym and empirically demonstrate that it outperforms a state-of-the-art exploration tool, can successfully verify many common distributed protocols, and can scale to multiple real-world industrial case studies across

Learning-based sensitivity analysis and feedback design for drug delivery of mixed therapy of cancer in the presence of high model uncertainties

In this paper, a methodology is proposed that enables to analyze the sensitivity of the outcome of a therapy to unavoidable high dispersion of the patient specific parameters on one hand and to the choice of the parameters that define the drug delivery feedback strategy on the other hand. More precisely, a method is given that enables to extract and rank the most influent parameters that determine the probability of success/failure of a given feedback therapy for a given set of initial conditions over a cloud of realizations of uncertainties. Moreover predictors of the expectations of the amounts of drugs being used can also be derived. This enables to design an efficient stochastic optimization framework that guarantees safe contraction of the tumor while minimizing a weighted sum of the quantities of the different drugs being used. The framework is illustrated and validated using the example of a mixed therapy of cancer involving three combined drugs namely: a chemotherapy drug, an immunology vaccine and an immunotherapy drug. Finally, in this specific case, it is shown that dash-boards can be built in the 2D-space of the most influent state components that summarize the outcomes’ probabilities and the associated drug usage as iso-values curves in the reduced state space.

Parallel improved DPSA algorithm for medium-term optimal scheduling of large-scale cascade hydropower plants

Medium-term optimal scheduling of hydropower plants (MOSHPP) should be frequently updated for recurring extreme precipitation events in China to reduce spillage and increase power generation; however, the curse of dimensionality makes obtaining a satisfactory solution in an acceptable time difficult. In this paper, a parallel improved dynamic programming with successive approximation (PIDPSA) for MOSHPPs is proposed to improve the solution quality and meet time requirements. The solution quality is improved by successive approximation of multiple plants instead of one plant for considering more hydraulic connections, a state space reduction strategy combined with constraint preprocessing is adopted to reduce unnecessary calculations and the fine-grained parallelism based on the fork/join framework is employed to greatly shorten the solution time. The proposed method was applied to 11 hydropower cascade plants in the Lancang River with a time horizon of 30 days. The results showed that higher quality solutions can be obtained by IDPSA than conventional methods and DPSA, and the space reduction strategy can effectively improve the solution efficiency. The computation time of PIDPSA decreased from 81345 s for the single-core environment to 6073 s for the 32-core environment, which proves the high efficiency and practical value of PIDPSA for solving MOSHPP problems.

Multi-band oscillations emerge from a simple spiking network.

In the brain, coherent neuronal activities often appear simultaneously in multiple frequency bands, e.g., as combinations of alpha (8-12 Hz), beta (12.5-30 Hz), and gamma (30-120 Hz) oscillations, among others. These rhythms are believed to underlie information processing and cognitive functions and have been subjected to intense experimental and theoretical scrutiny. Computational modeling has provided a framework for the emergence of network-level oscillatory behavior from the interaction of spiking neurons. However, due to the strong nonlinear interactions between highly recurrent spiking populations, the interplay between cortical rhythms in multiple frequency bands has rarely been theoretically investigated. Many studies invoke multiple physiological timescales (e.g., various ion channels or multiple types of inhibitory neurons) or oscillatory inputs to produce rhythms in multi-bands. Here, we demonstrate the emergence of multi-band oscillations in a simple network consisting of one excitatory and one inhibitory neuronal population driven by constant input. First, we construct a data-driven, Poincaré section theory for robust numerical observations of single-frequency oscillations bifurcating into multiple bands. Then, we develop model reductions of the stochastic, nonlinear, high-dimensional neuronal network to capture the appearance of multi-band dynamics and the underlying bifurcations theoretically. Furthermore, when viewed within the reduced state space, our analysis reveals conserved geometrical features of the bifurcations on low-dimensional dynamical manifolds. These results suggest a simple geometric mechanism behind the emergence of multi-band oscillations without appealing to oscillatory inputs or multiple synaptic or neuronal timescales. Thus, our work points to unexplored regimes of stochastic competition between excitation and inhibition behind the generation of dynamic, patterned neuronal activities.

Autonomous Platoon Control With Integrated Deep Reinforcement Learning and Dynamic Programming

Deep Reinforcement Learning (DRL) is regarded as a potential method for car-following control and has been mostly studied to support a single following vehicle. However, it is more challenging to learn a stable and efficient car-following policy when there are multiple following vehicles in a platoon, especially with unpredictable leading vehicle behavior. In this context, we adopt an integrated DRL and Dynamic Programming (DP) approach to learn autonomous platoon control policies, which embeds the Deep Deterministic Policy Gradient (DDPG) algorithm into a finite-horizon value iteration framework. Although the DP framework can improve the stability and performance of DDPG, it has the limitations of lower sampling and training efficiency. In this paper, we propose an algorithm, namely Finite-Horizon-DDPG with Sweeping through reduced state space using Stationary approximation (FH-DDPG-SS), which uses three key ideas to overcome the above limitations, i.e., transferring network weights backward in time, stationary policy approximation for earlier time steps, and sweeping through reduced state space. In order to verify the effectiveness of FH-DDPG-SS, simulation using real driving data is performed, where the performance of FH-DDPG-SS is compared with those of the benchmark algorithms. Finally, platoon safety and string stability for FH-DDPG-SS are demonstrated.

Data-Integrity Aware Stochastic Model for Cascading Failures in Power Grids

The reliable operation of power grids during cascading failures is heavily dependent on the interdependencies between the power grid components and the supporting communications and control networks. Moreover, the system operators’ expertise in dealing with cascading failures can play a pivotal role during contingencies. In this paper, a dynamical probabilistic model is developed based on Markov-chains, which captures the dynamics of cascading failures in the power grid. Specifically, a previously developed Markov-chain based model is extended to capture the trade-off between the benefits of having a robust communication infrastructure and its vulnerability from data integrity (e.g., cyber-attacks). State-space reduction of the complex interactions between power grids, communication networks and system operators is achieved by judiciously specifying the state variables of the Markov chain. The impact of system operators’ probability of error during a cascade-mitigation action is incorporated into the model as a function of the state variables of the Markov chain. A point of diminishing returns is observed beyond which the effect of information infidelity outweighs the benefits of having more information. For a given level of cyber threat, an optimal size of a communication network is observed that minimizes the expected number of transmission-line failures before the cascade stops.

Optimal Greedy Control in Reinforcement Learning.

We consider the problem of dimensionality reduction of state space in the variational approach to the optimal control problem, in particular, in the reinforcement learning method. The control problem is described by differential algebraic equations consisting of nonlinear differential equations and algebraic constraint equations interconnected with Lagrange multipliers. The proposed method is based on changing the Lagrange multipliers of one subset based on the Lagrange multipliers of another subset. We present examples of the application of the proposed method in robotics and vibration isolation in transport vehicles. The method is implemented in FRUND-a multibody system dynamics software package.

Contested logistics simulation output analysis with approximate dynamic programming: a proposed methodology

PurposeRapid sensitivity analysis and near-optimal decision-making in contested environments are valuable requirements when providing military logistics support. Port of debarkation denial motivates maneuver from strategic operational locations, further complicating logistics support. Simulations enable rapid concept design, experiment and testing that meet these complicated logistic support demands. However, simulation model analyses are time consuming as output data complexity grows with simulation input. This paper proposes a methodology that leverages the benefits of simulation-based insight and the computational speed of approximate dynamic programming (ADP).Design/methodology/approachThis paper describes a simulated contested logistics environment and demonstrates how output data informs the parameters required for the ADP dialect of reinforcement learning (aka Q-learning). Q-learning output includes a near-optimal policy that prescribes decisions for each state modeled in the simulation. This paper's methods conform to DoD simulation modeling practices complemented with AI-enabled decision-making.FindingsThis study demonstrates simulation output data as a means of state–space reduction to mitigate the curse of dimensionality. Furthermore, massive amounts of simulation output data become unwieldy. This work demonstrates how Q-learning parameters reflect simulation inputs so that simulation model behavior can compare to near-optimal policies.Originality/valueFast computation is attractive for sensitivity analysis while divorcing evaluation from scenario-based limitations. The United States military is eager to embrace emerging AI analytic techniques to inform decision-making but is hesitant to abandon simulation modeling. This paper proposes Q-learning as an aid to overcome cognitive limitations in a way that satisfies the desire to wield AI-enabled decision-making combined with modeling and simulation.