Mixed State Space Research Articles

Computation of the steady-state probability of Markov chain evolving on a mixed state space

Abstract The partitioning algorithm is an iterative procedure that computes explicitly the steady-state probability of a finite Markov chain 𝑋. In this paper, we propose to adapt this algorithm to the case where the state space E := C ∪ D E:=C\cup D is composed of a continuous part 𝐶 and a finite part 𝐷 such that C ∩ D = ∅ C\cap D=\emptyset . In this case, the steady-state probability 𝜋 of 𝑋 is a convex combination of two steady-state probabilities π C \pi_{C} and π D \pi_{D} of two Markov chains on 𝐶 and 𝐷 respectively. The obtained algorithm allows to compute explicitly π D \pi_{D} . If π C \pi_{C} cannot be computed explicitly, our algorithm approximates it by numerical resolution of successive integral equations. Some numerical examples are studied to show the usefulness and proper functioning of our proposal.

An Integrated Model for Autonomous Speed and Lane Change Decision-Making Based on Deep Reinforcement Learning

The implementation of autonomous driving is inseparable from developing intelligent driving decision-making models, which are facing high scene complexity, poor decision-making coupling, and the inability to guarantee decision-making safety. This paper starts with the priority and logic of lane change and car-following decision-making, considering driving efficiency, safety, and comfort, then constructs a double-layer decision-making model. This paper uses two deep reinforcement learning algorithms for the upper and lower layers to process large-scale mixed state space and ensure the composite action output of lane-changing decisions and car-following decisions. In the upper layer model, we use the D3QN algorithm to distinguish the potential value of the environment and the value of selecting lane-changing actions when making lane-changing decisions. Different from the traditional mechanisms that only use negative rewards, the lane changing benefit function and dangerous action shielding mechanism are used to eliminate collisions. DDPG algorithm is adopted in the lower layer model to process car-following decisions and output continuous vehicle speed control. Besides, coupled training is taken for the two algorithms to improve the coordination of the double-layer model. This paper selected mixed standard driving cycle conditions to build a highly complex training environment and used NGSIM data to reconstruct scenes to test our model. Simulations in SUMO are presented that the double-layer model can increase the driving speed of the original data by 23.99%, which has higher effectiveness than other models.

Dynamic Pricing for EV Charging Stations: A Deep Reinforcement Learning Approach

Dynamic pricing, which aims to dynamically adjust the charging price in a timely fashion to unlock the flexibility of electric vehicle (EV) customers, has been extensively studied with the rapid development of charging technologies. Many existing works on dynamic pricing focus on maximizing the social welfare of charging service providers and EV customers. Cases of high-dimensional charging environments, which are often encountered with the rapid growth of EV market penetration, have been rarely considered to date. This article proposes a new dynamic pricing framework for EV charging stations that can offer multiple charging options to customers over a finite-time horizon. The charging price can be dynamically adjusted to maximize the quality of service (QoS) with a differentiated service requirement level (SRL) whenever the arrival rates and queuing system capacities of the charging systems are given at the end of a time period. The dynamic pricing problem is formulated as a finite-discrete horizon Markov decision process (MDP) with a mixed state space. A customized deep reinforcement learning (DRL) approach is employed to solve the examined EV dynamic pricing problem. The simulation results demonstrate the effectiveness of the proposed method.

Optimal replacement of continuously degrading systems in partially observed environments

ABSTRACTMotivated by wind energy applications, we consider the problem of optimally replacing a stochastically degrading component that resides and operates in a partially observable environment. The component's rate of degradation is modulated by the stochastic environment process, and the component fails when it is accumulated degradation first reaches a fixed threshold. Assuming periodic inspection of the component, the objective is to minimize the long‐run average cost per unit time of performing preventive and reactive replacements for two distinct cases. The first case examines instantaneous replacements and fixed costs, while the second considers time‐consuming replacements and revenue losses accrued during periods of unavailability. Formulated and solved are mixed state space, partially observable Markov decision process models, both of which reveal the optimality of environment‐dependent threshold policies with respect to the component's cumulative degradation level. Additionally, it is shown that for each degradation value, a threshold policy with respect to the environment belief state is optimal if the environment alternates between two states. The threshold policies are illustrated by way of numerical examples using both synthetic and real wind turbine data. © 2015 Wiley Periodicals, Inc. Naval Research Logistics 62: 395–415, 2015

Mixed-State Models for Nonstationary Multiobject Activities

We present a mixed-state space approach for modeling and segmenting human activities. The discrete-valued component of the mixed state represents higher-level behavior while the continuous state models the dynamics within behavioral segments. A basis of behaviors based on generic properties of motion trajectories is chosen to characterize segments of activities. A Viterbi-based algorithm to detect boundaries between segments is described. The usefulness of the proposed approach for temporal segmentation and anomaly detection is illustrated using the TSA airport tarmac surveillance dataset, the bank monitoring dataset, and the UCF database of human actions.

Free-vibration analysis of laminated plates with embedded shear-mode piezoceramic layers

This paper presents a three-dimensional exact mixed state space solution and a parametric analysis for the free-vibration of simply-supported laminates with embedded transverse shear mode piezoceramic layers. The formulation retains, as state variables, the standard mechanical displacements and transverse stresses augmented with the electric transverse displacement and potential. It considers both non-electroded and electroded inner x-poled piezoceramic laminae. The solution validation is made through some demonstrated expected results. Then, a parametric study is conducted for various parameters variations. It was found that the thickness distributions of the electric state variables depend strongly on the electric boundary conditions and mode type. Moreover, in contrary to the common practice of the mechanical community, the small difference between short- and open-circuit natural frequencies should not be neglected but can be used to assess the piezoelectric effect through the so-called effective modal electromechanical coupling coefficient. Its level was found to depend on the poling direction, mode type, plate length-to-thickness ratio and piezoceramic lamina thickness and position in the laminate. This exact solution and its obtained results can be used for future reference and shear-mode piezoelectric vibration behavior comprehension purposes.

Piezoelectric Transverse Shear Actuation and Sensing of Plates, Part 1: A Three-Dimensional Mixed State Space Formulation

This paper, in two parts, presents in this first part an exact three-dimensional solution of transverse shear actuation and sensing of simply-supported piezoelectric laminated plates. For this, the piezoelectric laminae are considered initially polarized along their first materialaxis (x1) and subjected to through-thickness electric field. Hence, transverse shear strains can be induced or measured under electric or mechanical loading, respectively. The formulation is based on a new mixed state space approach that retains the standard state mechanical displacement and transverse stress variables augmented with the electric potential and transverse displacement. The latter are condensed analytically at the piezoelectric lamina transfer matrix level. Thus, the solution of the assembled system is reduced to that of only a third-order one. Both open- and closed-circuit electric boundary conditions are considered for sensing and actuation problems, respectively. Corresponding computational procedures are also detailed. In part two of the paper, the present approach is applied to transverse actuation and sensing of sandwich plates. Parametric studies are also conducted for physical understanding of these mechanisms. Presented analytical results are useful for future reference and comparison with other approximate laminate electromechanics.

Piezoelectric Transverse Shear Actuation and Sensing of Plates, Part 1: A Three-Dimensional Mixed State Space Formulation

Piezoelectric Transverse Shear Actuation and Sensing of Plates, Part 1: A Three-Dimensional Mixed State Space Formulation

Piezoelectric Transverse Shear Actuation and Sensing of Plates, Part 2: Application and Analysis

An exact three-dimensional mixed state space formulation has been presented in Part 1 of this paper for the piezoelectric transverse shear actuation and sensing of plates. In this second part, its application to three-layer adaptive plates with piezoceramic core is studied theoretically and numerically for validation purpose. Then, a detailed parametric analysis is conducted to investigate the influence of several parameters such as, the plate thickness ratio, the piezoceramic core thickness, the mechanical and electric loading amplitudes, and the piezoceramic sensor/actuator position through a multilayer cross-ply hybrid laminate. It was found that, to represent both sensing and actuation behaviors, mechanical in-plane displacements and transverse deflection should be taken layerwise linear (in “zig-zag”) and constant through the plate thickness, respectively. The electric potential was found always linear inside the piezoceramic layer. Variation of the piezoceramic core thickness did not affect much the distributions patterns. However, the induced plate voltage and center deflection under varied mechanical and electric loading amplitudes were found to be proportional to the applied mechanical load and actuation voltage. The optimum position of the piezoceramic transverse shear sensor/actuator through-the-thickness of a multilayer cross-ply hybrid plate was found to be at its mid-plane. These results are useful for future reference and comparison with other approximate analytic and numerical solutions. They can also be used to construct such two-dimensional models.