Time-varying Markov Research Articles

The relationship between renewable energy attention and volatility: A HAR model with markov time-varying transition probability

This study investigated whether a renewable energy attention (REA) index, based on natural language processing, Google search volume data, and dimensionality reduction methodology, can predict crude oil volatility. Considering the possible non-linear and time-varying effects, we adopted the time-varying transition probability Markov switching heterogeneous autoregressive-realized volatility (TVTP-MS-HAR-realized volatility (RV)) model. To further represent the impacts of REA, we developed an asymmetric TVTP-MS-HAR-RV model based on this model framework, i.e., the ASTVTP-MS-HAR-RV model. According to the results, the in-sample estimates indicated that West Texas Intermediate (WTI) volatility is more affected by negative REA shocks than by positive ones. Moreover, REA predicted WTI volatility better during low-volatility periods than in high ones. According to the out-of-sample findings, the ASTVTP-MS-HAR-RV-F model outperformed other competing models, indicating that time-varying transition probabilities and REA information can significantly improve volatility forecasting performance.

Integral sliding mode control and stability for Markov jump systems with structured perturbations and time-varying delay driven by fractional Brownian motion

The issues of stability and sliding mode control (SMC) for time-varying delay Markov jump systems (MJSs) with structured perturbations constrained by fractional Brownian motion (fBm) are explored. First, constructing a novel Lyapunov–Krasovskii functional (LKF) with exponential terms that contain the double-integral term, the pth moment exponential stability conditions are derived by utilizing the generalized fractional Itoˆ formula and conditional mathematical expectation. Subsequently, by designing the innovative integral sliding mode surface (SMS) associated with time-varying delay and the SMC law, the state trajectories of the dynamic systems can reach the designed SMS within a finite time. Ultimately, the numerical experiment is executed to confirm and ensure the accuracy and reliability of the obtained results.

Do shipping freight markets impact commodity markets?

The Baltic Dry Index (BDI) is mostly perceived as the leading indicator of shipping freight market and economic activity. In this study, we investigate the dependence structure and distributional predictability between BDI and nineteen major commodity investable indices to offer insights on the predictive ability of BDI for commodity assets. In addition, we explore the effects of policy uncertainty and geopolitical risk on the dynamic dependence structure between BDI and commodities futures. To this end, we apply time-varying Markov switching copula models and non-parametric causality in quantile approach to daily closed price indices of commodities futures running from 23rd May 2006 to 20th November 2020. We provide for the first-time empirical evidence of time-varying Markov tail dependence structure dynamics between marine activities and commodities futures. The nature of the dependence is positives except for petroleum, all metals, livestock wheat, cattle and softs markets. Furthermore, we show the significant effects of global uncertainties, policy uncertainty and geopolitical risk on the dependence between sea transport and commodity assets. Finally, we note from the causality-in-mean and causality-in-variance analysis that the daily change of BDI is an efficient predictor (informational efficiency) of the daily changes of most of the commodity indices. Finally, we show that markets.

Generating and understanding human daily activity sequences using Time-Varying Markov Chain models

Human activity patterns have raised broad interests across fields and activity-based modeling has been applied to analyze travel behaviors, effects of policies, etc. In this paper, we examined daily activity sequences from large volumes of datasets collected in the Tokyo Metropolitan Area over thirty years. We show that, using a simple time-varying (inhomogeneous) Markov Chain model, activity sequences at the individual level can be generated with high accuracy. The predictive power of the model is comparable to neural network models when generating activity sequences for each administrative region. The results are surprising considering the huge number of potential activity sequences and the complicated process of activity planning. A fundamental relationship between time-varying Markov Chain models and Multinomial Logit models is recognized to understand the unexpected accuracy. A new mechanism is proposed on how people make daily activity plans from the perspective of cost and utility, where the probability of one plan being chosen is the same as the probability of it being generated by the Markov Chain model. From this perspective, the Markov process becomes a utility-based model. Furthermore, given the average time-varying transition probabilities over a large spatial and temporal scale, the Markov Chain model for any specific area is reconstructed with six parameters. The generation accuracy using the reconstructed model is examined. Moreover, the statistical characteristics of the costs assigned to the Markov Chain model are investigated.

Time-Varying Gaussian Markov Random Fields Learning for Multivariate Time Series Clustering

Time-Varying Gaussian Markov Random Fields Learning for Multivariate Time Series Clustering

Robust [formula omitted] dynamic output feedback control for nonlinear time-varying delay rectangular descriptor Markov jump systems

The robust H∞ dynamic output feedback (DOF) control for a class of nonlinear time-varying delay rectangular descriptor Markov jump systems (DMJSs) is discussed in this paper, there are no restrictions on the rows and columns of the differential matrix E. The rectangular DOF controller is designed to make the closed-loop system as square DMJSs. Firstly, by utilizing a mode-dependent and delay-dependent Lyapunov-Krasovskii (L-K) functional and implicit function theorem, sufficient conditions are given to guarantee that the augmented closed-loop systems are robust stochastically admissible with an H∞ performance and have unique solution simultaneously. Then based on certain matrix decoupling techniques, a new strict linear matrix inequality (LMI) sufficient condition for the existence of a rectangular DOF controller is obtained, and the controller is given. Finally, numerical examples are presented to verify the validity of the results.

Alcohol use and the longitudinal HIV care continuum for people with HIV who enrolled in care between 2011 and 2019

PurposeWe described the impact of alcohol use on longitudinal engagement in HIV care including loss to follow-up, durability of viral suppression, and death. MethodsWe followed a cohort of 1781 people with HIV from enrolled in care at one of seven US clinics, 2011–2019 through 102 months. We used a multistate, time-varying Markov process and restricted mean time to summarize engagement in HIV care over follow-up according to baseline self-reported alcohol use (none, moderate, or unhealthy). ResultsOur sample (86% male, 54% White) had median age of 35 years. Over 102 months, people with no, moderate, and unhealthy alcohol use averaged 62.3, 61.1, and 59.5 months virally suppressed, respectively. People who reported unhealthy or moderate alcohol use spent 5.1 (95% confidence intervals (CI): 0.8, 9.3) and 7.6 (95%CI: 3.1, 11.7) more months lost to care than nondrinkers. Compared to no use, unhealthy alcohol use was associated with 3.4 (95%CI: −5.6, −1.6) fewer months in care, not virally suppressed. There were no statistically significant differences after adjustment for demographic and clinical characteristics. ConclusionsModerate or unhealthy drinking at enrollment in HIV care was associated with poor retention in care. Alcohol use was not associated with time spent virally suppressed.

Intra-hour photovoltaic forecasting through a time-varying Markov switching model

This work presents a Markov switching model with a time-varying transition matrix to forecast intra-hour photovoltaic (PV) power output, aiming at providing forecasting flexibility. First, the proposed methodology captures images of the sky employing a self-made, low-cost all-sky imager. Second, the goal is to limit exposure problems in those images via the exposure fusion technique. Third, the proposed algorithm identifies groups of pixels forming clouds through a super paramagnetic clustering algorithm. Finally, we model the problem with a homogeneous Poisson process and forecast the cloud location and the shadowed area on a PV plant for the coming minutes. The shadowed area together with meteorological data are the inputs to this model. In the case study, our approach shows better performance than the persistence method, in particular for changing cloud conditions.

Energy scheduling of a fuel cell based residential cogeneration system using stochastic dynamic programming

Decarbonization of residential systems is one of the effective ways to achieve the low carbon goal. Due to the advantages of cleanness, efficiency and flexibility, hydrogen fuel cells play a key role in driving the transition to low carbon or carbon-neutral systems. This paper studies the optimal energy scheduling of an off-grid residential cogeneration system composed of a fuel cell, photovoltaic (PV) device, battery, thermal energy storage (TES) and heat pump. With the objective function being fuel cost, the economic optimization scheduling model is developed based on security constraints of devices and power balances of supply and demand. To handle the system uncertainties and achieve global optimality, the stochastic dynamic programming (SDP) algorithm is used, where the random variables are modeled as time-varying Markov chains. Simulation results under the typical winter day scenarios show that the SDP algorithm can achieve efficient scheduling while enhancing the device sustainability. Besides, comparison results with dynamic programming (DP) in scenarios with different uncertainty degrees demonstrate the optimality and robustness of SDP. Additionally, the impact of TES capacity on the system operation is also discussed, which has certain instructive significance for system capacity configuration in terms of economy and sustainability.

Probabilistic Control of Heterogeneous Swarms Subject to Graph Temporal Logic Specifications: A Decentralized and Scalable Approach

We develop a probabilistic control algorithm, GTLProCo, for swarms of agents with heterogeneous dynamics and objectives, subject to high-level task specifications. The resulting algorithm not only achieves decentralized control of the swarm but also significantly improves scalability over state-of-the-art existing algorithms. Specifically, we study a setting in which the agents move along the nodes of a graph, and the high-level task specifications for the swarm are expressed in a recently-proposed language called graph temporal logic (GTL). By constraining the distribution of the swarm over the nodes of the graph, GTL can specify a wide range of properties, including safety, progress, and response. GTLProCo, with a computational complexity agnostic to the number of agents comprising the swarm, controls the density distribution of the swarm in a decentralized and probabilistic manner. To this end, it synthesizes a time-varying Markov chain modeling the time evolution of the density distribution under the GTL constraints. We first identify a subset of GTL, namely reach-avoid specifications, for which we can reduce the synthesis of such a Markov chain to either linear or semi-definite programs. Then, in the general case, we formulate the synthesis of the Markov chain as a mixed-integer nonlinear program (MINLP).We exploit the structure of the problem to provide an efficient sequential mixed-integer linear programming scheme with trust regions to solve the MINLP. We empirically demonstrate that our sequential scheme is at least three orders of magnitude faster than off-the-shelf MINLP solvers and illustrate the effectiveness of GTLProCo in several swarm scenarios.

Impact of updated trial data on the cost-effectiveness of percutaneous mitral repair.

When updated clinical trial data becomes available reassessing the cost-effectiveness of technologies may modify estimates and influence decision-making. We investigated the impact of updated trial outcomes on the cost-effectiveness of percutaneous mitral repair (PR) for secondary mitral regurgitation. We updated our previous three-state time-varying Markov model to assess the cost-effectiveness of PR + guideline directed medical treatment (GDMT) versus GDMT alone. Key clinical inputs (overall survival (OS) and heart failure hospitalisations (HFH)) were obtained using the 3-year trial findings from the COAPT (Cardiovascular Outcomes Assessment of the MitraClip Percutaneous Therapy) RCT. We calculated incremental cost-effectiveness ratios (ICER) and report how these differ between analyses based on early (2-year) and updated (3-year) evidence. Updated trial data showed an increase in mortality in the intervention arm between two and three years follow-up that was not seen in the control arm. Deterministic and multivariate cost-effectiveness modelling yielded incremental cost effectiveness ratios ICERs of €38,123 and €31,227 /QALY. Compared to our 2-year based estimate (€21,918 / QALY) these results imply an approximate 1.5-fold increase in ICER. The availability of updated survival analyses from the COAPT pivotal trial suggests previous estimates based on 2-year trial findings were over optimistic for the intervention.

Resource Management for Edge Intelligence (EI)-Assisted IoV Using Quantum-Inspired Reinforcement Learning

Recent developments in the Internet of Vehicles (IoV) enable interconnected vehicles to support ubiquitous services. Various emerging service applications are promising to increase the Quality of Experience (QoE) of users. On-board computation tasks generated by these applications have heavily overloaded the resource-constrained vehicles, forcing it to offload on-board tasks to other edge intelligence (EI)-assisted servers. However, excessive task offloading can lead to severe competition for communication and computation resources among vehicles, thereby increasing the processing latency, energy consumption, and system cost. To address these problems, we investigate the transmission-awareness and computing-sense uplink resource management problem and formulate it as a time-varying Markov decision process. Considering the total delay, energy consumption, and cost, quantum-inspired reinforcement learning (QRL) is proposed to develop an intelligence-oriented edge offloading strategy. Specifically, the vehicle can flexibly choose the network access mode and offloading strategy through two different radio interfaces to offload tasks to multiaccess edge computing (MEC) servers through WiFi and cloud servers through 5G. The objective of this joint optimization is to maintain a self-adaptive balance between these two aspects. Simulation results show that the proposed algorithm can significantly reduce the transmission latency and computation delay.

AUV Formation Coordination Control Based on Transformed Topology under Time-Varying Delay and Communication Interruption

Aiming at the control problem of autonomous underwater vehicle (AUV) pilot-following formation with communication delay and communication interruption, a controller based on feedback linearization and the PD control method is designed in this paper. Firstly, the nonlinear, strongly coupled vehicle model is transformed into a second-order model via the feedback linearization method, and then the formation coordination controller is designed based on consistency theory and the PD control method. The Markov random jump process is used to simulate the formation topology in the event of communication interruption. The condition of stable convergence of the AUV pilot-following formation is analyzed in the presence of time-varying delay and Markov transformation topology. A Lyapunov–Krasovskii equation is established, and linear matrix inequality (LMI) is used to solve the problem of communication interruption and communication delay. The boundary conditions of error convergence of the control system are obtained. Finally, the effectiveness of the formation coordination controller based on the second-order integral model under the unstable conditions of underwater acoustic communication is verified by simulation.

An Automatic Partition Time-Varying Markov Model for Reliability Evaluation

As the service time of mechanical devices is getting longer and longer, the safe and reliability evaluation during operation is highlighted. Moreover, real-time reliability evaluation with consideration of multi-state performance degradation becomes increasingly important nowadays, since the consequences of sudden failures are more unacceptable than ever before. The Markov process is a commonly used model in multi-state reliability evaluation. However, little research of the Markov model can deal with multi-source monitoring data and time-varying properties of device performance degradation, as well as the scientific state number determination. In this article, a real-time reliability evaluation model based on automatic partition and the time-varying Markov chain is proposed to solve the problems of the scientific state number selection and time-varying properties description with the state transition matrix of the Markov process, together with taking advantage of multi-source information. The effectiveness of the proposed algorithm is demonstrated on the bearing with life-long vibration and temperature data. It shows that the proposed automatic partition time-varying Markov model can decide the state number automatically according to the trend of life-long data, and evaluate real-time reliability based on equipment operating hours and operating status. The result of predicted remaining useful life obtained by the proposed model is more accurate, and it also shows great superiority in conformity with reality.

A competitive search game with a moving target

We introduce a discrete-time search game, in which two players compete to find an invisible object first. The object moves according to a time-varying Markov chain on finitely many states. The players are active in turns. At each period, the active player chooses a state. If the object is there then he finds the object and wins. Otherwise the object moves and the game enters the next period. We show that this game admits a value, and for any error-term ε>0, each player has a pure (subgame-perfect) ε-optimal strategy. Interestingly, a 0-optimal strategy does not always exist. We derive results on the analytic and structural properties of the value and the ε-optimal strategies. We devote special attention to the important time-homogeneous case, where we show that (subgame-perfect) optimal strategies exist if the Markov chain is irreducible and aperiodic.

Congestion-Aware Policy Synthesis for Multirobot Systems

Multirobot systems must be able to maintain performance when robots get delayed during execution. For mobile robots, one source of delays is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">congestion</i> . Congestion occurs when robots deployed in shared physical spaces interact, as robots present in the same area simultaneously must maneuver to avoid each other. Congestion can adversely affect navigation performance and increase the duration of navigation actions. In this article, we present a multirobot planning framework that utilizes learnt probabilistic models of how congestion affects navigation duration. Central to our framework is a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">probabilistic reservation table</i> , which summarizes robot plans, capturing the effects of congestion. To plan, we solve a sequence of single-robot <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">time-varying Markov automata</i> , where transition probabilities and rates are obtained from the probabilistic reservation table. We also present an iterative model refinement procedure for accurately predicting execution-time robot performance. We evaluate our framework with extensive experiments on synthetic data and simulated robot behavior.

OP135 Impact Of Updated Trial Data On The Cost-effectiveness Of Health Technologies: A Case Study On Percutaneous Mitral Repair

IntroductionExtrapolation methods are commonly used to model the cost-effectiveness of health technologies beyond observed data. Reassessing cost-effectiveness estimates using updated clinical trial data has the potential to reduce uncertainty and optimize decision-making. We present a case study based on percutaneous repair (PR) with the Mitraclip system, a technology to treat severe secondary mitral regurgitation (MR). For the study purpose, we considered the COAPT trial that evaluated the effectiveness of adding PR to medical treatment versus medical treatment alone.MethodsWe developed a time-varying Markov model to assess the cost-effectiveness of PR. Clinical inputs were based on reconstructed individual patient data from the COAPT trial results reported at 2 years, and at 3 years.We developed parametric modeling for overall survival (OS) and heart failure hospitalizations (HFH) to obtain clinically plausible extrapolations beyond observed data. We adopted the French perspective and used a 30-year time horizon. We expressed incremental cost-effectiveness ratios (ICERs) as cost per quality-adjusted life year (QALY).ResultsBased on 2 year-data, preferred parametric models for OS and HFH were exponential and log-logistic respectively, yielding an ICER of EUR21,918/QALY and &gt;0.5 probability of PR being cost-effective (EUR50,000/QALY threshold).Updated analyses at 3 years showed a change of OS trajectory for PR that justified the use of piecewise modelling, yielding an updated ICER that went up to EUR77,904/QALY (base-case), and to a minimum of EUR58,175/QALY (scenario analysis). Using data at 3 years, PR had &lt;0.5 probability of being cost-effective.ConclusionsIn this case study, the availability of updated survival analyses of the main trial is likely to have some impact on decision-making and/or pricing discussion as part of health-technology assessment (HTA). We aim to provide further updated analyses as 4 years results of the COAPT study become available.More broadly, original technology appraisals are frequently undertaken when mid/long-term follow-up trial data may be lacking. Our example suggests the need for continuous HTA review as new clinical data are released.

Re-examination of risk-return dynamics in international equity markets and the role of policy uncertainty, geopolitical risk and VIX: Evidence using Markov-switching copulas

This study re-examines the empirical relationship between risk and return from 1994m12 to 2020m08 in fifteen international equity markets employing the novel time-varying Markov switching copula models. We provide first-time insightful evidence of time-varying Markov tail dependence structure and dynamics between risk and return in international equity markets. Results show that the dependence structure is positive for USA, UK, Germany, Italy, Brazil, Australia, Taiwan, Canada, Mexico, Japan, France and South Africa and negative for Singapore, India, Japan and China. Finally, we document the effects of policy uncertainty, geopolitical risk and VIX conditional on different markets states.

Robust Stability of Time-Varying Markov Jump Linear Systems with Respect to a Class of Structured, Stochastic, Nonlinear Parametric Uncertainties

This note is devoted to a robust stability analysis, as well as to the problem of the robust stabilization of a class of continuous-time Markovian jump linear systems subject to block-diagonal stochastic parameter perturbations. The considered parametric uncertainties are of multiplicative white noise type with unknown intensity. In order to effectively address the multi-perturbations case, we use scaling techniques. These techniques allow us to obtain an estimation of the lower bound of the stability radius. A first characterization of a lower bound of the stability radius is obtained in terms of the unique bounded and positive semidefinite solutions of adequately defined parameterized backward Lyapunov differential equations. A second characterization is given in terms of the existence of positive solutions of adequately defined parameterized backward Lyapunov differential inequalities. This second result is then exploited in order to solve a robust control synthesis problem.

Forecasting crude oil volatility with geopolitical risk: Do time-varying switching probabilities play a role?

This study examines whether geopolitical risk (GPR) exhibits an ability to forecast crude oil volatility from a time-varying transitional dynamics perspective. Unlike previous studies that assume an oversimplification of the fixed transition probabilities for crude oil volatility, we develop an asymmetric time-varying transition probability Markov regime switching (AS-TVTP-MS) GARCH model. In-sample estimated results show that GPR yields strong evidence of regime switching behavior on crude oil volatility and that the negative shocks of GPR result in greater effects on switching probabilities than positive shocks. Out-of-sample results indicate that the AS-TVTP-MS GARCH model containing the GPR index outperforms other models, suggesting that the consideration of GPR information and time-varying regime switching together results in superior predictive performance. Moreover, the predictability of oil volatility is further verified to be economically significant in the framework of portfolio allocation. In addition, our results are robust to various settings.