semi-Markov Process Research Articles

A leader-following consensus of multi-agent systems with actuator saturation and semi-Markov switching topologies.

The leader-following consensus (LFC) issue is investigated in this paper for multi-agent systems (MASs) subject to actuator saturation with semi-Markov switching topologies (SMST). A new consensus protocol is proposed by using a semi-Markov process to model the switching of network topologies. Compared to the traditional Markov switching topologies, the SMST is more general and practical because the transition rates are time-varying. By using the local sector conditions and a suitable Lyapunov-Krasovskii functional, some sufficient conditions are proposed such that the leaderfollowing mean-square consensus is locally achieved. Based on the derived sufficient conditions, an optimization problem is analyzed to determine the consensus feedback gains and to find a maximal estimate of the domain of consensus attraction (DOCA) of a closed-loop model. At the end, a numerical case is presented to verify the performance of the design method.

Method of determining time for preventive diagnosis in city bus operation systems

The article presents a mathematical model of the process of diagnosing and repairing preventive public transport buses. The mathematical model was developed using the theory of semi-Markov processes. The presented model considers two types of corrective repairs - carried out by the Technical Emergency Service and at Service Stations, and preventive repair. The presented model of the exploitation process system has been described to a small extent in the literature so far. The considered system is analyzed on the basis of two criteria: the availability of public transport buses and the profit per unit time. For the adopted assumptions, the conditions for the existence of an extremum of the criterion function were formulated. In the article, the presented model is illustrated based on sample results.

Two-Layer Asynchronous Control for a Class of Nonlinear Jump Systems: An Interval Segmentation Approach.

This article proposes the two-layer asynchronous control scheme for a class of networked nonlinear jump systems. For the constructed system in a network environment, the data transmission may suffer from many restrictions, such as incomplete acceptable mode information and transition information, nonlinearity of system and inadequate bandwidth resources, etc. Then, the two-layer asynchronous controller is developed to stabilize the plant constructed by Takagi-Sugeno (T-S) fuzzy method and semi-Markov theory (SMT). Herein, the hidden semi-Markov process with time-varying emission probability is introduced to establish the relation between the system modes and the controller modes, in which the interval segmentation method is presented to deal with this time-varying probability. Compared with some published results, this method can make full use of the transition rate information, which may lead to the reduction of conservatism in the proposed asynchronous control design. At the same time, the limited bandwidth problem in the communication channel is addressed by introducing the bilateral quantization strategy, and the new sufficient conditions are derived on the stochastic stability of the nonlinear jump system with/without incomplete transition and sojourn-time information. Finally, the numerical simulation examples about DC motor illustrate the effectiveness and the feasibility of the proposed approach.

Stochastic model of the smart power grid section functioning

A feature of large systems, in particular electric power systems, is that the entire system physical state of the is affected by a large number of interacting elements that are randomly in different states. The work relevance is caused by to the fact that construction of mathematical models taking into account the structure and features of the power grid functioning allows us to solve important problems of the energy intellectualization. The goal of the study is due to the desire to show the advantages that arise when using the apparatus of the hidden Markov models theory to semi-Markov models of intelligent electrical systems. To do this, we build a semi-Markov model of a section of a distribution power grid (intelligent electrical grid). Using the theory of semi-Markov processes with a common phase space of states, it is possible to determine the temporal characteristics of the system reliability and obtain a more adequate model. However, territorial dispersion, inaccessibility of some sections of electrical networks and the widespread introduction of automation tools give rise to new problems. For a timely response to various factors, it becomes necessary to as-sess and forecast the states of the system (sections of the system) depending on the signals received in the course of its operation. This can be achieved by applying the hidden Markov models theory. Reliability characteristics are determined and, using the algorithm of stationary phase enlargement, an enlarged semi-Markov model of the section of the intelligent distribution grid is built, which allows passing to the finite state space of the model. Using the merged model, the parameters are determined and a hidden Markov model is developed, for which the most probable states corresponding to a given signal vector are found, the elements subsequent states of the modeled system and signals are predicted. The final part of the article provides an example of finding the reliability characteristics of the system and solving a number of problems in hidden Markov models theory.

Disturbance-observer-based control for non-homogeneous semi-Markov jump systems with time-varying delays

This paper discusses the issue of disturbance-observer-based control (DOBC) for a kind of continuous-time non-homogeneous semi-Markov jump systems (SMJSs) with time-varying delays. Unlike the existing results, the system examined in this article is about non-homogeneous SMJSs, which indicates that its transition rates (TRs) is time-varying. The versions of TRs matrices are selected by a switching signal, which is independent of the semi-Markov process that decides on the system modes. The time-varying TRs are established to be of two situations: stochastic process and arbitrary switching, respectively. First, we construct a novel class of Lyapunov function that is related to both system mode and switching signal mode, thus it can make more use of information about the system and help reduce conservatism. Second, some conditions are given to guarantee the exponentially mean square stable of the SMJSs and H∞ output tracking performance index. Finally, two examples are provided to demonstrate the efficacy of research methods.

Reliably Controlling Massive Traffic between a Sensor Network End Internet of Things Device Environment and a Hub Using Transmission Control Protocol Mechanisms

The constant growth of Internet of Things traffic is ensured by the ongoing evolution of the hierarchy of all hardware links of sensor networks. At the same time, the implementation of the Edge computing ideology regulates the complexity of the “first-mile” section (from the sensors array to the peripheral server). Here, the authors suggest paying attention to the growing share of massive traffic from target sensors in the total traffic of the sensors array. This circumstance makes it expedient to introduce an additional link to the peripheral server for summarizing massive traffic from target sensors. The authors present a sensor network end IoT device (SNEIoTD), implemented grounded on a reliable and cheap Raspberry Pi computing platform, as such a link. The introduction of this SNEIoTD makes it possible to reduce the probability of information loss from the critical infrastructure of a smart city and increase the flexibility of controlling the massive traffic of the first mile. In this context, the urgent task is the reliable control of information transfer from the SNEIoTD environment to a hub, which the authors formalize based on Transmission Control Protocol (TCP). This article proposes a mathematical model of the interaction of the main mechanisms of the TCP in the form of a queuing system. As part of this model, a semi-Markov process of an information transfer with a unified speed is selected and its stationary distribution is analytically formalized. A computationally efficient information technology for determining the TCP Window Size is formulated, taking into account the interaction of TCP mechanisms in the process of massive traffic control. Using the example of TCP Westwood+ protocol modification, it is shown that the results of the application of information technology permit increases in the stability of data transfer under the circumstances of increasing Round-Trip Times.

Semi-Markov processes in open quantum systems. II. Counting statistics with resetting.

A semi-Markov process method for obtaining general counting statistics for open quantum systems is extended to the scenario of resetting. The simultaneous presence of random resets and wave function collapses means that the quantum jump trajectories are no longer semi-Markov. However, focusing on trajectories and using simple probability formulas, general counting statistics can still be constructed from reset-free statistics. An exact tilted matrix equationis also obtained. The inputs of these methods are the survival distributions and waiting-time density distributions instead of quantum operators. In addition, a continuous-time cloning algorithm is introduced to simulate the large-deviation properties of open quantum systems. Several quantum optics systems are used to demonstrate these results.

Asymptotic large deviations of counting statistics in open quantum systems.

We use a semi-Markov process method to calculate large deviations of counting statistics for three open quantum systems, including a resonant two-level system and resonant three-level systems in the Λ and V configurations. In the first two systems, radical solutions to the scaled cumulant generating functions are obtained. Although this is impossible in the third system, since a general sixth-degree polynomial equationis present, we still obtain asymptotically large deviations of the complex system. Our results show that, in these open quantum systems, the large deviation rate functions at zero current are equal to two times the largest nonzero real parts of the eigenvalues of operator -iH[over ̂], where H[over ̂] is a non-Hermitian Hamiltonian, while at a large current these functions possess a unified formula.

Analyzing the Effects of Repairman Failure in Stochastic Models of Non-Identical Redundant Systems

A stochastic analysis is presented here of a repairable system containing two non-identical units, one of which is a duplicate and the other is a main unit. When the units are operative, they have a direct complete failure. During his work with the main unit, a single repairman may encounter problems that may cause his work to fail. During the repair of the duplicate unit, the repairman encounters no problems. Duplicate units receive priority for repair and operation over primary units. After receiving treatment, the repairman can perform his duties with full efficiency. For failure rates of the units, negative exponential distributions were used, while for repair and treatment rates, arbitrary distributions were used. The semi-Markov process and regenerative point technique studied various possible transition states. The reliability characteristics of the system model have been examined numerically and graphically for various parameter values. The profit of the system has also been analyzed for some fixed values of repair and other maintenance costs.

Efficacy of the Single Server Markovian Two-Phase Symmetric Queue with Encouraged Stationary-Queue-Size Analysis

The efficacy of the Markovian single server symmetric queue is examined in this paper together with the suggested encouraged stationary queue-size analysis. First, we propose the stochastic Markov renewal process with encouraged arrival see time averages and encouraged arrival-stationary-queue-size analysis. The server takes further vacations after two extra service phases or until the server discovers a new set of clients in the service. Finally, we discuss the utility of the Bernoulli two-vacation symmetric queue, and it is clearly shown that the degree of efficiency rises when the encouraged arrival is taken into account.

Nonfragile state estimation for semi-Markovian switching CVNs with general uncertain transition rates: An event-triggered scheme

This paper tackles the problem of nonfragile state estimation for semi-Markovian switching complex-valued networks with time-varying delay. The concerned transition rates of the semi-Markov process are uncertain, including both the completely unknown ones and the inaccurately known ones with known bounds. To reduce the communication burden, a particular event-triggered generator is constructed, which depends on the latest available measurement output and a predefined positive threshold. Combining the stochastic analysis method with the Lyapunov stability theory, some less conservative criteria are obtained to ascertain the global asymptotic stability of the estimation error system in the mean-square sense. In addition, by solving some matrix inequalities, the desired nonfragile estimator gains are explicitly designed. Finally, a numerical example with simulations is given to illustrate effectiveness of the established estimation scheme.

Competing risks-based resilience approach for multi-state systems under multiple shocks

Effective measurement of system resilience provides a comprehensive understanding of the system's characteristics. However, little research has been devoted to the resilience of a technical system that is affected by both the degradation process of the system and the external shocks. In addition, existing studies have mainly evaluated the resilience of systems without considering competing risks, and rarely investigated the transient resilience evaluation subjected to the interaction of shocks and maintenance. In this paper, a new resilience model is proposed for the systems under competing risks. The paper first introduces a competing risk model to depict the failure modes of the single-component system, and then uses semi-Markov processes to describe the state transition process when the system suffers the attacks of multiple shocks. Then, according to the multi-state division of the system, the resistibility index, absorbability index and recoverability index are proposed and the overall resilience is then introduced. Considering that the system needs to meet the reliability requirement, constrained by limit budget for maintenance, a reliability and cost-based resilience model is proposed. Finally, the case of a radar system subjected to shocks of one type and multiple types of shocks is given to illustrate the concept developed in this paper.

Performance modeling and analysis for randomly walking mobile users with Markov chains

We treat user equipments (UEs) and mobile edge clouds (MECs) as M/G/1 queueing systems, which are the most suitable, powerful, and manageable models. We propose a computation offloading strategy which can satisfy all UEs served by an MEC and develop an efficient method to find such a strategy. We use discrete-time Markov chains, continuous-time Markov chains, and semi-Markov processes to characterize the mobility of UEs, and calculate the joint probability distribution of the locations of UEs at any time. We extend our Markov chains to incorporate mobility cost into consideration, and are able to obtain the average response time of a UE with location change penalty. We can algorithmically predict the overall average response time of tasks generated on a UE and also demonstrate numerical data and examples. We consider the power constrained MEC speed setting problem and develop an algorithm to solve the problem for two power consumption models.

Estimation of non-monotonic transition rates inasemi-Markov process with covariates adjustments andapplication to caregivers' stress data.

With the large ongoing number of aged people and Alzheimer's disease (AD) patients worldwide, unpaid caregivers have become the primary sources of their daily caregiving. Alzheimer's family caregivers often suffer from physical and mental morbidities owing to various reasons. The aims of this paper were to develop alternate methods to understand the transition properties, the dynamic change, and the long-run behavior of AD caregivers' stress levels, by assuming their transition to the next level only depends on the duration of the current stress level. In this paper, we modeled the transition rates in the semi-Markov Process with log-logistic hazard functions. We assumed the transition rates were non-monotonic over time and the scale of transition rates depended on covariates. We also extended the uniform accelerated expansion to calculate the long-run probability distribution of stress levels while adjusting for multiple covariates. The proposed methods were evaluated through an empirical study. The application results showed that all the transition rates of caregivers' stress levels were right skewed. Care recipients' baseline age was significantly associated with the transitions. The long-run probability of severe state was slightly higher, implying a prolonged recovery time for severe stress patients.

An Application of Homogenous Semi-Markov Model for Assessing HIV/AIDS Progression to ART Patients in Namibia

Background: The progression of HIV infection to AIDS and then to death can be considered a stochastic process. Disease progression can be broken down into a finite number of intermediate states, based on CD4 counts. The five states of the Markov process of HIV/AIDS progression are commonly defined as: S1: CD4 count > 500 cells/microliter; S2: 350 < CD4 count ≤ 500 cells/microliter; S3: 200 < CD4 count ≤ 350 cells/microliter; S4: CD4 count ≤ 200 cells/microliter; and D: Death.
 Objectives: The objective of this study was to model the progression of HIV/AIDS disease of patients under ART follow-up in Namibia using homogenous semi-Markov processes, using the data obtained from Ministry of Health and Social Services.
 Methods: A retrospective study design was used to obtain data on 2422 patients who were observed 11028 times. The semi-Markov model was employed to estimate the transition probabilities and transition intensity rate. Time Homogeneous Semi-Markov model was fitted to assess effectiveness of ART by comparing the forward transition and reverse transitions.
 Results: As expected the probabilities of transiting from good states to worse states increased with time (from state 1 to state 3 and 4 after 6 months is 0.023 and 0.004, after 12 months is 0.059 and 0.010 respectively). As time increase the probabilities of remaining in the same state is decreasing (probabilities of remaining in state 1 after 6, 12 and 18 months is 0.804, 0.698 and 0.633). As expected the intensity indicates that the rate of transiting from good states to worst states is decreasing (the intensity of transiting from state 1 to 3 and 4 is p<0.001).
 The strongest predictor of transition from state 1 to 2 is TDF/3TC/EFV, which has a hazard ratio of 1.338 (with p value of 0.002). Patients who were prescribed TDF/3TC/EFV, are over 1.338 times more likely to transit from state 1 to state 2 than patients who did not receive TDF/3TC/EFV. A hazard ratio of 0.678 for the predictor variable female shows that female were less likely to transit from state 2 to 3 than their male counterparts. The hazard ratios of females from a bad state to a better state are more than 1, which is an indication that females are less likely to respond to treatment compared to males.
 Conclusions: HIV can progress to AIDS without delay if there is no intervention. Early ART initiation is crucial to reduce the probabilities of transiting from good states to worse states.

A Dynamic-Memory Event-Triggered Protocol to Multiarea Power Systems With Semi-Markov Jumping Parameter.

This work deals with the dynamic-memory event-triggered-based load frequency control issue for interconnected multiarea power systems (IMAPSs) associated with random abrupt variations and deception attacks. To facilitate the transient faults, a semi-Markov process is addressed to model the dynamic behavior of IMAPSs. In order to modulate transmission frequency, a novel area-dependent dynamic-memory event-triggered protocol (DMETP) is scheduled by resorting to a set of the historically released packets (HRPs), which ensures better dynamic performance. From the viewpoint of the defender, the randomly occurring deception attack is taken into account, which is regulated by a Bernoulli-distributed scalar. Benefitting from the DMETP scheduling, a novel framework of the memory-based asynchronous control strategy is formulated, in which the hidden semi-Markov model is adopted to reveal the mode mismatches. Based on the Lyapunov theory, sufficient conditions are established to ensure the stochastic stability of the resulting systems. In the end, the simulation result is presented to reveal the efficiency of the proposed dynamic-memory event-triggered-based approach.

Understanding NFV-Enabled Vehicle Platooning Application: A Dependability View

This paper aims to use analytical modeling technique to quantitatively study the dependability of Vehicle Platooning Application, which consists of Multiple Sub-Services (VPP-MSS) to achieve its functionality. Each sub-service (SS), based on network function virtualization technology, is executed in a container. Both SSes and OSes which SSes run on can suffer from software aging after a long and continuous running, reducing VPP-MSS dependability. Rejuvenation techniques are usually used to combat software aging, but they require the support of backup components. Quantitative study of VPP-MSS dependability enables in-depth understanding of the effectiveness of rejuvenation techniques based on analytical models. In contrast to the existing studies, we develop a semi-Markov process (SMP) model to jointly analyze the impact of rejuvenation technique trigger intervals (RTTIs), backup components' behaviors, time-dependent interactions between various behaviors and the number of active SSes deployed on an OS on the effectiveness of rejuvenation technique. Sensitivity analysis helps identify key parameters for improving the dependability of VPP-MSS. Extensive numerical experiments demonstrate the necessity of considering backup components' behaviors and investigating non-exponentially distributed failure times. We also determine both the optimal RTTI combination and the optimal combination of SSes and OSes, which can maximize VPP-MSS dependability.

A Hierarchical Deep Learning Approach for Optimizing CCA Threshold and Transmit Power in Wi-Fi Networks

The nodes, e.g., access points and clients, in current WiFi networks rely on carrier sense multiple access (CSMA) for channel access. This means they rely on a clear channel assessment (CCA) threshold to determine when it is safe to transmit a packet. In practice, this threshold is fixed, and nodes do not adapt their CCA threshold given varying channel conditions. To this end, this paper proposes and studies the use of deep reinforcement learning (DRL) to optimize the CCA threshold and transmit power of an AP. It allows an AP to learn the optimal policy for setting its CCA threshold and transmit power given its queue length and interference level. We outline a semi-Markov decision process (Semi-MDP) for the problem at hand and solve it using a decentralized deep hierarchical reinforcement learning (HRL) approach. We then evaluate its performance using traffic generated via a Poisson distribution and also from a trace file. The results show that when compared to two competing solutions, an AP with HRL reduces its average queue length by 24.4% and 52.4%, respectively.

Observer-Based Control for Discrete-Time Hidden Semi-Markov Jump Systems

In this note, the control problem for discrete-time hidden semi-Markov jump systems is investigated under the situation that the system modes are unavailable directly. Compared with the previous results, the case when both system modes and states are immeasurable simultaneously is fully considered and an observer-based control scheme is established for discrete-time hidden semi-Markov jump systems. The random parameters of the systems are subject to a two-layer stochastic process and the underlying stochastic process is assumed to be subject to a semi-Markov process. Furthermore, a novel switched Lyapunov function is used, which may be monotonically increasing or decreasing at the instant of non-modal switching. Moreover, by means of the stochastic stability theory and semi-Markov kernel method, some novel sufficient conditions ensuring <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -error mean-square stability of the closed-loop system are obtained by a power term reduction method. Finally, an illustrative example is introduced to verify the feasibility of the proposed control method.

An Efficient Multi-AUV Cooperative Navigation Method Based on Hierarchical Reinforcement Learning

Positioning errors introduced by low-precision navigation devices can affect the overall accuracy of a positioning system. To address this issue, this paper proposes a master-slave multi-AUV collaborative navigation method based on hierarchical reinforcement learning. First, a collaborative navigation system is modeled as a discrete semi-Markov process with defined state and action sets and reward functions. Second, trajectory planning is performed using a hierarchical reinforcement learning-based approach combined with the polar Kalman filter to reduce the positioning error of slave AUVs, realizing collaborative navigation in multi-slave AUV scenarios. The proposed collaborative navigation method is analyzed and validated by simulation experiments in terms of the relative distance between the master and slave AUVs and the positioning error of a slave AUV. The research results show that the proposed method can not only successfully reduce the observation and positioning errors of slave AUVs in the collaborative navigation process but can also effectively maintain the relative measurement distance between the master and slave AUVs within an appropriate range.