Markov Regenerative Process Research Articles

Stochastic modeling of enhancing energy efficiency for HAP in HAP-LEO systems using Markov regenerative process

The sixth-generation (6G) communication systems heavily rely on space-air-terrestrial networks (SAT), which integrate high-altitude platforms (HAPs) and low Earth orbit (LEO) satellites. LEO satellites play a crucial role in ensuring global coverage and supporting a large number of users, while HAPs are strategically deployed to cover regions with high demand, providing a more reliable communication service to ground users. The principal aim of the proposed model is to explore techniques for energy conservation and efficient energy harvesting within the multi-layered system for HAP. In this energy harvesting scenario, it is assumed that HAPs will store the energy units generated through solar power. Moreover, the presented model incorporates two energy-saving states: standby and sleep states, during which HAPs consume minimal or zero energy. To analyze the described multi-layer system, a stochastic model is formulated using Markov regenerative process (MRGP), with consideration for the mean sojourn times of energy-saving states following a non-exponential distribution. The numerical results vividly demonstrate the significant contribution of energy-saving states in enhancing the energy efficiency of the HAP system. Also, trade-offs have been presented to analyze the impact of energy harvesting and energy saving in the proposed model. These findings are visually presented in graphs, and to validate the proposed model, simulation results have been carried out.

On the sensitivity of stationary solutions of Markov regenerative processes

Markov regenerative process (MRGP) is favored for modeling and evaluating system dependability due to its high power and flexibility. However, its analysis presents challenges because of its inherent renewal nature. The embedded Markov chain (EMC) method offers a stationary solution to the MRGP, while the phase expansion approach delivers both stationary and transient solutions. From these solutions, one can derive performance or dependability measures as outputs from the MRGP model. It is crucial to conduct a sensitivity analysis on MRGP to understand the influence of input factor changes on model outputs, aiding efficient system optimization. Yet, a clear analytical method for sensitivity analysis of MRGP models is currently lacking. Filling this gap, this paper introduces an analytical approach to assess parametric sensitivity for steady-state MRGP, utilizing the EMC method for obtaining the stationary solution. Specifically, since system availability closely correlates with the average system available duration, this paper also shifts its focus from mere model parameters to representative values, like the average available time of a system.

A Markov Regenerative Process Model for the Dependability and Performance of a Two-Unit Multi-State System under Maintenance

Preventive maintenance (PM) is a key aspect for industrial, production and generally technological systems, since it can prevent unexpected breakdowns and their consequences. However, PM incurs additional cost and thus system designers should carefully examine how and when to implement it. Beyond this, modeling correctively the structure and the evolution of the system is of critical significance when designing the appropriate PM strategy. In this paper, a two-unit multi-state parallel system with one repair facility, which can model various industrial systems like power generation or pump systems, production systems or even computing systems, is considered, in which PM is enabled to prevent failures and improve system's dependability and performance. Given some specific system characteristics, a Markov Regenerative Process is proposed to be used to accurately model the evolution of the system in time. The asymptotic availability, the expected downtime and the expected operational cost are evaluated as measures for the system dependability and performance correspondingly. The aim is to determine the PM policies that optimize these measures either separately or jointly, through formulating and solving the appropriate optimization problem. The theoretical framework is illustrated through a numerical example for a pumping station consisting of two centrifugal pumps working in parallel.

Sensitivity analysis for a Markov regenerative software rejuvenation model

This article considers the parametric sensitivity of a software rejuvenation model for transaction systems whose system behavior is described by a quasi-birth-and-death (QBD) process with Markovian arrivals, and the software rejuvenation model is represented by a Markov regenerative process (MRGP). The stationary analysis of the MRGP model is based on the embedding Markov chain (EMC) approach. Then the sensitivities of the stationary solution and the performance criteria of interest, i.e., the loss probability of transactions, with respect to all model parameters, are formulated. Finally, a numerical experiment was conducted to illustrate the critical parameters for a transaction system with software rejuvenation. The numerical results showed that the proposed approach effectively revealed critical model parameters and provided significant insights into performance optimization and parameter estimation.

The ORIS tool

ORIS is a tool for quantitative modeling and evaluation of concurrent systems with non-Markovian durations. It provides a Graphical User Interface (GUI) for model specification as Stochastic Time Petri Nets (STPNs), validation by interactive simulation, and evaluation by several techniques, computing instantaneous and cumulative rewards. It also provides an open-source Java Application Programming Interface (API) to automate the workflow, and it can be used as a toolkit for derivation and evaluation of STPNs in model driven engineering. As distinguishing features, ORIS implements transient and steady-state analysis of STPNs with underlying Markov Regenerative Process (MRP), and transient analysis of STPNs with underlying Generalized Semi- Markov Process (GSMP). It also implements nondeterministic analysis of Time Petri Nets (TPNs), simulation of STPNs, and solution methods for Continuous-Time Markov Chains (CTMCs) and MRPs with at most one non-exponential timer in each state. The well-engineered software architecture of ORIS supports agile implementation of new STPN features, new modeling formalisms, and new analysis methods.

The ORIS Tool: Quantitative Evaluation of Non-Markovian Systems

We present the next generation of ORIS, a toolbox for quantitative evaluation of concurrent models with non-Markovian timers. The tool shifts its focus from timed models to stochastic ones, it includes a new graphical user interface, new analysis methods and a Java Application Programming Interface (API). Models can be specified as Stochastic Time Petri Nets (STPNs) through the graphical editor, validated using an interactive token game, and analyzed through several techniques to compute instantaneous or cumulative rewards. STPNs can also be exported as Java code to conduct extensive parametric studies through the Java library, now distributed as open-source. A well-engineered software architecture allows the user to implement new features for STPNs, new modeling formalisms, and new analysis methods. The most distinctive features of ORIS include transient and steady-state analysis of STPNs modeling Markov Regenerative Processes (MRPs), and transient analysis of STPNs modeling generalized semi-Markov processes. ORIS also supports state-space analysis of Time Petri Nets (TPNs), simulation of STPNs, and standard analysis techniques for continuous-time Markov chains or MRPs with at most one non-exponential timer in each state. We illustrate the general workflow for the application of ORIS to the modeling and evaluation of non-functional requirements of software-intensive systems.

A Markov regenerative process model for phased mission systems under internal degradation and external shocks

Recently, phased mission systems (PMSs) have been widely studied due to their wide range of applications, such as man-made satellites or spacecraft. The lifetime of the PMSs can be separated into several phases, in which their tasks, system configurations, and failure criteria could be different. Meanwhile, the randomly occurred shocks, such as the space radiation from outer space, will cause additional wear and fatal damage on the electronic devices in these systems, which will obviously affect the system reliability. To consider these shocks’ effect in the reliability modeling of the PMSs, a Markov regenerative process (MRGP) based model is proposed in this paper. Firstly, shock models for components and the basic conceptions of MRGP are introduced. Then, a simple cold standby system is used to show the proposed MRGP model for systems under mixed shocks. And a Monte Carlo simulation procedure is applied to verify the results. At last, by integrating the proposed MRGP model and the modular method for PMSs, the reliability of an AOCS in a spacecraft is assessed to show the proposed method.

A Markov regenerative process with recurrence time and its application

This study proposes a non-homogeneous continuous-time Markov regenerative process with recurrence times, in particular, forward and backward recurrence processes. We obtain the transient solution of the process in the form of a generalized Markov renewal equation. A distinguishing feature is that Markov and semi-Markov processes result as special cases of the proposed model. To model the credit rating dynamics to demonstrate its applicability, we apply the proposed stochastic process to Standard and Poor’s rating agency’s data. Further, statistical tests confirm that the proposed model captures the rating dynamics better than the existing models, and the inclusion of recurrence times significantly impacts the transition probabilities.

Availability Analysis of Software Systems with Rejuvenation and Checkpointing

In software reliability engineering, software-rejuvenation and -checkpointing techniques are widely used for enhancing system reliability and strengthening data protection. In this paper, a stochastic framework composed of a composite stochastic Petri reward net and its resulting non-Markovian availability model is presented to capture the dynamic behavior of an operational software system in which time-based software rejuvenation and checkpointing are both aperiodically conducted. In particular, apart from the software-aging problem that may cause the system to fail, human-error factors (i.e., a system operator’s misoperations) during checkpointing are also considered. To solve the stationary solution of the non-Markovian availability model, which is derived on the basis of the reachability graph of stochastic Petri reward nets and is actually not one of the trivial stochastic models such as the semi-Markov process and the Markov regenerative process, the phase-expansion approach is considered. In numerical experiments, we illustrate steady-state system availability and find optimal software-rejuvenation policies that maximize steady-state system availability. The effects of human-error factors on both steady-state system availability and the optimal software-rejuvenation trigger timing are also evaluated. Numerical results showed that human errors during checkpointing both decreased system availability and brought a significant effect on the optimal rejuvenation-trigger timing, so that it should not be overlooked during system modeling.

Service Availability Analysis in a Virtualized System: A Markov Regenerative Model Approach

With the rapid and wide development and deployment of system virtualization, service availability analysis has become increasingly important in a virtualized system (VS) which suffers from software aging. Software rejuvenation techniques can be applied to improve service availability but its effectiveness depends on the rejuvenation policy, which defines when and where to rejuvenate, and which rejuvenation technique to be triggered. This article aims to analyze the optimal inspection time interval for maximizing application service (AS) availability under a three-level rejuvenation policy, in which rejuvenation techniques are deployed at each level, namely, AS, virtual machine (VM), and virtual machine monitor (VMM) levels. We first apply Markov regenerative process to construct an analytical model for the VS. Experiments of injecting memory leaks are conducted to measure aging-related parameters. Furthermore, numerical analysis is carried out to study the quantitative relationship between AS availability and inspection time interval, and determine the approximate optimal inspection time interval.

Adaptive Admission Control for IoT Applications in Home WiFi Networks

Assuring the required quality of service, despite the growing volume and variety of Internet of Things (IoT) traffic, has remained an immense challenge in the popular WiFi networks. The IoT devices using short TCP flows often attain very different levels of service due to the complicated interactions between the transport layer protocol and the shared dynamic wireless medium. We develop in this paper a novel queue management policy by using a transient model to capture the interactions of IoT (short TCP) flows over traditional traffic in WiFi networks. Based on the Markov regenerative processes coupled with fluid model, we discover that the adaptive admission control (AAC) mechanism at the WiFi access point (AP) improves the response time and fairness of IoT traffic over the lossy WiFi links. To this end, under the proposed AAC, packets are admitted into the AP in such a way that the fractions of packets in the AP buffer belonging to the different IoT devices are balanced resulting in a comparable level of service. We furthermore prove the stability of the system dynamics under AAC which provides important practical insights in designing home WiFi IoT system.

A phase expansion for non-Markovian availability models with time-based aperiodic rejuvenation and checkpointing

This paper presents a stochastic framework, consisting of stochastic reward net (SRN) for capturing the transient behaviors of the system and its related non-Markovian state transition diagram, to model an operational software system that undergoes aperiodic time-based rejuvenation and checkpointing schemes, and further to investigate whether there exists the optimal rejuvenation schedule that maximizes the system steady-state availability. A phase expansion approach is adopted to solve the non-Markovian availability models, which are actually neither the semi-Markov processes nor the Markov regenerative processes. Our numerical results show an appropriate rejuvenation trigger timing range, resulting in the positive improvement effect on the system availability of a database system, and that there exists the optimal rejuvenation trigger timing maximizing the system availability.

Markov Regenerative Models of WebServers for Their User-Perceived Availability and Bottlenecks

The Internet world is moving toward a scenario where users and applications have very diverse service expectation, making the current best-effort model inadequate and limiting. To be able to design high-availability service systems, it is essential to consider not only the actual failure and recovery behavior of the service infrastructure, but also the behavioral aspects of its user and their subjective perceptions and reactions in the wake of failure events. In this paper, we propose to use Markov regenerative process (MRGP) models to study the availability of Internet-based services perceived by a Web user on two different online service scenarios: (1) single-user-single-host and (2) single-user-multiple-host. The MRGP models capture the interactions between the service facility and the user. We also detect its parameter bottlenecks by applying the formal sensitivity analysis technique. The trends of the users’ perceived unavailability are analyzed with the changed different parameter values, and the necessity of the sophisticated MRGP modeling is evidenced by the comparisons with the corresponding continuous time Markov chain (CTMC) models, which show that the popular convenient CTMC models tend to overestimate user-perceived service unavailability. Finally, controlled experiments are carried out on a real Web service to demonstrate the proposed approach.

Two-Level Rejuvenation for Android Smartphones and Its Optimization

The Android operating system (OS) is a sophisticated man-made system and is the dominant OS in the current smartphone market. Due to the accumulation of errors in the system internal state and the incremental consumption of resources, such as the Dalvik heap memory of software applications and the physical memory, software aging is observed frequently and recognized as a chronic problem of Android smartphones. To mitigate this problem, we propose a two-level software rejuvenation, with the two levels referring to software applications and the OS, in this paper. Based on this strategy, a Markov regenerative process model is constructed to evaluate the steady-state availability and to optimize the time required to trigger rejuvenation for Android smartphones. The parameters of the model, such as the degradation rate and failure rate of software applications and the Android OS, are obtained via our testing platform. Experiments on two real Android applications show that the availability of an Android smartphone increases by 10.81% and 10.18% for the two subjects in our experiments, respectively. An empirical study comparing our two-level strategy with one-level strategies (single application-level and system-level rejuvenation) further verifies the effectiveness of our approach.

Optimal periodic software rejuvenation policies based on interval reliability criteria

Software aging often affects the performance of software systems and may eventually cause them to fail. A complementary approach to handle transient software failures due to the software aging is called software rejuvenation. It is a preventive and proactive solution that is particularly useful for counteracting the phenomenon of software aging. In this paper, we consider the optimal software rejuvenation policies maximizing the interval reliability using the Markov regenerative process formalism. We derive analytically the optimal software rejuvenation timing that maximizes the limiting interval reliability or the interval reliability with exponentially distributed operation time. Further, we examine numerically the transient behavior of the interval reliability at an arbitrary operation time. Our results under the interval reliability criteria are extensions of some earlier papers, since the interval reliability is a comprehensive measure that specializes to pointwise system availability and to system reliability.

Reliability assessment of phased-mission systems under random shocks

Phased-mission systems (PMSs) are widely used, especially in the aerospace industry. As in the outer space there are many kinds of cosmic rays, such as the Galactic Cosmic Rays (GCR), randomly hitting on these systems and causing significant impact on the electronics inside or outside the equipment, a reliability model for PMSs considering both finite and infinite random shocks is proposed in this paper. The modularization method is used to simplify the state space model for each phase and reduce the amount of system states, and the Markov regenerative process (MRGP) is used to describe the hybrid components’ lifetime distributions and the dynamic behaviors within the modules. Then, two kinds of scenarios, finite and infinite random shocks effect, are both integrated into the dynamic modules. For demonstration, a phased altitude and orbit control system (AOCS) subjected to infinite random shocks is illustrated to demonstrate the procedure of the proposed Monte Carlo simulation. Thirdly, the evaluated system reliability under infinite random shocks is compared with the same system without considering random shocks. At last, a sensitivity analysis is also provided for completion.

Efficient model checking of the stochastic logic CSLTA

CSLTA is a stochastic temporal logic for continuous-time Markov chains (CTMCs) which includes the well known CSL. CSLTA properties are defined using single-clock Deterministic Timed Automata (DTAs). The model checking of CSLTA amounts, in the worst-case, to the computation of the steady-state probability of a non-ergodic Markov Regenerative Process (MRgP) of size of |CTMC|×|DTA|. Various MRgP solution techniques are available in the literature, and we shall use the Component Method, which computes the steady state distribution of a non-ergodic MRgP by recognizing that, in an MRgP, certain components may actually be solved at a lower cost (same cost as that of a CTMC solution). Unfortunately the technique still requires the construction of the whole MRgP. This paper applies the Component Method to devise various CSLTA model checking algorithms, forward and backward. The Component Method can be applied to the MRgP constructed from the CTMC and the DTA, which is a rather straightforward application of the method, or to the MRgP constructed from the CTMC and the region graph of the DTA, a construction that takes into account the timed reachability in the DTA and that allows, in most cases, a significant reduction in the considered MRgP states. In both cases the whole MRgP is built. The primary result of this paper is instead to devise a model checking algorithm in which the component identification is based only on the region graph of the DTA. The MRgP components are generated “on-the-fly”, when needed, starting from the components of the region graph; they are then solved with the cheapest available solution method. Once a component has been solved it is discarded, therefore the whole MRgP is never constructed nor solved. The on-the-fly algorithm is “adaptive”: the time and space used depend on the formula, and, when the DTA actually expresses a CSL property, the algorithm complexity reduces, seamlessly, to that of standard CSL model checking algorithms.

Performability Modeling for RAID Storage Systems by Markov Regenerative Process

This paper presents a performability model for RAID storage systems using Markov regenerative process to compare different RAID architectures. While homogeneous Markov models are extensively used for reliability analysis of RAID storage systems, the memory-less property of the sojourn time assumed in such models is not satisfied in reality, especially in disk rebuild process whose progress is not interrupted even at an event of another disk failure. In this paper, we use Markov regenerative process which allows us to model the generally distributed rebuild times providing a needed extension of the traditional Markov models. The Markov regenerative process is then used to assess the performability of the storage system by assigning reward rates to each state based on the real storage benchmark results. Our numerical study characterizes the performability advantage of RAID6 architecture over RAID10 architecture in terms of sequential read access. Our findings include that the effect of exponential assumption for the rebuild times has practically negligible effect when we focus on data availability. However, the effect this approximation on performability prediction may not be negligible especially when the performance level drastically changes in degraded states. Our MRGP model provides more accurate prediction of performability in such cases.

Markov regenerative credit rating model

PurposeCredit ratings serve as an important input in several applications in risk management of the financial firms. The level of credit rating changes from time to time because of random credit risk and, thus, can be modeled by an appropriate stochastic process. Markov chain models have been widely used in the literature to generate credit migration matrices; however, emergent empirical evidences suggest that the Markov property is not appropriate for credit rating dynamics. The purpose of this article is to address the non-Markov behavior of the rating dynamics.Design/methodology/approachThis paper proposes a model based on Markov regenerative process (MRGP) with subordinated semi-Markov process (SMP) to obtain the estimates of rating migration probability matrices and default probabilities. Numerical example is given to illustrate the applicability of the proposed model with the help of historical Standard & Poor’s (S&P) credit rating data.FindingsThe proposed model implies that rating of a firm in the future not only depends on its present rating, but also on its previous ratings. If a firm gets a rating lower than its previous ratings, there are higher chances of further downgrades, and the issue is called the rating momentum. The model also addresses the ageing problem of credit rating evolution.Originality/valueThe contribution of this paper is a more general approach to study the rating dynamics and overcome the issues of inappropriateness of Markov process applied in rating dynamics.

Optimization of Two-Granularity Software Rejuvenation Policy Based on the Markov Regenerative Process

Software rejuvenation is a proactive software control technique that is used to improve a computing system performance when it suffers from software aging. In this paper, a two-granularity inspection-based software rejuvenation policy, which works as a closed-loop control technique, is proposed. This policy mitigates the negative impact of two-level software aging. The two levels considered are the user-level applications and the operating system. A Markov regenerative process model is constructed based on the system condition. We obtain the degradation rate of the application software and operating system from fault injection experiments. The diagnostic accuracy of the adopted monitor and analysis system, which is applied to inspect the application software and operating system, is considered as we provide the optimal rejuvenation strategies. Finally, the availability and the overall loss probability with their corresponding optimal inspection time intervals are obtained numerically based on the parameter values estimated from the experiments. Experimental results show that two-granularity software rejuvenation is much more effective than traditional single-level software rejuvenation. In our experi-mental study, when two-granularity software rejuvenation is used, the unavailability and the overall loss probability of the system were reduced by 17.9% and 2.65%, respectively, in comparison with the single-level rejuvenation.