Markov Stochastic Process Research Articles

Reliability modeling and statistical analysis of accelerated degradation process with memory effects and unit-to-unit variability

A reasonable description of the degradation process is essential for credible reliability assessment in accelerated degradation testing. Existing methods usually use Markovian stochastic processes to describe the degradation process. However, degradation processes of some products are non-Markovian due to the interaction with environments. Misinterpretation of the degradation pattern may lead to biased reliability evaluations. Besides, owing to the differences in materials and manufacturing processes, products from the same population exhibit diverse degradation paths, further increasing the difficulty of accurately reliability estimation. To address the above issues, this paper proposes an accelerated degradation model incorporating memory effects and unit-to-unit variability. The memory effect in the degradation process is captured by the fractional Brownian motion, which reflects the non-Markovian characteristic of degradation. The unit-to-unit variability is considered in the acceleration model to describe diverse degradation paths. Then, lifetime and reliability under normal operating conditions are presented. Furthermore, to give an accurate estimation of the memory effect, a new statistical analysis method based on the expectation maximization algorithm is devised. The effectiveness of the proposed method is verified by a simulation case and a real-world tuner reliability analysis case. The simulation case shows that the estimation of the memory effect obtained by the proposed statistical analysis method is much more accurate than the traditional one. Moreover, ignoring unit-to-unit variability can lead to a highly biased estimation of the memory effect and reliability. From the tuner reliability analysis case, the proposed model is superior in both deterministic degradation trend predictions and degradation boundary quantification compared to existing models, which can provide more credible reliability assessment.

Efficiency bounds for bipartite information-thermodynamic systems.

In this paper, we introduce an approach to derive a lower bound for the entropy production rate of a subsystem by utilizing the Cauchy-Schwarz inequality. It extends to establishing comprehensive upper and lower bounds for the efficiency of two subsystems. These bounds are applicable to a wide range of Markovian stochastic processes, which enhances the accuracy in depicting the range of energy conversion efficiency between subsystems. Empirical validation is conducted using a two-quantum-dot system model, which serves to confirm the effectiveness of our inequality in refining the boundaries of efficiency.

Imperfect diffusion-controlled reactions for stochastic processes with memory

Many physical processes are controlled by the time that a random walker needs to reach a target. In many practical situations, such as reaction kinetics, this target is imperfect: multiple random encounters may be necessary to actually trigger a reaction. So far, most analytical approaches of imperfect reaction kinetics have been limited to Markovian (memoryless) stochastic processes. However, as soon as the random walker interacts with its environment, its motion becomes effectively non-Markovian. Here, we present a theory that provides the mean reaction time for a non-Markovian Gaussian random walker in a large confining volume in the presence of a spatially localized reaction rate or a gated target. Remarkably, in the weakly reactive regime, for strongly subdiffusive processes, our theory predicts that the deviation of the mean reaction time to the reaction controlled time displays a non-trivial scaling with the reactivity, which we identify analytically. This effect illustrates how the memory of past passages to the target influences the statistics of next-return times, to the difference of Markovian processes. The theory is developed in one and two dimensions and agrees with stochastic simulations. These results provide a refined understanding of how non-Markovian transport and local reactivity influence the kinetics of diffusion controlled reactions.

Distributed consensus control based on double event-triggered mechanism for multi-agent systems with irregular switching topologies under malicious attacks

This paper studies the consensus control of leader-following multi-agent systems (MASs) with irregular switching communication topology and time-varying delay under malicious attacks. The Markov stochastic process is used to describe the irregular switching of the communication topology caused by random factors, and it is verified that the communication protocol is still feasible when the communication channel is partially broken. At the same time, in order to filter out more efficient status signals and reduce the burden of network communication resources under malicious attacks, the double event-triggered mechanism (DETM) are adopted to make the number of state signals reaching the actuator much less and the state signals more efficient. Then, using the Lyapunov–Krasovskii functional (LKF) and integral inequality methods, the LKF contains the information of the lower bound of delay. Lastly, the stability criterion of MASs stochastic asymptotic consensus with time-varying delay is given, the gain of the consistent controller and the event triggered matrix parameters are solved by using the linear matrix inequality toolbox. Finally, four numerical simulations are given to verify that the MASs with irregular switching topology under malicious attacks can still achieve leader-following consensus.

Cylindrically Symmetric Diffusion Model for Relativistic Heavy‐Ion Collisions

AbstractA relativistic diffusion model with cylindrical symmetry, which propagates an initial state based on quantum chromodynamics in time toward a thermal equilibrium limit, is derived from nonequilibrium–statistical considerations: adapting an existing framework for Markovian stochastic processes representing relativistic phase‐space trajectories, a Fokker–Planck equation is obtained for the time evolution of particle‐number distribution functions with respect to transverse and longitudinal rapidity. The resulting partially‐evolved distribution functions are transformed to transverse‐momentum and pseudorapidity space, and compared with charged‐hadron data from the CERN Large Hadron Collider.

Approximation of multistage stochastic programming problems by smoothed quantization

We present an approximation technique for solving multistage stochastic programming problems with an underlying Markov stochastic process. This process is approximated by a discrete skeleton process, which is consequently smoothed down by means of the original unconditional distribution. Approximated in this way, the problem is solvable by means of Markov Stochastic Dual Dynamic Programming. We state an upper bound for the nested distance between the exact process and its approximation and discuss its convergence in the one-dimensional case. We further propose an adjustment of the approximation, which guarantees that the approximate problem is bounded. Finally, we apply our technique to a real-life production-emission trading problem and demonstrate the performance of its approximation given the “true” distribution of the random parameters.

Stochastic model of Alzheimer's disease progression using two-state Markov chains.

In 2016, Hao and Friedman developed a deterministic model of Alzheimer's disease progression using a system of partial differential equations. This model describes the general behavior of the disease, however, it does not incorporate the molecular and cellular stochasticity intrinsic to the underlying disease processes. Here we extend the Hao and Friedman model by modeling each event in disease progression as a stochastic Markov process. This model identifies stochasticity in disease progression, as well as changes to the mean dynamics of key agents. We find that the pace of neuron death increases whereas the production of the two key measures of progression, Tau and Amyloid beta proteins, decelerates when stochasticity is incorporated into the model. These results suggest that the non-constant reactions and time-steps have a significant effect on the overall progression of the disease.

Stochastic modeling of the receiver clock parameter in Galileo-only and multi-GNSS PPP solutions

In Precise Point Positioning (PPP), the receiver clock parameter is typically estimated independently in each observation epoch, which increases the noise of the estimated station coordinates and troposphere parameters due to correlations. Applying stochastic modeling to the receiver clock parameter stabilizes PPP solutions and reduces clock noise for the time transfer. However, the receiver clock modeling is possible only for the GNSS receivers connected to the utmost stable atomic clocks. We propose receiver clock modeling that involves the Markov stochastic process in the form of a random walk. We test different levels of random walk constraints for GNSS stations equipped with different types of clocks for Galileo-only and multi-GNSS solutions in kinematic and static PPP. In multi-GNSS solutions, the common clock parameter is derived with inter-system biases (ISBs). This raises the question of the constraints that should be imposed on the common clock only or also on the ISBs. We found that similar results can be achieved by imposing constraints on the common clock parameter and estimating ISB as a constant parameter and when constraining the common clock parameter and ISBs with a ratio of 1:100. Other ratios of clock-to-ISB constraints, such as 1:1 and 1:10, give inferior results. In the kinematic PPP, stochastic clock modeling has a marginal impact on the North and East coordinate components, whereas the Up component is substantially improved for GNSS receivers equipped with hydrogen masers. In the static PPP, the clock modeling improves the time transfer, due to the reduced noise of the clocks.

Probabilistic modeling of hardware and software interactions for system reliability assessment

An important aspect of complex hardware-software systems reliability is the interaction between hardware and software. Most of the existing work has assumed either independence between hardware (HW) and software (SW) or a fixed proportion of hardware reported failures to represent HW/SW interactions. These assumptions do not necessarily reflect reality. In this paper, probabilistic HW/SW interactions, in conjunction with hardware and software reliability, are considered to model and assessment overall hardware-software systems reliability. However, incorporating uncertainty demands the use of stochastic programming to estimate the parameters of the hardware reliability model. To capture the HW/SW interactions, a Markov Stochastic process model with uncertainty in transition rates is proposed and solved using Monte Carlo sampling. By combining hardware, software, and probabilistic HW/SW interactions, the overall system reliability can be predicted not only as a point estimator but as quantiles or confidence intervals. The proposed methodology was demonstrated with data from a real computer system provided by Los Alamos National Laboratory. Reliability predictions for a 1,095-day horizon were carried out with an R script in approximately 1.5 s running on a laptop, which demonstrates the feasibility and convenience of the approach. In the four cases analyzed, the 97.5% lower bound of system reliability estimation was above the reliability calculated assuming independent hardware and software reliability.

Modelling intermittent anomalous diffusion with switching fractional Brownian motion

The stochastic trajectories of molecules in living cells, as well as the dynamics in many other complex systems, often exhibit memory in their path over long periods of time. In addition, these systems can show dynamic heterogeneities due to which the motion changes along the trajectories. Such effects manifest themselves as spatiotemporal correlations. Despite the broad occurrence of heterogeneous complex systems in nature, their analysis is still quite poorly understood and tools to model them are largely missing. We contribute to tackling this problem by employing an integral representation of Mandelbrot’s fractional Brownian motion that is compliant with varying motion parameters while maintaining long memory. Two types of switching fractional Brownian motion are analysed, with transitions arising from a Markovian stochastic process and scale-free intermittent processes. We obtain simple formulas for classical statistics of the processes, namely the mean squared displacement and the power spectral density. Further, a method to identify switching fractional Brownian motion based on the distribution of displacements is described. A validation of the model is given for experimental measurements of the motion of quantum dots in the cytoplasm of live mammalian cells that were obtained by single-particle tracking.

Modelling of wind and photovoltaic power output considering dynamic spatio-temporal correlation

The simulation technology of wind and solar power output can provide data support for the planning of new energy stations and the optimization and scheduling of power systems. In order to solve the problem that the existing output models can not accurately describe the dynamic spatio-temporal dependence between wind and solar output, a dynamic spatiotemporal correlation model of wind and solar output based on Markov stochastic process and Copula function theory is proposed. Firstly, wind power and photovoltaic output are regarded as a stochastic process, and the time autocorrelation models of wind power and photovoltaic output are constructed based on a one-dimensional Markov chain and hybrid Copula function. Next, two one-dimensional Markov chains were coupled to form a two-dimensional Markov chain, and a spatial correlation model between wind and solar output was constructed using the dynamic SJC Copula function. Take the measured data of adjacent wind farms and photovoltaic power stations in Hami, Xinjiang as an example for simulation. The simulation results show that the proposed model can effectively reflect the spatio-temporal correlation of the original data and reflect the dynamic changes in the correlation between wind and solar energy.

Generative abstraction of Markov population processes

Markov population models are a widespread formalism used to model the dynamics of complex systems, with applications in systems biology and many other fields. The associated Markov stochastic process in continuous time is often analyzed by simulation, which can be costly for large or stiff systems, particularly when a massive number of simulations has to be performed, e.g. in a multi-scale model. A strategy to reduce computational load is to abstract the population model, replacing it with a simpler stochastic model, faster to simulate. Here we pursue this idea, exploring and comparing state-of-the-art generative models, which are flexible enough to automatically learn distributions over entire trajectories, rather than single simulation steps, from observed realizations of the system. In particular, we compare a Generative Adversarial setting with a Score-based Diffusion approach and show how the latter outperforms the former both in terms of accuracy and stability at the cost of slightly higher simulation times. To improve the accuracy of abstract samples, we develop an active learning framework to enrich our dataset with observations whose expected satisfaction of a temporal requirement differs significantly from the abstract one. We experimentally show how the proposed abstractions are well suited to work on multi-scale and data-driven scenarios, meaning that we can infer a (black-box) dynamical model from a pool of real data.

Adaptive Residual Useful Life Prediction for the Insulated-Gate Bipolar Transistors with Pulse-Width Modulation Based on Multiple Modes and Transfer Learning

Currently, residual useful life (RUL) prediction models for insulated-gate bipolar transistors (IGBT) do not focus on the multi-modal characteristics caused by the pulse-width modulation (PWM). To fill this gap, the Markovian stochastic process is proposed to model the mode transition process, due to the memoryless properties of the grid operation. For the estimation of the mode transition probabilities, transfer learning is utilized between different control signals. With the continuous mode switching, fractional Weibull motion (fWm) of multiple modes is established to model the stochasticity of the multi-modal IGBT degradation. The drift and diffusion coefficients are adaptively updated in the proposed RUL prediction model. In the case study, two sets of the real thermal-accelerated IGBT aging data are used. Different degradation modes are extracted from the meta degradation data, and then fused to be a complex health indicator (CHI) via a multi-sensor fusion algorithm. The RUL prediction model based on the fWm of multiple modes can reach a maximum relative prediction error of 2.96% and a mean relative prediction error of 1.78%. The proposed RUL prediction model with better accuracy can reduce the losses of the power grid caused by the unexpected IGBT failures.

Electrical Response Estimation of Vibratory Energy Harvesters via Hilbert Transform Based Stochastic Averaging

Abstract Converting mechanical vibrations into electrical power with vibratory energy harvesters can ensure the portability, efficiency, and sustainability of electronic devices and batteries. Vibratory energy harvesters are typically modeled as nonlinear oscillators subject to random excitation, and their design requires a complete characterization of their probabilistic responses. However, simulation techniques such as Monte Carlo are computationally prohibitive when the accurate estimation of the response probability distribution is needed. Alternatively, approximate methods such as stochastic averaging can estimate the probabilistic response of such systems at a reduced computational cost. In this paper, the Hilbert transform based stochastic averaging is used to model the output voltage amplitude as a Markovian stochastic process with dynamics governed by a stochastic differential equation with nonlinear drift and diffusion terms. Moreover, the voltage amplitude dependent damping and stiffness terms are determined via an appropriate equivalent linearization, and the stationary probability distribution of the output voltage amplitude is obtained analytically by solving the corresponding Fokker–Plank equation. Two examples are used to demonstrate the accuracy of the obtained analytical probability distributions via comparisons with Monte Carlo simulation data.

Measuring the variability of local characteristics in complex networks: Empirical and analytical analysis.

We examine the dynamics for the average degree of a node's neighbors in complex networks. It is a Markov stochastic process, and at each moment of time, this quantity takes on its values in accordance with some probability distribution. We are interested in some characteristics of this distribution: its expectation and its variance, as well as its coefficient of variation. First, we look at several real communities to understand how these values change over time in social networks. The empirical analysis of the behavior of these quantities for real networks shows that the coefficient of variation remains at high level as the network grows. This means that the standard deviation and the mean degree of the neighbors are comparable. Then, we examine the evolution of these three quantities over time for networks obtained as simulations of one of the well-known varieties of the Barabási-Albert model, the growth model with nonlinear preferential attachment (NPA) and a fixed number of attached links at each iteration. We analytically show that the coefficient of variation for the average degree of a node's neighbors tends to zero in such networks (albeit very slowly). Thus, we establish that the behavior of the average degree of neighbors in Barabási-Albert networks differs from its behavior in real networks. In this regard, we propose a model based on the NPA mechanism with the rule of random number of edges added at each iteration in which the dynamics of the average degree of neighbors is comparable to its dynamics in real networks.

Stable stadial and interstadial states of the last glacial's climate identified in a combined stable water isotope and dust record from Greenland

Abstract. During the last glacial interval, the Northern Hemisphere climate was punctuated by a series of abrupt changes between two characteristic climate regimes. The existence of stadial (cold) and interstadial (milder) periods is typically attributed to a hypothesised bistability in the glacial North Atlantic climate system, allowing for rapid transitions from the stadial to the interstadial state – the so-called Dansgaard–Oeschger (DO) events – and more gradual yet still fairly abrupt reverse shifts. The physical mechanisms driving these regime transitions remain debated. DO events are characterised by substantial warming over Greenland and a reorganisation of the Northern Hemisphere atmospheric circulation, which are evident from concomitant shifts in the δ18O ratios and dust concentration records from Greenland ice cores. Treating the combined δ18O and dust record obtained by the North Greenland Ice Core Project (NGRIP) as a realisation of a two-dimensional, time-homogeneous, and Markovian stochastic process, we present a reconstruction of its underlying deterministic drift based on the leading-order terms of the Kramers–Moyal equation. The analysis reveals two basins of attraction in the two-dimensional state space that can be identified with the stadial and interstadial regimes. The drift term of the dust exhibits a double-fold bifurcation structure, while – in contrast to prevailing assumptions – the δ18O component of the drift is clearly mono-stable. This suggests that the last glacial's Greenland temperatures should not be regarded as an intrinsically bistable climate variable. Instead, the two-regime nature of the δ18O record is apparently inherited from a coupling to another bistable climate process. In contrast, the bistability evidenced in the dust drift points to the presence of two stable circulation regimes of the last glacial's Northern Hemisphere atmosphere.

Spontaneous Stochasticity and Renormalization Group in Discrete Multi-scale Dynamics

We introduce a class of multi-scale systems with discrete time, motivated by the problem of inviscid limit in fluid dynamics in the presence of small-scale noise. These systems are infinite-dimensional and defined on a scale-invariant space-time lattice. We propose a qualitative theory describing the vanishing regularization (inviscid) limit as an attractor of the renormalization group operator acting in the space of flow maps or respective probability kernels. If the attractor is a nontrivial probability kernel, we say that the inviscid limit is spontaneously stochastic: it defines a stochastic (Markov) process solving deterministic equations with deterministic initial and boundary conditions. The results are illustrated with solvable models: symbolic systems leading to digital turbulence and systems of expanding interacting phases.

Ensuring SDN Resilience under the Influence of Cyber Attacks: Combining Methods of Topological Transformation of Stochastic Networks, Markov Processes, and Neural Networks

The article proposes an approach to ensuring the functioning of Software-Defined Networks (SDN) in cyber attack conditions based on the analytical modeling of cyber attacks using the method of topological transformation of stochastic networks. Unlike other well-known approaches, the proposed approach combines the SDN resilience assessment based on analytical modeling and the SDN state monitoring based on a neural network. The mathematical foundations of this assessment are considered, which make it possible to calculate the resilience indicators of SDN using analytical expressions. As the main indicator, it is proposed to use the correct operation coefficient for the resilience of SDN. The approach under consideration involves the development of verbal models of cyber attacks, followed by the construction of their analytical models. In order to build analytical models of cyber attacks, the method of topological transformation of stochastic networks (TTSN) is used. To obtain initial data in the simulation, the SDN simulation bench was justified and deployed in the EVE-NG (Emulated Virtual Environment Next Generation) virtual environment. The result of the simulation is the time distribution function and the average time for the cyber attack implementation. These results are then used to evaluate the SDN resilience indicators, which are found by using the Markov processes theory. In order to ensure the resilience of the SDN functioning, the article substantiates an algorithm for monitoring the state of controllers and their automatic restructuring, built on the basis of a neural network. When one is choosing a neural network, a comparative evaluation of the convolutional neural network and the LSTM neural network is carried out. The experimental results of analytical modeling and simulation are presented and their comparative evaluation is carried out, which showed that the proposed approach has a sufficiently high accuracy, completeness of the obtained solutions and it took a short time to obtain the result.

A Spatio-Temporal Approach For Determining Individual's Covid-19 Risks

Current state of art approaches such as the susceptible-infected-removed model and machine learning models are not optimized for modeling the risks of individuals and modeling the effects of local restrictions. To improve the drawback of these approaches, the feedback processing framework is proposed where previously accumulated global statistics and the model estimates generated from the spatial-temporal data are combined to improve the performance of the local prediction. The proposed framework is evaluated in three processing stages: generation of the simulation dataset, feedback analysis, and evaluation for the spatial-temporal and real-time pandemic analysis. In the data generation stage, the corresponding state of the illness for each person is modeled by a Markov stochastic process. In this stage, the parameters such as the reproduction rate, symptomatic rate, asymptomatic rate, population count, infected count, and the average mobility rate are used to update the individual's Covid-19 status and the individual's movements. The movement data of each person is generated randomly for several places of interest. In the feedback analysis stage, both the aggregated statistics and the local event data are combined in a linear model to infer a score for the Covid-19 probability of the person. In this respect, a stochastic model can be used to approximate the local statistics. In the evaluation stage, the result of the feedback analysis for all the interactions is used to classify the state of the individuals periodically. Later the accuracy of the evaluation for each person is obtained by comparing the individual's prediction with the real data generated in the same time interval. The Kappa scores independent from different populations, locations, and mobility rates obtained for every interaction indicate a significant difference from the random statistics.

Generative Datalog with Continuous Distributions

Arguing for the need to combine declarative and probabilistic programming, Bárány et al. (TODS 2017) recently introduced a probabilistic extension of Datalog as a “purely declarative probabilistic programming language.” We revisit this language and propose a more principled approach towards defining its semantics based on stochastic kernels and Markov processes—standard notions from probability theory. This allows us to extend the semantics to continuous probability distributions, thereby settling an open problem posed by Bárány et al. We show that our semantics is fairly robust, allowing both parallel execution and arbitrary chase orders when evaluating a program. We cast our semantics in the framework of infinite probabilistic databases (Grohe and Lindner, LMCS 2022) and show that the semantics remains meaningful even when the input of a probabilistic Datalog program is an arbitrary probabilistic database.