Markov Approaches Research Articles

Reconstructing Microstructures From Statistical Descriptors Using Neural Cellular Automata

The problem of generating microstructures of complex materials in silico has been approached from various directions including simulation, Markov, deep learning and descriptor-based approaches. This work presents a hybrid method that is inspired by all four categories and has interesting scalability properties. A neural cellular automaton is trained to evolve microstructures based on local information. Unlike most machine learning-based approaches, it does not directly require a data set of reference micrographs, but is trained from statistical microstructure descriptors that can stem from a single reference. This means that the training cost scales only with the complexity of the structure and associated descriptors. Since the size of the reconstructed structures can be set during inference, even extremely large structures can be efficiently generated. Similarly, the method is very efficient if many structures are to be reconstructed from the same descriptor for statistical evaluations. The method is formulated and discussed in detail by means of various numerical experiments, demonstrating its utility and scalability.

PCN158 Considerations and Challenges for Economic Modeling in NON-Metastatic NON-SMALL CELL LUNG Cancer (NMNSCLC)

Recent regulatory approvals in neoadjuvant non-metastatic cancer treatment have been contingent on early endpoints (e.g., event-free survival [EFS] or pathological complete response [pCR]), but an estimate of overall survival (OS) benefit is required for a health technology assessment model. The objective of this study was to understand challenges in estimating the cost-effectiveness of therapies in non-metastatic cancer using partitioned survival analysis (partSA) and Markov approaches. Neoadjuvant treatment of nmNSCLC was selected as the case study. A targeted literature review revealed no prior cost-effectiveness analyses in this indication. Thus, we developed a de novo model that could estimate the long-term survival outcomes of patients with nmNSCLC using two approaches: three-state partSA and Markov state-transition (health states: EFS, locoregional recurrence [LR], post-recurrence remission, distant metastasis [DM], and death). In both approaches, EFS was stratified by pCR, and inputs were informed by published data evaluating neoadjuvant platinum-doublet chemotherapy. Mortality rates are lower in non-metastatic vs. metastatic cancer; multiple prognostic factors and subsequent treatments should be considered. Both approaches produced similar undiscounted EFS estimates (1.48 years) but different OS benefits (Markov: 2.92 life years [LY]; partSA: 3.42 LYs) over a lifetime time horizon. Predicted 5-year OS was similar in both models (Markov: 17%; partSA: 19%). Data required to inform the Markov post-progression phase are sparse and from heterogeneous populations. PartSA approach produces greater uncertainty due to structural inability to explicitly model the post-progression pathway. Model results are sensitive to selected parametric distribution to extrapolate EFS, the appropriateness of the literature informing the model, and the impact of the extent of progression (e.g., LR vs. DM). Both modeling approaches face challenges. The Markov approach better articulates the post-progression setting but suffers from a lack of complete data; conversely, the PartSA approach is less data-driven, but the results are subject to uncertainty.

The pitch vulnerability of a typical wave energy converter geometry based on Melnikov and Markov approaches

ABSTRACT Many wave energy converter (WEC) designs emerging over the recent years are floating cylindrical geometries with position-keeping mooring system. However, the dynamic stability of these novel floating structures under random sea states has not been investigated adequately. The Melnikov function model and the Markov process model are two efficient approaches providing quantitative predictions of capsizing. In this study, application of the two methods to a moored floating cylinder representing a generic WEC, has been explored to predict the risks of pitchpoling under random excitation. Using the Melnikov approach, the rate of phase space flux was evaluated to quantify the dynamic stability. This approach is compared with the Markov approach, which evaluates the mean first escape (capsize) rate to quantify the dynamic stability. The two methods are investigated through studies using various parameters. The similarity of their results and the differences of their approximations will be explored.

Finding faults: A scoping study of fault diagnostics for Industrial Cyber–Physical Systems

Context:As Industrial Cyber–Physical Systems (ICPS) become more connected and widely-distributed, often operating in safety-critical environments, we require innovative approaches to detect and diagnose the faults that occur in them. Objective:We profile fault identification and diagnosis techniques employed in the aerospace, automotive, and industrial control domains. Each of these sectors has adopted particular methods to meet their differing diagnostic needs. By examining both theoretical presentations as well as case studies from production environments, we present a profile of the current approaches being employed and identify gaps. Methodology:A scoping study was used to identify and compare fault detection and diagnosis methodologies that are presented in the current literature. We created categories for the different diagnostic approaches via a pilot study and present an analysis of the trends that emerged. We then compared the maturity of these approaches by adapting and using the NASA Technology Readiness Level (TRL) scale. Results:Fault identification and analysis studies from 127 papers published from 2004 to 2019 reveal a wide diversity of promising techniques, both emerging and in-use. These range from traditional Physics-based Models to Data-Driven Artificial Intelligence (AI) and Knowledge-Based approaches. Hybrid techniques that blend aspects of these three broad categories were also encountered. Predictive diagnostics or prognostics featured prominently across all sectors, along with discussions of techniques including Fault trees, Petri nets and Markov approaches. We also profile some of the techniques that have reached the highest Technology Readiness Levels, showing how those methods are being applied in real-world environments beyond the laboratory. Conclusions:Our results suggest that the continuing wide use of both Model-Based and Data-Driven AI techniques across all domains, especially when they are used together in hybrid configuration, reflects the complexity of the current ICPS application space. While creating sufficiently-complete models is labor intensive, Model-free AI techniques were evidenced as a viable way of addressing aspects of this challenge, demonstrating the increasing sophistication of current machine learning systems. Connecting ICPS together to share sufficient telemetry to diagnose and manage faults is difficult when the physical environment places demands on ICPS. Despite these challenges, the most mature papers present robust fault diagnosis and analysis techniques which have moved beyond the laboratory and are proving valuable in real-world environments.

Estimating the Case Fatality Rate for COVID-19: A Markov Model Application

Europe has became the new epicentre of the COVID-19 pandemic, according to the WHO on 13th March 2020. Sums and ratios of death and confirmed cases were reported daily, however, such statistics vary significantly by country and it is therefore challenging to understand and measure the risk and severity of the novel disease. Prior to the European outbreak, the COVID-19 virus infected more than 80,000 people in China since late 2019 and took the life of several thousands during the past few months. In this paper, a 3-state model Markov model is applied on the data from China to study the dynamics of the disease and the impact of containment strategies. The long-run stable transition probability obtained from the Markov model provides a convenient approach to estimate the case fatality rate of the COVID-19. Also, the estimated life expectancy give a reasonable estimate of time between first symptom and death. Considering the containment strategy implemented in China, the analysis is done for Hubei province and the rest of China respectively. Comparison of daily estimated results over the whole observation period highlight the impact of the strategy while supporting the measures and controls in place. The proposed Markov model produce reasonable and intuitive estimates that help to measure the virulence of the disease and understand the prevalence overtime. While uncertainty persists as the pandemic goes on, our results show that the Markov approaches provide a useful tool for prognosis and epidemic control.

Solving Ill-Conditioned State-Estimation Problems in Distribution Grids With Hidden-Markov Models of Load Dynamics

Future smart grid operations must rely upon accurate state estimation (SE). Estimation at the distribution level is very challenging since the customary monitoring infrastructure inhibits access to real-time widespread information, precluding redundancy. To overcome the lack of redundancy, we propose to use Markov information on load dynamics instead of load pseudo-measurements directly. The proposed SE methodology Combines a (i) regularization-based method for solving ill-conditioned load-estimation problems with (ii) a Markov model for restricting load variations to probable time-varying load changes. Two Markov approaches are implemented. First, Markov information is used to estimate load one-step ahead, allowing a simple forward dynamic estimation. Then, the SE problem is formulated as a Hidden Markov problem and solved with the Viterbi algorithm to estimate load evolution. Several monitoring contexts are finally analyzed to illustrate the approaches proposed and to discuss the limitations of their application in realistic contexts.

Parametric roll vulnerability of ships using Markov and Melnikov approaches

The designs of modern container ships, roll-on–roll-off vessels and cruise vessels have evolved over the years, and in recent times, some of them have been observed to experience dynamic instabilities during operation in the open ocean. These catastrophic events demonstrate that satisfying prescriptive stability rules set forth by International Maritime Organization (IMO), national authorities (e.g., Coast Guard) and other classification societies are not sufficient to ensure dynamic stability of ships at sea. In light of these events, IMO is organizing efforts to make way toward a second generation of intact stability criteria that are better equipped to deal with these dynamic instabilities. This paper discusses the development of such a tool for parametric rolling in a realistic random seaway, which is one of the critical phenomena identified by IMO. In this study, a previously developed analytical model for roll restoring moment, which was found to be effective in modeling the problem of parametric roll, is analyzed using the Melnikov approach. The stability of the system is quantified in terms of rate of phase space flux of the system. This approach is further compared with another technique known as the Markov approach that is based on stochastic averaging and quantifies stability in terms of mean first passage time. The sensitivity of both of these metrics to environmental parameters is investigated. Finally, the nature of random response is analyzed using Lyapunov exponents to determine whether the vessel exhibits any chaotic dynamics.

Comparing Markov and non-Markov alternatives for cost-effectiveness analysis: Insights from a cervical cancer case

Markov model allows medical prognosis to be modeled with health state transitions over time and are particularly useful for decisions regarding diseases where uncertain events and outcomes may occur. To provide sufficient detail for operations researchers to carry out a Markov analysis, we present a detailed example of a Markov model with five health states with monthly transitions with stationary transition probabilities between states to model the cost and effectiveness of two treatments for advanced cervical cancer. A different approach uses survival curves to directly model the fraction of patients in each state at each time period without the Markov property. We use this alternative method to analyze the cervical cancer case and compare the Markov and non-Markov approaches. These models provide useful insights about both the effectiveness of treatments and the associated costs for healthcare decision makers.

Clustering based interest prediction in social networks

Efficient interest prediction for social networks is critical for both users and service providers for behavior analysis and a series of extension services. However, most existing approaches are inefficient, incomplete or isolated. In this paper, we propose combination of Gaussian and Markov approaches (namely, GAM) as typical soft computing technology for interest prediction of social intelligent multimedia systems. GAM model considers “the number of posted messages” as the only parameter, and defines selection logic to implement either Gaussian or Markov based approaches. Our proposed solution takes the advantage of Gaussian model in prediction accuracy and computation complexity, and advantage of Markov model in high availability. Further experiments illustrate that our solution achieves higher prediction accuracy of 94.3% (without considering the influence of swing users), with the best result achieved ever.

Cohort versus patient level simulation for the economic evaluation of single versus combination immuno-oncology therapies in metastatic melanoma

Background: Model structure, despite being a key source of uncertainty in economic evaluations, is often not treated as a priority for model development. In oncology, partitioned survival models (PSMs) and Markov models, both types of cohort model, are commonly used, but patient responses to newer immuno-oncology (I-O) agents suggest that more innovative model frameworks should be explored.Objective: A discussion of the theoretical pros and cons of cohort level vs patient level simulation (PLS) models provides the background for an illustrative comparison of I-O therapies, namely nivolumab/ipilimumab combination and ipilimumab alone using patient level data from the CheckMate 067 trial in metastatic melanoma. PSM, Markov, and PLS models were compared on the basis of coherence with short-term clinical trial endpoints and long-term cost per QALY outcomes reported.Methods: The PSM was based on Kaplan-Meier curves from CheckMate 067 with 3-year data on progression free survival (PFS) and overall survival (OS). The Markov model used time independent transition probabilities based on the average trajectory of PFS and OS over the trial period. The PLS model was developed based on baseline characteristics hypothesized to be associated with disease as well as significant mortality and disease progression risk factors identified through a proportional hazards model.Results: The short-term Markov model outputs matched the 1–3 year clinical trial results approximately as well as the PSMs for OS but not PFS. The fixed (average) cohort PLS results corresponded as well as the PSMs for OS in the combination therapy arm and PFS in the monotherapy arm. Over the lifetime horizon, the PLS produced an additional 5.95 quality adjusted life years (QALYs) associated with combination therapy relative to ipilimumab alone, resulting in an incremental cost-effectiveness ratio (ICER) of £6,474 per QALY, compared with £14,194 for the PSMs which gave an incremental benefit of between 2.2 and 2.4 QALYs. The Markov model was an outlier (∼ £49,000 per QALY in the base case).Conclusions: The 4- and 5-state versions of the PSM cohort model estimated in this study deviate from the standard 3-state approach to better capture I-O response patterns. Markov and PLS approaches, by modeling state transitions explicitly, could be more informative in understanding I-O immune response, the PLS particularly so by reflecting heterogeneity in treatment response. However, both require a number of assumptions to capture the immune response effectively. Better I-O representation with surrogate endpoints in future clinical trials could yield greater model validity across all models.

Microscopic treatment of energy dissipation and decoherence via many-body Lindblad superoperators

Starting from a recent reformulation of the Markov limit, we apply the mean- field approximation to the resulting Lindblad-type many-electron dynamics, and derive a closed equation of motion for the electronic single-particle density matrix in the presence of one- and two-body scattering mechanisms. The proposed formulation preserves the positive- definite character of the single-particle density matrix. This result is in striking contrast with conventional Markov approaches, where the single-particle mean-field equations can lead to positivity violations and therefore to unphysical results.

Petri net modelling of bridge asset management using maintenance-related state conditions

Bridge structures are an important part of the UK transportation network. They are also experiencing increasing rates of deterioration due to the increasing traffic volume and load intensity. Available bridge models have many restrictions due to the assumptions of the analytical method used and the means by which the model states are defined to represent the condition of the structure. These models also lack the complexity to allow detailed maintenance and renewal options to be explored. This paper presents a bridge model developed based on the Petri net (PN) approach. The method allows for detailed modelling of the individual components in the structure whilst maintaining the size of the analytical problem to a manageable size and resulting in an efficient analysis. The bridge model is formed from sub-models of each of the bridge components and takes into consideration the component deterioration process, the interaction and dependency between different component deterioration processes, along with the inspection and maintenance processes. The model states are defined based on actual degraded component conditions which they experience. It is therefore easy to relate these to the appropriate maintenance options. This gives a considerable advantage over those models based on the condition scores or ratings. The state residence times between changes in state resulting from deterioration and maintenance are governed by appropriate Weibull distributions. Thus, avoiding the restriction of constant failure rates used in Markov approaches which are rarely appropriate to model deteriorating asset conditions. The application of the model is demonstrated on a typical bridge structure where the PN model is solved using Monte Carlo simulation, the model results are also presented and discussed.

Computationally Efficient, Real-Time, and Embeddable Prognostic Techniques for Power Electronics

Power electronics are increasingly important in new generation vehicles as critical safety mechanical subsystems are being replaced with more electronic components. Hence, it is vital that the health of these power electronic components is monitored for safety and reliability on a platform. The aim of this paper is to develop a prognostic approach for predicting the remaining useful life of power electronic components. The developed algorithms must also be embeddable and computationally efficient to support on-board real-time decision making. Current state-of-the-art prognostic algorithms, notably those based on Markov models, are computationally intensive and not applicable to real-time embedded applications. In this paper, an isolated-gate bipolar transistor (IGBT) is used as a case study for prognostic development. The proposed approach is developed by analyzing failure mechanisms and statistics of IGBT degradation data obtained from an accelerated aging experiment. The approach explores various probability distributions for modeling discrete degradation profiles of the IGBT component. This allows the stochastic degradation model to be efficiently simulated, in this particular example ~1000 times more efficiently than Markov approaches.

Probabilistic model–based multi-integration formulas for quantifying a generalized minimal cut sequence

Many sequence- and function-dependent failure behaviors may exist in fault-tolerant systems, in which the failure of the systems depends not only on the combination of basic events but also on their failure sequences. Such systems can be modeled by dynamic fault tree with Priority AND gate, Spare gate, Sequence Enforcing gate and Functional Dependence gate. The existing analytical methods for analyzing these dynamic gates are mainly Markov-based or inclusion–exclusion-based approaches. Compared with Markov approaches, the inclusion–exclusion-based method presents notable advantages. However, to the author’s knowledge, no references by far have presented general multi-integration formulas applicable for solving a generalized minimal cut sequence, which is an unavoidable problem in inclusion–exclusion-based method. In this article, new multi-integration formulas are proposed based on traditional probabilistic models, which are applicable for solving a generalized minimal cut sequence as well as any dynamic gate. The rigorous derivation processes are given. The application and advantages of our proposed approach are demonstrated through a case study and the results indicate our proposed approach is reasonable.

Derivation of nonlinear single-particle equations via many-body Lindblad superoperators: A density-matrix approach

A recently proposed Markov approach provides Lindblad-type scattering superoperators, which ensure the physical (positive-definite) character of the many-body density matrix. We apply the mean-field approximation to such a many-body equation, in the presence of one- and two-body scattering mechanisms, and we derive a closed equation of motion for the electronic single-particle density matrix, which turns out to be nonlinear as well as non-Lindblad. We prove that, in spite of its nonlinear and non-Lindblad structure, the mean-field approximation does preserve the positive-definite character of the single-particle density matrix, an essential prerequisite of any reliable kinetic treatment of semiconductor quantum devices. This result is in striking contrast with conventional (non-Lindblad) Markov approaches, where the single-particle mean-field equations can lead to positivity violations and thus to unphysical results. Furthermore, the proposed single-particle formulation is extended to the case of quantum systems with spatial open boundaries, providing a formal derivation of a recently proposed density-matrix treatment based on a Lindblad-like system-reservoir scattering superoperator.

Modeling of spatio-temporal dynamics of land use and land cover in a part of Brahmaputra River basin using Geoinformatic techniques

An attempt has been made to explore and evaluate the Cellular Automata (CA) Markov modelling to monitor and predict the future land use and land cover (LULC) scenario in a part of Brahmaputra River basin using LULC maps derived from multi-temporal satellite images. CA Markov is a combined cellular automata/Markov chain/multi-criteria/multi-objective land allocation (MOLA) LULC prediction procedure that adds an element of spatial contiguity as well as knowledge base of the likely spatial distribution of transitions to Markov chain analysis. Evidence likelihood map was used for as knowledge base of the likely spatial procedure in CA Markov model. The predicting quantity and predicting location change have been analysed and statistically evaluated. The validation statistics indicated how well the comparison map agreed and disagreed with the reference map. Predicted results accuracy is slightly higher when compare to others studies of LULC change using CA Markov approaches.

Model-based Tracking for Mobile Ad Hoc Networks

We consider the problem of tracking of a mobile target node in a mobile ad hoc network (MANET) set-up. We propose a lightweight probabilisticmodel-based tracking protocol (PMBT) that achieves a short catching path with a high success rate. PMBT is a probabilistic online tracking algorithm that computes information utilities at each step, and then chooses the next step toward the target based on the maximum expected utility. Our algorithm avoids the need to maintain a tracking data structure (such as a hierarchical directory look-up structure) and the need to send periodic update messages about the target’s location. Simulation results show, by taking a hybrid approach that integrates a gradient model and a Markov model, our algorithm significantly outperforms both gradient-based and Markov approaches alone.

Comparative Interactivity Analysis in Multiview Video Coding Schemes

In a multiview video system, interactivity is important for users and should be considered in the design of multiview video coding (MVC). In this paper, we present an interactivity evaluation model for MVC schemes by using both weighted random graph and Markov approaches. The main factors that affect both the interactivity and rate-distortion (RD) performances of MVC schemes are analyzed and discussed in detail. By taking these factors into consideration, a new MVC scheme is proposed for high interactivity and RD gains. Experimental results show that the proposed scheme has a significant interactivity gain with little coding loss, compared to the state-of-the-art benchmark. As an extension to RD performance analysis, the interactivity evaluation model can be used as a design tool of alternative schemes for a future interactive multiview video system.

Markov Approach to Percolation Theory Based Propagation in Random Media

For line-of-sight links in random media or urban areas, propagation may be approximated through sequential reflections of an optical ray in a two-dimensional medium of disordered lossless scatterers. Franceschetti approximated such percolation-based optical-ray propagation by a Markov process with two absorbing barriers, provided numerical solutions for the probability of a ray passing through the percolation lattice and solved-both approximately and exactly-a corresponding problem based on the theory of martingales. In this paper we solve exactly the Markov-theoretical formulation of the problem and prove that both the Markov and martingale approaches are equivalent. Our proof is an application of the Perron-Frobenius theory which provides an elegant framework for the study of the asymptotic behavior of stochastic processes. We demonstrate that for a wide range of vacancies and incident angles the exact solution of the Markov-theoretical formulation performs significantly better than the commonly used Wald approximation in the martingale approach. This has a number of implications on the accuracy of the model, especially for low density propagation media.

Modeling and Specification of Time-Limited Dispatch Categories for Commercial Aircraft

Time-limited dispatch allows the degraded redundancy dispatch of aircraft. Aircraft can be dispatched with certain control system faults and fault combinations for specified periods of time if the failure rates from those configurations meet certification requirements. The various system faults and fault combinations are assigned to dispatch categories according to these failure rates. This gives the dispatch criteria for the system. The overall failure rate of the system can then be calculated according to the dispatch criteria. Dispatch criteria are allocated to a small example system, and the system is subsequently modeled using a reduced-state Markov approach currently recommended in SAE ARP5107. An alternative method of setting dispatch criteria and modeling systems, using Monte Carlo simulation, is proposed in this paper, and this technique is also applied to the example system. Dispatch criteria applied to the different models are seen to differ, as are the system failure rates calculated using the different models. A method for setting the dispatch criteria for a system using a Monte Carlo simulation approach is introduced. The method is applied to a simple system, giving auditable results that exhibit the expected behavior for such a system. Because restrictive assumptions in the mathematics are unnecessary with Monte Carlo simulation, it is expected to give more accurate results in comparison to Markov approaches. Also, the results of the reduced-state Markov model appear to be largely dependent on failure rates, which are very difficult to determine.