Markovian Modeling Research Articles

Stochastic modeling of multiple-server charging stations for electric vehicle networks using feedback strategies: A queueing-theoretic approach

Nowadays, electric vehicles (EVs) significantly affect transportation as they provide a more environmentally friendly alternative to traditional fossil-fueled automobiles. Electric vehicles, which depend on energy stored in batteries, significantly contribute to environmental preservation and comply with worldwide efforts to tackle climate change. However, the growing demand for electric vehicles causes traditional power grids under pressure emphasizing the necessity of establishing a suitable infrastructure for charging electric vehicles. Charging stations are becoming increasingly critical since they allow for the recharging of electric vehicles and play a significant role in stabilizing the power system. In order to optimize charging station infrastructure with multiple servers, the current research incorporates a Markovian queueing modeling approach. The primary objective of the study is to address queue management concerns and boost overall productivity. Considering the real-world challenges, a queue-based stochastic model for multi-server EV systems and individual feedback strategies is developed. Subsequently, a transition state diagram is provided by balancing the input-output rates between the adjacent states. Next, the system of Chapman-Kolmogorov differential-difference equations is formulated to help understand mathematical modeling better. The matrix method is employed to demonstrate the state probability distribution in equilibrium. The infographics are utilized and incorporated for better visualization of the research findings. For a better understanding from an individual's point of view, numerous managerial insights are provided. Lastly, several concluding remarks and future perspectives are provided that can help decision-makers and practitioners to construct and analyze economic strategies based on EV management systems.

A Kernelized Classification Approach for Cancer Recognition Using Markovian Analysis of DNA Structure Patterns as Feature Mining.

Nucleotide-based molecules called DNA and RNA are essential for several biological processes that affect both normal and cancerous cells. They contain the critical genetic material needed for normal cell growth and functioning. The DNA structure patterns that make up the genetic code affect cells' growth, behavior, and control. Different DNA structure patterns indicate different physiological effects in the cell. Knowledge of these patterns is necessary to identify the molecular origins of cancer and other disorders. Analyzing these patterns can help in the early detection of diseases, which is essential for the effectiveness of cancer research and therapy. The novelty of this study is to examine the patterns of dinucleotide structure in many genomic regions, including the non-coding region sequence (N-CDS), coding region sequence (CDS), and whole raw DNA sequence (W.R. sequence). It provides an in-depth discussion of dinucleotide patterns related to these diverse genetic environments and contains malignant and non-malignant DNA sequences. The Markovian modeling that predicts dinucleotide probabilities also reduces feature complexity and minimizes computational costs compared to the approaches of Kernelized Logistic Regression (KLR) and Support Vector Machine (SVM). This technique is effectively evaluated in essential case studies, as indicated by accuracy metrics and 10-fold cross-validation. The classifier and feature reduction, which are generated by Markovian probability, operate well together and can help predict cancer. Our findings successfully distinguish DNA sequences related to cancer from those diagnostics of non-cancerous diseases by analyzing the W.R. DNA sequence as well as its CDS and N-CDS regions.

Cooperative Localization under Ionospheric Scintillation Events

Ionospheric scintillation causes major impairments to Global Navigation Satellite System (GNSS) in low-latitude regions. In severe scenarios, this event can lead to complete loss of lock, thus making GNSS measurements unusable for navigation. In this paper, we derive a cooperative localization algorithm where a set of partially connected aircraft exchange messages with neighboring nodes on the network to improve their own position estimates. We consider the scintillation events as abrupt changes in the measurement variance, which are modeled by a discrete-valued Markov process at the nodes which have access to GNSS measurements. Simulation results show that Markovian modeling and cooperation via factor graph message passing reduce the average 3D root mean square localization error and yield an average vertical position error that meets civil aviation standards for approach and landing.

Markov State Models and Perturbation-Based Approaches Reveal Distinct Dynamic Signatures and Hidden Allosteric Pockets in the Emerging SARS-Cov-2 Spike Omicron Variant Complexes with the Host Receptor: The Interplay of Dynamics and Convergent Evolution Modulates Allostery and Functional Mechanisms.

The new generation of SARS-CoV-2 Omicron variants displayed a significant growth advantage and increased viral fitness by acquiring convergent mutations, suggesting that the immune pressure can promote convergent evolution leading to the sudden acceleration of SARS-CoV-2 evolution. In the current study, we combined structural modeling, microsecond molecular dynamics simulations, and Markov state models to characterize conformational landscapes and identify specific dynamic signatures of the SARS-CoV-2 spike complexes with the host receptor ACE2 for the recently emerged highly transmissible XBB.1, XBB.1.5, BQ.1, and BQ.1.1 Omicron variants. Microsecond simulations and Markovian modeling provided a detailed characterization of the functional conformational states and revealed the increased thermodynamic stabilization of the XBB.1.5 subvariant, which can be contrasted to more dynamic BQ.1 and BQ.1.1 subvariants. Despite considerable structural similarities, Omicron mutations can induce unique dynamic signatures and specific distributions of the conformational states. The results suggested that variant-specific changes of the conformational mobility in the functional interfacial loops of the receptor-binding domain in the SARS-CoV-2 spike protein can be fine-tuned through crosstalk between convergent mutations which could provide an evolutionary path for modulation of immune escape. By combining atomistic simulations and Markovian modeling analysis with perturbation-based approaches, we determined important complementary roles of convergent mutation sites as effectors and receivers of allosteric signaling involved in modulation of conformational plasticity and regulation of allosteric communications. This study also revealed hidden allosteric pockets and suggested that convergent mutation sites could control evolution and distribution of allosteric pockets through modulation of conformational plasticity in the flexible adaptable regions.

Analyzing White-rumped Vulture breeding behavior using Markovian modeling

Understanding wildlife behavior, including accurate identification, processing, and interpretation of activities or cues, is important to behavioral biology and corresponding conservation strategies. We characterized the breeding activities of the critically endangered White-rumped Vulture Gyps bengalensis following a sequential pattern from courtship to fledging. We recorded 4,160 visual observations of 20 behaviors of eight pairs of White-rumped Vultures from September 2021–April 2022 and constructed Markov chain models to model three composite behaviors (i.e., breeding, foraging, and roosting). We found that vultures at four nests displayed >70% of the time in breeding behavior, and each nest produced offspring, indicating a potential correlation between breeding behavior and successful reproductive outcomes. Our model explained each composite behavior with high accuracy. Identifying behaviors White-rumped Vulture have practical applications for developing management plans for their conservation, including the timing of critical reproductive events. Our findings and approach can improve our understanding of White-rumped Vulture behavioral ecology and conservation and have applications for other species.

Markovian modeling of match results with an application

Our main goal in this paper is to propose a mathematical model in order to use it to study the historical achievements of a particular team in order to stand on the evolutionary behavior of the team and to be able to predict the future results of the team by measuring the team’s performance through the total number of victories, losses and draws in official matches approved. Our main idea is to use a Markov model to find the results of matches, so that the outcome of each match is one of the specified cases: win or lose or draw.

New Diagnostic Tool for Ion Channel Activity Hidden Behind the Dwell-Time Correlations.

The patch-clamp technique is a powerful tool that allows for a long observation of transport protein activity in real time. Experimental traces of single-channel currents can be considered as a record of the channel’s conformational switching related to its activation and gating. In this work, we present a mathematically simple method of patch-clamp data analysis that assesses the connectivity and occupancy of distinct conformational substates of the channel. The proposed approach appears to be a big step forward due to its possible applications in the determination of channel substates related to disease and in the analysis of drug–channel interactions on the level of repetitive sequences of channel conformations. This is especially important in cases when molecular dynamics docking is impossible and Markovian modeling requires ambiguous optimization tasks.

Personalized next-best action recommendation with multi-party interaction learning for automated decision-making.

Automated next-best action recommendation for each customer in a sequential, dynamic and interactive context has been widely needed in natural, social and business decision-making. Personalized next-best action recommendation must involve past, current and future customer demographics and circumstances (states) and behaviors, long-range sequential interactions between customers and decision-makers, multi-sequence interactions between states, behaviors and actions, and their reactions to their counterpart’s actions. No existing modeling theories and tools, including Markovian decision processes, user and behavior modeling, deep sequential modeling, and personalized sequential recommendation, can quantify such complex decision-making on a personal level. We take a data-driven approach to learn the next-best actions for personalized decision-making by a reinforced coupled recurrent neural network (CRN). CRN represents multiple coupled dynamic sequences of a customer’s historical and current states, responses to decision-makers’ actions, decision rewards to actions, and learns long-term multi-sequence interactions between parties (customer and decision-maker). Next-best actions are then recommended on each customer at a time point to change their state for an optimal decision-making objective. Our study demonstrates the potential of personalized deep learning of multi-sequence interactions and automated dynamic intervention for personalized decision-making in complex systems.

Time-Slot Reservation and Channel Switching Using Markovian Model for Multichannel TDMA MAC in VANETs

Efficient communication in the intelligent transport system requires the vehicles to transmit safety and non-safety messages timely without any loss. In multi-channel MAC, SCH channels have different properties in terms of the frequency range, frequency limit, power limit, distance covered, and channel parameters. Based on these channel properties, messages must be transmitted using the most appropriate channel. Existing multi-channel MAC approaches randomly choose the available SCH channel to transmit the messages. Random selection of the channel leads to overuse of the channel resources or lack of the resources due to inappropriate channel selection. The proposed approach provides a way to reserve the most appropriate SCH leading to optimized resource utilization. For collision-free allocation of resources to a large number of contending vehicles, we have proposed a variable size TDMA approach for SCH channel allocation. The paper also focuses on reducing the transmission collision and re-usability of the time slot for timely message transmission. Markovian modeling determines the probability of successfully reserving the time slot and switching to the required SCH. Simulation results prove that the probability of time slot reservation in the required SCH is approximately 0.6 for the proposed approach and approximately 0.2 for other approaches.

Toward Human-Centered Design of Automated Vehicles: A Naturalistic Brake Policy

While safety is the ultimate goal in designing Connected and Automated Vehicles (CAVs), current automotive safety standards fail to explicitly define rules and regulations that ensure the safety of CAVs or those interacting with such vehicles. This study investigates CAV safety in mixed traffic environments with both human-driven and automated vehicles, focusing particularly on rear-end collisions at intersections. The central hypothesis is that the primary reason behind these crashes is the potential mismatch between CAVs’ braking decisions and human drivers’ expectations. To test this hypothesis, various Artificial Intelligence (AI) techniques, along with specialized statistical methods are adopted to learn and model the braking behavior of human drivers at intersections and compare the results to that of CAVs. Findings suggest systematical differences in CAVs’ and humans’ braking trajectories, revealing a mismatch between their braking patterns. Accordingly, a Markovian decision modeling framework is adopted to design a novel CAV braking profile that ensures 1) compatibility with human expectation, and 2) safe and comfortable maneuvers by CAVs in mixed driving environments. The findings of this study are expected to facilitate the development of higher levels of vehicle automation by providing guidelines to prevent rear-end collisions caused by existing differences in CAVs’ and humans’ braking strategies.

A finite state method in improvement and design of lean Bernoulli serial production lines

Production systems have a significant impact on national and global economies. It is, therefore, of particular interest to develop, master, and apply new methodologies improving the performance of the production systems. A systematic approach to the improvability and design for lean production in the case of the serial Bernoulli production lines is presented in this research using the finite state method and application of the Markovian modeling approach. A new concept of the bottleneck identification procedure is presented based on the analytical relationship between line properties and performance measures. Also, a new design methodology combining features of the finite state method and genetic evolution algorithm is developed to enable the efficient lean design of the serial Bernoulli production lines. The developed procedures are applied in the case of an automotive paint shop system including bottleneck identification concerning the production rate, the work-in-process, and the probabilities of blockage and starvation. Finally, the lean automotive paint shop system is obtained in an efficient and designer-friendly way.

Fuzzy Markovian modeling of machining system with imperfect coverage, spare provisioning and reboot

Reliability prediction of fault-tolerant system (FTS) supported by a backup unit and operating in a fuzzy environment has been investigated by developing Markov models. The FTS is supported by redundancy, reboot provisioning, and a facility of a single server for repairing the failed components. The failures in the FTS are not covered successfully due to a lack of fault detection of the failed unit and recovered automatically by the reboot process. The parametric non-linear programming (P-NLP) method is used for the assessment of the reliability, availability, and maintainability (RAM) in a fuzzified environment. The failure and repair rates having the lack of exactness are fuzzified using fuzzy triangular numbers. The $$\alpha$$ -cut is used to analyze the fuzzy Markov model. P-NLP approach is proposed in order to facilitate the required defuzzified system performance indices. The performance indices obtained analytically are examined by computing numerical results.

Markovian modeling for dependent interrecurrence times in bladder cancer

A methodology to model a process in which repeated events occur is presented. The context is the evolution of non‐muscle‐invasive bladder carcinoma (NMIBC), characterized by recurrent relapses. It is based on the statistical flowgraph approach, a technique specifically suited for semi‐Markov processes. A very useful feature of the flowgraph framework is that it naturally incorporates the management of censored data. However, this approach presents two difficulties with the process to be modeled. On one hand, the management of covariates is not straightforward. However, it is of great interest to know how the characteristics of a certain patient influence the evolution of the disease. On the other hand, repeated events on the same subject are generally not independent, in which case the semi‐Markov framework is not sufficient because the semi‐Markov assumption implies independence among waiting time distributions. We solve this issue by extending the flowgraph methodology using the Markovian arrival process (MAP), which does successfully model the dependence between events. Along the way, we provide a procedure to consider covariates and censored times in MAPs, a pending task needed in this field. In short, we have managed to extend the flowgraph methodology beyond the semi‐Markovian framework, simplifying the incorporation of covariates and keeping the management of censored times. All of which has allowed us to build a multistate model of the evolution of NMIBC. The developed model allows us to compute the Survival function for any evolution of a patient with specific clinic‐pathological characteristics in this primary tumor.

Modeling the Switching Behavior of Functional Connectivity Microstates (FCμstates) as a Novel Biomarker for Mild Cognitive Impairment.

The need for designing and validating novel biomarkers for the detection of mild cognitive impairment (MCI) is evident. MCI patients have a high risk of developing Alzheimer’s disease (AD), and for that reason the introduction of novel and reliable biomarkers is of significant clinical importance. Motivated by recent findings on the rich information of dynamic functional connectivity graphs (DFCGs) about brain (dys) function, we introduced a novel approach of identifying MCI based on magnetoencephalographic (MEG) resting state recordings. The activity of different brain rhythms {δ, 𝜃, α1, α2, β1, β2, γ1, γ2} was first beamformed with linear constrained minimum norm variance in the MEG data to determine 90 anatomical regions of interest (ROIs). A DFCG was then estimated using the imaginary part of phase lag value (iPLV) for both intra-frequency coupling (8) and cross-frequency coupling pairs (28). We analyzed DFCG profiles of neuromagnetic resting state recordings of 18 MCI patients and 22 healthy controls. We followed our model of identifying the dominant intrinsic coupling mode (DICM) across MEG sources and temporal segments, which further leads to the construction of an integrated DFCG (iDFCG). We then filtered statistically and topologically every snapshot of the iDFCG with data-driven approaches. An estimation of the normalized Laplacian transformation for every temporal segment of the iDFCG and the related eigenvalues created a 2D map based on the network metric time series of the eigenvalues (NMTSeigs). The NMTSeigs preserves the non-stationarity of the fluctuated synchronizability of iDCFG for each subject. Employing the initial set of 20 healthy elders and 20 MCI patients, as training set, we built an overcomplete dictionary set of network microstates (n μstates). Afterward, we tested the whole procedure in an extra blind set of 20 subjects for external validation. We succeeded in gaining a high classification accuracy on the blind dataset (85%), which further supports the proposed Markovian modeling of the evolution of brain states. The adaptation of appropriate neuroinformatic tools that combine advanced signal processing and network neuroscience tools could properly manipulate the non-stationarity of time-resolved FC patterns revealing a robust biomarker for MCI.

Estimating the earthquake occurrence rates in Corinth Gulf (Greece) through Markovian arrival process modeling

ABSTRACTThe Markovian Arrival Process (MAP) is applied as a candidate model to describe the time-varying earthquake activity in Corinth Gulf, Greece. To the best of our knowledge, this is the first attempt to study the earthquake temporal evolution with the specific class of MAPs. A complete catalogue is used for the earthquake temporal distribution investigation, along with data sets of different magnitude cutoffs. The study area is divided into its western and eastern subareas, and possible variations in the earthquake occurrence times were sought. Hidden states of MAPs correspond to different levels of seismicity, and hence various numbers of states are examined. Akaike and Bayes information criteria are implemented for identifying the best model, and comparison to the most known and broadly accepted theoretical interevent time distributions is provided. In all cases, the fitted MAPs with phase type distributed intearrival times outperform the models with other distributions. Important indicators of the underlying Markov process are computed, and the earthquake frequency is approximated by the counting process. The analysis demonstrates high index of burstiness for the earthquake generation in the eastern part, i.e. long quiescent periods alternate with short ones of intense seismic activity.

Markovian Software Reliability Modeling with Change-Point

We discuss a Markovian modeling approach for software reliability assessment with the effects of change-point and imperfect debugging environment. Testing-time when the characteristic of the software failure-occurrence or fault-detection phenomenon changes notably is called change-point. Taking into account the effect at change-point in software reliability growth modeling is important to improve the accuracy of software reliability assessment. Our modeling approach describes a software reliability growth process with not only the effect of change-point but also the imperfect debugging activities based on a semi-Markov process for reflecting actual situation of debugging activities. Finally, we show numerical examples of our model for software reliability analysis and check the performance of our model with an existing Markovian software reliability growth model by using actual data.

Oriented Triplet Markov Fields

Hidden Markov Field modeling is widely used for image segmentation. However, it sometimes lacks power to handle complex situations, e.g. correlated noise, textures or non-stationarities. This is why Pairwise, and then Triplet Markov Fields were introduced to handle in a generic fashion more complex observations. In this paper, we tackle the problem of anisotropic image modeling by introducing an Oriented Triplet Markov Field model, able to explicitly deal with oriented structures. Using oriented features in the framework of Triplet Markov Field modeling, we compare the behavior of this model towards other Markovian modeling on images containing such oriented pattern. We present experiments on synthetic data for segmentation, and application to real data from remote sensing images.

Explicit Fourth-Order Runge\u2013Kutta Method on Intel Xeon Phi Coprocessor

This paper concerns an Intel Xeon Phi implementation of the explicit fourth-order Runge–Kutta method (RK4) for very sparse matrices with very short rows. Such matrices arise during Markovian modeling of computer and telecommunication networks. In this work an implementation based on Intel Math Kernel Library (Intel MKL) routines and the authors’ own implementation, both using the CSR storage scheme and working on Intel Xeon Phi, were investigated. The implementation based on the Intel MKL library uses the high-performance BLAS and Sparse BLAS routines. In our application we focus on OpenMP style programming. We implement SpMV operation and vector addition using the basic optimizing techniques and the vectorization. We evaluate our approach in native and offload modes for various number of cores and thread allocation affinities. Both implementations (based on Intel MKL and made by the authors) were compared in respect of the time, the speedup and the performance. The numerical experiments on Intel Xeon Phi show that the performance of authors’ implementation is very promising and gives a gain of up to two times compared to the multithreaded implementation (based on Intel MKL) running on CPU (Intel Xeon processor) and even three times in comparison with the application which uses Intel MKL on Intel Xeon Phi.

Markovian and Non-Markovian Modeling of Membrane Dynamics with Milestoning.

We exploit atomically detailed simulations and the milestoning theory to extract coarse grained models of membrane kinetics and thermodynamics. Non-Markovian and Markovian theories for the phosphate group displacements are used to coarsely represent membrane motions. The construction of the two models makes it possible to examine their consistency and accuracy. The equilibrium and fluctuations of the phosphate groups along the normal to the membrane plane are estimated, and milestoning equations are constructed and solved. An optimal Markovian model is constructed that reproduces exactly the equilibrium and mean first passage time (MFPT) of the non-Markovian model. The equilibrium solution of both models is favorably compared to distributions obtained from straightforward molecular dynamics simulations. The picture for the kinetics is complex. Multiple local relaxation times of the mass density are illustrated emphasizing the non-Markovian characteristics of the process. In Markovian modeling, only a single relaxation time is assumed for a state. Mapping of particle dynamics to the dynamics of a field density offers a new way of coarse graining complex systems as membranes that may bridge between atomically detailed models and phenomenological descriptions of macroscopic membranes.

Markovian modeling of seismic damage accumulation

SummaryAnalysis of civil structures at the scale of life‐cycle requires stochastic modeling of degradation. Phenomena causing structures to degrade are typically categorized as aging and point‐in‐time overloads. Earthquake effects are the members of the latter category this study deals with in the framework of performance‐based earthquake engineering (PBEE). The focus is structural seismic reliability, which requires modeling of the stochastic process describing damage progression, because of subsequent events, over time. The presented study explicitly addresses this issue via a Markov‐chain‐based approach, which is able to account for the change in seismic response of damaged structures (i.e. state‐dependent seismic fragility) as well as uncertainty in occurrence and intensity of earthquakes (i.e. seismic hazard). The state‐dependent vulnerability issue arises when the seismic hysteretic response is evolutionary and/or when the damage measure employed is such that the degradation increment probabilistically depends on the conditions of the structure at the time of the shock. The framework set up takes advantage also of the hypotheses of classical probabilistic seismic hazard analysis, allowing to separate the modeling of the process of occurrence of seismic shocks and the effect they produce on the structure. It is also discussed how the reliability assessment, which is in closed‐form, may be virtually extended to describe a generic age‐ and state‐dependent degradation process (e.g. including aging and/or when aftershock risk is of interest). Illustrative applications show the options to calibrate the model and its potential in the context of PBEE. Copyright © 2015 John Wiley & Sons, Ltd.