Order Markov Model Research Articles

Call combination patterns in Icelandic killer whales (Orcinus orca)

Acoustic sequences have been described in a range of species and in varying complexity. Cetaceans are known to produce complex song displays but these are generally limited to mysticetes; little is known about call combinations in odontocetes. Here we investigate call combinations produced by killer whales (Orcinus orca), a highly social and vocal species. Using acoustic recordings from 22 multisensor tags, we use a first order Markov model to show that transitions between call types or subtypes were significantly different from random, with repetitions and specific call combinations occurring more often than expected by chance. The mixed call combinations were composed of two or three calls and were part of three call combination clusters. Call combinations were recorded over several years, from different individuals, and several social clusters. The most common call combination cluster consisted of six call (sub-)types. Although different combinations were generated, there were clear rules regarding which were the first and last call types produced, and combinations were highly stereotyped. Two of the three call combination clusters were produced outside of feeding contexts, but their function remains unclear and further research is required to determine possible functions and whether these combinations could be behaviour- or group-specific.

Description of all limit distributions of some Markov chains with memory 2

There are situations where the data sequence does not depend on past values. As can be expected, the additional history of memory often has the advantage of offering a more precise predictive value. By bringing more memory into the random process, one can build a higher order Markov model. In this paper we consider 2-state Markov chain with memory 2 generated by Hamiltonian with competing interactions and describe its all limit distributions.

Improving Prediction of Tobacco Use Over Time: Findings from Waves 1-4 of the Population Assessment of Tobacco and Health Study.

First-order Markov models assume future tobacco use behavior is dependent on current tobacco use and are often used to characterize patterns of tobacco use over time. Higher-order Markov models that assume future behavior is dependent on current and prior tobacco use may better estimate patterns of tobacco use. This study compared Markov models of different orders to examine whether incorporating information about tobacco use history improves model estimation of tobacco use and estimated tobacco use transition probabilities. We used data from four waves of the Population Assessment of Tobacco and Health Study. In each Wave, a participant was categorized into one of the following tobacco use states: never smoker, former smoker, menthol cigarette smoker, non-menthol cigarette smoker, or e-cigarette/dual user. We compared first-, second-, and third-order Markov models using multinomial logistic regression and estimated transition probabilities between tobacco use states. The third-order model was the best fit for the data. The percentage of former smokers, menthol cigarette smokers, non-menthol cigarette smokers, and e-cigarette/dual users in Wave 3 that remained in the same tobacco use state in Wave 4 ranged from 63.4% to 97.2%, 29.2% to 89.8%, 34.8% to 89.7%, and 20.5% to 80.0%, respectively, dependent on tobacco use history. Individuals who were current tobacco users, but former smokers in the prior two years, were most likely to quit. Transition probabilities between tobacco use states varied widely depending on tobacco use history. Higher-order Markov models improve estimation of tobacco use over time and can inform understanding of trajectories of tobacco use behavior. Findings from this study suggest that transition probabilities between tobacco use states vary widely depending on tobacco use history. Tobacco product users (cigarette or e-cigarette/dual users) who were in the same tobacco use state in the prior two years were least likely to quit. Individuals who were current tobacco users, but former smokers in the prior two years, were most likely to quit. Quitting smoking for at least two years is an important milestone in the process of cessation.

Traffic Prediction and Fast Uplink for Hidden Markov IoT Models

In this work, we present a novel traffic prediction and fast uplink (FU) framework for IoT networks controlled by binary Markovian events. First, we apply the forward algorithm with hidden Markov models (HMMs) in order to schedule the available resources to the devices with maximum likelihood activation probabilities via the FU grant. In addition, we evaluate the regret metric as the number of wasted transmission slots to evaluate the performance of the prediction. Next, we formulate a fairness optimization problem to minimize the Age of Information (AoI) while keeping the regret as minimum as possible. Finally, we propose an iterative algorithm to estimate the model hyperparameters (activation probabilities) in a real-time application and apply an online-learning version of the proposed traffic prediction scheme. Simulation results show that the proposed algorithms outperform baseline models, such as time-division multiple access (TDMA) and grant-free (GF) random-access in terms of regret, the efficiency of system usage, and AoI.

Factors That Influence the Choice of Markov Model Order in Discriminating DNA Sequences from Different Sources.

Markov models have frequently been used in genetic sequence analysis. The number of parameters of a Markov model increases exponentially with model order, so it is often recommended that the order be chosen based on the size of data being modeled, lower orders for small and higher orders for large dataset sizes. Approaches based on model selection criterion have also been proposed. An important problem in microbiology and evolutionary biology is to decipher chimeric genomes of microbes, particularly, identify segments of distinct ancestries in genomes and reconstruct the plausible evolutionary scenarios that might have shaped the chimeric genomes in the microbial world. In this study, we assessed a Markov model-based segmentation method for its ability to detect compositionally disparate segments in chimeric sequence constructs as a function of model order, sequence length, and phylogenetic divergence. Our results show that the choice of Markov model order depends on both sequence size and composition. Higher order Markov models were found to be more effective in delineating sequence segments arising from closely related organisms in longer constructs; on the other hand, lower order Markov models were found to be more appropriate in delineating sequence segments arising from distantly related organisms in shorter constructs. These findings are important and timely, with broad implications in fields such as epidemiology that has to deal with the emergence of novel pathogenic chimeras that arise by foreign DNA acquisition, and ecology where chimeric structures may arise in various ecosystems, necessitating more robust approaches for their deconstruction and interpretation.

When History and Heterogeneity Matter: A Tutorial on the Impact of Markov Model Specifications in the Context of Colorectal Cancer Screening.

Markov models are used in health research to simulate health care utilization and disease states over time. Health phenomena, however, are complex, and the memoryless assumption of Markov models may not appropriately represent reality. This tutorial provides guidance on the use of Markov models of different orders and stratification levels in health decision-analytic modeling. Colorectal cancer (CRC) screening is used as a case example to examine the impact of using different Markov modeling approaches on CRC outcomes. This study used insurance claims data from commercially insured individuals in Oregon to estimate transition probabilities between CRC screening states (no screen, colonoscopy, fecal immunochemical test or fecal occult blood test). First-order, first-order stratified by sex and geography, and third-order Markov models were compared. Screening trajectories produced from the different Markov models were incorporated into a microsimulation model that simulated the natural history of CRC disease progression. Simulation outcomes (e.g., future screening choices, CRC incidence, deaths due to CRC) were compared across models. Simulated CRC screening trajectories and resulting CRC outcomes varied depending on the Markov modeling approach used. For example, when using the first-order, first-order stratified, and third-order Markov models, 30%, 31%, and 44% of individuals used colonoscopy as their only screening modality, respectively. Screening trajectories based on the third-order Markov model predicted that a higher percentage of individuals were up-to-date with CRC screening as compared with the other Markov models. The study was limited to insurance claims data spanning 5 y. It was not possible to validate which Markov model better predicts long-term screening behavior and outcomes. Findings demonstrate the impact that different order and stratification assumptions can have in decision-analytic models. This tutorial uses colorectal cancer screening as a case example to provide guidance on the use of Markov models of different orders and stratification levels in health decision-analytic models.Colorectal cancer screening trajectories and projected health outcomes were sensitive to the use of alternate Markov model specifications.Although data limitations precluded the assessment of model accuracy beyond a 5-y period, within the 5-y period, the third-order Markov model was slightly more accurate in predicting the fifth colorectal cancer screening action than the first-order Markov model.Findings from this tutorial demonstrate the importance of examining the memoryless assumption of the first-order Markov model when simulating health care utilization over time.

Context dependency of nucleotide probabilities and variants in human DNA

BackgroundGenomic DNA has been shaped by mutational processes through evolution. The cellular machinery for error correction and repair has left its marks in the nucleotide composition along with structural and functional constraints. Therefore, the probability of observing a base in a certain position in the human genome is highly context-dependent.ResultsHere we develop context-dependent nucleotide models. We first investigate models of nucleotides conditioned on sequence context. We develop a bidirectional Markov model that use an average of the probability from a Markov model applied to both strands of the sequence and thus depends on up to 14 bases to each side of the nucleotide. We show how the genome predictability varies across different types of genomic regions. Surprisingly, this model can predict a base from its context with an average of more than 50% accuracy. For somatic variants we show a tendency towards higher probability for the variant base than for the reference base. Inspired by DNA substitution models, we develop a model of mutability that estimates a mutation matrix (called the alpha matrix) on top of the nucleotide distribution. The alpha matrix can be estimated from a much smaller context than the nucleotide model, but the final model will still depend on the full context of the nucleotide model. With the bidirectional Markov model of order 14 and an alpha matrix dependent on just one base to each side, we obtain a model that compares well with a model of mutability that estimates mutation probabilities directly conditioned on three nucleotides to each side. For somatic variants in particular, our model fits better than the simpler model. Interestingly, the model is not very sensitive to the size of the context for the alpha matrix.ConclusionsOur study found strong context dependencies of nucleotides in the human genome. The best model uses a context of 14 nucleotides to each side. Based on these models, a substitution model was constructed that separates into the context model and a matrix dependent on a small context. The model fit somatic variants particularly well.

Dynamic order Markov model for categorical sequence clustering

Markov models are extensively used for categorical sequence clustering and classification due to their inherent ability to capture complex chronological dependencies hidden in sequential data. Existing Markov models are based on an implicit assumption that the probability of the next state depends on the preceding context/pattern which is consist of consecutive states. This restriction hampers the models since some patterns, disrupted by noise, may be not frequent enough in a consecutive form, but frequent in a sparse form, which can not make use of the information hidden in the sequential data. A sparse pattern corresponds to a pattern in which one or some of the state(s) between the first and last one in the pattern is/are replaced by wildcard(s) that can be matched by a subset of values in the state set. In this paper, we propose a new model that generalizes the conventional Markov approach making it capable of dealing with the sparse pattern and handling the length of the sparse patterns adaptively, i.e. allowing variable length pattern with variable wildcards. The model, named Dynamic order Markov model (DOMM), allows deriving a new similarity measure between a sequence and a set of sequences/cluster. DOMM builds a sparse pattern from sub-frequent patterns that contain significant statistical information veiled by the noise. To implement DOMM, we propose a sparse pattern detector (SPD) based on the probability suffix tree (PST) capable of discovering both sparse and consecutive patterns, and then we develop a divisive clustering algorithm, named DMSC, for Dynamic order Markov model for categorical sequence clustering. Experimental results on real-world datasets demonstrate the promising performance of the proposed model.

The probability of response after each subcutaneous injection of esketamine in treatment-resistant depression

IntroductionThe administration of multiple esketamine doses has shown efficacy for unipolar and bipolar treatment-resistant depression (TRD). Nevertheless, the probability of responding or not after each dose in the real-world remains unknown. This study aimed to estimate it throughout four doses of esketamine, administrated via subcutaneous (SC). Material and methodsWe conducted a retrospective analysis of a case series of 70 patients with TRD who received treatment from the esketamine assistance program at Federal University of Sao Paulo, between April 2017 and December 2018. The SC injections were administrated weekly at a dose of 0.5–1.0mg/kg, in conjunction with patients’ psychotropic drugs. Response was defined as a decrease of at least 50% in the Montgomery-Åsberg Depression Rating Scale between baseline and 24h after dose. We used hidden Markov modeling in order to estimate de probability of response after each esketamine injection. ResultsThe probability of a patient that was a “non-responder” to become a “responder” following a SC injection of esketamine was 17.30% and the probability that this patient remains a “non-responder” was 82.70%. The probability of a patient that was a “responder” to remain as a “responder” was 95%. ConclusionsPatients with TRD who had not responded after the first dose of esketamine, still had a chance of responding after the subsequent dose administrated via SC.

A Hybrid Intelligent Text Watermarking and Natural Language Processing Approach for Transferring and Receiving an Authentic English Text Via Internet

Abstract Due to the rapid increase in the exchange of text information via internet networks, the security and the reliability of digital content have become a major research issue. The main challenges faced by researchers are authentication, integrity verification, and tampering detection of the digital contents. In this paper, a Robust English Text Watermarking and Natural Language Processing Approach (RETWNLPA) is proposed based on word mechanism and first level order of Markov model to improve the accuracy of tampering detection of sensitive English text. The RETWNLPA approach embeds and detects the watermark logically without altering the original text document. Based on the hidden Markov model (HMM), the first-level order of word mechanism is used to analyze the interrelationship between English text. The extracted features are used as watermark information and integrated with text zero-watermarking techniques. To detect eventual tampering, RETWNLPA has been implemented and validated with attacked English text. Experiments were performed on four datasets of varying sizes under random locations of common tampering attacks. The simulation results prove the tampering detection accuracy of our method against all kinds of tampering attacks. Comparison results show that RETWNLPA outperforms baseline approaches HNLPZWA (an intelligent hybrid of natural language processing and zero-watermarking approach) and ZWAFWMMM (Zero-Watermarking Approach based on Fourth level order of Word Mechanism of Markov Model) in terms of tampering detection accuracy.

Prediction of resource contention in cloud using second order Markov model

Prediction of resource contention in cloud using second order Markov model

Webpage Recommendation System Based on the Social Media Semantic Details of the Website

The web page recommendation is generated by using the navigational history from web server log files. Semantic Variable Length Markov Chain Model (SVLMC) is a web page recommendation system used to generate recommendation by combining a higher order Markov model with rich semantic data. The problem of state space complexity and time complexity in SVLMC was resolved by Semantic Variable Length confidence pruned Markov Chain Model (SVLCPMC) and Support vector machine based SVLCPMC (SSVLCPMC) meth-ods respectively. The recommendation accuracy was further improved by quickest change detection using Kullback-Leibler Divergence method. In this paper, socio semantic information is included with the similarity score which improves the recommendation accuracy. The social information from the social websites such as twitter is considered for web page recommendation. Initially number of web pages is collected and the similari-ty between web pages is computed by comparing their semantic information. The term frequency and inverse document frequency (tf-idf) is used to produce a composite weight, the most important terms in the web pages are extracted. Then the Pointwise Mutual Information (PMI) between the most important terms and the terms in the twitter dataset are calculated. The PMI metric measures the closeness between the twitter terms and the most important terms in the web pages. Then this measure is added with the similarity score matrix to provide the socio semantic search information for recommendation generation. The experimental results show that the pro-posed method has better performance in terms of prediction accuracy, precision, F1 measure, R measure and coverage.

Comparison of Recursive Neural Network and Markov Chain Models in Facies Inversion

Predicting the spatial configuration of geological facies is a key step in the reservoir modeling process. The productivity of a reservoir depends not only on the facies proportions but also on the spatial patterns of the facies sequence. The recent developments in seismic to facies inversion techniques use $$1\mathrm{st}$$ -order Markov models to improve the geological realism of the inferred facies profiles. However, the emergence of deep learning techniques such as recursive neural networks shows promising results in predictive modeling of event sequences as shown by the successful applications in complex modeling problems, such as natural language processing. In this work, a comparison between hidden Markov models and recursive neural networks is presented to highlight their advantages and disadvantages. The results are discussed according to the prior assumptions related to facies proportions and sequence patterns. Then, an innovative approach integrating recursive neural networks and the state-of-the-art seismic to facies inversion, known as the convolutional hidden Markov model, is proposed in order to predict geologically more realistic facies sequences based on seismic data. The proposed inversion technique is validated using synthetic seismic data in the context of a complex geological environment.

Reduced-Dimension Filtering in Triplet Markov Models

This article presents an optimal reduced-dimension Kalman filter for a family of triplet Markov models (TMMs). The problem is to estimate the state vector in the case when the auxiliary process in the TMM can be eliminated. Sufficient conditions for this elimination to be feasible are established and we give a selection of illustrative real-life TMM examples, where these conditions are satisfied. We subsequently show that the original TMM boils down to a pairwise Markov model (PMM) of second order. Then, we derive a new optimal Kalman filter applicable to any linear PMM of second order. Our numerical results confirm that the proposed estimator can provide substantial complexity reduction with either no or minor accuracy loss, depending on the use of model approximation.

Profiling Readmissions Using Hidden Markov Model - the Case of Congestive Heart Failure

ABSTRACT Reducing costly hospital readmissions of patients with Congestive Heart Failure (CHF) is important. We analyzed 4,661 CHF patients (from 2007 to 2017) using Hidden Markov Models in order to profile CHF readmission risk over time. This method proved practical in identifying three patient groups with distinctive characteristics, which might guide physicians in tailoring personalized care to prevent hospital readmission. We thus demonstrate how applying appropriate AI analytics can save costs and improve the quality of care.

The probability of response after each subcutaneous injection of esketamine in treatment-resistant depression.

The administration of multiple esketamine doses has shown efficacy for unipolar and bipolar treatment-resistant depression (TRD). Nevertheless, the probability of responding or not after each dose in the real-world remains unknown. This study aimed to estimate it throughout four doses of esketamine, administrated via subcutaneous (SC). We conducted a retrospective analysis of a case series of 70 patients with TRD who received treatment from the esketamine assistance program at Federal University of Sao Paulo, between April 2017 and December 2018. The SC injections were administrated weekly at a dose of 0.5-1.0mg/kg, in conjunction with patients' psychotropic drugs. Response was defined as a decrease of at least 50% in the Montgomery-Åsberg Depression Rating Scale between baseline and 24h after dose. We used hidden Markov modeling in order to estimate de probability of response after each esketamine injection. The probability of a patient that was a "non-responder" to become a "responder" following a SC injection of esketamine was 17.30% and the probability that this patient remains a "non-responder" was 82.70%. The probability of a patient that was a "responder" to remain as a "responder" was 95%. Patients with TRD who had not responded after the first dose of esketamine, still had a chance of responding after the subsequent dose administrated via SC.

Cost-Effectiveness of a Continuous Glucose Monitoring Mobile App for Patients With Type 2 Diabetes Mellitus: Analysis Simulation.

BackgroundApps for real-time continuous glucose monitoring (CGM) on smartphones and other devices linked to CGM systems have recently been developed, and such CGM apps are also coming into use in Japan. In comparison with conventional retrospective CGM, the use of CGM apps improves patients’ own blood glucose control, which is expected to help slow the progression of type 2 diabetes mellitus (DM) and prevent complications, but the effect of their introduction on medical costs remains unknown.ObjectiveOur objective in this study was to perform an economic appraisal of CGM apps from the viewpoint of assessing public medical costs associated with type 2 DM, using the probability of developing type 2 DM–associated complications, and data on medical costs and utility value to carry out a medical cost simulation using a Markov model in order to ascertain the cost-effectiveness of the apps.MethodsWe developed a Markov model with the transition states of insulin therapy, nephrosis, dialysis, and cardiovascular disease, all of which have a major effect on medical costs, to identify changes in medical costs and utility values resulting from the introduction of a CGM app and calculated the incremental cost-effectiveness ratio (ICER).ResultsThe ICER for CGM app use was US $33,039/quality-adjusted life year (QALY).ConclusionsSensitivity analyses showed that, with the exception of conditions where the transition probability of insulin therapy, utility value, or increased medical costs increases, the ICER for the introduction of CGM apps was below the threshold of US $43,478/QALY used by the Central Social Insurance Medical Council. Our results provide basic data on the cost-effectiveness of introducing CGM apps, which are currently starting to come into use.

PKM3: an optimal Markov model for predicting future navigation sequences of the web surfers

Predicting the browsing behavior of the user on the web has gained significant importance, as it improves the productivity of the website owners and also raises the interest of web users. The Markov model has been used immensely for user’s web navigation prediction. To enhance the coverage and accuracy of the Markov model, higher order Markov models are integrated with lower order models. However, this integration results in large state-space complexity. To reduce the state-space complexity, this paper proposes a novel technique, namely Pruned all-Kth modified Markov model (PKM3). PKM3 eliminates the irrelevant states from a higher order model, which have a negligible contribution toward prediction. The proposed model is evaluated on four standard weblogs: BMS, MSWEB, CTI and MSNBC. PKM3 performance was optimal for the website in which pages were closely placed and share high interlinking. This pruning-based optimal model achieves a significant reduction in state-space complexity while maintaining comparable accuracy.

A zero watermarking approach for content authentication and tampering detection of Arabic text based on fourth level order and word mechanism of Markov model

Text information is a natural language dependent. Therefore, improving security and reliability of text information exchanged via Internet network has become one of the most challenges faced by researchers. Content authentication, integrity verification and tampering detection of digital contents have become a major concern in the communication and information exchange area via the Internet.In this paper, an intelligent text Zero-Watermarking approach called ZWAFWMMM (Zero-Watermarking Approach based on Fourth level order of Word Mechanism of Markov Model) has been proposed for content authentication and tampering detection of Arabic text contents. In ZWAFWMMM, watermark embedding and tampering detection were performed logically in which no need to alter the original text document to embed a watermark. Fourth-level order of word mechanism based on hidden Markov model has been integrated with digital zero-watermarking techniques to improve performance, accuracy, capacity and robustness of the previous literature proposed by researchers. Moreover, Fourth-Level order of Markov model is used as a soft computing technique for analysing Arabic text and extracting the interrelationship features from the context of a text, and then utilizing the extracted features as a watermark information to be validated later with attacked Arabic text to detect any tampering that may occur in the text.ZWAFWMMM has been implemented by using PHP with VS code IDE. Robustness, effectiveness, and applicability of ZWAFWMMM are proved with several scenarios of simulations and experiments using four datasets of different lengths under multiple random locations of insertion, reorder and deletion attacks of experimental datasets. The experiment results show that ZWAFWMMM is more sensitive for all kinds of tampering attacks with a high-level accuracy in tampering detection.

Inference for the Analysis of Ordinal Data with Spatio-Temporal Models.

In this work, we propose a spatio-temporal Markovian-like model for ordinal observations to predict in time the spread of disease in a discrete rectangular grid of plants. This model is constructed from a logistic distribution and some simple assumptions that reflect the conditions present in a series of studies carried out to understand the dissemination of a particular infection in plants. After constructing the model, we establish conditions for the existence and uniqueness of the maximum likelihood estimator (MLE) of the model parameters. In addition, we show that, under further restrictions based on Partially Ordered Markov Models (POMMs), the MLE of the model is consistent and normally asymptotic. We then employ the MLE's asymptotic normality to propose methods for testing spatio-temporal and spatial dependencies. The model is estimated from the real data on plants that inspired the model, and we used its results to construct prediction maps to better understand the transmission of plant illness in time and space.