This work introduces a novel deep learning-based architecture, termed the Deep Belief Markov Model (DBMM), which provides efficient, model-formulation agnostic inference in Partially Observable Markov Decision Process (POMDP) problems. The POMDP framework allows for modeling and solving sequential decision-making problems under observation uncertainty. In complex, high-dimensional, partially observable environments, existing methods for inference based on exact computations (e.g., via Bayes' theorem) or sampling algorithms do not always scale well. Furthermore, ground truth states may not be available for learning the exact transition dynamics. DBMMs extend deep Markov models into the partially observable decision-making framework and allow efficient belief inference entirely based on available observation data via variational inference methods. By leveraging the potency of neural networks, DBMMs can infer and simulate non-linear relationships in the system dynamics and naturally scale to problems with high dimensionality and discrete or continuous variables. In addition, neural network parameters can be dynamically updated efficiently based on data availability. DBMMs can thus be used to infer a belief variable, thus enabling the derivation of POMDP solutions over the belief space. We evaluate the efficacy of the proposed methodology by evaluating the capability of model-formulation agnostic inference of DBMMs in benchmark problems that include discrete and continuous variables. Finally, we demonstrate the practical utility of the inferred beliefs in a downstream decision-making task, showing that an RL agent guided by DBMMs beliefs significantly outperforms powerful model-free baselines and achieves near-optimal performance.1.

Understanding the householder solar panel consumer: A Markovian model and its societal implications

Uncovering the Cognitive Mechanisms of Risk Decision-Making among ICU Nurses in Complex Clinical Contexts.

Dynamics of Hidden Brain States in Subcortical Vascular Cognitive Impairment: Linking Neural Activity to Neurotransmitter Systems and Genetic Pathways.

Cost-effectiveness analysis of oral semaglutide versus subcutaneous dulaglutide in patients with type 2 diabetes mellitus: A Markov model study.

Hidden Markov Model and long short-term memory models for therapy response prediction in septic patients using routinely collected sequential data

Interacting Partially Observable DBN to model the dynamics of partially observable metapopulations: Opportunities and open challenges.

A novel Markov model for reliability evaluation of communication systems in wide-area protection

Dynamically agile operation of mobile apps: a hidden Markov model

Collision-avoidance on-demand spectrum access based on Indian Buffet Markov model in C-V2X

Cost-Utility and Budget Impact Analysis of Tumor Necrosis Factor-Alpha Inhibitors for the Treatment of Refractory Nonsystemic Juvenile Idiopathic Arthritis in Thailand.

Chronic lymphocytic leukaemia (CLL) is a common adult leukaemia, and selecting the most effective first-line treatment is crucial for optimising patient outcomes and managing healthcare costs. While chemoimmunotherapy (CIT) has been the standard approach, targeted therapies offer promising alternatives for treatment-naïve CLL patients. The objective of this study was to evaluate the cost effectiveness of chemoimmunotherapy compared to targeted therapies for treatment-naïve CLL patients in South Africa. A cost-effectiveness analysis was conducted using a Markov model based on three health states: progression-free survival (PFS), progression, and death. The model employed a 15-year time horizon and a 1-month cycle length. Patient-level data were reconstructed, and parametric estimation was used to project long-term clinical outcomes. Cost estimates were derived from national tariffs, reflecting a South African public healthcare perspective, while utilities were sourced from published literature. Outcomes were measured in total costs and quality-adjusted life years (QALYs). Incremental cost-effectiveness ratios (ICERs) were calculated and compared to a willingness-to-pay (WTP) threshold, and sensitivity analyses were conducted to test the robustness of the results. Among the evaluated treatment strategies, chlorambucil-plus-obinutuzumab (ClbO) had the lowest cost and served as the reference comparator. Both CIT regimens were cost effective, with fludarabine, cyclophosphamide, and rituximab (FCR) yielding an ICER of US$1645.52 per QALY gained and bendamustine-plus-rituximab (BR) had an ICER of US$1716.79 per QALY gained, both below the US$3407 WTP threshold, under the model assumptions. Ibrutinib generated the highest QALYs but at a higher cost, resulting in an ICER of US$19,679.52 per QALY gained and a 0% probability of being cost effective at the WTP threshold, while venetoclax-plus-obinutuzumab (VenO) was extendedly dominated, and therefore eliminated from the results. Sensitivity analyses confirmed the robustness of the findings across variations in key parameters. In the South African public healthcare setting, CIT regimens (FCR and BR) represent cost-effective first-line treatment strategies for symptomatic, treatment-naïve CLL. Bendamustine-plus-rituximab emerged as the most decision-robust option under uncertainty, while FCR yielded the lowest point-estimate ICER. Among CIT regimens, FCR may be preferred in fit patients, while BR represents a more decision-robust option in older or unfit populations. Targeted therapies such as ibrutinib and VenO, despite superior clinical efficacy, are not cost effective at current prices. Substantial price reductions, generic entry, or targeted use in high-risk subgroups may improve their value and enable equitable access within South Africa's resource-constrained health system.

Cost-effectiveness of Add-On Ivabradine Versus Standard Pharmacotherapy in the Treatment of Heart Failure in Vietnam: An Analysis From a Health System Perspective.

Due to the rapid growth of sequence generation, which has surpassed the expert curators ability to manually review and annotate them, the computational annotation of proteins remains a significant challenge in bioinformatics nowadays. The Pfam database contains a large collection of proteins that are annotated with domain families through profile Hidden Markov models (pHMMs). Using the aligned sequences of a curated family, one HMM is trained independently for each family, missing the opportunity of learning patterns across families, that is, from a complete view of all the dataset. As an alternative, some deep learning (DL) models have been recently proposed, nevertheless with simple representations of the inputs and moderate improvements in performance. In this work we present ET-Pfam, a novel approach based on transfer learning and ensembles of multiple DL classifiers to predict functional families in the Pfam database. Several base DL models are first trained using learned representations from protein large language models. Then, the base models are integrated using classical ensemble strategies and novel voting approaches by learning weights for each model and for each Pfam family. Results demonstrate that the proposed ET-Pfam method can consistently diminish error rates compared to individual DL models, boosting prediction performance. Among the novel ensemble strategies presented here, the learned weights by family voting achieved the best performance, with the lowest error rate (7.00%), significantly surpassing the best individual base model error (12.91%) and competitors of the state-of-the-art. Data and source code are available at https://github.com/sinc-lab/ET-Pfam. Supplementary data are available at Bioinformatics online.

ABSTRACTAdvances in fine‐scale movement modeling of soaring birds can aid efforts to understand and resolve the impacts of anthropogenic activities on such birds. Soaring birds often rely on underlying terrain and low‐altitude updrafts to govern their flights at rotor‐swept altitudes (≤ 200 m above ground level), which puts them at risk of collision with wind turbines. We developed a data‐driven Markov model at 1‐s resolution that predicts the fine‐scale flight behavior of golden eagles (Aquila chrysaetos) as a function of ecological covariates at the current location as well as those within an eagle's line of sight. We only considered ecological covariates that are readily available in real‐time (ground elevation and wind conditions). Latent factors (age, sex, species, behavioral intent, migratory status) were intentionally left out of the model. We calibrated the model using golden eagle telemetry data collected in two different ecoregions of the United States. Given a starting location, the calibrated model simulates multiple stochastic 3D paths to produce a time‐explicit and spatially explicit risk map of turbine collisions. We discovered an empirical relation between the rate of change of heading and the orographic updraft conditions within an eagle's line of sight. Our model performed most effectively when predicting predominantly‐soaring flights at rotor‐swept altitudes during wind conditions in which turbines are likely to be operational. The calibrated model could be used in concert with automated eagle detection and turbine curtailment technologies. Specifically, once an eagle is detected by those systems, our model could then provide accurate predictions of turbines the eagle is likely to interact with in the near term.

Pharmacotherapy combinations have been shown to improve survival and reduce hospitalisations in adults with chronic heart failure with reduced ejection fraction (HFrEF); however, their cost-effectiveness when used as first-line treatment remains uncertain. A lifetime cohort Markov model was developed from the perspective of the NHS in England to assess the cost-effectiveness of five first-line pharmacotherapy combinations: (i) angiotensin-converting enzyme inhibitor (ACEI) or angiotensin receptor blocker (ARB) and beta-blocker (BB) (NICE-recommended treatment at the time of analysis); (ii) ACEI/ARB, BB and mineralocorticoid receptor antagonists (MRA); (iii) angiotensin receptor-neprilysin inhibitor (ARNI), BB and MRA; (iv) ACEI/ARB, BB, MRA and sodium-glucose cotransporter-2 inhibitor (SGLT2i); and (v) ARNI, BB, MRA and SGLT2i. Baseline hospitalisation and mortality rates were informed by real-world data, while treatment effects (HRs) were derived from a review of randomised controlled trials. Among individuals able to tolerate an ACEI, the combination of ACEI, BB, MRA and SGLT2i (cost, £12 124; quality-adjusted life years (QALYs), 5.72) was found to be the most cost-effective first-line treatment option with an incremental cost-effectiveness ratio (ICER) of £7699.Among individuals unable to tolerate an ACEI, the combination of ARNI, BB, MRA and SGLT2i (cost, £18 950; QALYs, 6.04) was found to be the most cost-effective first-line treatment option with an ICER of £15 821. The next most cost-effective first-line treatment option was the combination of ARB, BB, MRA and SGLT2i (cost, £11 842; QALYs, 5.59). These findings were primarily driven by the greater relative QALY gain of ARNI compared with ARB. This study demonstrates that a first-line quadruple pharmacotherapy combination is cost-effective compared with a stepwise approach for treating people with HFrEF, suggesting that wider adoption of early initiation of quadruple pharmacotherapy may improve health outcomes and optimise healthcare resource use.

Abstract Functional neuroimaging techniques allow us to estimate functional networks that underlie cognition. However, these functional networks are often estimated at the group level and do not allow for the discovery of, nor benefit from, subpopulation structure in the data, i.e. the fact that some recording sessions maybe more similar than others. Here, we propose the use of embedding vectors (c.f. word embedding in Natural Language Processing) to explicitly model individual sessions while inferring networks across a group. This vector is effectively a “fingerprint” for each session, which can cluster sessions with similar functional networks together in a learnt embedding space. We apply this approach to estimate dynamic functional networks using a hierarchical Hidden Markov Model (HMM). We call this approach HIVE (HMM with Integrated Variability Estimation). Using simulated data, we show that HIVE can uncover true subpopulation structure and show improved performance over existing approaches. Using real magnetoencephalography data, we show the learnt embedding vectors (session fingerprints) reflect meaningful sources of variation across a population. Overall, HIVE provides a powerful new approach for modelling individual sessions while leveraging information available across an entire group.

The challenges presented by market fluctuations and environmental events such as lightning are addressed in this paper by integrating fuzzy logic with Markov models. The challenges presented by market fluctuations and environmental events such as lightning are addressed in this paper by integrating fuzzy logic with Markov models. This integration is critically needed because market uncertainties (e.g., prices) often follow probabilistic patterns, while lightning impacts involve imprecise, linguistic assessments. A unified Fuzzy-Markov framework is therefore essential to holistically manage these hybrid uncertainties and enhance decision-making in smart grid self-scheduling. The objective of this research is to create a dependable framework for improving the predictability and stability of smart grid systems under unforeseen circumstances. The proposed Fuzzy-Markov approach facilitates the proactive decision-making process and the effective forecasting of future market conditions by categorizing complex numerical data into fuzzy states and analyzing the transition probabilities between these states. One of the most significant contributions is the successful classification of financial metrics, including price, revenue, and sales, into qualitative fuzzy states. The Markov transition matrix's construction and analysis provide critical insights into state transitions, with the model attaining an accuracy of 56.13%. Although this accuracy is moderate, it illustrates the model's effectiveness in predicting future conditions, superseding random conjecture and establishing a strong foundation for strategic planning. The research also emphasizes significant findings through rolling statistics, which are crucial for risk management. The novelty of this work is its distinctive integration of fuzzy logic and Markov models to address both market and environmental uncertainties in smart grids.

To estimate the cost effectiveness of the Support and Services at Home (SASH) program for health improvements associated with cardiovascular risk factors. Located in affordable housing units, SASH uses wellness approaches to prevent illness, manage chronic conditions and coordinate care delivery by connecting older adults and individuals with disabilities with community-based services. We calculated total quality-adjusted life years (QALYs) gained from cardiovascular risk reduction and program costs using a Markov model. Data on changes in health status, health outcomes, and programmatic costs were drawn from SASH (primary) data sources from the statewide enrolled population in the original (Vermont) program. Data were collected from 2017 to 2023. SASH reduced total cardiovascular risk factors including increases in appropriate medication use and reductions in systolic blood pressure. The cost per QALY gained ranged from $8344 to $4013 depending on gender and diabetes. SASH is a cost-effective approach to improving the health of older adults and individuals with disabilities through a housing-based community partnership. SASH is emblematic of the "wrong pocket" problem, so replication and funding of the model are challenging. For greater system efficiency and equity, finding ways to incorporate programs outside the healthcare system will be required.

Association Between Sarcopenia and Frailty Transition in Chinese Older Adults: A Multistate Markov Model Study.