State Transition Probabilities Research Articles

D.2 Cost-effectiveness analysis of efgartigimod vs chronic IVIg for treatment of patients with generalized myasthenia gravis in Canada

Background: Efgartigimod is a human IgG1 antibody Fc fragment recently approved by Health Canada for patients with acetylcholine receptor antibody positive (AChR-Ab+) generalized myasthenia gravis (gMG). We assessed cost-effectiveness of efgartigimod vs chronic IVIg for adult patients with AChR-Ab+ gMG. Methods: A Markov model estimated costs (treatment and administration, disease monitoring, complications from chronic corticosteroid use, exacerbation and crisis management, adverse events, end-of-life care) and benefits (quality-adjusted life-years [QALYs]). The analysis was conducted from the Canadian healthcare system perspective. Health state transition probabilities were estimated using data from ADAPT, ADAPT+, and a network meta-analysis comparing efgartigimod with chronic IVIg. Utility values were obtained from MyRealWorld MG. Patients with MG-ADL ≥5 who did not die/discontinue were assumed to receive treatment every 4 weeks or every 3 weeks over the lifetime horizon. Results: Over the lifetime horizon, efgartigimod and chronic IVIg were predicted to have total discounted QALYs of 16.80 and 13.35, and total discounted costs of $1,913,294 and $2,170,315, respectively. Efgartigimod dominated chronic IVIg with incremental QALYs of 3.45 and cost savings of $257,020 over the lifetime horizon. Conclusions: Efgartigimod may provide greater benefit at lower costs than chronic IVIg for Canadian patients with AChR-Ab+ gMG, with substantial healthcare system savings over the lifetime horizon.

Physics Package Based on Intracavity Laser Cooling 87Rb Atoms for Space Cold Atom Microwave Clock

Abstract This article proposes a new physics package to enhance the frequency stability of the space cold atom clock with the advantages of a microgravity environment. Clock working processes, including atom cooling, atomic state preparation, microwave interrogation, and transition probability detection, are integrated into the cylindrical microwave cavity to achieve a high-performance and compact physics package for the space cold atom clock. We present the detailed design and ground-test results of the cold atom clock physics package in this article, which demonstrates a frequency stability of 1.2 × 10-12 τ-1/2 with a Ramsey linewidth of 12.5 Hz, and a better performance is predicted with a 1 Hz or a narrower Ramsey linewidth in microgravity environment. The miniaturized cold atom clock based on intracavity cooling has great potential for achieving space high-precision time-frequency reference in the future.

Path Planning Analysis of Mobile Robot with Improved Ant Colony Algorithm

Aiming at the problems of poor convergence and local optimization in path planning using the basic ant colony algorithm, this paper studies an improved ant colony algorithm to enhance the effect of mobile robot path planning. Firstly, the state transition probability of the ant colony algorithm is modified, and the influence of the angle on node selection is increased by adding a new angle index heuristic function. Then, the sorting and elite ant colony algorithm strategies are fused to research a path length difference pheromone update method, improving the efficiency of the ant colony algorithm in planning the optimal path. Finally, through a comparison with the basic ant colony algorithm simulation on MATLAB, the feasibility and effectiveness of the improved ant colony algorithm are verified.

Early altered directionality of resting brain network state transitions in the TgF344-AD rat model of Alzheimer's disease.

Alzheimer's disease (AD) is a progressive neurodegenerative disease resulting in memory loss and cognitive decline. Synaptic dysfunction is an early hallmark of the disease whose effects on whole-brain functional architecture can be identified using resting-state functional MRI (rsfMRI). Insights into mechanisms of early, whole-brain network alterations can help our understanding of the functional impact of AD's pathophysiology. Here, we obtained rsfMRI data in the TgF344-AD rat model at the pre- and early-plaque stages. This model recapitulates the major pathological and behavioral hallmarks of AD. We used co-activation pattern (CAP) analysis to investigate if and how the dynamic organization of intrinsic brain functional networks states, undetectable by earlier methods, is altered at these early stages. We identified and characterized six intrinsic brain states as CAPs, their spatial and temporal features, and the transitions between the different states. At the pre-plaque stage, the TgF344-AD rats showed reduced co-activation of hub regions in the CAPs corresponding to the default mode-like and lateral cortical network. Default mode-like network activity segregated into two distinct brain states, with one state characterized by high co-activation of the basal forebrain. This basal forebrain co-activation was reduced in TgF344-AD animals mainly at the pre-plaque stage. Brain state transition probabilities were altered at the pre-plaque stage between states involving the default mode-like network, lateral cortical network, and basal forebrain regions. Additionally, while the directionality preference in the network-state transitions observed in the wild-type animals at the pre-plaque stage had diminished at the early-plaque stage, TgF344-AD animals continued to show directionality preference at both stages. Our study enhances the understanding of intrinsic brain state dynamics and how they are impacted at the early stages of AD, providing a nuanced characterization of the early, functional impact of the disease's neurodegenerative process.

On State Transition Probability and Performance of Direct Acyclic Graph Based Ledgers

Direct acyclic graph (DAG) based ledgers with multi-chain structures aim to solve the technical bottlenecks associated with classical blockchain technologies in the Internet of Things (IoT). The basic working principle of DAG-based ledgers is to validate new transactions by previous transactions in order to be added to the system. During the tip selection process in the unsteady regime, the state transition probability refers to the probability of a transaction changing from the initial state to an arbitrary state. The state transition probability plays an indispensable role in the performance and security analysis of the IoT relying on DAG-based ledgers. In this paper, we derive the exact expression and an approximate expression of the state transition probability, which both are in closed form. In addition, we propose and analyze three performance metrics, i.e., the expected cumulative weight, the expected number of steps, and the confirmation failure probability, which are derived from the state transition probability and greatly enrich the performance analysis and evaluation of the IoT. Markov chain Monte Carlo (MCMC) simulations are carried out to verify the derived analytical results and provide insight into the IoT using DAG-based ledgers.

Joint optimization of maintenance and spares inventory policy for a series-parallel system considering dependent failure processes

This paper investigates the joint optimization of condition-based opportunistic maintenance and spare inventory policies for a series-parallel system consisting of software and components. Software failure follows a homogeneous Poisson process. Components are subject to mutually dependent competing failure processes. When the software failure occurs, an opportunistic inspection will be implemented for the components. Besides the maintenance policy, the (s, S) inventory policy is applied for components. The maintenance and order decisions will be made according to the conditions of the components and inventory level at each inspection. The state transition probabilities and expected sojourn times can be obtained by the formulated semi-Markov decision process. The optimal inspection interval, the preventive replacement threshold of the component, and the order point can be derived by minimizing the expected average cost rate. Finally, the effectiveness and superiority of the proposed joint optimization model are verified by an illustrative example.

Simulation Modeling Using Markovian Decision Theory on Cash Flow Analysis of Central Bank of Nigeria

This research aims to investigate the cash flow analysis of the Central Bank of Nigeria using Markovian decision theory. The specific objectives include determining the (i) cash flow solvency ratio, (ii) cash flow adequacy ratio, (iii) sufficiency ratio, (iv) cash flow profitability ratio, and (v) estimating the optimal policy of cash flow ratios performance in CBN. The identified problems are the effects of (i) insolvency, (ii) inadequacy, (iii) insufficiency, and (v) Unprofitability on Central Bank of Nigeria (CBN) cash flows performance.The methodology involves a research design tailored toward collecting, arranging, and determining cash flow data for model prediction and optimization. The Markov chain is introduced as an operator to evaluate the distribution of cash flow ratios in the long term, using initial state vectors and state transition probabilities for forecasting behavior. Data validation is performed using graphical and Pearson moment correlation coefficient methods. The pre-model analysis of CBN cash flows problem during the period of January 2012 to December 2017 identifies six finite current states. State-2012 cash flows performance was exceptional (above the zero line), reflecting 100%, while State-2013 and 2015 reflect 75%, and State-2014, 2016 & 2017 reflect 50% healthy cash flows status. The model results introduce the Markovian Cash Flow Ratios Monitoring Curve (MCFRMC), specifying the minimum values for healthy status. The research explores the present status of cash flow ratios, presenting forecasted ratios in the form of an optimum policy or solution. Pearson moment correlation coefficient validation of the prototype and model results in a coefficient of 1.0, indicating a 100% higher performance of the model. Further research reveals strategic cash inflows policy allocation to the cash flows indicators, with (i) operational activity receiving 24%, (ii) investment active e

An enhanced ant colony optimization algorithm for global path planning of deep-sea mining vehicles

Due to the complex variations in slope within deep-sea mining areas, effective path planning for mining vehicle operations is crucial for minimizing energy consumption. However, traditional ant colony algorithms (ACO) neglect the effect of a terrain slope in mining areas. Additionally, these algorithms exhibit limitations such as slow convergence and susceptibility to local optima. To address these issues, this study proposes an enhanced ant colony algorithm, called DYACO, for mining vehicle path optimization. This algorithm dynamically adjusts heuristic information, pheromone volatilization factor, pheromone update strategy, and state transition probability during the iterative process to enhance traditional ACO. Simulation experiments were conducted to comprehensively assess the proposed model, revealing that DYACO not only generates optimal solutions but also demonstrates significant advantages in terms of convergence speed and turning times. Furthermore, DYACO converts the time required for mining vehicles to traverse different slope regions into distances, then incorporating slope effects to path planning for deep-sea mining vehicles. In comparison to ACO, DYACO achieves a 15.3% reduction in the length of an optimal path and a 70.0% decrease in the number of turn times.

Dipolar Coupling as a Mechanism for Fine Control of Magnetic States in ErCOT-Alkyl Molecular Magnets.

The design of molecular magnets has progressed greatly by taking advantage of the ability to impart successive perturbations and control vibronic transitions in 4fn systems through the careful manipulation of the crystal field. Herein, we control the orientation and rigidity of two dinuclear ErCOT-based molecular magnets: the inversion-symmetric bridged [ErCOT(μ-Me)(THF)]2 (2) and the nearly linear Li[(ErCOT)2(μ-Me)3] (3). The conserved anisotropy of the ErCOT synthetic unit facilitates the direction of the arrangement of its magnetic anisotropy for the purposes of generating controlled internal magnetic fields, improving control of the energetics and transition probabilities of the electronic angular momentum states with exchange biasing via dipolar coupling. This control is evidenced through the introduction of a second thermal barrier to relaxation operant at low temperatures that is twice as large in 3 as in 2. This barrier acts to suppress through-barrier relaxation by protecting the ground state from interacting with stray local fields while operating at an energy scale an order of magnitude smaller than the crystal field term. These properties are highlighted when contrasted against the mononuclear structure ErCOT(Bn)(THF)2 (1), in which quantum tunneling of the magnetization processes dominate, as demonstrated by magnetometry and ab initio computational methods. Furthermore, far-infrared magnetospectroscopy measurements reveal that the increased rigidity imparted by successive removal of solvent ligands when adding bridging methyl groups, along with the increased excited state purity, severely limits local spin-vibrational interactions that facilitate magnetic relaxation, manifesting as longer relaxation times in 3 relative to those in 2 as temperature is increased.

A cost and demand sensitive adjustment algorithm for service function chain in data center network

The introduction of Network Function Virtualization (NFV) and Software-Defined Network (SDN) architectures has significantly reduced the Operational Expenditure (OPEX) and Capital Expenditure (CAPEX) of network system. However, NFV orchestration management also brings about challenges. After the initial deployment of VNFs, due to the volatility of network requests, the original deployment may not be able to meet user resource demands. The key issue is how to readjust resources dynamically to accommodate more network requests without violating Quality of Service (QoS) for users. Several existing techniques can be used to achieve this goal, such as horizontal scaling, vertical scaling, and virtual network function (VNF) migration. However, these techniques inevitably incur some overhead, such as the cost of instantiating VNF and link rerouting. Additionally, resource adjustment may also result in unbalanced distribution of network resources. In this paper, an Intelligent Service Function Chain Dynamic Adjustment Algorithm (ISFCDAA) is proposed to address the above challenges. Firstly, an Integer Linear Programming (ILP) model is established with the objective of minimizing the long-term adjustment cost and reducing the imbalance of resource distribution. Then we transform the optimization process into a Markov Decision Process (MDP). Secondly, to solve the problems that the state and action space is too large and the state transition probability is uncertain in MDP, a SFC dynamic adjustment algorithm based on deep reinforcement learning is proposed. This algorithm can obtain an approximate optimal adjustment strategy for ILP model. The simulation results show that ISFCDAA can reduce the adjustment overhead and maintain a balanced distribution of network resources while ensuring the QoS. Compared with the existing algorithms, the average standard deviation of resource distribution of ISFCDAA is reduced by up to 9.90%, the average acceptance rate of ISFCDAA is improved by up to 39.57%, and the average long-term profit is improved by up to 42.92%. The incorporation of cost and demand-sensitive considerations into ISFCDAA enhances its responsiveness to fluctuating network demands, solidifying its effectiveness in dynamic resource management scenarios.

Finite-state Markov modeling of Gamma-Gamma fading FSO links with pointing errors

The knowledge of atmospheric channel is quite imperative for free-space optical (FSO) communications. To capture the time-varying essence of intensity fluctuations in an FSO connection, we propose a finite-state Markov (FSM) model for FSO links affected by Gamma-Gamma distributed atmospheric turbulence and pointing errors. First, the level crossing rate (LCR) of the unified intensity fluctuation is derived considering the non-generic channel autocorrelation characteristics, based on which the state transition probability (STP) matrix of the FSM model is calculated. Then, numerical simulations are presented for different strength parameters and frequency parameters of the channel. The results show that the misalignment strength does not affect the ratio of the level where the LCR reaches its maximum to the mean level. Finally, a series of well-designed experiments are carried out to verify the theory. As a result, the theoretical predictions conform to the experimental data, with averaged STP relative errors below 6.7%.

Construction of Cybersecurity and Risk Prediction Model for New Energy Power Plants under Machine Learning Algorithm

Abstract Addressing the complex equipment and network challenges in new energy power plant industrial control systems, this study introduces a Markov time-varying machine learning algorithm, leveraging classification-constrained Boltzmann machines for real-time network security risk prediction. By employing a hybrid training mode for innovative feature extraction and classification, the algorithm forecasts future security risk states through an up-to-date state transition probability matrix. Integrating with the Markov time-varying model enhances the efficiency over traditional Boltzmann machines, facilitating nuanced network state analyses. The proposed model demonstrates high effectiveness against various network attacks, with average precision, recall, and F1 scores of 0.96, 0.93, and 0.94, respectively, and maintains over 80% accuracy under noise levels up to 40 dB. This research provides a solid foundation for proactive security defense mechanisms in industrial control systems.

A predictor-informed multi-subject bayesian approach for dynamic functional connectivity.

Dynamic functional connectivity investigates how the interactions among brain regions vary over the course of an fMRI experiment. Such transitions between different individual connectivity states can be modulated by changes in underlying physiological mechanisms that drive functional network dynamics, e.g., changes in attention or cognitive effort. In this paper, we develop a multi-subject Bayesian framework where the estimation of dynamic functional networks is informed by time-varying exogenous physiological covariates that are simultaneously recorded in each subject during the fMRI experiment. More specifically, we consider a dynamic Gaussian graphical model approach where a non-homogeneous hidden Markov model is employed to classify the fMRI time series into latent neurological states. We assume the state-transition probabilities to vary over time and across subjects as a function of the underlying covariates, allowing for the estimation of recurrent connectivity patterns and the sharing of networks among the subjects. We further assume sparsity in the network structures via shrinkage priors, and achieve edge selection in the estimated graph structures by introducing a multi-comparison procedure for shrinkage-based inferences with Bayesian false discovery rate control. We evaluate the performances of our method vs alternative approaches on synthetic data. We apply our modeling framework on a resting-state experiment where fMRI data have been collected concurrently with pupillometry measurements, as a proxy of cognitive processing, and assess the heterogeneity of the effects of changes in pupil dilation on the subjects' propensity to change connectivity states. The heterogeneity of state occupancy across subjects provides an understanding of the relationship between increased pupil dilation and transitions toward different cognitive states.

UAV path planning based on a dual-strategy ant colony optimization algorithm

With the rapid development of modern communication and automatic control technologies, unmanned aerial vehicles (UAVs) have increasingly gained importance in both military and civilian domains. Path planning, a critical aspect for achieving autonomous aerial navigation, has consistently been a focal point in UAV research. However, traditional ant colony algorithms need to be improved for the drawbacks of susceptibility to local optima and weak convergence capabilities. Consequently, a novel path planning methodology is proposed based on a dual-strategy ant colony algorithm. In detail, an improved state transition probability rule is introduced, redefining ant movement rules by integrating the state transition strategy of deterministic selection during the iterative process. Additionally, heuristic information on adjacent node distance and mountain height is added to further improve the search efficiency of the algorithm. Then, a new dynamically adjusted pheromone update strategy is proposed. The update strategy is continuously adjusted during the iteration process, which is beneficial to the algorithm’s global search in the early stage and accelerated convergence in the later stage, preventing the algorithm from falling into local optimality and improving its convergence. Based on the above improvements, a new variation of ant colony optimization (ACO) called dual-strategy ACO algorithm is formed. Experimental results prove that dual-strategy ACO has superior global search capabilities and convergence characteristics from four key aspects: path length, fitness values, iteration number, and running time.

Prediction of Consumers’ Adoption Behavior of Products with Water Efficiency Labeling Based on Hidden Markov Model

Water efficiency labeling, an environmental protection label, can provide consumers with products’ water-saving information, guide consumers to choose high-efficiency water-saving products, and encourage residents to save water. Data were obtained from a questionnaire survey among 526 respondents in Zhengzhou, China. The study constructed a Hidden Markov Model (HMM) to predict consumers’ adoption behavior of products with water efficiency labeling. The adoption probability of consumers at different perceptions and water-saving awareness for products with water efficiency labeling and the state transition probability of consumers’ adoption behavior of products with water efficiency labeling were calculated. Consumers’ adoption behavior of products with water efficiency labeling in different situations is predicted. The research found: (1) When the degree of perception remains unchanged, the improvement of consumers’ water-saving awareness will promote their adoption of products with water efficiency labeling and increase the probability of continuous adoption behavior. (2) The state of consumers’ adoption behavior depends on the last adoption behavior. Furthermore, as the state level increases, the dependency on adoption behavior becomes more vigorous. (3) HMM is suitable for predicting consumers’ adoption behavior of products with water efficiency labeling and has high accuracy. The research findings can provide a theoretical basis and decision-making reference for the government and enterprises to formulate relevant guiding policies and promotion strategies.

APPLICATION OF EXPECTED CREDIT LOSS MODEL AND MARKOV CHAIN TO CALCULATE NET SINGLE PREMIUM OF UNSECURED CREDIT INSURANCE

Transferring credit risk to an insurance company is a way to mitigate risk. Premiums should be calculated accurately to attain economic value for both the lender and the guarantor. The aim of this study was to determine the net single premium (NSP) values for an unsecured credit insurance product using the expected credit loss (ECL) method from IFRS 9. This study used data generated through simulation of insurance policies issued in 2015 or 2016. Their state classifications were monthly observed from 2016 to 2020. The probability of disbursed claim (PDC) parameter replaced the probability of default parameter on the ECL model, whereas the PDC model was constructed based on the components of a state-transition probability matrix, obtained with the Markov chain approach using the cohort method: = 0.999181, = 0.000130, and = 0.000689. The PDC model validation showed relatively decent results, whereas MSE = 2.457% and zs = 0.608 with a = 5%. These results indicated that the PDC model was a good fit to calculate ECL. 5,000 iterations were done as part of the cash flow simulation process, whereas debtors’ loan amounts were randomly generated during each iteration, and the average NPV of these iterations was -Rp564.419.305. Based on model sensitivity analysis, cash flow values were most sensitive to the variable used to construct the PDC model (). Thus, the 5,000-iteration process was repeated with the newly adjusted PDC value, which were = 0.998924 and = 0.000946. The new average NPV of these iterations was Rp409,877,840, indicating that the constructed ECL model was a good fit to calculate NSP values for unsecured credit insurance products.

Theoretical substantiation of the requirements for an intelligent video monitoring system for analyzing the physiological state of cows and predicting its changes

At the present stage of development of dairy farming, the main task facing the industry is the effective management of all processes that ensure the operation of a farm with a large livestock. This task will not be achievable without the use of digital intelligent production control and management systems, tracking the health of animals and their productivity. There are many systems that integrate into production and allow you to monitor the health of cows on a dairy farm. These systems are based on the identification, tracking and collection of information about the physiological state of animals through various sensors. However, all these systems are expensive, demanding to maintain, often fail due to aggressive operating conditions. The article considers the possibility of using an intelligent video monitoring system to analyze the physiological state of cows and predict its changes. To substantiate the requirements for an intelligent video monitoring system of the physiological state of cows, a mathematical model of the life of productive cows in the form of a graph of states is proposed. This method graphically displays the possible states of the system and their possible transitions from state to state. Having constructed a graph of states, relying on the central probability theorem for event flows, it is possible to determine the probability and intensity of state transitions using differential equations. For the functioning of the video monitoring system for analyzing the physiological state of cows and predicting its changes, the proposed mathematical model can be the basis of the system and be based on the collection and analysis of video information about animal behavior. When covering the entire territory of the animal's living space with video cameras, the intelligent video monitoring system will be able to provide early diagnosis of changes in the physiological state of the observed animals and issue a warning, which will allow for timely therapeutic and preventive measures.

Markov decision process based value chain calculation of water distribution network scheduling

Precise value of scheduling decisions forms the cornerstone of water distribution network (WDN) scheduling optimization, which aims at conserving energy and enhancing network operational efficiency. This article proposes a computational methodology for evaluating the value chain of scheduling decisions in WDN. The scheduling process is modeled as a Markov decision process with immediate reward function, action and state space. Due to the periodicity of water supply and sequential nature of scheduling, the calculation quantifies cumulative value of scheduling decisions by incorporating state transition probability with expected value. The effectiveness and applicability of the proposed evaluation method are demonstrated using scheduling data from a real world WDN. The method provides rational values on scheduling period and strategies, offering practical feedback for scheduling decisions.

Optimal online energy management strategy of a fuel cell hybrid bus via reinforcement learning

An energy management strategy (EMS) based on reinforcement learning is proposed in this study to enhance the fuel economy and durability of a fuel cell hybrid bus (FCHB). Firstly, a comprehensive powertrain system model for the FCHB is established, mainly including the FCHB’s power balance, fuel cell system (FCS) efficiency, and aging models. Secondly, the state–action space, state transition probability matrix (TPM), and multi-objective reward function of Q-learning algorithm are designed to improve the fuel economy and the durability of power sources. The FCHB’s demand power and battery state of charge (SOC) serve as the state variables and the FCS output power is used as the action variable. Using the demonstration FCHB data, a state TPM is created to represent the overall operation. Finally, an EMS employing Q-learning is formulated to optimize the fuel economy of FCHB, maintain battery SOC, suppress FCS power fluctuations, and enhance FCS lifetime. The proposed EMS is tested and verified through hardware-in-the-loop (HIL) tests. The simulation results demonstrate the effectiveness of the proposed strategy. Compared to a rule-based EMS, the Q-learning-based EMS can improve the energy economy by 7.8%. Furthermore, it is only a 3.7% difference to the best energy economy under dynamic optimization, while effectively reducing the decline and enhancing the durability of the FCS.

Disrupted brain state dynamics in opioid and alcohol use disorder: attenuation by nicotine use.

Substance use disorder (SUD) is a chronic relapsing disorder with long-lasting changes in brain intrinsic networks. While most research to date has focused on static functional connectivity, less is known about the effect of chronic drug use on dynamics of brain networks. Here we investigated brain state dynamics in individuals with opioid use (OUD) and alcohol use disorder (AUD) and assessed how concomitant nicotine use, which is frequent among individuals with OUD and AUD, affects brain dynamics. Resting-state functional magnetic resonance imaging data of 27 OUD, 107 AUD, and 137 healthy participants were included in the analyses. To identify recurrent brain states and their dynamics, we applied a data-driven clustering approach that determines brain states at a single time frame. We found that OUD and AUD non-smokers displayed similar changes in brain state dynamics including decreased fractional occupancy or dwell time in default mode network (DMN)-dominated brain states and increased appearance rate in visual network (VIS)-dominated brain states, which were also reflected in transition probabilities of related brain states. Interestingly, co-use of nicotine affected brain states in an opposite manner by lowering VIS-dominated and enhancing DMN-dominated brain states in both OUD and AUD participants. Our finding revealed a similar pattern of brain state dynamics in OUD and AUD participants that differed from controls, with an opposite effect for nicotine use suggesting distinct effects of various drugs on brain state dynamics. Different strategies for treating SUD may need to be implemented based on patterns of co-morbid drug use.