Probabilistic State Transition Model Research Articles

Endurance Prediction Based on Hidden Markov Model and Programming Optimization for 28nm 1Mbit Resistive Random Access Memory Chip

We proposed a state transition probability model based on Hidden Markov Model (HMM), which can predict the lifetime for different endurance failure modes. The prediction span of this model is 500 cycles, and the accuracy rate can reach 83.6 %. Based on the prediction results, we proposed an optimized programming algorithm to rescue the failing cells during endurance. The failure rate of memory chip is reduced by 38 % after 5000 cycles. The forecasting model is effective and of practical value since it is based on the data from 28nm 1Mbit Resistive Random Access Memory (RRAM) chips, including the peripheral circuit noise, machine noise and device randomness. This prediction model is significant for promoting RRAM application and improving memory chip utilization.

Cross-Layer Defense Methods for Jamming-Resistant CBTC Systems

Communication-based Train Control (CBTC) systems are the burgeoning directions for developing future train control systems. With the adoption of wireless communication and network techniques, train control systems are more vulnerable to cyber-attacks. Notably, the jamming attacks, aiming at the handoff process that is the weakest part of train ground communication systems, will cause long disruption of communication. It will have a severe impact on train control operation efficiency. Current research regarding industry control system security is hard to model the impact of the jamming attacks on the train control system quantitatively, and current countermeasure schemes against jamming attacks are not designed for the operating mechanism of train control systems. This paper first builds the train control security state transition probability model under jamming attacks. A cross-layer defense scheme is then proposed from the aspect of the physical layer, the cyber layer and the management layer. In the physical layer, this paper designs a model prediction control algorithm to track dynamic target signals, in the hopes of eventually tracking the dynamic target quickly and smoothly. In the cyber layer, a multi-stage and zero-sum stochastic game model is built for the channel selection for the attack and the defense, whereby the channel selection randomized policy will be obtained. In the management layer, a dynamic train travel speed profile generation algorithm is proposed to mitigate the jamming attacks’ impact on train control systems. Extensive simulation results are shown that jamming attack impact on CBTC can be mitigated effectively with our proposed cross-layer defense scheme.

Cost-Effectiveness of a Pharmacogenetic Test to Guide Treatment for Major Depressive Disorder.

Recent clinical trials indicate that pharmacogenetic-guided treatment of major depressive disorder (MDD) results in higher treatment response rates by genetically matching patients to medications and avoiding a trial-and-error process. To evaluate the cost-effectiveness of a pharmacogenetic test (IDGx) that has demonstrated effectiveness compared with standard of care (SOC) medication management among patients with varied MDD severity. Data from a large prospective, randomized controlled trial of treatment-naive patients or patients with inadequately controlled MDD in general practice and psychiatric treatment settings were used to build a Markov state-transition probability model. Analyses were conducted from the societal perspective. Treatment response rates, mortality rates, direct and indirect medical costs, and utility inputs were derived from the reference study and published scientific literature. The cost of the pharmacogenetic test was $2,000. A 3% discount rate was used to discount costs and effects. Univariate one-way sensitivity analyses were performed to determine the effect of input parameters on net monetary benefit. For moderate to severe MDD, the model estimated a cumulative effect over 3 years of 2.07 quality-adjusted life-years (QALYs) for the pharmacogenetic-guided treatment group and 1.97 QALYs for the SOC group, including a lower probability of death from suicide (0.328% and 0.351%, respectively). Total costs over 3 years were $44,697 (IDGx) and $47,295 (SOC). This difference includes a savings of $2,918 in direct medical costs and $1,680 in indirect costs. Results were more pronounced when only severely depressed patients were evaluated. Pharmacogenetic testing among moderate to severe MDD patients improved QALYs and resulted in cost savings. Sensitivity analyses supported the robust nature of the current findings of the dominant IDGx test to guide treatment. Funding for this analysis was provided by AltheaDx, which is the manufacturer of the IDgenetix test. AltheaDx personnel assisted in the study design, data collection, and review of the manuscript. Maciel and Garces are employed by AltheaDx. Groessl has received funding as a consultant from American Specialty Health.

Stochastic traffic modelling and decentralised signal control based on a state transition probability model

In this study, the authors develop a state transition probability model for an isolated intersection on the basis of link-based state transition probability matrix. The state of an intersection is defined as the combination of its input links’ states which represent their respective congestion level by occupancy rate. This way of modelling is further extended to a compact region by likening the region to a virtual node with several virtual input links to avoid dimension expansion. Based on the state transition probability model, the stochastic signal control problem for both intersections and compact regions is formulated as a Markov decision process with the specified definition of state, action, probability and reward. A sensitivity-based policy iteration algorithm is employed to solve the Markov decision process in real-time, which has a great advantage in computational efficiency. The results of the numerical study on a calibrated network of Caohejing District in Shanghai indicate that the authors’ proposed method outperforms the fixed-time and actuated signal control at high loads in terms of many indices and greatly decreases the variability in the traffic performance. In addition, the compact region control can improve the optimisation efficiency while providing similar performance to the intersection control at different loads.

Cost-effectiveness of eye care services in Zambia

ObjectiveTo estimate the cost-effectiveness of cataract surgery and refractive error/presbyopia correction in Zambia.MethodsPrimary data on costs and health related quality of life were collected in a prospective cohort study of 170 cataract and 113 refractive error/presbyopia patients recruited from three health facilities. Six months later, follow-up data were available from 77 and 41 patients who had received cataract surgery and spectacles, respectively. Costs were determined from patient interviews and micro-costing at the three health facilities. Utility values were gathered by administering the EQ-5D quality of life instrument immediately before and six months after cataract surgery or acquiring spectacles. A probabilistic state-transition model was used to generate cost-effectiveness estimates with uncertainty ranges.ResultsUtility values significantly improved across the patient sample after cataract surgery and acquiring spectacles. Incremental costs per Quality Adjusted Life Years gained were US$ 259 for cataract surgery and US$ 375 for refractive error correction. The probabilities of the incremental cost-effectiveness ratios being below the Zambian gross national income per capita were 95% for both cataract surgery and refractive error correction.ConclusionIn spite of proven cost-effectiveness, severe health system constraints are likely to hamper scaling up of the interventions.

Application of FTRF in Cascading Failure Probability Model in Adverse Weather

In allusion to the issues of system line state transfer that may arise in adverse weather, a new method of probability calculation is proposed. In a statistical analysis, this article firstly defines that failure probability of the first line subjects to Poisson distribution. Secondly, we figure out the power flow transferring distribution after first line fault, according to the method of Flow Transferring Relativity Factor (FTRF), and combine with the protective possibility so as to build the probability model between the load rate and protection action. Then, the method defines the severity of line load rate. Finally, the approach constructs the line state transition probability model considering direct and indirect factors in adverse weather. The effectiveness and correctness of the proposed method are verified by simulation based on IEEE 39-node system.

Cost-effectiveness of the 21-gene recurrence score assay in the setting of multifactorial decision making for chemotherapy in early-stage breast cancer.

1525 Background: The objective of this study was to incorporate evidence from two recently-published studies to reevaluate the cost-effectiveness of the 21-gene Recurrence Score (RS) assay (Oncotype DX) in the context of multifactorial decision making to guide the use of chemotherapy for node-negative, estrogen receptor–positive breast cancer in the United States from the societal and healthcare system perspectives. Methods: We developed a decision-analytic model to first cross-classify hypothetical patients by clinicopathologic characteristics according to the Adjuvant! using risk groups and RS risk groups. We generated estimates of long-term costs, survival, and quality-adjusted survival for the RS-guided and non–RS-guided strategies using a probabilistic state transition model. In addition to costs for the 21-gene assay, we assigned attributable costs for chemotherapy, hormonal therapy, monitoring for disease recurrence, and distant recurrence. For the societal perspective, we also considered incremental patient time costs. Costs and survival were discounted at 3% annually. Results: With the RS-guided strategy, 40.4% of patients were expected to receive chemotherapy relative to 47.3% in the non–RS-guided strategy. Targeted use of chemotherapy in the RS-guided strategy was expected to increase survival by 0.19 years (95% CI, 0.09 to 0.32) and 0.16 QALYs (95% CI, 0.08 to 0.28). Lifetime direct medical costs were expected to be $2692 (95% CI, 1546 to 3821) higher with the RS-guided strategy. The incremental cost-effectiveness ratios (ICERs) were $14,059 per life-year saved (95% CI, $6840-$28,912) and $16,677 per QALY (95% CI, $7613-$37,219). When incorporating lower indirect costs of $950 per patient, the ICERs were $9095 per life-year saved (95% CI, dominant-$23,397) and $10,788 per QALY (95% CI, $6840-$30,265). In probabilistic sensitivity analysis, more than 99% of the ICERs were less than $50,000 per life-year saved and per QALY. Conclusions: Our updated cost-effectiveness estimates are supportive of the economic value of the 21-gene RS assay in the setting of node-negative, estrogen receptor–positive breast cancer.

Quantification of the Fragmentation of Rest-Activity Patterns in Elderly Individuals Using a State Transition Analysis

Recent interest in the temporal dynamics of behavioral states has spurred the development of analytical approaches for their quantification. Several analytical approaches for polysomnographic data have been described. However, polysomnography is cumbersome, perturbs behavior, and is limited to short recordings. Although less physiologically comprehensive than polysomnography, actigraphy is nonintrusive, amenable to long recordings, and suited to use in subjects' natural environments, and provides an indirect measure of behavioral state. We developed a probabilistic state transition model to quantify the fragmentation of human rest-activity patterns from actigraphic data. We then applied this to the study of the temporal dynamics of rest-activity patterns in older individuals. Cross-sectional. Community-based. 621 community-dwelling individuals without dementia participating in the Rush Memory and Aging Project. We analyzed actigraphic data collected for up to 11 days. We processed each record to give a series of transitions between the states of rest and activity, calculated the probabilities of such transitions, and described their evolution as a function of time. From these analyses, we derived metrics of the fragmentation of rest or activity at scales of seconds to minutes. Regression modeling of the relationship of these metrics with clinical variables revealed significant associations with age, even after adjusting for sex, body mass index, and a broad range of medical comorbidities. Probabilistic analyses of the transition dynamics of rest-activity data provide a high-throughput, automated, quantitative, and noninvasive method of assessing the fragmentation of behavioral states suitable for large scale human and animal studies; these methods reveal age-associated changes in the fragmentation of rest-activity patterns akin to those described using polysomnographic methods.

Spatial succession modeling of biological communities: a multi-model approach

Strong spatial correlation may exist in the spatial succession of biological communities, and the spatial succession can be mathematically described. It was confirmed by our study on spatial succession of both plant and arthropod communities along a linear transect of natural grassland. Both auto-correlation and cross-correlation analyses revealed that the succession of plant and arthropod communities exhibited a significant spatial correlation, and the spatial correlation for plant community succession was stronger than arthropod community succession. Theoretically it should be reasonable to infer a site's community composition from the last site in the linear transect. An artificial neural network for state space modeling (ANNSSM) was developed in present study. An algorithm (i.e., Importance Detection Method (IDM)) for determining the relative importance of input variables was proposed. The relative importance for plant families Gramineae, Compositae and Leguminosae, and arthropod orders Homoptera, Diptera and Orthoptera, were detected and analyzed using IDM. ANNSSM performed better than multivariate linear regression and ordinary differential equation, while ordinary differential equation exhibited the worst performance in the simulation and prediction of spatial succession of biological communities. A state transition probability model (STPM) was proposed to simulate the state transition process of biological communities. STPM performed better than multinomial logistic regression in the state transition modeling. We suggested a novel multi-model framework, i.e., the joint use of ANNSSM and STPM, to predict the spatial succession of biological communities. In this framework, ANNSSM and STPM can be separately used to simulate the continuous and discrete dynamics.

状態遷移モデルを逐次推定する強化学習の学習加速

状態遷移モデルを逐次推定する強化学習の学習加速

An improved contour extractor for bottlenose dolphin whistles

The time-frequency contours of bottlenose dolphin (Tursiops truncatus) whistles are commonly extracted features for the classification and comparison of these signals. We have previously developed a method to extract the whistles’ fundamental frequency contours [J. Acoust. Soc. Am. 108, 2635 (2000)] by sequentially tracking the contour. These contours are often extracted from recorded rather than real-time signals, which allows us to read ahead or access the ‘‘future’’ signal. This talk extends our previous work to propose a technique to estimate the fundamental frequency at each time from the ‘‘future’’ values of the recorded signal in addition to the past and present signals. This technique is similar to the Kalman smoother except that the states are probability densities, and it uses a probabilistic state transition model with a Bayesian projection to produce the maximum a posteriori probability estimate of the frequency contour based on all recorded data, not just the present and past signal. Experimental evaluation indicates that this method greatly improves the reliability of contour extraction from noisy signals, with considerable resistance to strong harmonics, some dropouts and echolocation clicks, when compared to our previous technique. A huge improvement is achieved from the technique proposed by Buck and Tyack [J. Acoust. Soc. Am. 94, 2497–2506 (1993)].

Extraction and tracking of bottlenose dolphin whistle contours

The time-frequency contours of bottlenose dolphin (Tursiops truncatus) whistles are commonly extracted as features for the classification and comparison of these signals. This research presents a new method to extract and track the whistles’ fundamental frequency contours. This method takes a spectrogram as an input and uses a Bayesian inference approach with a random walk model for the frequency transitions. The output of the method is the maximum a posteriori probability estimate of the frequency contour. This technique is similar to the Kalman filter except that the states are probability densities, and it uses a probabilistic state transition model. For each time segment, the method combines the prior estimate of the probability distribution with the current observation to obtain the current estimate of the probability distribution. Experimental evaluation indicates that this method greatly improves the reliability of contour extraction from noisy signals when compared to the ad hoc contour extractor proposed by Buck and Tyack [J. Acoust. Soc. Am. 94 (1993)]. This technique has previously applied to target tracking in radar [Bethel and Rahikka, IEEE Trans. Aerosp. 23(6) (1987)]. [Work supported by NSF Ocean Sciences.]