Constrained Markov Decision Process Modeling for Optimal Sensing of Cardiac Events in Mobile Health

Abstract

Rapid advances in the smartphone, wearable sensing, and wireless communication provide an unprecedented opportunity to develop mobile systems for smart health management. Mobile cardiac sensing collects health-related data from individuals and enables the extraction of information pertinent to cardiac conditions. However, wireless sensors in ambulatory care settings operate on batteries. All-time sensing and monitoring will result in fast depletion of the battery in the mobile system. There is an urgent need to develop optimal sensing schemes that will reduce energy consumption while satisfying the requirements in the detection of cardiac events. In this article, we develop a constrained Markov decision process (CMDP) framework to optimize mobile electrocardiography (ECG) sensing under the constraint of the energy budget. We first characterize the cardiac states from ECG signals using the heterogeneous recurrence analysis. Second, we model the stochastic dynamics in cardiac processes as a continuous-time Markov chain (CTMC). Third, we optimize the ECG sensing through a CMDP framework under the constraint of energy budget. Finally, we validate and evaluate the performance of our CMDP policy in both simulation and real-world case studies. Experimental results demonstrate that the proposed CMDP policy significantly outperforms the traditional uniform and mean-time-to-event (MTTE) policies. Specifically, the error of state estimation is reduced by 34.0% in the real-world case study for energy-constrained sensing of cardiac events. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article is motivated by the markedly increasing applications of mobile health (mHealth) in cardiac care. MHealth systems enable the real-time monitoring, tracking, and transmitting of heart-health information but suffer from the problem with limited battery life. This article presents a novel energy-efficient framework for mobile ECG sensing by integrating ECG signal analysis with sensing-policy optimization under the constraint of the energy budget. Experimental results demonstrate the effectiveness of the proposed framework to realize energy-efficient mobile sensing for cardiac events detection.