RENAL2: Wired Room: Digital Sound, Image and Motion Analysis for Enhanced Medical Diagnostics from A Patient Encounter

Abstract

Background: In-person health care encounters are critical for obtaining patient data needed for accurate diagnosis. However, in recent years with increasing patient-to-doctor ratios, physicians are spending less time face-to-face with their patients, often restrained by requirements of electronic health care record (EHR) entry. We developed a system to supplement in-person health care encounters with both real time audio/visual data capture and analysis, thus streamlining patient-physician interactions, as well as allowing for aggregation of data that may otherwise be lost. Method: A machine-vision-based motion tracking program - Google MediaPipe was utilized for the motion subsystem. Time invariant cross correlation was used to compare MediaPipe’s ability to measure joint angle and angular acceleration to an established system MOCA 2.0 - previously advanced by our group as a smartphone-based motion tracker. A Python-based sound analysis program was developed and utilized to process audio data into signal components for statistical analysis. Regression analysis of signal components was run to identify audio samples with speech and cough from those without. Two trials with six subjects were conducted with subject performing a range of movements: speaking and generating a cough. Results: Initial motion tracking analysis across 2 trials with 6 subjects showed a 98.3% (±0.5) correlation between joint angle measurements between MediaPipe and MOCA 2.0. The reliability of acceleration measurements was less robust. Sound analysis, using amplitude range, identified speech and cough with 94.5% accuracy. Conclusion: A patient encounter room, wired and equipped with motion and sound detection systems was designed. The system demonstrated ability to capture and process a range of data otherwise lost with conventional health provider encounters. Further development of the system is focus on enhanced signal processing to isolate patient from ambient noise, and to allow on-boarding of a catalog of correlation between digital signatures and physiological conditions, to generate and refine digital biomarkers. New digital biomarkers many not only augment the in-person diagnostic process, but may provide a useful foundation for significant advances in telemedicine and remote patient monitoring.