Development of a biofeedback system using harmonic musical intervals to control heart rate variability with a generative adversarial network

Abstract

Biofeedback methods supply biological information to subjects in real-time. Target biomedical/physiological variables are measured and fed back either directly, or transformed beforehand. Biofeedback based on Heart Rate Variability (HRV) has shown to have positive effects in managing anger, sleep, general quality of life and the restoration of physiological homeostasis. The literature suggests the use of new technologies to deal with interpreting the information and determining how Generative Adversarial Networks (GANs) might help in transforming information that flows through a biofeedback system. Given the wide use of Musical Instrument Digital Interface (MIDI) and piano-rolls, and the importance of HRV in the fields of health and perception, a GAN-based biofeedback system was proposed to generate MIDI sequences of harmonic musical intervals (HMI) from HRV signals. The proposed system seeks to produce, by changing HMI sounds, desirable respective HRV responses. The system was tested simulating HRV responses to HMI stimuli (the global health threat excluded real-life testing). Simulation was performed using a one-dimensional GAN. Rather than from latent space, one GAN generator was fed from raw HRV and the other from sound signals. An association was distinguished between the captured HRV signals and the stimuli, since GAN can generate representations similar to the stimuli employed. As a biofeedback system, the subjects provide HRV to feed the GANs and can be stimulated with sounds produced by an HMI generator. This system opens doors in the fields of perception and stress management, and might be applied in neuromarketing and human–computer interfaces.