A multi-dimensional visualization tool for understanding the role of EMG signals in head movement anticipation

Abstract

One of the main problems Virtual Environment (VE) applications face is the perceptible time latency users experience as the graphics engine updates the scene. This latency causes undesirable effects on users such as lack of accuracy, cyber sickness, and an overall degraded performance. Ongoing research in our laboratory deals with minimizing the lag produced by head tracking devices [Barniv et al. 2004], [Hasanbelliu et al. 2004]. Based on previous research, we know that muscle contraction is preceded by muscle activation. We use Electromyography (EMG) signals from neck muscles in conjunction with angular velocity to anticipate head motion, consequently reducing this time latency. Our system is capable of anticipating significantly well up to 70 ms compared to the 20 ms request set by NASA.In our research, we need to understand the activity of neck muscles, their role in controlling head motion, and more specifically the role that EMG signals play in head movement anticipation. For these reasons, we simultaneously gathered ground truth data from a rate-gyro head tracker in order to determine head motion and EMG signals from four pairs of muscles around the neck (identified experimentally) to measure muscle activity. The amount of data we are dealing with is quite voluminous; three gyro and eight EMG channels with over one hundred twenty thousand data points each per experiment. Our efforts require an understanding of the temporal characteristics of the EMG signals. Simultaneously, we want to visualize each muscle's role in head motion; more specifically, how it acts with or against other muscles. Furthermore, we need to be able to visualize the anticipation results of our system as well as compare its performance against other existing prediction mechanisms. Therefore, to better understand the large amount of disparate data we are dealing with, it was necessary to develop a powerful visualization tool.The tool we developed is shown in Figure 1. Here we use CG models of a skull and thorax to animate the experimental sequences. To define the actual head position and rotation at a particular time, we utilize the rate-gyro data gathered during the experiments. As a result, using computer animation, we are able to replicate the actual head movement of each subject.To understand the EMG signals in the context of head movement, in tandem with this animation we created eight graphs to represent the EMG activity recorded during the experiments. We use an animated cursor (as well as the blue to red color switch illustrated on the left inset of Fig. 1) to indicate the point on EMG tracks that corresponds to the currently animated head motion.In addition, we need to visualize which muscles are contributing to a particular movement and how important they are to the specific motion. We developed CG models for the eight muscles attached to the 3D skull/thorax model. To visualize a muscle's activity we correlate its size and color to the strength of the corresponding EMG signals. The muscle size increases and the flesh color is modulated from a pale flesh tone to the more saturated color assigned to the particular muscle as the amplitude and duration of the corresponding EMG signal increases.To visualize the anticipation results, we use three arrows. The first arrow is attached to the skull and represents the forward direction of the head during motion; the second represents the direction of the ideal prediction, which we compute by shifting the measured direction by a few ms into the future; and the third represents the position anticipated by our system. Through the animation of these arrows, we can readily assess the performance of the system by measuring the temporal gap between the arrows. To better depict the perceptual effects of anticipation, we also introduced a simplified world represented as a set of concentric squares that simulate the user's view as shown in Figure 2. On the left we see two sets of concentric squares, in green indicating the appearance of the world where the user is actually looking, and in blue the appearance where the system anticipates the user is looking. In this manner, the visualization tool provides us with a powerful means for assessing the anticipation results and understanding the capabilities of the system across movements.This work presents a number of unique contributions, in particular, the ability to simultaneously visualize a number of disparate information sources to permit an intuitive understanding of them as a whole. The temporal and spatial data visualization in tandem with the 3D animation immersed in a simulated world allows for a more effective management of information content.