Continuous fine‐grained arm action recognition using motion spectrum mixture models

Abstract

Motion sensors in smart wristbands/watches have been widely used to sense users’ level of movement and animation. Some studies have further recognised activity contexts using these sensors, such as walking, sitting and running. However, in applications requiring understanding of more complex activities such as interactions with other people or objects, it is necessary to recognise the fine-grained arm action during user interactions with other people or objects. A method to recognise a set of arm actions on a fine-grained level (e.g. checking the wristband, drinking water etc.) is proposed. Motion signals from the accelerometer and gyroscope are transformed into the frequency domain using the short-time Fourier transform. Then, the action patterns are represented by the motion spectrum mixture model and action dynamics are modelled by continuous density hidden Markov models. Tested on a dataset collected from 23 subjects, the method shows satisfying performance and efficiency.