Martial Arts Training in Virtual Reality with Full-body Tracking and Physically Simulated Opponents

Abstract

Virtual reality (VR) training is an increasingly popular topic in sports science [1]. We introduce a VR application for martial arts training, which utilizes physics-based full-body interaction that extends our previous work on the subject [2]. Our use of full-body tracking and user-worn VR equipment without external tethering enables training of techniques that employ lower body: e.g. kicking, grappling, and leg sweeps. Hence a wide variety of martial art styles – from stand-up fighting to ground fighting – can be trained.The training application features virtual opponents and a motion tracked user avatar, which is implemented using RUIS toolkit [3]. The avatar’s and the virtual opponents’ body segments are part of physics simulation that determine their final motion. This enables dynamic hand-to-hand combat, where the user’s punches, takedowns, holds, and other techniques affect the opponents in a convincing manner. The user can engage any number of virtual opponents at full power in no-holds-barred matches without risk of injury to fellow practitioners.In addition to partner training, sparring, and virtual target mitts, the application also includes novel VR training features: performance playback, rhythm game, and controls for adjusting virtual opponent speed, strength, and knockout resistance. User performed techniques can be recorded and then replayed dynamically by the virtual opponent using the physics simulation. Thus, the user can be their own training partner, and even fight himself in the virtual arena.