Prospects of pose estimation as an input in gaming

Abstract

Although there have been attempts to use Gesture-Based User Interaction in interfaces, not just in games, they require some special hardware. For example, Xbox’s Kindest controller. Although it is very good at what it does, the cost and its proprietary nature restrict its use drastically. With this research, we aim to bridge the experience of already existing expensive and proprietary technologies with the cost-effectiveness of cheap accessories like basic webcams and nominal processing for image processing and gesture recognition. A pose estimation-based gaming interaction system would require a media capturing device, a processing unit, and a display device to show the output. We expect to use the inbuilt webcam on our laptops as the capturing media device. Our dedicated GPU or integrated GPUs have become powerful enough to sustain loads to perform pose estimation and gesture recognition while running a game in parallel. We get in-depth and look at the different approaches that are present in pose estimation. There is 2D Pose Estimation, and there is 3D pose estimation. We carry out intense research in implementing these two techniques and implementing them in a Unity game environment on a Game Object and making it interactive with the environment better.