FPGA accelerator for wireless AR/VR display

Abstract

Nowadays, Virtual reality(VR) and Aug- mented reality(AR) have become one of the most popular format in many fields, for example video gaming, medical training and even aviation. VR and AR technique simulates images in an edge device, it gives an immersive experience to the users. AR/VR requires high resolution and high FPS for good experience. However, most of the AR/VR devices are made of embedded device due to the limitation of the size and weight of the headset. It is hard to render high quality frames in headset. Many popular VR/AR applications utilize the desktop and server to render the frames and transmit the frames to VR/AR for display. Data transmission from a more powerful device to the VR/AR device requires high transmission speed (1.6GB/s for Oculus quest 2), it is hard to provide the bandwidth with wireless protocol (WIFI/5G). HDMI or DP cable can be applied, but they limit the use case of the VR/AR devices. In this paper, we proposed a latency sensitive super sampling hardware accelerator for VR/AR devices based on machine learning which can significantly reduce the bandwidth requires to transmit frames to VR/AR. In our experiment, the super sampling can deliver high-resolution frames with 25% bandwidth which enable the wireless protocal for VR/AR devices. We implemented the accelerators in RTL and synthesis it with 130 nm skywater pkd. The power consumption of our accelerator at normal data rate for VR/AR devices is 20.97 w and the area is 299.602 mm2.