A Customized VR Rendering with Neural-Network Generated Frames for Reducing VR Dizziness

Abstract

AbstractThis research is to develop a NN system that generates smooth frames for VR experiences, in order to reduce the hardware requirement for VR therapy. VR is introduced as a therapy to many symptoms, such as Acrophobia, for a long, but not yet broadly used. One of the main reason is that a high and stable frame rate is a must for a comfortable VR experience. Low fresh rate in VR could cause some adverse reactions, such as disorientation or nausea. These adverse reactions are known as “VR dizziness”. We plan to develop a NN system to compress the frame data and automatically generate consecutive frames with extra information in real-time computation. Provided with the extra information, the goal of compressing NN is to conclude and abandon textures that are dispensable. Thereafter, it would compress left and right frames in VR into a data frame with all necessary information. Subsequently, we send it to clients, the client put the data frame into rebuilding GAN system and get three pairs of frames. After applying some CV methods such as filtering and alias, these frames are shown on users’ HMD. Furthermore, the compressed data can be transported easily, which allows medical facilities to build a VR computing center for multiple clients, making VR more broadly available for medical usages.KeywordsComputer visionVirtual RealityNeural NetworkGenerative adversarial network