Considering the increasing demand for the integration of haptic feedback into virtual reality (VR) devices, we studied and developed a multi-sensor and actuator array for real-time perception in virtual reality for training first responders. The sensors respond to users’ interactions in the VR world and actuate the response in the real world to create realistic sensory perception [1–2]. Though several haptic device applications have been created [3], very few of them have investigated the integration of an all-in-one sensory feedback mechanism. This paper presents a novel haptic feedback glove that helps with virtual reality training applications by utilizing all-in-one sensory feedback of heat, and cold. We investigated heat and cold sensing by incorporating peltiers and microheaters into the multi-functional haptic glove to generate the radiation of heat and cold. This version of the glove functions on three components: a microcontroller (ESP32), thermocouple modules that work as sensor elements (-55°C to 125°C), polyimide heater plate films, and micro Peltiers (12V max at 12W max) that work as actuators. These three modules are responsible for creating thermal haptic feedback, which communicates with the virtual environment world and the local environment to create the sensation of heat and cold, as shown in Fig. 1(a). When the user touches a hot or cold object in virtual reality, the hand tracking recognizes the element triggering the actuators, which are controlled by the sensor to regulate the temperature. We utilized Unreal Engine 5 (UE5) and two frameworks to enable serial and Bluetooth data communication between UE5 and the microcontroller for the high-risk training simulation. Based on Fig. 2 (a), after carefully testing, the distance between the data transfer start and maximum temperature output was about a full arm’s length in real life. This could be translated to 2 to 3 feet in distance. In the case of Fig. 2(b), we see multiple negative voltages applied to the Peltier device, producing the following equation:(1).In eq.1, A is the ambient temperature, T is the temperature of the object, D is the max distance of when the user starts feeling the object’s temperature, and t is the temperature of the finger because each Peltier is individually controlled. The heating ramping rate was 4 seconds from room temperature to 60°C and for cooling, the ramping time was 8 seconds from room temperature to 10°C. The currently developed glove demonstrates for the first time the integration of heat and cold sensing for VR applications. The developed prototype glove will be used for implementing training in VR for safety officers, military, and first responders. Figure 1
Read full abstract