SmokeNav: Millimeter‐Wave‐Radar/Inertial Measurement Unit Integrated Positioning and Semantic Mapping in Visually Degraded Environments for First Responders

Abstract

First responders often face hazardous and life‐threatening situations in environments filled with smoke, posing significant risks to their safety. The existing perception solutions, such as camera or light detection and ranging (LiDAR)‐based methods, are inadequate when faced with visually degraded conditions caused by smoke. In this work, SmokeNav, a novel system that combines data from an inertial sensor and millimeter‐wave (mmWave) radar, is proposed to enhance situational awareness for first responders in smoky environments. SmokeNav utilizes an inertial positioning module that exploits the human motion constraints with a foot‐mounted inertial measurement unit to provide accurate user localization. By integrating this location information with mmWave radar data, it employs a probabilistic occupancy map construction to reconstruct an accurate metric map. To enable semantic understanding of the environment, a DNN‐based semantic segmentation model that incorporates radar reflectivity and employs focal loss to improve performance is introduced. Herein, extensive real‐world experiments in smoky environments is conducted to demonstrate that SmokeNav precisely localizes the user and generates detailed maps with semantic segmentation. In this work, potentials are held for enhancing the safety and effectiveness of first responders in hazardous conditions.