Abstract
This paper presents Kinect-based vision system of mine rescue robot working in illuminous underground environment. The somatosensory system of Kinect is used to realize the hand gesture recognition involving static hand gesture and action. AK-curvature based convex detection method is proposed to fit the hand contour with polygon. In addition, the hand action is completed by using the NiTE library with the framework of hand gesture recognition. In addition, the proposed method is compared with BP neural network and template matching. Furthermore, taking advantage of the information of the depth map, the interface of hand gesture recognition is established for human machine interaction of rescue robot. Experimental results verify the effectiveness of Kinect-based vision system as a feasible and alternative technology for HMI of mine rescue robot.
Highlights
Rescue is very urgent after coal mine accident because after 48 hours, victim mortality drastically increases owing to exposure to bad air and lack of food, water, and medical treatment [1, 2]
It can be attained that hand gesture recognition depends more on depth map, while body posture is liable to focus on body skeleton joint data captured from Kinect
The corresponding command is sent to control the movement direction of rescue robots through WiFi network
Summary
Rescue is very urgent after coal mine accident because after 48 hours, victim mortality drastically increases owing to exposure to bad air and lack of food, water, and medical treatment [1, 2]. When explosive conditions exist or heavy smoke enters the mine roadway, robots can become an invaluable tool These mobile robots navigating deep into rubbles can search for survivors and transfer on-site video and atmospheric monitoring information for rescuers to confirm safe state or identify potentially hazardous conditions for mine rescue team [3]. It can be attained that hand gesture recognition depends more on depth map, while body posture is liable to focus on body skeleton joint data captured from Kinect. The paper proposes a Kinect-based vision system for rescue robot underground mining tunnel and achieves hand gesture recognition combining depth map and skeleton joint information.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
