Abstract
To fulfill the tasks of human-robot interaction (HRI), how to detect the specific human (SH) becomes paramount. In this paper, the deep learning approach by the integration of Single-Shot Detection, FaceNet, and Kernelized Correlation Filter (SSD-FN-KCF) is developed. From the outset, the SSD is employed to detect the human up to 8m using the RGB-D camera with 320 × 240 resolution. Afterward the omnidirectional mobile robot (ODMR) is driven to the neighborhood of 2.5~3.0 m such that the depth image can accurately estimate the detected human's pose. Subsequently, the ODMR is commanded to the vicinity of 1.0m and the orientation inside -60~60° with respect to the optical axis to identify whether he/she is the SH by the FaceNet. To reduce the computation time of the FaceNet and extend the SH's tracking, the KCF is employed to achieve the task of HRI (e.g., human following). Based on the image processing result, the required pose for searching or tracking (specific) human is accomplished by the image-based adaptive finite-time hierarchical constraint control. Finally, the experiment with the SH, who is far from and on the backside of the ODMR, validates the effectiveness and robustness of the proposed approach.
Highlights
Human-robot interaction has received increasing attention in the last decades, since robots may act as both helpers and companions for the elderly and impaired people, especially for an aging population [1], [2]
The contributions of this study are summarized as follows: (i) The learning of single shot detector (SSD) can effectively detect the human beyond the general recognized distance of RGB-D camera system (e.g., 8m). (ii) The FaceNet is effectively learned to recognize the different faces with the recognition rate over 95% under suitable distance (0.75∼1.25m), different view angle (−60∼60◦), different light angle (-80∼80◦), and some occlusions. (iii) To avoid the repeated calculation of the FaceNet, extend the tracking distance, and reduce the computation time, the kernelized correlation filter (KCF) is employed to track the specific human (SH) such that human-robot interactions are achieved by the suggested IB-adaptive finitetime hierarchical constraint control (AFTHCC)
The omnidirectional mobile robot (ODMR) is controlled to the desired orientation φd (t) by the IB-AFTHCC such that human is at the central position of field of view (FOV). (vii) The vertical position of the ODMR is controlled by the IB-AFTHCC
Summary
Human-robot interaction (or collaboration) has received increasing attention in the last decades, since robots may act as both helpers and companions for the elderly and impaired people, especially for an aging population [1], [2]. Hwang et al.: Interactions Between Specific Human and ODMR Using Deep Learning Approach: SSD-FN-KCF but acceptable. After the identification of the SH, he/she is tracked by kernelized correlation filter (KCF) to avoid the repeated calculation of the FaceNet, extend the tracking distance, and reduce the computation time. The contributions of this study are summarized as follows: (i) The learning of SSD can effectively detect the human beyond the general recognized distance of RGB-D camera system (e.g., 8m). (iii) To avoid the repeated calculation of the FaceNet, extend the tracking distance, and reduce the computation time, the KCF is employed to track the SH such that human-robot interactions (e.g., human following) are achieved by the suggested IB-AFTHCC.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
