Person Finding: An Autonomous Robot Search Method for Finding Multiple Dynamic Users in Human-Centered Environments

Abstract

Robot search for multiple dynamic users within a multi-room environment is important for social robots to find and engage in various human–robot interaction scenarios with these users. In this paper, we present a novel autonomous person search technique for a robot finding a group of dynamic users before a deadline. The uniqueness of our approach is that unlike existing robot search methods, we consider activity information to predict where, when, and for how long a user will be in a specific room. This allows for the generation of search plans without any assumption on the frequency of user movements. We represent our search problem as an extension of the orienteering problem (OP), which we define herein as the robot person search OP (PSOP). User activity information is represented as spatial–temporal user activity probability density functions (APDFs). We solve the PSOP using APDFs to generate a search plan to maximize the expected number of users found before the deadline. The solution of the PSOP is obtained in two steps. First, by solving a variant of the multiperiod knapsack problem to determine which rooms should be searched and for how long these rooms should be searched. Then, we solve the traveling salesman problem to obtain the order in which to search these rooms. Experiments were conducted to validate the performance of our robot search method in finding different numbers of multiple dynamic users for varying environment sizes and search durations. We also compared our method with two coverage planners and a Markov decision process planner. On average, our planner found more users than the other planners for a variety of scenarios. Finally, we performed experiments that introduced uncertainty into both the APDFs as well as during the search to validate the robustness of our overall approach. Note to Practitioners —The majority of current social robot applications either consider users being collocated with the robot in the same region or users being static within another region in the environment. However, several applications exist where users are dynamic within their environments and for which a robot needs to find them in order to provide assistance, for example, in office buildings, airports, museums, hospitals, and long-term care facilities. In general, these users are performing activities within these regions. We uniquely consider such activity information in order to model user location probabilities. We developed a robot search planner that uses these probabilities to find users of interest in multi-room environments. The planner is novel as it reasons about when and which regions to search and for how long, as well as if the same region needs to be searched multiple times as users can perform multiple activities during the search time frame in the same region or revisit a region to perform a new activity. We have integrated the search planner within a robot system architecture. The robot travels to each region and then uses a local planner to navigate to locations within the region. At each location, a person identification technique is used to identify the target users in order to engage in human–robot interactions. Experiments were performed for two search applications: 1) a simulated Blueberry robot finding multiple residents in a virtual representation of one of our collaborating long-term care facilities and 2) the physical Blueberry robot finding multiple staff/students on a physical floor of a university building. For both experiments, plans were generated on the robot’s onboard Lenovo Thinkpad X230 using the robot operating system (ROS) in Ubuntu. User activity data and maps used for the experiments in the care facility can be found on our website ( http://asblab.mie.utoronto.ca/research-areas/person-search-human-centered-environments ), under multi-user robot search. The physical Blueberry robot was also equipped with an ASUS Xtion IR depth camera, a Logitech pro c920 RGB camera, and a Hokuyo laser range finder for person identification and navigation in the environment. The results showed that our system was effective in finding multiple dynamic users under varying environment sizes and search durations. Our search planner also outperformed other planners and was robust to uncertainties in the user model. Future work will consider environments with multiple floors and crowded regions, planners that directly reason about environment dynamics, and local planners that reason about user location probabilities within regions.