Domestic Robots Research Articles

Socially CompliAnt Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation

Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a “socially compliant” manner in the presence of other intelligent agents such as humans. With the emergence of autonomously navigating mobile robots in human-populated environments (e.g., domestic service robots in homes and restaurants and food delivery robots on public sidewalks), incorporating socially compliant navigation behaviors on these robots becomes critical to ensuring safe and comfortable human-robot coexistence. To address this challenge, imitation learning is a promising framework, since it is easier for humans to demonstrate the task of social navigation rather than to formulate reward functions that accurately capture the complex multi-objective setting of social navigation. The use of imitation learning and inverse reinforcement learning to social navigation for mobile robots, however, is currently hindered by a lack of large-scale datasets that capture socially compliant robot navigation demonstrations in the wild. To fill this gap, we introduce Socially CompliAnt Navigation Dataset ( SCAND )–a large-scale, first-person-view dataset of socially compliant navigation demonstrations. Our dataset contains 8.7 hours, 138 trajectories, 25 miles of socially compliant, human tele-operated driving demonstrations that comprises multi-modal data streams including 3D lidar, joystick commands, odometry, visual and inertial information, collected on two morphologically different mobile robots–a Boston Dynamics Spot and a Clearpath Jackal–by four different human demonstrators in both indoor and outdoor environments. We additionally perform preliminary analysis and validation through real-world robot experiments and show that navigation policies learned by imitation learning on SCAND generate socially compliant behaviors.

Interaction with a reactive partner improves learning in contrast to passive guidance

Many tasks such as physical rehabilitation, vehicle co-piloting or surgical training, rely on physical assistance from a partner. While this assistance may be provided by a robotic interface, how to implement the necessary haptic support to help improve performance without impeding learning is unclear. In this paper, we study the influence of haptic interaction on the performance and learning of a shared tracking task. We compare in a tracking task the interaction with a human partner, the trajectory guidance traditionally used in training robots, and a robot partner yielding human-like interaction. While trajectory guidance resulted in the best performance during training, it dramatically reduced error variability and hindered learning. In contrast, the reactive human and robot partners did not impede the adaptation and allowed the subjects to learn without modifying their movement patterns. Moreover, interaction with a human partner was the only condition that demonstrated an improvement in retention and transfer learning compared to a subject training alone. These results reveal distinctly different learning behaviour in training with a human compared to trajectory guidance, and similar learning between the robotic partner and human partner. Therefore, for movement assistance and learning, algorithms that react to the user’s motion and change their behaviour accordingly are better suited.

Solution of World Robot Challenge 2020 Partner Robot Challenge (Real Space)

The World Robot Challenge is an international competition for the social implementation of robots. Among them, the Partner Robot Challenge (Real Space) is a category that focuses on domestic service robots, competing to achieve simple tasks such as tidying a room, avoiding small obstacles, grasping a specified object from the shelf, and delivering an object to a waving person. In this category, we focused on the theme of ‘Keep Moving’ and worked on researching and developing technologies for object recognition, grasping, and other tasks. For object recognition, we propose an automatic dataset generator using a physics simulator and generate about 490,000 images in two hours. We trained it with an instance segmentation model, achieving an accuracy of about 79.3% in all games. For object grasping, we installed a three-dimensional sensor on the robot hand, and achieved successful grasping with high accuracy of about 79.6% (88.9% excluding failures due to hardware problems) in all games. In addition, we propose a motion synthesis method that simultaneously performs movement and posture transition to achieve high-speed motion. When navigation is executed, the robot changes to the navigation posture while aiming for the goal. Before reaching the goal, it changes the posture for the following action. As a result, we realized a speedup of about 1.32 times in the pick-and-place task. In the tidy-up task, the robot could clear an average of 14 objects and a maximum of 16 objects in 15 minutes. In the obstacle avoidance task, the robot succeeded five times out of six games, and in the delivery task, it succeeded four times out of six games. Throughout all the tasks, we scored 800 points twice, the highest score in the world, and won the championship. This research is aimed at the practical application of domestic service robots. We believe that the results presented in this paper demonstrate the practicality of domestic service robots in a well-developed environment. The video of the finals is available at https://youtu.be/ElUb8bfSC34?t=5511 (right side). The our source code using the simulator is available at https://github.com/Hibikino-Musashi-Home/hma_wrs_sim_ws.

Indoor Positioning Technology and Control of Mobile Home Service Robot

With the rapid development of the market economy, location information services have shifted from outdoor to indoor, requiring higher system accuracy. Indoor positioning and control is an indispensable technical support for mobile home service robots to perform system tasks, and this technology is also an important parameter that marks the level of intelligence of a home service robot. However, in indoor places, how to obtain the target node location information timely and accurately and how to control and path planning should be the first issues to be considered in the design of home robots at this stage. This article gave a brief overview of the current indoor positioning technology and control, analyzed the more widely and frequently used indoor positioning methods, and deeply studied the ultrasonic positioning technology. On this basis, a mobile home service robot system was designed, and simulation experiments were carried out, respectively, to test the receiving time and arrival time of the node signal, the node positioning accuracy, the optimal degree of route planning, and the absolute and relative errors of navigation and path estimation. The experimental results showed that the accuracy of the robot system positioning proposed in this article reached more than 80%, and the relative error was only 4.21%, which verified its feasibility and effectiveness in practical applications. Popularizing it in the market can effectively improve the system performance and intelligence of home service robots.

Explanations and trust: What happens to trust when a robot partner does something unexpected?

Performance within Human-Autonomy Teams (HATs) is influenced by the effectiveness of communication between humans and robots. Communication is particularly important when robot teammates engage in behaviors that were not anticipated by the human teammate which could degrade trust. However, the literature on trust repair focuses on the role of apologies which may not be appropriate for an unexpected behavior since this behavior may not be an error. Explanations are one method that can be used by robot teammates to avoid costly trust degradation when human expectations are violated. The current study used an Urban Search and Rescue (USAR) scenario to examine the role of explanation in a context wherein a robot teammate deviated from an expected behavior. The current study examined how trust, trustworthiness, and responsibility attribution were influenced by observing the robot teammate deviate from an expected behavior. Participants (N = 148) used an online platform to view videos of the robot: 1) following a planned search path, and 2) deviating from a planned search path. A debriefing event between the human and the robot followed each search activity. Four explanation conditions were tested in the debriefing phase following the behavioral violation. Results showed that trust and trustworthiness (ability, benevolence, and integrity) declined following the unexpected behavior. Accordingly, responsibility attribution shifted from the human to the robot following the unexpected behavior. Explanation strategies that focused on communicating why the event occurred by highlighting the robot's environmental awareness were most effective at buffering the decline in trust and trustworthiness.

On the Use of Assistive Technology during the COVID-19 Outbreak: Results and Lessons Learned from Pilot Studies.

As a consequence of the COVID-19 emergency, frail citizens felt isolated because of social isolation, suspended and/or strongly reduced home assistance, and limited access to hospitals. In this sense, assistive technology could play a pivotal role in empowering frail older adults reducing their isolation, as well as in reinforcing the work of formal caregivers and professionals. In this context, the goal of this paper is to present four pilot studies—conducted from March 2020 to April 2021—to promptly react to COVID-19 by providing assistive technology solutions, aiming to (1) guarantee high-quality service to older adults in-home or in residential facility contexts, (2) promote social inclusion, and (3) reduce the virus transmission. In particular, four services, namely, telepresence service, remote monitoring service, virtual visit, and environmental disinfection, were designed, implemented, and tested in real environments involving 85 end-users to assess the user experience and/or preliminary assess the technical feasibility. The results underlined that all the proposed services were generally accepted by older adults and professionals. Additionally, the results remarked that the use of telepresence robots in private homes and residential facilities increased enjoyment reducing anxiety, whereas the monitoring service supported the clinicians in monitoring the discharged COVID-19 patients. It is also worth mentioning that two new services/products were developed to disinfect the environment and to allow virtual visits within the framework of a hospital information system. The virtual visits service offered the opportunity to expand the portfolio of hospital services. The main barriers were found in education, technology interoperability, and ethical/legal/privacy compliance. It is also worth mentioning the key role played by an appropriate design and customer needs analysis since not all assistive devices were designed for older persons.

Team Tidyboy at the WRS 2020: a modular software framework for home service robots

This paper describes Team Tidyboy's approach to build a functional, robust, yet highly modular software framework for home service robots. Home service tasks require a number of software modules to work together, which includes verbal and nonverbal human robot interactions, 3D perception of known and unknown objects and manipulation of various objects using movement of both the manipulator and the base of the robot. World Robot Summit (WRS) 2020 Partner Robot Challenge (PRC) rule has raised the bar higher by asking for a reliable, long-term operation of such a system without human intervention. We present how we have overcome such challenges by robust software modules, repeated experiments in randomly initialized simulated environments, and finally introduction of various failure detection and recovery algorithms. Our software is successfully validated in RoboCup@Home 2021 and WRS 2020 PRC remote participation league, which led us to obtaining the first places in both competitions.

What the Home of the Future is Missing Today

Abstract A few household robots are on the market today. But, none of those could be confused with Rosie the Robot, the muliply capable robot on The Jetsons. That raises the question: As technology advances, what will it take for the promise of the home of the future—powered by intelligent domestic robots—to finally be realized?

World robot challenge 2020 – partner robot: a data-driven approach for room tidying with mobile manipulator

Tidying up a household environment using a mobile manipulator poses various challenges in robotics, such as adaptation to large real-world environmental variations, and safe and robust deployment in the presence of humans. The Partner Robot Challenge in World Robot Challenge (WRC) 2020, a global competition held in September 2021, benchmarked tidying tasks in real home environments, and, importantly, tested for full system performances. For this challenge, we developed an entire household service robot system, which leverages a data-driven approach to adapt to numerous edge cases that occur during the execution, instead of classical manual pre-programmed solutions. In this paper, we describe the core ingredients of the proposed robot system, including visual recognition, object manipulation, and motion planning. Our robot system won the second prize, verifying the effectiveness and potential of data-driven robot systems for mobile manipulation in home environments.

Information technology in agriculture: import substitution, economic challenges and technological alternatives

The article provides an overview of possible mechanisms for import substitution of information technologies in the agro-industrial complex (AIC). Changes in the field of international relations directly affected the agro-industrial complex of the Russian Federation. The withdrawal of Western partners from the market, all kinds of restrictions and disruption of supply chains have become a real challenge for Russian agribusiness. These difficulties will become a powerful impetus for the development of domestic production, which will make it possible to occupy vacant niches, reorient cargo flows, increase import substitution, accelerate automation and digitalization of processes, and, ultimately, reach a new level of economic development of the agro-industrial complex. To a large extent, it is information technology (IT) that will become an indicator that determines how successfully enterprises in the industry will develop in the short and long term. The development of the agro-industrial complex is a fundamental direction in any economy, as it is an important strategic factor influencing the socio-economic stability of society as a whole. Comprehensive and effective development of the agro-industrial complex is impossible without the use of advanced digital technologies. Digital transformation will contribute to the development of domestic robotics and the entire IT market. The growth rate of digitalization of the agro-industrial complex in recent years, and its share in Russia's GDP, is still lagging behind the leading economies of the world. Currently, in the Russian Federation, through the implementation of the national program "Digital Economy of the Russian Federation" and the departmental project "Digital Agriculture" of the Ministry of Agriculture of Russia, as well as by creating conditions for advanced development and import substitution, the number of investment projects for the construction and reconstruction of agricultural facilities is growing. The latest equipment and technologies require a high degree of automation and the use of advanced digital solutions, and therefore the digital transformation of food production and, in general, all areas of the agro-industrial complex is gaining momentum.

Long-term object search using incremental scene graph updating

AbstractEffective searching for target objects in indoor scenes is essential for household robots to perform daily tasks. With the establishment of a precise map, the robot can navigate to a fixed static target. However, it is difficult for mobile robots to find movable objects like cups. To address this problem, we establish an object search framework that combines navigation map, semantic map, and scene graph. The robot updates the scene graph to achieve a long-term target search. Considering the different start positions of the robots, we weigh the distance the robot walks and the probability of finding objects to achieve global path planning. The robot can continuously update the scene graph in a dynamic environment to memorize the position relation of objects in the scene. This method has been realized in both simulation and real-world environments. The experimental results show the feasibility and effectiveness of this method.

Deep understanding of everyday activity commands for household robots

Going from natural language directions to fully specified executable plans for household robots involves a challenging variety of reasoning steps. In this paper, a processing pipeline to tackle these steps for natural language directions is proposed and implemented. It uses the ontological Socio-physical Model of Activities (SOMA) as a common interface between its components. The pipeline includes a natural language parser and a module for natural language grounding. Several reasoning steps formulate simulation plans, in which robot actions are guided by data gathered using human computation. As a last step, the pipeline simulates the given natural language direction inside a virtual environment. The major advantage of employing an overarching ontological framework is that its asserted facts can be stored alongside the semantics of directions, contextual knowledge, and annotated activity models in one central knowledge base. This allows for a unified and efficient knowledge retrieval across all pipeline components, providing flexibility and reasoning capabilities as symbolic knowledge is combined with annotated sub-symbolic models.

Two ways to make your robot proactive: Reasoning about human intentions or reasoning about possible futures.

Robots sharing their space with humans need to be proactive to be helpful. Proactive robots can act on their own initiatives in an anticipatory way to benefit humans. In this work, we investigate two ways to make robots proactive. One way is to recognize human intentions and to act to fulfill them, like opening the door that you are about to cross. The other way is to reason about possible future threats or opportunities and to act to prevent or to foster them, like recommending you to take an umbrella since rain has been forecast. In this article, we present approaches to realize these two types of proactive behavior. We then present an integrated system that can generate proactive robot behavior by reasoning on both factors: intentions and predictions. We illustrate our system on a sample use case including a domestic robot and a human. We first run this use case with the two separate proactive systems, intention-based and prediction-based, and then run it with our integrated system. The results show that the integrated system is able to consider a broader variety of aspects that are required for proactivity.

Towards general purpose service robots: World Robot Summit – Partner Robot Challenge

Although the skills required to solve specific robotics problems are reaching amazing performances recently, we propose the evaluation of such individual solutions in fully integrated robot systems tested in real daily situations like those presented at international robotics competitions. The World Robot Summit (WRS)– Partner Robot Challenge surges from the necessity to standardise and spread the research on Domestic Service Robots where a series of solutions can be tested to solve a general purpose task in a standard domestic environment. We propose the Tidy-Up task as a benchmark and present the key features behind this challenge as well as our main findings through the several stages in the design process. This approach has been proven successful in different international competitions, namely, the RoboCup Japan Open, the Mexican Tournament of Robotics, the RoboCup Worldwide and the WRS.

CLEAN SWEEP: The Floor Cleaning Robot

Abstract: Modern households are getting increasingly automated and intelligent. Home automation makes life more convenient and gives individuals more free time. Domestic robots are becoming more prevalent in homes and in people's daily lives, yet the market is still young and in its infancy. But a rise is anticipated, and the use of household robots is changing. There are many robotic vacuum cleaners on the market, but very few of them wet-clean floors. This project's goal is to create and use a vacuum robot with two cleaning modes. Through a phone application, autonomous and manual modes are available. Instead of using a manual vacuum, a robot vacuum cleaner is designed to make cleaning easier. This project's goal is to build and put into action a vacuum robot with two cleaning modes: autonomous and manual, the latter of which may be accessed via a phone application.

Generalizable Task Planning Through Representation Pretraining

The ability to plan for multi-step manipulation tasks in unseen situations is crucial for future home robots. But collecting sufficient experience data for end-to-end learning is often infeasible in the real world, as deploying robots in many environments can be prohibitively expensive. On the other hand, large-scale scene understanding datasets contain diverse and rich semantic and geometric information. But how to leverage such information for manipulation remains an open problem. In this letter, we propose a learning-to-plan method that can generalize to new object instances by leveraging object-level representations extracted from a synthetic scene understanding dataset. We evaluate our method with a suite of challenging multi-step manipulation tasks inspired by household activities (Srivastava, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> , 2022) and show that our model achieves measurably better success rate than state-of-the-art end-to-end approaches.

1039 RECEIVING ASSISTANCE FROM DOMESTIC ROBOTS: WHAT ARE ELDERS’ PREFERENCES?

Abstract Introduction Aging leads to decline in physical and functional ability. Strengthening the domestic support enables elders to manage their health and living at home and reduce their need for residential service. Domestic robots can potentially provide a broad range of support to the elders. However, relatively little research attention has investigated elders’ preferences on it. This study aims to investigate elders’ preferences on receiving assistance from domestic robots. Methods This cross-sectional descriptive study recruited a convenient sample of 365 robotic inexperienced elders (65-99 years old). It used the Assistance Preference Checklist to assess participants’ preferences on receiving assistance from domestic robots in 48 home-based tasks under six aspects including personal care, leisure activities, health assistance, chores, information management, and manipulating objects. It used t-test and one-way analysis of variance to compare the difference in preferences between participants with different demographic and health characteristics. Results Overall, participants preferred domestic robots to assist in tasks under the aspect on chores, information management, and manipulating objects. Specifically, males indicated a higher preference for domestic robots to assist in maintaining lawn or raking leaves than females (p = 0.05). Married participants indicated a higher preference for domestic robots to assist in getting information on weather/news than unmarried participants (p = 0.049). In contrast, participants who were partially dependent and fully dependent indicated a higher preference for domestic robots to assist in tasks under the personal care aspect such as shaving, bathing, washing/combing hair, getting dressed, walking, and brushing teeth (p &amp;lt; 0.001), comparing to participants who were physically independent. Conclusion This study provides insight for manufacturers to develop domestic robots with specific capabilities which are important to support elders’ living in the community. Moreover, it helps the care providers to identify domestic robots with the required capabilities that meet elders’ specific preferences.

1040 VALIDATION OF THE CHINESE VERSION OF THE ASSISTANCE PREFERENCE CHECKLIST

Abstract Introduction Domestic robots have great potential to be developed as a useful and supportive device for elders in domestic setting. To achieve the intended purpose, the designs of domestic robots should make reference to elders’ preferences. The Assistance Preference Checklist is developed to assess elders’ preferences on receiving assistance from domestic robots. It comprises of 48 home-based task under six categories including personal care, leisure activities, health assistance, chores, information management, and manipulating objects. Altogether, these tasks are considered important for the elders to fulfil their general health needs and maintain their homes. The Assistance Preference Checklist is valid and reliable. To adapt it in the Chinese population, the Assistance Preference Checklist has been translated from English to Chinese according to the standard translation model. This study aims to validate the Chinese version of the Assistance Preference Checklist (the checklist). Method This was a validation study. Semantic and content equivalence of the checklist was evaluated by an expert panel (five academic specialized on instrument validation or translation and 15 nurses specialized on geriatrics). Content validity of the checklist was assessed by another expert panel (one academic specialized on instrument validation and two nurses specialized on geriatrics). Internal consistency of the checklist was determined by Cronbach's method on a convenience sample of 50 Chinese-speaking elders in Hong Kong. Results The checklist demonstrated good semantic and content equivalence with the original English version. The experts agreed that it was appropriately translated. The checklist demonstrated good content validity by having a content validity index of 1.00. Additionally, the checklist reported satisfactory internal consistency by having a Cronbach's alpha of 0.95. Conclusion The Chinese version of the Assistance Preference Checklist is a useful instrument for assessing elders’ preferences on receiving assistance from domestic robots. Findings provide evidence on its validity and reliability.

The growing importance of lidar technology

PurposeThis paper aims to provide an insight into light detection and ranging (lidar) technology and its growing applications in robotics.Design/methodology/approachFollowing a short introduction, this paper first describes the main lidar techniques and then provides details of a selection of recent academic and corporate research and development activities. This is followed by a discussion of existing and emerging applications. Finally, conclusions are drawn.FindingsLidar technology has been the topic of extensive development activity and several principles which differ from the original concept have been commercialised. Lidars are used in all manner of autonomous mobile robots (AMRs) across a broad sector of industries for navigation and have recently started to penetrate the domestic robot market. They have the potential to play a central role in the emerging families of driverless passenger cars and commercial vehicles. In the future, the markets for lidar are expected to expand dramatically as the technology continues to evolve and improve and autonomous vehicles, AMRs and drones become ever-more commonplace.Originality/valueThis paper illustrates the growing importance of lidar to robotics by providing details of the technology, developments and applications.

Strategies for implementing pet robots in care homes and nursing homes for residents with dementia: protocol for a modified Delphi study

BackgroundPet robots are a type of technology-based innovation that have shown positive psychosocial benefits for people with dementia in residential facilities, such as improving mood and social interaction and reducing agitation. Nevertheless, little is known about how pet robots can be implemented in care homes and nursing homes for dementia care in real-world practice. The objectives of this study are to (1) identify contextualised implementation strategies for implementing pet robots into care homes and nursing homes for dementia care and (2) achieve consensus on the most relevant strategies.MethodThis study is informed by a preceding scoping review and qualitative study, which used the Consolidated Framework of Implementation Research (CFIR) to identify multi-level determinants of implementation (i.e. barriers and facilitators). We will use the CFIR-ERIC matching tool to identify relevant implementation strategies from the Expert Recommendations for Implementing Change (ERIC) taxonomy to address these determinants. Data from the scoping review and qualitative study will be used to contextualise the generic ERIC strategies for our setting. After that, a group of key stakeholders will be consulted to further contextualise and refine these strategies. Next, a two-round modified Delphi process will be conducted. Fifty-four international expert participants including healthcare professionals and organisational leaders from care homes and nursing homes and academic researchers will be recruited through purposive sampling. During the first Delphi round, participants will be invited to rate the relevance of each implementation strategy on a 9-point Likert scale and provide comments or suggestions. Descriptive statistics will be used to identify whether consensus has been obtained. Inductive qualitative content analysis will be used to analyse and summarise textual responses for any new statements suggested by participants. Statements that do not reach consensus and new statements suggested in round 1 will be taken to the next round, which will follow the same rating process.DiscussionThis study will identify strategies for implementing pet robots in care homes and nursing homes for residents with dementia, which will have practical utility for clinicians, organisations and researchers. It will also demonstrate the practical application (and adaptation) of the CFIR-ERIC tool to identify and contextualise ERIC strategies.