Home Service Robot Research Articles

A new action recognition method by distinguishing ambiguous postures

One of the most important aspects of promoting the intelligence of home service robots is to reliably recognize human actions and accurately understand human behaviors and intentions. In the task of action recognition, there are many common ambiguous postures, which affect the recognition accuracy. To improve the reliability of the service provided by home service robots, this article presents a method of probabilistic soft-assignment recognition scheme based on Gaussian mixture models to recognize similar actions. First, we generate a representative posture dictionary based on the standard bag-of-words model; then, a Gaussian mixture model is introduced for the similar poses. Finally, combined with the Naive Bayesian principle, the method of weighted voting is used to recognize the action. The proposed scheme is verified by recognizing four types of daily actions, and the experimental results show its effectiveness.

Research on Home Service Robot for support of laundry operation.

Research on Home Service Robot for support of laundry operation.

RiSH: A robot-integrated smart home for elderly care

This article presents the development of a robot-integrated smart home (RiSH) which can be used for research in assistive technologies for elderly care. The RiSH integrates a home service robot, a home sensor network, a body sensor network, a mobile device, cloud servers, and remote caregivers. A layered architecture is proposed to guide the design and implementation of the RiSH software. Basic service functions are developed to allow the RiSH to recognize human body activity using an inertial measurement unit (IMU) and the home service robot to perceive the environment through audio signals. Based on these functions, we developed two low-level applications: (1) particle filter-based human localization and tracking using wearable motion sensors and distributed binary sensors; (2) Dynamic Bayesian Network-based human activity recognition using microphones and distributed binary sensors. Both applications extend the robot’s perception beyond its onboard sensors. Utilizing the low-level applications, a high-level application is realized that detects and responds to human falls. We conducted experiments in our RiSH testbed to evaluate auditory perception services, human body activity recognition, human position tracking, sound-based human activity monitoring, and fall detection and rescue. Twelve human subjects were asked to conduct daily activities in the testbed and mimic falls. All data of their movement, body activities, and sound events were collected by the robot. Human trajectories were estimated with a root mean square error of less than 0.2 m. The robot was able to recognize 37 human activities through sound events with an average accuracy of 88% and detect falling sounds with an accuracy of 80% at the frame level. The experiments show the operation of the various components in the RiSH and the capabilities of the home service robot in monitoring and assisting the resident.

Particle-based perception of garment folding for robotic manipulation purposes

The research focuses on the development of a robust home service robot that is capable of doing multiple types of household chores. This work considers the problem of garment perception and folding...

Off the Shelf Cloud Robotics for the Smart Home: Empowering a Wireless Robot through Cloud Computing.

In this paper, we explore the possibilities offered by the integration of home automation systems and service robots. In particular, we examine how advanced computationally expensive services can be provided by using a cloud computing approach to overcome the limitations of the hardware available at the user’s home. To this end, we integrate two wireless low-cost, off-the-shelf systems in this work, namely, the service robot Rovio and the home automation system Z-wave. Cloud computing is used to enhance the capabilities of these systems so that advanced sensing and interaction services based on image processing and voice recognition can be offered.

FPGA-Based Robust Wireless Speech Motion Control for Home Service Robot Subject to Environmental Noises

In this paper, a robust speech recognition system for the recognition of the speeches subjected to environmental noise is designed and implemented on FPGA to control a home service robot wirelessly. An empirical mode decomposition is used to separate the clean speeches from the speech signals contaminated by environmental noise. To improve the recognition speed, instead of continuous hidden Markov model (CHMM), Discrete HMM (DHMM) is used here to reduce the computation load during speech recognition. However, to compensate the decreased speech recognition rate using DHMM, this paper uses fuzzy vector quantization (FVQ) on the modeling of DHMM to improve the speech recognition rates. It will be shown that the computation time just increases a little, while the speech recognition rates increase much when the FVQ is applied. Finally, combining a wireless module, a FPGA-based speech recognition system is designed to control the motions of a home service robot wirelessly via speech commands under some environmental noises. The performance of the designed system will be demonstrated in the end of this paper.

Human-assisted sound event recognition for home service robots.

This paper proposes and implements an open framework of active auditory learning for a home service robot to serve the elderly living alone at home. The framework was developed to realize the various auditory perception capabilities while enabling a remote human operator to involve in the sound event recognition process for elderly care. The home service robot is able to estimate the sound source position and collaborate with the human operator in sound event recognition while protecting the privacy of the elderly. Our experimental results validated the proposed framework and evaluated auditory perception capabilities and human–robot collaboration in sound event recognition.

Grasping Posture Control Design for a Home Service Robot Using an ABC-Based Adaptive PSO Algorithm

In this paper, a grasping posture control for a robotic arm is developed based on novel adaptive particle swarm optimization (PSO) for the home service robot. To grasp an object using the robotic arm of the home-service robot, both the spatial coordinates of the target and the appropriate collocation of the grasping posture should be examined. In this paper, we present another method for dealing with this problem, which integrates the artificial bee colony (ABC) algorithm into the adaptive particle swarm optimization (APSO) algorithm, where the mutation concept of the scout bee in the ABC algorithm is used to increase the diversity of the particles. In addition, adaptive acceleration coefficients and adaptive inertia weight are presented to ameliorate the convergence rate of the PSO algorithm. We name this control scheme AIWCPSO-S, which represents Adaptive Inertia Weight and acceleration Coefficients PSO with the aid of the Scout bee. Performance comparisons of existing ABC, global ABC, adaptive inertia weight PSO, low-discrepancy sequence initialized PSO algorithm with high-order nonlinear time-varying inertia weight (LHNPSO), oscillating triangular inertia weight PSO (OTIWPSO) and AIWCPSO-S algorithms are conducted by computer simulations. The experiment results show that the presented algorithm gives the most correct and fastest convergence capability.

Human Detection and Face Recognition Using 3D Structure of Head and Face Surfaces Detected by RGB-D Sensor

<div class=""abs_img""><img src=""[disp_template_path]/JRM/abst-image/00270006/11.jpg"" width=""300"" /> Summary of proposed method</div>Home service robots must possess the ability to communicate with humans, for which human detection and recognition methods are particularly important. This paper proposes methods for human detection and face recognition that are based on image processing, and are suitable for home service robots. For the human detection method, we combine the method proposed by Xia et al. based on the use of head shape with the results of region segmentation based on depth information, and use the positional relations of the detected points. We obtained a detection rate of 98.1% when the method was evaluated for various postures and facing directions. We demonstrate the robustness of the proposed method against postural changes such as stretching the arms, resting the chin on one’s hands, and drinking beverages. For the human recognition method, we combine the elastic bunch graph matching method proposed by Wiskott et al. with Face Tracking SDK to extract facial feature points, and use the 3D information in the deformation computation; we obtained a recognition rate of 93.6% during evaluation.

Integrated particle swarm optimization algorithm based obstacle avoidance control design for home service robot

This paper presents a new particle swarm optimization (PSO) algorithm, called the PSO-IAC algorithm, to resolve the goal of reaching with the obstacle avoidance problem for a 6-DOF manipulator of the home service robot. The proposed PSO-IAC algorithm integrates the improved adaptive inertia weight and the constriction factor with the standard PSO. Both the free-space and obstacle avoidance states are established for evaluations in computer simulations and real-time experiments. The performance comparisons of the PSO-IAC algorithm with respect to the existing inertia weighted PSO (PSO-W), constriction factor based PSO (PSO-C), constriction factor and inertia weighted PSO (PSO-CW), and adaptive inertia weighted PSO (PSO-A) algorithms are examined. Simulation results indicate that the PSO-IAC algorithm provides the fastest convergence capability. Finally, the proposed control scheme can make the manipulator of the home service robot arrive at the goal position with and without obstacles in all real-time experiments.

A novel method for estimating the heading angle for a home service robot using a forward-viewing mono-camera and motion sensors

This paper presents a novel method for estimating the heading angle of a home service robot. The images captured from a forward viewing mono-camera, gyroscope data, and wheel encoder data are used as sensory inputs. The data rates of camera, gyroscope, and encoder are 1 Hz, 50 Hz, and 50 Hz, respectively. The camera is slightly tilted by 16.5° to increase the amount of visual information obtained. The context of this study is simultaneous localization and mapping (SLAM) in an indoor home environment. The ceiling lines, i.e., the intersection between the ceiling and wall, are used as the only landmarks for the heading angle estimation. In our method, since the ceiling lines are used as the sole landmarks, line parameterization and data association problem can be simplified when compared to the general line based methods. In home environment, the ceiling line features can be easily detected regardless of the robot rotation. The developed method can be used to accurately estimate the global heading angle while the robot is in motion. A ceiling line map is also reconstructed. The reconstructed ceiling line map provides intuitively understandable spatial information about the environment using only a few features. The experimental results demonstrate that it is feasible to use the proposed method for an indoor home environment.

2P1-E08 ハンドアームと自律走行車の連携による冷蔵庫への横付けと開扉動作の実装

We aim to develop a home service robot which can take foods or drinks to users. The robot should be able to move to a refrigerator and open the door even if the actual refrigerator's pose is slightly different from the environmental map which has been created beforehand. In this paper, we focus on two functions of the robot: approaching the front of the refrigerator correctly and opening the door by pulling the door ' s handle. We propose a method of determining a goal pose of the robot according to an actual environment by recognition of a refrigerator ' s pose. This paper shows an implementation of these functions by cooperation of vehicle motion and hand-arm operation using ROS.

Design of Embedded Control System for Window-Cleaning Robot

Considering increasing importance of home service robot, the embedded control system based on TI msp430 and μC/OS-II is designed for window-cleaning robot. This paper includes three aspects, i.e., functions, hardware and software design. In the development of the system, the robot can be controlled by smart home system. PID controller is used to control DC motor stably. Meanwhile, a complete coverage path planning is proposed for efficiency.

3D Elastic Deformable Object Model for Robot Manipulation Purposes

This paper presents a mass-spring model applied to the manipulation of an elastic deformable object for home service robot application. A system is also proposed that is used to fold a piece of rectangular cloth from a specific initial condition using a robot. The cloth is modeled as a three-dimensional object in a two-dimensional quadrangular mesh based on a massspring system, and its state is estimated using an explicit integration scheme that computes the particle position as a function of the internal and external forces acting on the elastic deformable object. The current state of the elastic deformable object under robot manipulation is tracked based on the trajectory of the mass points in the mass-spring system model in a self-developed simulator, which integrates a massspring model and a five-degree-of-freedom articulated robotic arm. To test the reliability of the model, the simulator is used to predict the best possible paths for using the robotic arm to fold a rectangular cloth into two. In the test, the state of the object is derived from the model and then compared with the results of a practical experiment. Based on the test, the error is found to be generally acceptable. Thus, this model can be used as an estimator for the vision-based tracking of the state of an elastic deformable object for manipulation by home service robots.

Needs and attitudes of Singaporeans towards home service robots: a multi-generational perspective

The paper presents a study aiming to assess how users belonging to different generations perceive service robots in the home context, in order to gain insights of multiple aspects of user needs and attitudes towards robots as to inform the design of robots that are universally acceptable. Surveys with 30 subjects from 10 Singaporean families were conducted. These subjects belonged to three generations, namely young adults, working adults, and elderly people. The survey had two phases; during the first phase, interviews were conducted in users' homes to assess their general attitudes and preferences of service robots. The second phase involved an interaction process between the users and a prototype robot in a simulated smart home environment. Users' feedback was collected according to multiple aspects of robot usability and acceptability. It was found that users of all ages were wishing for service robots to help handle home chores, and they expected the robot to complete the task with good reliability and efficiency. However, users of different age groups showed different patterns of concerns over robot usage. A multi-generational perspective is helpful to understand users' needs and attitudes that reflect age differences. It also provides insights into how members of a family share attitudes towards robots.

Facial Expression Recognition of Home Service Robots

It is of great significance that a home service robot can recognize facial expressions of a human being. This thesis suggests that features of facial expressions be extracted with PCA, and facial expressions be recognized by distance-based Hashing K-nearest neighbor classification. First, Haar-like feature and AdaBoost algorithm is adopted to detect a face and preprocess the face image; then PCA is applied to extract features of the facial expression, those features will be inserted into the hash table; finally, the facial expression can be recognized by K-nearest neighbor classification algorithm. As concluded, recognition efficiency can be greatly improved after reconstructing the feature database into hash tables.

가속도 센서기반의 변형된 외란 관측기를 이용한 로봇 매니퓰레이터의 위치 제어의 보상

This paper presents a modified disturbance observer (MDOB) for controlling two arms of a manipulator designed for a home service robot. The MDOB is slightly different from the original DOB in that it uses an accelerometer to measure acceleration of the robot arm. Then it uses the acceleration to estimate the disturbance to cancel out in the control loop. Relying on the acceleration information of the robot arm, a partial model-based control structure is formed. Experimental studies of position control of 2 DOF robot arm are conducted to evaluate the performance of the proposed position control by an MDOB method.

한국형 가정용 서비스 로봇의 설계 및 작업 공간 분석

본 논문은 가사지원 한국형 서비스 로봇의 설계 및 작업공간의 분석에 대해 기술하였다. 한국형 서비스 로봇의 주안점은 바닥작업에 용이하도록 로봇을 설계하는 것이다. 로봇 팔의 길이도 바닥 작업에 용이 하도록 설계하였으며, 그에 따른 작업 공간을 분석하였다. 로봇 설계에 있어서도 매니퓰레이션 작업 영역의 확장을 위해 어깨와 허리에 높이 조절이 가능한 메커니즘과 슬라이딩 메커니즘 등을 적용하였다. 매니퓰레이터를 설계하고 및 제작하였으며, 제작된 매니퓰레이션 시스템의 작업 공간을 분석하였다. 실제로 로봇의 바닥잡업실험을 수행하였다. This paper presents design and workspace analysis of a Korean home service robot. The robot is designed with specific needs of floor-task applications suitable for Korean life style. Shoulder and waist of the robot are adjusted by sliding mechanism to increase the workspace of the robot arms. Manipulators are designed and built and their workspaces are analyzed. Experimental studies of the robot for cleaning the floor are conducted.

Memory-efficient real-time map building using octree of planes and points

This paper presents an octree-based map building algorithm for mobile home-service robots. The robot is equipped with a time-of-flight camera, which produces point clouds of the environment surfaces. Given the successive input of point clouds, a 3D map is incrementally computed in real time. The map is accurate and memory-efficient because the octree nodes containing points on a plane are merged and represented simply by an index to the plane. The real-time performance is achieved largely due to the parallel processing capability of the many-core Graphics Processing Unit used for plane extraction.

Experiences in Developing an Experimental Robotics Course Program for Undergraduate Education

An interdisciplinary undergraduate-level robotics course offers students the chance to integrate their engineering knowledge learned throughout their college years by building a robotic system. Robotics is thus a core course in system and control-related engineering education. This paper summarizes the experience of developing robotics courses presented in the literature and shares the author's experiences through many years of teaching and developing robotics courses with other educators in the Department of Mechatronics, Chungnam National University (CNU), Daejeon, Korea. First, the CNU robotics course described here has classroom and laboratory sections. In class, students learn the theories behind robotics and practice them by performing simulation studies. In parallel, students perform robotics exercises in the laboratory. Second, the lab exercises are focused on hands-on experiments on robot systems; these include an experimental kit, LEGO robots, humanoid robots, industrial robots, and home service robots. Third, competition-based learning is explored by assigning a class project to develop a boxing robot, which covers both manipulation and mobility. Finally, the course introduces robotics-associated outreach activities. The analysis of several years of student evaluation is presented.