Personal Assistant Robot Research Articles

Customer service chatbot enhancement with attention-based transfer learning

Customer service is an important and expensive aspect of business, often being the largest department in most companies. With growing societal acceptance and increasing cost efficiency due to mass production, service robots are beginning to cross from the industrial domain to the social domain. Currently, customer service robots tend to be digital and emulate social interactions through on-screen text, but state-of-the-art research points towards physical robots soon providing customer service in person. This article explores the feasibility of Transfer Learning different customer service domains to improve chatbot models. In our proposed approach, transfer learning-based chatbot models are initially assigned to learn one domain from an initial random weight distribution. Each model is then tasked with learning another domain by transferring knowledge from the previous domains. To evaluate our approach, a range of 19 companies from domains such as e-Commerce, telecommunications, and technology are selected through social interaction with X (formerly Twitter) customer support accounts. The results show that the majority of models are improved when transferring knowledge from at least one other domain, particularly those more data-scarce than others. General language transfer learning is observed, as well as higher-level transfer of similar domain knowledge. For each of the 19 domains, the Wilcoxon signed-rank test suggests that 16 have statistically significant distributions between transfer and non-transfer learning. Finally, feasibility is explored for the deployment of chatbot models to physical robot platforms including “Pepper”, a semi-humanoid robot manufactured by SoftBank Robotics, and “Temi”, a personal assistant robot.

Designing Socially Assistive Robots Exploring Israeli and German Designers’ Perceptions

Socially assistive robots (SARs) are becoming more prevalent in everyday life, emphasizing the need to make them socially acceptable and aligned with users’ expectations. Robots’ appearance impacts users’ behaviors and attitudes toward them. Therefore, product designers choose visual qualities to give the robot a character and to imply its functionality and personality. In this work, we sought to investigate the effect of cultural differences on Israeli and German designers’ perceptions of SARs’ roles and appearance in four different contexts: a service robot for an assisted living/retirement residence facility, a medical assistant robot for a hospital environment, a COVID-19 officer robot, and a personal assistant robot for domestic use. The key insight is that although Israeli and German designers share similar perceptions of visual qualities for most of the robotics roles, we found differences in the perception of the COVID-19 officer robot’s role and, by that, its most suitable visual design. This work indicates that context and culture play a role in users’ perceptions and expectations; therefore, they should be taken into account when designing new SARs for diverse contexts.

Personal assistant robot using reinforcement learning: DARWIN-OP2 as a case study

Personal assistant robot using reinforcement learning: DARWIN-OP2 as a case study

J.A.R.V.I.S: A Personal Assistant Robot

Abstract: Personal assistant robots are becoming increasingly popular as versatile devices that can assist individuals in various aspects of daily life. These robots are designed to be small and portable, with user-friendly interaction features that allow them to navigate tight spaces and interact closely with users. Equipped with sensors, cameras, and AI algorithms, personal assistant robots can perceive and respond to their environment, recognize faces and voices, and perform tasks such as setting reminders, providing information, and controlling smart home devices. They offer a range of benefits in different settings, including smart homes, healthcare facilities, and public spaces, where they can serve as personal assistants, health monitors, and information guides. The ability of personal assistant robots to learn and adapt to users' preferences and behaviors over time enables them to provide personalized and intuitive assistance, making them suitable for individuals of all ages. As advancements in their design and capabilities continue, personal assistant robots are expected to have an even greater impact on enhancing daily life in the future

Robot Ethics

Robot Ethics

Robot Ethics

Robot Ethics

A Brain-Controlled Quadruped Robot: A Proof-of-Concept Demonstration.

Coupling brain-computer interfaces (BCIs) and robotic systems in the future can enable seamless personal assistant systems in everyday life, with the requests that can be performed in a discrete manner, using one's brain activity only. These types of systems might be of a particular interest for people with locked-in syndrome (LIS) or amyotrophic lateral sclerosis (ALS) because they can benefit from communicating with robotic assistants using brain sensing interfaces. In this proof-of-concept work, we explored how a wireless and wearable BCI device can control a quadruped robot-Boston Dynamics' Spot. The device measures the user's electroencephalography (EEG) and electrooculography (EOG) activity of the user from the electrodes embedded in the glasses' frame. The user responds to a series of questions with YES/NO answers by performing a brain-teaser activity of mental calculus. Each question-answer pair has a pre-configured set of actions for Spot. For instance, Spot was prompted to walk across a room, pick up an object, and retrieve it for the user (i.e., bring a bottle of water) when a sequence resolved to a YES response. Our system achieved at a success rate of 83.4%. To the best of our knowledge, this is the first integration of wireless, non-visual-based BCI systems with Spot in the context of personal assistant use cases. While this BCI quadruped robot system is an early prototype, future iterations may embody friendly and intuitive cues similar to regular service dogs. As such, this project aims to pave a path towards future developments in modern day personal assistant robots powered by wireless and wearable BCI systems in everyday living conditions.

Innovative Personal Assistance: Speech Recognition and NLP-Driven Robot Prototype

This paper presents the development and evaluation of a personal assistant robot prototype with advanced speech recognition and natural language processing (NLP) capabilities. Powered by a Raspberry Pi microprocessor, it is the core component of the robot's hardware. It is designed to receive commands and promptly respond by performing the requested actions, utilizing integrated speech recognition and NLP technologies. The prototype aims to enhance meeting efficiency and productivity through audio-to-text conversion and high-quality image capture. Results show excellent performance, with accuracy rates of 100% in Indonesian and 99% in English. The efficient processing speed, averaging 9.07 seconds per minute in Indonesian and 15.3 seconds per minute in English, further enhances the robot's functionality. Additionally, integrating a high-resolution webcam enables high-quality image capture at 1280 x 720 pixels. Real-time integration with Google Drive ensures secure storage and seamless data management. The findings highlight the prototype's effectiveness in facilitating smooth interactions and effective communication, leveraging NLP for intelligent language understanding. Integrating NLP-based speech recognition, visual documentation, and data transfer provides a comprehensive platform for managing audio, text, and image data. The personal assistant robot prototype presented in this research represents a significant advancement in human-robot interaction, particularly in meeting and collaborative work settings. Further refinements in NLP can enhance efficiency and foster seamless human-robot interaction experiences.

An Emotional Model Based on Fuzzy Logic and Social Psychology for a Personal Assistant Robot

Personal assistants and social robotics have evolved significantly in recent years thanks to the development of artificial intelligence and affective computing. Today’s main challenge is achieving a more natural and human interaction with these systems. Integrating emotional models into social robotics is necessary to accomplish this goal. This paper presents an emotional model whose design has been supervised by psychologists, and its implementation on a social robot. Based on social psychology, this dimensional model has six dimensions with twelve emotions. Fuzzy logic has been selected for defining: (i) how the input stimuli affect the emotions and (ii) how the emotions affect the responses generated by the robot. The most significant contribution of this work is that the proposed methodology, which allows engineers to easily adapt the robot personality designed by a team of psychologists. It also allows expert psychologists to define the rules that relate the inputs and outputs to the emotions, even without technical knowledge. This methodology has been developed and validated on a personal assistant robot. It consists of three input stimuli, (i) the battery level, (ii) the brightness of the room, and (iii) the touch of caresses. In a simplified implementation of the general model, these inputs affect two emotions that generate an externalized emotional response through the robot’s heartbeat, facial expression, and tail movement. The three experiments performed verify the correct functioning of the emotional model developed, demonstrating that stimuli, independently or jointly, generate changes in emotions that, in turn, affect the robot’s responses.

Impact of Online Education and Sentiment Analysis from Twitter Data using Topic Modeling Algorithms

During a pandemic, all industries suffer greatly, and every sector of the world suffers in some way, including the education sector. Internet expressions reflect users' feelings about a product or service. The polarity of information in source data toward a subject under investigation is determined by sentiment analysis processes. The goal of this study is to examine social media expressions about online teaching and learning, as online education will become a part of everyday life in the future. We collected data from Twitter using keywords related to online education and Google form from engineering undergraduate students for prototype implementation. This analysis will assist teachers, parents, and the student community in understanding the benefits and drawbacks of the education industry, allowing for further improvement in educational outcomes. We used aspect-based sentiment analysis and topic modeling to determine sentiment polarity and important topics for education sector stakeholders. To begin, we used TextBlob Python package to determine sentiment polarity, and Bag of Words, LDA and LSA model for discovering topics. After modeling topics from the collected data, topic Coherence is used to assess the degree of semantic similarity between high scoring words in the topic. The word cloud and LDAvis are used to visualize data. The experimental results are promising and it will assist education stakeholders in addressing the concerns that have been identified as social media expressions to work on. Since the boom in science and technology, humans have been trying to invent machines that could reduce their efforts in day to day activities. In this paper, we develop a personal assistant robot that could pick up objects and return it to the user. The robot is controlled using an android application in mobile phones. The robot can listen to user’s command and then respond in the best way possible. The user can command the robot to move to given location, capture images and pick objects. The robot is equipped with ultrasonic sensor and web camera that helps it to move to different location effectively. It is also equipped with sleds that play important role in object picking process. The robot uses a tiny YOLOv3 model which is rigorously trained on several images of the object. There are some possible improvements that can be achieved which could help this robot to be used in several other fields as well.

Review of Robotics Systems Available to aid Visually Impaired Persons

The need for assistance robots has risen dramatically in this time of age for persons with disabilities particularity for visually impaired persons as their number continues to rise. For them independent living can be performed by an aid robot which is one of the key assistive technology devices that can help to regain dignity and self-confidence. This paper reviews the existing design and development of a personal assistant robots that uses a certain scientific algorithm to detect and estimate the relative location of an objects in an indoor environment using voice instructions. These semi-humanoid robots have built in multiple HD cameras located on different parts of the robots. Autonomous movement, object detection, distance measuring, and motion planning are all done with the cameras. Furthermore, the robot’s utility is increased by keeping the user informed about the results of its actions.

Prospective Customer Preference Toward Service Robots: A Discrete Choice Model Approach

Abstract. One major technological development pertains to the service robot industry. In Indonesia, the development of robots is increasing. Telkom will begin to explore new opportunities with companion robots, which can respond to and chat with humans and move around. Therefore, more analysis is needed based on market research and prospective user surveys. The present study aims to obtain information on the features of service robots significantly influencing customers’ desire to purchase them and on the types of service robots they are likely to choose. The discrete choice model method is used to understand and predict a decision from a discrete alternative, based on a set of choices constructed with the help of the Ngene software and the D-efficient design method. We processed questionnaire results using the Python-Biogeme software to look for significant attributes and their elasticity. The significant attributes of companion robots defined by T-test are basic robot look (3.86), battery life (5.45), robot price (9.16), walking-computer look (6.10), and robot weight (8.38). The suggested strategy, among the three types of companion robots, is to focus on the personal assistant robot with the highest T-test score while paying attention to the resources a company already has. Keywords: Service Robot, Discrete Choice Model, Significant Attribute, Elasticity

Personal Assistant Robot

Since the boom in science and technology, humans have been trying to invent machines that could reduce their efforts in day to day activities. In this paper, we develop a personal assistant robot that could pick up objects and return it to the user. The robot is controlled using an android application in mobile phones. The robot can listen to user’s command and then respond in the best way possible. The user can command the robot to move to given location, capture images and pick objects. The robot is equipped with ultrasonic sensor and web camera that helps it to move to different location effectively. It is also equipped with sleds that play important role in object picking process. The robot uses a tiny YOLOv3 model which is rigorously trained on several images of the object. There are some possible improvements that can be achieved which could help this robot to be used in several other fields as well.

Real-Time Motion Control of a Humanoid Robot Using Deep Learning

This paper discusses the research work done for controlling the humanoid robot manually using deep learning. For teaching, personal assistance, search and rescue humanoid robot are used. Controlling manually makes it to do any task without any explicitly programming. Existing technique for manually controlling the humanoid are heavily dependent on hardware and they are not cost efficient. This paper proposes a novel method for controlling the humanoid using a 2D camera. The image from the 2D camera is processed and skeleton of the human body is captured using deep learning. Then the skeleton is used to control the actuators present in the humanoid robot using image classifier and ROS. As a proof of concept the upper body of the humanoid robot is controlled in real time using this method.

Comparison between Recurrent Networks and Temporal Convolutional Networks Approaches for Skeleton-Based Action Recognition.

Action recognition plays an important role in various applications such as video monitoring, automatic video indexing, crowd analysis, human-machine interaction, smart homes and personal assistive robotics. In this paper, we propose improvements to some methods for human action recognition from videos that work with data represented in the form of skeleton poses. These methods are based on the most widely used techniques for this problem—Graph Convolutional Networks (GCNs), Temporal Convolutional Networks (TCNs) and Recurrent Neural Networks (RNNs). Initially, the paper explores and compares different ways to extract the most relevant spatial and temporal characteristics for a sequence of frames describing an action. Based on this comparative analysis, we show how a TCN type unit can be extended to work even on the characteristics extracted from the spatial domain. To validate our approach, we test it against a benchmark often used for human action recognition problems and we show that our solution obtains comparable results to the state-of-the-art, but with a significant increase in the inference speed.

Human interaction behavior modeling using Generative Adversarial Networks

Recently, considerable research has focused on personal assistant robots, and robots capable of rich human-like communication are expected. Among humans, non-verbal elements contribute to effective and dynamic communication. However, people use a wide range of diverse gestures, and a robot capable of expressing various human gestures has not been realized. In this study, we address human behavior modeling during interaction using a deep generative model. In the proposed method, to consider interaction motion, three factors, i.e., interaction intensity, time evolution, and time resolution, are embedded in the network structure. Subjective evaluation results suggest that the proposed method can generate high-quality human motions.

Personal Assistant Robot for Paraplegic People with Hand Gesture

Personal Assistant Robot for Paraplegic People with Hand Gesture

Simulation of Lidar-Based Robot Detection Task using ROS and Gazebo

In the last few decades, the robotics world has seen great progress at all levels, from personal assistant robots to multi-robotic and intelligent swarm systems. Simulation platforms play a critical role in this improvement due to efficiency, flexibility, and fault tolerance they provide during the cycles of developing and testing new strategies and algorithms. In this paper, we model a new mobile robot equipped with a 2D Lidar using Robot Operating System (ROS) and use this robot model to develop a robot detection method in Gazebo simulation environment. Detecting surrounding objects and distinguishing robots from these objects (kin detection) are essential in multi-robot and swarm robotic applications. In this paper, we use Lidar to handle this task by applying the following steps: (1) acquisition of laser data and pre-processing, (2) segmentation of data using the point-distance-based segmentation method, (3) classification of segments by applying two levels of filtering: filtering by segment diameter which aims to eliminate segments that don’t fit a certain size (Lidar size) using features for each segment, filtering by segment shape to check remaining segments to test if they fit the Lidar's shape (which is a circle with known radius) or not by using the circle fitting method and (4) identify the position of kin relative to the observer robot. Two different scenarios are discussed in the experiments section. In the first scenario, many cylindrical objects with radius different from the robot’s Lidar were used in addition to a robot, and thus objects are distinguished from the robot in the first level of filtering without using the second one which may be a complex operation. In the second scenario, various objects with a similar radius were used, and due to the similarity in the radius between the Lidar and the objects, it was necessary to apply all the method’s steps to detect the kin robots. It was noticed from experiments that the accuracy of the results depends on two main factors: the distance between the observer robot and other objects or robots and the amount of noise in the Lidar measurements.

Computer Vision in Autonomous Unmanned Aerial Vehicles—A Systematic Mapping Study

Personal assistant robots provide novel technological solutions in order to monitor people’s activities, helping them in their daily lives. In this sense, unmanned aerial vehicles (UAVs) can also bring forward a present and future model of assistant robots. To develop aerial assistants, it is necessary to address the issue of autonomous navigation based on visual cues. Indeed, navigating autonomously is still a challenge in which computer vision technologies tend to play an outstanding role. Thus, the design of vision systems and algorithms for autonomous UAV navigation and flight control has become a prominent research field in the last few years. In this paper, a systematic mapping study is carried out in order to obtain a general view of this subject. The study provides an extensive analysis of papers that address computer vision as regards the following autonomous UAV vision-based tasks: (1) navigation, (2) control, (3) tracking or guidance, and (4) sense-and-avoid. The works considered in the mapping study—a total of 144 papers from an initial set of 2081—have been classified under the four categories above. Moreover, type of UAV, features of the vision systems employed and validation procedures are also analyzed. The results obtained make it possible to draw conclusions about the research focuses, which UAV platforms are mostly used in each category, which vision systems are most frequently employed, and which types of tests are usually performed to validate the proposed solutions. The results of this systematic mapping study demonstrate the scientific community’s growing interest in the development of vision-based solutions for autonomous UAVs. Moreover, they will make it possible to study the feasibility and characteristics of future UAVs taking the role of personal assistants.

Understanding of Human Behavior with a Robotic Agent Through Daily Activity Analysis

Personal assistive robots to be realized in the near future should have the ability to seamlessly coexist with humans in unconstrained environments, with the robot’s capability to understand and interpret the human behavior during human–robot cohabitation significantly contributing towards this end. Still, the understanding of human behavior through a robot is a challenging task as it necessitates a comprehensive representation of the high-level structure of the human’s behavior from the robot’s low-level sensory input. The paper at hand tackles this problem by demonstrating a robotic agent capable of apprehending human daily activities through a method, the Interaction Unit analysis, that enables activities’ decomposition into a sequence of units, each one associated with a behavioral factor. The modelling of human behavior is addressed with a Dynamic Bayesian Network that operates on top of the Interaction Unit, offering quantification of the behavioral factors and the formulation of the human’s behavioral model. In addition, light-weight human action and object manipulation monitoring strategies have been developed, based on RGB-D and laser sensors, tailored for onboard robot operation. As a proof of concept, we used our robot to evaluate the ability of the method to differentiate among the examined human activities, as well as to assess the capability of behavior modeling of people with Mild Cognitive Impairment. Moreover, we deployed our robot in 12 real house environments with real users, showcasing the behavior understanding ability of our method in unconstrained realistic environments. The evaluation process revealed promising performance and demonstrated that human behavior can be automatically modeled through Interaction Unit analysis, directly from robotic agents.