Cloud Robotics Research Articles

Formula omitted]-Hub: Large-scale video learning, storage, and retrieval on heterogeneous hardware platforms

The burgeoning of Internet of Things (IoT) and camera-equipped mobile devices contributes a tremendous amount of video data generated at the edge of the network. At the same time, we have witnessed the fast deployment of many video-based application services, such as plate recognition for public safety, intelligent transportation, Industry 4.0 and so on. The success of these services, in turn, requires large-scale video data being learned, stored, and retrieved in a more efficient way. A generic software and hardware framework for large-scale IoT video analysis and service support is still missing. To address this challenge, we present π-Hub, PerceptIn’s robotic cloud solution which supports large-scale video data analysis, storage, and query by implementing the learn-store-retrieve paradigm. Interestingly, we found that among the learning, storage, and retrieval services each of them stresses one type of resources on heterogeneous computing servers, i.e., GPU, CPU, and Memory, respectively, therefore it is extremely cost-efficient to co-locate these services together to fully utilize the resources. In addition, several optimization techniques for data writing, reading, and data reduction are proposed and evaluated. The evaluation results show that these techniques improve the performance of the learning, storage and retrieval services significantly as well as notably reduce the cost of the system. We also verify π-Hub’s scalability by reliably running a 1000-machine deployment to support up to one million users. Finally, we conclude the paper by discussing several lessons learned from this study and future work.

An Separation of Different Facial Expression Recognition System

Face recognition is one of the hot topics in the current world and one of the popular topics of computer studies. Today face recognition in the network society and access to digital data is gaining more attention. The facial recognition system technology is a biometric assessment of a human's face. There are many facial recognition techniques that are intended depending on facial expressions extraction, one of which is 3D facial recognition, as well as their fusion,is difficult. During preprocessing measures for picture recognition to remove only expression-specific characteristics from the face and prevent their issues with a convolution neural network. We can also use some theorems such as LBP and Taylor's theorem to model face recognition. In particular, for cloud robots, we can also use this facial recognition on robots. The robot can perform functions and share data between servers and devices. Seven fundamental expressions are used to identify and classify: happiness, shock, fear, disgust, sadness, rage, and a neutral condition. Until now, the recognition rate is quite up to the expectation stage, but it still tries to enhance. To enhance the recognition frequency of facial image recognition, feelings are chosen by the vibrant Bayesian network technique to depict the development of facial awareness in addition to various emotional operations of facial expressions. The ICCA techniques involve various multivariate sets of distinct facial features that could be eyes, nose, and mouth.

HDec-POSMDPs MRS Exploration and Fire Searching Based on IoT Cloud Robotics

The multi-robot systems (MRS) exploration and fire searching problem is an important application of mobile robots which require massive computation capability that exceeds the ability of traditional MRS’s. This paper propose a cloud-based hybrid decentralized partially observable semi-Markov decision process (HDec-POSMDPs) model. The proposed model is implemented for MRS exploration and fire searching application based on the Internet of things (IoT) cloud robotics framework. In this implementation the heavy and expensive computational tasks are offloaded to the cloud servers. The proposed model achieves a significant improvement in the computation burden of the whole task relative to a traditional MRS. The proposed model is applied to explore and search for fire objects in an unknown environment; using different sets of robots sizes. The preliminary evaluation of this implementation demonstrates that as the parallelism of computational instances increase the delay of new actuation commands which will be decreased, the mean time of task completion is decreased, the number of turns in the path from the start pose cells to the target cells is minimized and the energy consumption for each robot is reduced.

Cloud robotics law and regulation: Challenges in the governance of complex and dynamic cyber–physical ecosystems

This paper assesses some of the key legal and regulatory questions arising from the integration of physical robotic systems with cloud-based services, also called “cloud robotics.” The literature on legal and ethical issues in robotics has a strong focus on the robot itself, but largely ignores the background information processing. Conversely, the literature on cloud computing rarely addresses human–machine interactions, which raise distinctive ethical and legal concerns. In this paper, we investigate, from a European legal and regulatory perspective, the growing interdependence and interactions of tangible and virtual elements in cloud robotics environments. We highlight specific problems and challenges in regulating such complex and dynamic ecosystems and explore potential solutions. To illustrate practical challenges, we consider several examples of cloud robotics ecosystems involving multiple parties, various physical devices, and various cloud services. These examples illuminate the complexity of interactions between relevant parties. By identifying pressing legal and regulatory issues in relation to cloud robotics, we hope to inform the policy debate and set the scene for further research.

Energy-efficient optimal task offloading in cloud networked multi-robot systems

Task offloading plays a critical role in cloud networked multi-robot systems for leveraging computation support from cloud infrastructure and benefiting greatly from the well-developed cloud network facilities. However, considering the delay constraint, the extra costs of data transmission and remote computation, it is not trivial to make optimized offloading decisions. In particular, task offloading for robots is more complex due to their on-demand mobility and network connectivity that significantly influence the robot–cloud communication links. Moreover, for multi-robot systems, a suitable balance of workload between local network (robot–robot) and global cloud (robot–cloud) is also required, so as to attain proper utilization of resources. Therefore, it is essential to establish more comprehensive offloading schemes for modeling systems that can handle these higher level of complications. With that view, this paper aims to develop a novel multi-layer decision-making scheme for task offloading which jointly considers the following four aspects: (i) selection of task for offloading, (ii) selection of robot to offload a task, (iii) selection of location to offload/perform task, (iv) selection of access point for offloaded task. An integrated framework for cloud networked multi-robot systems is presented to enable our task offloading scheme where the primary robot can aid from additional local robots to improve the offloading process. In particular, we consider a warehouse scenario with 36 cell workspace where a 40 node taskflow is motivated from a “parcel sorting and distribution” application, to be completed by the primary robot. The offloading decision for each task is formulated as part of a joint optimization problem and it is solved by developing a multi-layer genetic algorithm scheme that takes into account motion, network connectivity and local sharing for its offloading decisions. We evaluate the results of the scheme via comparison with two validated benchmarks. The outcome highlights a significant improvement in overall system performance due to joint involvement of motion (path planning), connectivity (bandwidth estimation) and robot–robot communication (local offloading) that facilitates energy-efficient offloading to cloud, faster completion of tasks and better utilization of available resources.

Predictor-Based Motion Tracking Control for Cloud Robotic Systems with Delayed Measurements

This paper addresses the problem of motion prediction and tracking control for cloud robotic systems with time-varying delays in measurements. A novel method using an observer-based structure for position and velocity prediction is developed to estimate the real-time information of robot manipulator. The prediction error can converge to zero even if model uncertainties exist in the robot manipulator. Based on the predicted positions and velocities, some sufficient conditions are derived to design suitable tracking controllers such that semi-globally uniformly ultimately bounded tracking performance of the predictor–controller couple can be guaranteed. Finally, the effectiveness and robustness to model uncertainties of the proposed method are verified by a two degree-of-freedom (DOF) robot system.

The Pransky interview: Dr Ken Goldberg, Professor, Industrial Engineering and Operations Research, UC Berkeley; Inventor and Artist

Purpose The following article is a “Q&A interview” conducted by Joanne Pransky of Industrial Robot Journal as a method to impart the combined technological, business, and personal experience of a prominent, robotic industry PhD and inventor regarding his pioneering efforts and the commercialization of bringing a technological invention to market. The paper aims to discuss these issues. Design/methodology/approach The interviewee is Dr Ken Goldberg, an inventor working at the intersection of art, robotics, and social media. He joined the UC Berkeley faculty in 1995 where he is the UC Berkeley William S. Floyd Jr Distinguished Chair in Engineering and recently served as Chair of the Industrial Engineering and Operations Research Department. He has secondary appointments in UC Berkeley’s Electrical Engineering/Computer Science, Art Practice and the School of Information. Goldberg also holds an appointment at the UC San Francisco Medical School’s Department of Radiation Oncology where he pursues research in medical robotics. Goldberg is Director of the CITRIS “People and Robots” Initiative and the UC Berkeley’s Laboratory for Automation Science and Engineering (AUTOLAB) where he and his students research machine learning for robotics and automation in warehouses, homes, and operating rooms. In this interview, Goldberg shares some of his personal and business perspectives from his career-long pursuit of making robots less clumsy. Findings Goldberg earned dual BS degrees in Electrical Engineering and Economics from the University of Pennsylvania in 1984, and MS and PhD degrees in Computer Science from Carnegie Mellon University in 1990. Goldberg also studied at Edinburgh University and the Technion. From 1991-95 he taught at the University of Southern California, and in fall 2000, he was visiting faculty at the MIT Media Lab. Goldberg and his students pursue research in three primary areas: Geometric Algorithms for Automation, Cloud Robotics, and Robot Learning. Originality/value Goldberg developed the first complete algorithms for part feeding and part fixturing, and developed the first robot on the Internet. His inventions have been awarded nine US Patents. Goldberg has published over 250 peer-reviewed technical papers and edited four books. He co-founded and served as Editor-in-Chief of the IEEE Transactions on Automation Science and Engineering (T-ASE). He is also Co-Founder of the Berkeley AI Research (BAIR) Lab, the Berkeley Center for New Media (BCNM), the African Robotics Network (AFRON), the Center for Automation and Learning for Medical Robotics (CAL-MR), the CITRIS Data and Democracy Initiative (DDI), Hybrid Wisdom Labs, and Moxie Institute. He has presented over four hundred keynote and invited lectures. Goldberg's artwork, closely linked with his research, has appeared in over seventy venues. Ken was awarded the Presidential Faculty Fellowship in 1995 by Bill Clinton, the Joseph Engelberger Robotics Award in 2000, elected IEEE Fellow in 2005, and selected by the IEEE Robotics and Automation Society for the George Saridis Leadership Award in 2016.

Dynamic Path Selection for Cloud-based Multi-Hop Multi-Robot Wireless Networks

ABSTRACTIn a cloud robotics system, communication between robots and between robots and the cloud are crucial from the application prospective. Due to the nature of the network technology (predominantly wireless) and nature of the network devices, direct communication between robots and between robots and the cloud is not feasible. Multi-hop communication is the only feasible solution in such scenarios. In Multi-hop communication, finding the optimal route between the end points is a complicated task. This becomes further complicated when the communicating nodes are mobile and do not have enough communication and computational power as in the case of robots. Though robots can compute the end-to-end path, it is a complicated task for them and is sometimes infeasible in practice due to their low computation and communication power. To overcome this problem, we propose a multi-hop communication model called Dynamic Path Selection using Cloud-based Multi-hop Multi-Robot (DPS-CMM) model, where instead of the robots, the cloud decides the end-to-end communication path which the robots need to use to communicate among themselves. In this model, the cloud provides the optimal path information of the robots to the Access Points (APs) such that source robot establishes a reliable communication path through the APs to reach the destination robot. In this paper, the DPS-CMM model provides the information of network topology and computes the path selection at the cloud instead of computing the same in robots level thereby increasing the life time of robotic networks. To validate this model, we use combination of NS-3 and Matlab-based simulations and compare our scheme with the existing state-of-the-art solutions.

Polymer Optical Fiber-Based Sensor System for Smart Walker Instrumentation and Health Assessment

This paper presents a polymer optical fiber (POF)-based sensor system for smart walker (SW) instrumentation. The system comprises an oximetry sensor working in reflectance mode, which is embedded in a 3D printed handle for the SW. In addition, there is also a POF-based smart textile that can be integrated in the users’ clothes and is placed on their chest. In this case, the smart textile can detect the displacement caused by the respiration, vibration due to heartbeat and bending induced in the gait. Results show that the proposed sensor system can detect the oxygen saturation, breathing rate, gait cadence, and heart rate with errors of about 1%. In addition, it is proposed an architecture for the SW in which the POF-based sensor system can be applied not only for remote healthcare applications, but also as sensor feedback in cloud robotics for some control paradigms for human–robot interaction. Furthermore, the proposed instrumentation presents the possibility of novel SW control, where the health conditions of the user are considered in the human–robot interaction.

Probability machine-learning-based communication and operation optimization for cloud-based UAVs

This paper proposes a smart machine-learning-based unmanned aerial vehicle (UAV) control system that optimizes UAV operations in real time. The purpose of this system is to increase the efficiency of UAVs that need to operate with limited resources. This can be accomplished by allowing the UAVs in flight to identify their current state and respond appropriately. The proposed system, which is developed based on “cloud robotics,” benefits from the powerful computational capabilities of cloud computing and can therefore calculate many types of information received from various sensors in real time to maximize the performance of the UAV control system. The system learn about normal situations when creating models. That is, preprocessing data that is correlated with a particular situation and modeling it with a “multivariate Gaussian distribution.” Once the model is created, the UAV can be used to analyze the current situation in real time during flight. Of course, it is possible to recognize the situation based on the traditional RULE or the latest LSTM. However, this is not an appropriate solution for UAV situations where irregularities are severe and unpredictable. In this paper, we succeeded in recognizing the UAV flight status in real time by the proposed method and succeeded in optimizing it by adjusting communication cycle based on a recognized situation. Based on the results of this study, we expect to be able to stabilize and optimize systems that are highly irregular and unpredictable. In other words, this system will be extended to learn about various situations and create a model. A reliable and efficient smart system can be designed by judging the situation comprehensively.

Intelligent task scheduling strategy for cloud robot based on parallel reinforcement learning

Cloud computing is an emerging computing model that has been developed on the basis of grid computing. Its powerful computing capabilities have evoked great vigour in its integration with various industries. The integration of cloud computing and robotics has created the concept of cloud robots. This paper proposes an intelligent task scheduling strategy for cloud robots based on parallel reinforcement learning. Firstly, the cloud computing platform is used to divide the complex reinforcement learning problem into several sub-problems. Then, the task scheduling strategy is used to assign sub-tasks to the robot to learn and summarise the results. Finally, Cloudsim builds a cloud computing platform for simulation experiments to verify the effectiveness of the proposed method. The experimental results show that the proposed method reduces the time for the robot to learn the whole problem and improves the learning efficiency.

Robots and the return to collaborative intelligence

Ken Goldberg reflects on how four exciting sub-fields of robotics — co-robotics, human–robot interaction, deep learning and cloud robotics — accelerate a renewed trend toward robots working safely and constructively with humans.

Reliability of ROS Networked Mobile Robots

When working remotely with mobile robotics, a reliable wireless communication network becomes essential, especially in large operating regions. As most teleoperated robots rely on standard Wi-Fi communication, network behavior has a crucial effect on autonomous robot control. The main goal of this research is to measure and diagnose the system reliability, roaming issues, and bottlenecks of such data transmission. To study these significant factors, two measurement scenarios were conducted. Measurements consist of two Wi-Fi access points (AP) and a TurtleBot II robot used in two different system set-up configurations. In the first configuration, two APs are connected in bridge mode (LAN connection) and in the second configuration the APs are connected in WDS bridge (Wireless Data Distribution) mode. This article presents the results of Robot Operating System (ROS) IEEE 802.11 network measurements in roaming mode, in wireless bridge mode, and in an extended coverage area employed in WDS mode. Results of data transmission measurements, configurations, and evaluation of the entire system are also presented. All the measurements utilize the Open Cloud Robotic Platform (OpenCRP)1 an open-source cloud robotics ecosystem based on service-oriented PaaS architecture using the Ubuntu Linux operating system.

A Novel Cloud Platform for Service Robots

With the development of computer technology and artificial intelligence (AI), service robots are widely used in our daily life. While the manufacturing cost of the robots is too expensive for most small technology companies. The biggest technical limitations are the design of the robot service and the resources sharing of the robot groups. As far as we know, there is no complete and open-source service robot cloud service platform. To solve the above problems, in this paper, a novel robot cloud platform called cloud robotics intelligent cloud platform (CRICP) is designed, which consists of gateway layer, an interface layer, service pool, and algorithm layer. Gateway layer mainly solves the problem of robot access control and service invocation requests scheduling. In addition, a standardized access method is proposed to overcome the problem that heterogeneous service robots cannot access the cloud platform. Interface layer is responsible for protocol injection, including motoring protocol, a management protocol, and other algorithms invocation protocol or service invocation interfaces protocol. Service Pool consists of different kinds of robot services which could scale based on the historical data analysis. In Algorithm layer, we can implement machine learning (ML) algorithm, deep learning (DL) algorithm, distributed algorithm, and so on. Finally, voice recognition service invocation experiment and cloud service dynamic scaling test are taken as an example to verify the availability and accuracy of our platform. Moreover, the compared results with a local framework and SOA also verifies the superiority of our platform.

Intelligent task scheduling strategy for cloud robot based on parallel reinforcement learning

Intelligent task scheduling strategy for cloud robot based on parallel reinforcement learning

Evaluation of Peer Robot Communications using CryptoROS

The demand of cloud robotics makes data encryp-tion essential for peer robot communications. Certain types of data such as odometry, action controller and perception data need to be secured to prevent attacks. However, the introduction of data encryption caused increment of overhead for data stream communication. This paper presents an evaluation of CryptoROS architecture on Robot Operating System (ROS) which focused on peer-to-peer conversations between nodes with confidentiality and integrity violation. OpenSSL is used to create a private key and generate a Certificate Signing Request (CSR) that contains public key and a signature. The CSR is submitted to a Certificate Authority (CA) to chain the root CA certificate and encryption of RSA private key with AES-256 and a passphrase. The protected private key are securely backed up, transported, and stored. Experiments were carried out multiple times with and without the proposed protocol intervention to assess the performance impact of the Manager. The results for different number of messages transmitted each time increased from 100, 250 to 500 with performance impact 1.7%, 0.5% and 0.2%, respectively. It is concluded that CryptoROS capable of protecting messages and service requests from unauthorized intentional alteration with authenticity verification in all components.

Collaborative Optimization of Service Scheduling for Industrial Cloud Robotics Based on Knowledge Sharing

Industrial Cloud Robotics (ICR), which has the characteristics of resource sharing, convenient access and high efficiency, is the combination of Cloud Computing and Industrial Robots. In the current manufacturing workshop, most industrial robots that are not connected to each other use onboard processors and memories with limited resources, which leading to the constraints of multi-robot information sharing. However, knowledge sharing for multi-robot collaborative optimization is very important. In the service scheduling optimization of industrial robots oriented to workshop manufacturing tasks, the lack of knowledge sharing seriously restricts the further performance improvement of the collaborative optimization. Aiming at this problem, the collaborative optimization framework of service scheduling for industrial cloud robotics is built, and then a cloud-based knowledge sharing mechanism for industrial robots and a collaborative optimization method of service scheduling based on Deep Reinforcement Learning (DRL) are proposed, so as to realize a comprehensive performance improvement of the whole manufacturing system. Finally, a case study is implemented to verify the effectiveness of the proposed method.

Developing Context-Aware Dialoguing Services for a Cloud-Based Robotic System

In the development of service robots, building dialoguing services for robots to provide natural human-robot interactions and enhancing user experiences is now advocated. In this type of service, a robot can play the role of a knowledgeable consultant and deliver domain-specific knowledge to end users. Following our previous studies in constructing action-oriented robot services, in this paper, we adopt a service-oriented framework to develop a context-aware dialoguing service for a cloud robot. Our work has several unique features: it trains a deep neural model to generate answers in response to the users' questions, utilizes an external knowledge resource to further enrich knowledge for searching for answers, constructs a reasoning procedure to exploit the answers of similar questions, and most importantly, and develops an integrated approach that embeds both contextual information and dialoguing content in the same model. To evaluate the proposed approach, we conducted a series of experiments and implemented several strategies for performance comparison. The results confirmed the usefulness and effectiveness of the integrated context and content approach.

Comments: SPE’s New Publications

Editor's column Digitalization and unconventionals production are transforming the oil and gas industry. Shale oil output now accounts for the lion’s share of world production increases and has reshaped global energy markets and trading. Partly in response to the severe price downturn that began in 2014, companies are adopting the use of advanced data analytics, machine learning, visualization, and artificial intelligence to reduce costs and to make operations more productive and efficient. SPE’s mission is to give its members the most current information related to key industry topics and develop programs and publications that are of most relevance to petroleum professionals, offering valuable insight into current and future ways of doing business. A key part of the new SPE Strategic Plan, adopted earlier this year, is to promote expertise in new areas, as well as better understanding of new digital technologies, through increased content offerings. To support this effort, SPE is launching a new online publication and newsletter on data science and digital engineering and a new monthly newsletter on the unconventionals oil and gas sector. The new publication and newsletters are designed to give members in-depth information and insight into these increasingly important sectors, in addition to the coverage these topics receive in SPE’s current publications. The JPT Unconventional Insights newsletter debuted in November, containing staff-written and curated content (items of interest from non-SPE sources), as well as other information such as key insights from recent technical papers that caught the eye of subject matter experts. The Data Science and Digital Engineering publication and accompanying newsletter will debut in the first quarter of 2019. To receive the newsletters, members and nonmembers must sign up on the SPE preferences page located at www.spe.org/en/preferences/email_preferences. Both the unconventionals and digital science topics are likely to dominate industry discussion for the foreseeable future. October marked the 10th anniversary of the first completed horizontal well in the Eagle Ford Shale formation in south Texas. That discovery helped propel a boom over the next several years that turned the region into one of the world’s most active oil plays, and was further realization that shale development had even greater potential than originally thought. Now, with Permian Basin output growing so fast that it has outstripped transportation capacity, and huge new plays coming on line in Argentina and China, the shale sector will continue to play a major role in upstream oil and gas for years to come. The International Energy Agency reported last month that US crude oil production is expanding at an unprecedented rate thanks to shale. The US is expected to account for 75% of the world’s oil production gains to 2025, with an increase of 5 million B/D during the period to 18.5 million B/D, according to the organization’s World Energy Outlook 2018. Tight oil production alone will rise to 9.2 million B/D. Integrating digitalization and data science into upstream operations has emerged as a strategic priority and will have an inevitable impact on the industry. As consultancy Wood Mackenzie noted in a recent report: “Digitalization is happening. In our view, it is not a choice to do it or not; it is now about the speed and depth of adoption. …Cloud computing, automation and robotics, blockchain, 3D printing, machine learning, and the Internet of Things are all playing increasingly large roles, serving as a platform for data, analytics, and the connectivity of people, machines, and devices.”

Microservice-based cloud robotics system for intelligent space

Cloud robotics (CR) is a red-hot branch of the burgeoning field of service robots that is centered on the benefits of integrating infrastructure and shared services via a cloud computing environment. Although it extends the computation power and information sharing capabilities of the network robots, the development and operations (DevOps) of the CR system are currently limited for enterprise-scale projects due to the heavy framework. In fact, current developed CR systems are typical distributed monomer architectures followed by a “top-down” design. As the scale of the applications gets larger, the operation and maintenance of CR systems will become a very difficult task. In this paper, a new architecture for a microservice-based cloud robotics system in intelligent space is proposed to solve the present dilemma. To enable this, we design a service management architecture based on a microservice to provide a highly efficient and flexible development/deployment mechanism. The container technology based on the docker engine is then used to functionally decompose the application into a set of collaborating services to ensure the software design methods, based on microservice, easy for implementation. Finally, a real experiment on SLAM (Simulation localization and mapping) in an intelligent space is implemented to verify the proposed architecture. Compared with traditional monomer architectures, the results show that the proposed framework is more productive, flexible and cost effective.