Humanoid Platform Research Articles

Hybrid Sensor Fusion Mixed with Reinforcement Learning in Autonomous Dual-Arm Lifting Tasks Performed by Humanoid Robots

Humanoid robots often struggle with tasks such as lifting objects due to the complexity involved in identifying contact points, applying the correct force, and tracking task progress. We propose an integrated solution that leverages the dual-arm capability of humanoids and utilizes sensor fusion from vision and force sensors. Our system employs a computer vision algorithm to detect and characterize object properties (shape, size, position, orientation) and differentiate between parallel and non-parallel bi-manipulation tasks. The controller then identifies optimal contact points for the end effectors, generating trajectories fed into a closed-loop controller using force feedback. For parallel bi-manipulation, momentum cancellation is achieved through sensor fusion. For non-parallel surfaces, a reinforcement learning algorithm determines the appropriate lifting force to prevent slippage using only two contact points. Experimental validation on a real humanoid platform demonstrates the effectiveness of our approach in autonomously lifting objects, regardless of contact surface configuration. This advancement significantly enhances the reliability and versatility of humanoid robots in performing complex manipulation tasks, contributing to their practical deployment in human-oriented environments.

Miha: An Affordable Humanoid Platform for Research and Development

Miha: An Affordable Humanoid Platform for Research and Development

Control and evaluation of a humanoid robot with rolling contact joints on its lower body.

In this paper, we introduce a new teen-sized humanoid platform dubbed DRACO 3, custom-built by Apptronik and altered for practical use by the Human Centered Robotics Laboratory at The University of Texas at Austin. The form factor of DRACO 3 is such that it can operate safely in human environments while reaching objects at human heights. To approximate the range of motion of humans, this robot features proximal actuation and mechanical artifacts to provide a high range of hip, knee, and ankle motions. In particular, rolling contact mechanisms on the lower body are incorporated using a proximal actuation principle to provide an extensive vertical pose workspace. To enable DRACO 3 to perform dexterous tasks while dealing with these complex transmissions, we introduce a novel whole-body controller (WBC) incorporating internal constraints to model the rolling motion behavior. In addition, details of our WBC for DRACO 3 are presented with an emphasis on practical points for hardware implementation. We perform a design analysis of DRACO 3, as well as empirical evaluations under the lens of the Centroidal Inertia Isotropy (CII) design metric. Lastly, we experimentally validate our design and controller by testing center of mass (CoM) balancing, one-leg balancing, and stepping-in-place behaviors.

Development of the Kinematic Adjustable Humanoid Platform and its Driving Capability

Development of the Kinematic Adjustable Humanoid Platform and its Driving Capability

Constraint-consistent task-oriented whole-body robot formulation: Task, posture, constraints, multiple contacts, and balance

We present a comprehensive formulation to the problem of controlling a high-dimensional robotic system involving complex tasks subject to a variety of constraints, obstacles, balance, and contact challenges. Using intuitive and natural representations, the approach is initiated by establishing individual objectives for a task and its constraints. Simple independent controllers using artificial potential fields are then designed for each objective to reach goals while enforcing the constraints. Dynamically consistent projections in nullspaces associated with task and constraint representations are employed to deliver a coherent whole-body robot control. In multi-link multi-contact tasks, contact forces produce both resulting and internal forces. Internal forces play a critical role in robot balance and stability, achieved in this framework through modeling and controlling virtual linkages that explicitly describe the relationship between active/passive contact force, resultant force, controlled/uncontrolled internal force for multi-link multi-contact underactuated robots. Control of contacts with the environment involves material considerations such as friction and geometric constraints. Potential barriers direct the selection of contact forces ensuring stability and balance. This approach of dynamic projection and the Virtual Linkage Model addresses robot underactuation. In addition, the framework introduces a coordinate completion mechanism to establish a generalized coordinates representation of the task, removing redundancy and maintaining the full operational space dynamics description. This enables task-space dynamic control based on the relevant inertial properties. We present the experimental validation on a physical humanoid platform.

Whole-Body Compliant Control of Robot Arms Based on Distributed Flexible Tactile Electronic Skin

Robots are now working outside of industrial fences more and more closely with humans. Safety is the primary requirement for intimate human–robot interactions. Contacts could happen at any and multipoints of robot. Tactile sensors have great potentials for contact sensing. However, their implementation for the whole-body compliance of robots upon unknown contacts is still challenging. In this work, a systematic solution is proposed. A dual-arm humanoid platform is constructed with distributed tactile sensors on its arms and body. Cheap and easy accessed resistive flexible tactile sensors are used. A data collection and signal processing system is developed for the sensing system with scalable capabilities. External contacts on the robot can be monitored by a visualized system. The multipoint contact force is calculated with the sensor positions taken into consideration. A PD controller-based compliant force control algorithm is proposed in the joint space of the robot. Particularly, a geometry-based force propagation method is introduced in order to achieve overall whole-body compliance of the robot. Experiments verified the whole-body compliance of the robot arms. Safety could then be maintained for frequent human–robot interactions.

Learning to teleoperate an upper-limb assistive humanoid robot for bimanual daily-living tasks

Objective. Bimanual humanoid platforms for home assistance are nowadays available, both as academic prototypes and commercially. Although they are usually thought of as daily helpers for non-disabled users, their ability to move around, together with their dexterity, makes them ideal assistive devices for upper-limb disabled persons, too. Indeed, teleoperating a bimanual robotic platform via muscle activation could revolutionize the way stroke survivors, amputees and patients with spinal injuries solve their daily home chores. Moreover, with respect to direct prosthetic control, teleoperation has the advantage of freeing the user from the burden of the prosthesis itself, overpassing several limitations regarding size, weight, or integration, and thus enables a much higher level of functionality. Approach. In this study, nine participants, two of whom suffer from severe upper-limb disabilities, teleoperated a humanoid assistive platform, performing complex bimanual tasks requiring high precision and bilateral arm/hand coordination, simulating home/office chores. A wearable body posture tracker was used for position control of the robotic torso and arms, while interactive machine learning applied to electromyography of the forearms helped the robot to build an increasingly accurate model of the participant’s intent over time. Main results. All participants, irrespective of their disability, were uniformly able to perform the demanded tasks. Completion times, subjective evaluation scores, as well as energy- and time- efficiency show improvement over time on short and long term. Significance. This is the first time a hybrid setup, involving myoeletric and inertial measurements, is used by disabled people to teleoperate a bimanual humanoid robot. The proposed setup, taking advantage of interactive machine learning, is simple, non-invasive, and offers a new assistive solution for disabled people in their home environment. Additionnally, it has the potential of being used in several other applications in which fine humanoid robot control is required.

Humanoid navigation: A firefly based approach

AbstractHumanoid navigation has been the center of attraction among robotic researchers since many years by virtue of their increasing use in industrial automation and smart manufacturing. In the current research, a firefly based computer vision integrated navigational analysis has been performed on humanoid robots for smooth movement by negotiation with obstacles present in complicated terrains. Here, the logic of the firefly algorithm has been used to design the controller by careful consideration of the navigational parameters. A computer vision based method is integrated along with the developed controller to resolve some conflicting situations that may arise by encountering large sized obstacle or detection of an obstacle exactly in front of the robot where the robot becomes confused regarding the direction of turn. The developed navigational model has been tested in simulation environments by considering NAO robot as the humanoid platform. The simulation results are also verified through an experimental platform developed under research laboratory conditions. The data obtained from both the platforms are compared in terms of navigational parameters and are found to be in close proximity to each other. Finally, the developed navigational model is also authenticated against other existing models for validating better efficiency.

MaskUKF: An Instance Segmentation Aided Unscented Kalman Filter for 6D Object Pose and Velocity Tracking.

Tracking the 6D pose and velocity of objects represents a fundamental requirement for modern robotics manipulation tasks. This paper proposes a 6D object pose tracking algorithm, called MaskUKF, that combines deep object segmentation networks and depth information with a serial Unscented Kalman Filter to track the pose and the velocity of an object in real-time. MaskUKF achieves and in most cases surpasses state-of-the-art performance on the YCB-Video pose estimation benchmark without the need for expensive ground truth pose annotations at training time. Closed loop control experiments on the iCub humanoid platform in simulation show that joint pose and velocity tracking helps achieving higher precision and reliability than with one-shot deep pose estimation networks. A video of the experiments is available as Supplementary Material.

Nimbro-OP2X: Affordable Adult-Sized 3D-Printed Open-Source Humanoid Robot for Research

For several years, high development and production costs of humanoid robots restricted researchers interested in working in the field. To overcome this problem, several research groups have opted to work with simulated or smaller robots, whose acquisition costs are significantly lower. However, due to scale differences and imperfect simulation replicability, results may not be directly reproducible on real, adult-sized robots. In this paper, we present the NimbRo-OP2X, a capable and affordable adult-sized humanoid platform aiming to significantly lower the entry barrier for humanoid robot research. With a height of 135[Formula: see text]cm and weight of only 19[Formula: see text]kg, the robot can interact in an unmodified, human environment without special safety equipment. Modularity in hardware and software allows this platform enough flexibility to operate in different scenarios and applications with minimal effort. The robot is equipped with an on-board computer with GPU, which enables the implementation of state-of-the-art approaches for object detection and human perception demanded by areas such as manipulation and human–robot interaction. Finally, the capabilities of the NimbRo-OP2X, especially in terms of locomotion stability and visual perception, are evaluated. This includes the performance at RoboCup 2018, where NimbRo-OP2X won all possible awards in the AdultSize class.

Character Synthesis of Ballet Archetypes on Robots Using Laban Movement Analysis: Comparison Between a Humanoid and an Aerial Robot Platform with Lay and Expert Observation

Imitation of human movement by robots is an important area of research to create expressive systems that are able to interact with humans in social settings. One lens through which imitation may be investigated is that of character archetypes, commonly found in human story-telling. Classical ballet, a form of story-telling through movement, makes extensive use of such archetypes. This paper reviews how Laban Movement Analysis creates a descriptive taxonomy in which features of interest may be identified and reported from human performers to robotic platforms, specifically for a BeBop aerial drone and a Nao humanoid platform. To determine the success of this approach, the paper also presents evaluation by lay human viewers that validates the character transfer. Moreover, this paper offers more nuance into this evaluation by querying two specialized groups of experts, which frequently deviated from lay ratings and, thus, offer deeper insight into both the success of the character transfer and the importance of prior training in movement perception. This work may be useful in human-facing scenarios where character archetypes facilitate engagement, identification, and understanding between robots and human counterparts. Further, it highlights nuances in working with expert artists to create broadly applicable designs.

Low-Cost Interactive Humanoid Robot

Humanoid robots have been on the frontier of robotic science for several decades, where human alike capabilities have been replicated into electromechanical units. Humanoid robots hold promises in the field of rescue, quarantine, hazardous conditions, radiation leakage, medical trials, etc. Building a humanoid robot is very complicated as it has to deal with locomotion, power, drive train, sensors and computing at the real time. With the development of Single board computers (SBC), the cost of computers has drastically fallen in last 2 decades. At the same time the computation power (GF/Sec) has also increased exponentially. Similarly, MEMS and sensors have also become industrially available with micro sized, robust and reliable. The power source used by robots has also advanced from dry cell to Li-Ion batteries with 5 to 8 times more energy density, resulting in higher operation time. The objective of this paper is to propose a low-cost Humanoid platform comprising a computational platform, sensors, power unit and drive train to deliver basic human alike functions like speech, visual signs, and navigation. The proposed humanoid robot uses a single board computer (SBC) capable of executing python-based AI frameworks combined with Ultrasonic sensors, Li-ion battery and DC motor drives. A top mounted touch screen is used for human machine interface (HMI). This human robotics used in a mid-size campus to guide people to their respective destination, display brief information to new users and navigate to different locations. The humanoid robot adds an aesthetical value to the campus.

Biomimetic Energy-Based Humanoid Gait Design

One of the challenges facing humanoid robots is the design of a more human-like gait. In this paper, we propose a new paradigm for gait design for humanoids that is founded in the field of Kinesiology and is based on energy-exchange between potential and kinetic energies. Additionally, we propose an energy-based controller, which not only maintains the desired gait but is also more efficient than current controllers in terms of energy expenditure and joint motor torque exertion. Experiments were performed in simulation on Webots and on an actual humanoid platform, the Nao. Results indicate an improvement in mechanical energy consumption by 10% in simulations, and 1.8% on the Nao. Qualitatively, the proposed gait yielded motions that are more human-like.

Online Delayed Reference Generation for a Humanoid Imitating Human Walking Motion

In this article, we propose a delayed reference generation method for a humanoid to stably imitate human walking. An admissible condition for a robust imitation is first defined from the state of the robot and the target support polygon. In addition, a motion buffer is proposed and implemented to apply a time delay to the human motion according to the abovementioned condition for use as a reference to the robot. A data acquisition device and the HUBO2+ humanoid platform were used in teleoperation experiments to verify the effectiveness of the proposed method. We confirmed that the robot could stably imitate human footsteps under either static or dynamic walking when applying the proposed method.

A Study on Sparse Hierarchical Inverse Kinematics Algorithms for Humanoid Robots

In humanoid robotic platforms, classical inverse kinematics algorithms, based on L2-regularization of joint velocities or accelerations, tends to engage the motion of all the available degrees of freedom, resulting in movements of the whole robot structure, which are inherently not sparse. The role of sparsity in motion control has recently gained interest in the robotics community for various reasons, e.g., human-like motions, and human-robot interaction, actuation parsimony, yet an exhaustive mathematical analysis is still missing. In order to address this topic, we here propose and compare possible sparse optimization approaches applied to hierarchical inverse kinematics for humanoid robots. This is achieved through LASSO regression and MILP optimization to resolve the IK problem. A first order formulation of the sparse regression problem is further introduced to reduce chattering on the joint velocity profiles. This article presents the theory behind the proposed approaches and performs a comparison analysis based on simulated and real experiments on different humanoid platforms.

Humanoid Robots in Aircraft Manufacturing: The Airbus Use Cases

We report on the results of a collaborative project that investigated the deployment of humanoid robotic solutions in air-craft manufacturing for several assembly op erations where access by wheeled or railported robotic platforms is not possible. Recent de velopments in multicontact planning and control, bipedal walking, embedded simultaneous localization and mapping (SLAM), whole-body multisensory task-space optimization control, and contact detection and safety suggest that humanoids could be a plausible solution for automation, given the specific requirements in large-scale manufacturing sites. The main challenge is the integration of these scientific and technological advances into two existing humanoid platforms: the position-controlled Human Robotics Project (HRP-4) and the torque-controlled robot (TORO). This integration effort was demonstrated during a bracketassembly operation inside a 1:1-scale A350 mockup of the front part of the fuselage at the Airbus Saint-Nazaire site. We present and discuss the main results achieved in this project and provide recommendations for future work.

Intermediate Programming Methodologies for Manipulating Modern Humanoid Robots

The future for robotics in today's world is clear. We must automate our manufacturing processes to remain competitive in the global marketplace. Modern robotics is an important part of that automation. Humanoids, a more recent development in robotics, have a host of capabilities that may reveal potential new uses in industry and modern engineering education. This paper describes how one may advance from a beginning user of an NAO robot to that of an intermediate user of the Choregraphe development software. The authors seek to bridge the gap between the available documentation for the novice and the advanced users/programmers of the NAO humanoid platform within the Choregraphe user environment. The NAO platform was utilized due to its high impact of use by more than 13,000 educators and researchers from more than 70 countries worldwide [1].

Humanoid whole-body remote-control framework with delayed reference generator for imitating human motion

In this study, a humanoid whole-body remote-control framework with a delayed reference generator for imitating human motion is proposed. The framework comprises two main parts: delayed reference generator and optimization-based whole-body inverse kinematics (WBIK). In the delayed reference generator, a variable time delay is applied to the human motion according to the condition for imitating human motion, so that the robot can sequentially imitate the human motion that cannot be followed stably. In the optimization-based WBIK, the optimal joint angles of the robot are determined by using quadratic programming to satisfy the motion constraints, such as joint limits, self-collision, and balance stability while following the reference motion of the robot. Experiments using the data device and humanoid platform HUBO2+ demonstrate that the robot follows all the human motions using the proposed framework. Additionally, it is confirmed that it is possible to perform accurate positioning and manipulation using the robot.

A Variable Stiffness Actuator Module With Favorable Mass Distribution for a Bio-inspired Biped Robot.

Achieving human-like locomotion with humanoid platforms often requires the use of variable stiffness actuators (VSAs) in multi-degree-of-freedom robotic joints. VSAs possess 2 motors for the control of both stiffness and equilibrium position. Hence, they add mass and mechanical complexity to the design of humanoids. Mass distribution of the legs is an important design parameter, because it can have detrimental effects on the cost of transport. This work presents a novel VSA module, designed to be implemented in a bio-inspired humanoid robot, Binocchio, that houses all components on the same side of the actuated joint. This feature allowed to place the actuator's mass to more proximal locations with respect to the actuated joint instead of concentrating it at the joint level, creating a more favorable mass distribution in the humanoid. Besides, it also facilitated it's usage in joints with centralized multi-degree of freedom (DoF) joints instead of cascading single DoF modules. The design of the VSA module is presented, including it's integration in the multi-DoFs joints of Binocchio. Experiments validated the static characteristics of the VSA module to accurately estimate the output torque and stiffness. The dynamic responses of the driving and stiffening mechanisms are shown. Finally, experiments show the ability of the actuation system to replicate the envisioned human-like kinematic, torque and stiffness profiles for Binocchio.

Sparse Optimization of Contact Forces for Balancing Control of Multi-Legged Humanoids

Multi-legged humanoid platforms present an inherent redundancy in the number of end-effectors required to perform interaction tasks, such as balancing and manipulation. The most relevant possibility opened up by end-effector redundancy consists in using a subset of the available end-effectors to perform a primary task, while employing the remaining end-effectors to perform a secondary tasks. As a consequence, it necessarily requires a methodology to automatically find the smallest set of end-effectors required to perform a primary task. For the balancing control of a torque-controlled humanoid, this is equivalent to finding a sparse solution of a contact force distribution problem. To this end, two different sparse optimization approaches are presented and extensively discussed in this work. The effectiveness of the proposed approaches has been validated on a simulated model of the CENTAURO robot developed at the Istituto Italiano di Tecnologia .