Giant Swing Motion Research Articles

Motion control of underactuated linkage robot based on gymnastic skill

AbstractIn this paper, a reproduction of a swing‐up and a giant swing motion of underactuated robots based on technique of the horizontal bar gymnast is discussed with focusing on an equivalent center of mass (ECM) of underactuated robots and the gymnast. At first, the behaviors of the ECM of the gymnast (ECMG) are analyzed by using a motion capturing technique and an efficient motion of the ECMG for the swing‐up and for the giant swing motion are identified from analysis results, respectively. Next, a partial linearization method, which can realize that the ECM of the Acrobot (ECMA) replicates this efficient motion, is designed and reproduces the underactuated robots the swing‐up and giant swing motion like the gymnast. Finally, an effectiveness of the proposed controller is shown by numerical simulations.

Dynamic delayed feedback control for stabilizing the giant swing motions of an underactuated three-link gymnastic robot

This paper investigates the dynamics of the giant swing motions of an underactuated three-link gymnastic robot moving in a vertical plane by means of dynamic delayed feedback control (DDFC). DDFC, being one of useful methods to overcome the so-called odd number limitation in controlling a chaotic discrete-time system, is extended to control a continuous-time system such as a 3-link gymnastic robot with passive joint. Meanwhile, a way to calculate the error transfer matrix and the input matrix which are necessary for discretization is proposed, based on a Poincare section which is defined to regard the target system as a discrete-time system. Moreover, the stability of the closed-loop system by the proposed control strategy is discussed. Furthermore, some numerical simulations are presented to show the effectiveness in controlling a chaotic motion of the 3-link gymnastic robot to a periodic giant swing motion.

1A1-J04 強化学習を用いた大車輪ロボットの行動獲得と学習プロセスについて(進化・学習とロボティクス)

In our laboratory, we have succeeded in acquiring forward actions to various robot systems using Reinforcement Learning. We have also succeeded in acquiring a giant swing motion as dynamic task by devising its rewards. Then, the purpose of this study is to investigate the effect of probabilistic behavior which gives to the giant swing motion acquisition. Although the giant swing robot has a continuous dynamic motion such as its angle and angler velocity, its state of the motion must be divided into 216 states in order to apply the reinforcement learning. For this reason, this robot shows probabilistic behaviors. This study investigated the relationship between the performance of learning knowledge and rotation count of the learning process. Consequently, it became clear that the performance of the knowledge is deteriorated when the ratio of knowledge use of the learning process is large.

Cognitively inspired reinforcement learning architecture and its application to giant-swing motion control

Many algorithms and methods in artificial intelligence or machine learning were inspired by human cognition. As a mechanism to handle the exploration–exploitation dilemma in reinforcement learning, the loosely symmetric (LS) value function that models causal intuition of humans was proposed (Shinohara et al., 2007). While LS shows the highest correlation with causal induction by humans, it has been reported that it effectively works in multi-armed bandit problems that form the simplest class of tasks representing the dilemma. However, the scope of application of LS was limited to the reinforcement learning problems that have K actions with only one state (K-armed bandit problems). This study proposes LS-Q learning architecture that can deal with general reinforcement learning tasks with multiple states and delayed reward. We tested the learning performance of the new architecture in giant-swing robot motion learning, where uncertainty and unknown-ness of the environment is huge. In the test, the help of ready-made internal models or functional approximation of the state space were not given. The simulations showed that while the ordinary Q-learning agent does not reach giant-swing motion because of stagnant loops (local optima with low rewards), LS-Q escapes such loops and acquires giant-swing. It is confirmed that the smaller number of states is, in other words, the more coarse-grained the division of states and the more incomplete the state observation is, the better LS-Q performs in comparison with Q-learning. We also showed that the high performance of LS-Q depends comparatively little on parameter tuning and learning time. This suggests that the proposed method inspired by human cognition works adaptively in real environments.

Acquisition of a Gymnast-Like Robotic Giant-Swing Motion by Q-Learning and Improvement of the Repeatability

This paper proposes an application of Q-learning to a compact humanoid robot, aiming at acquisition of a gymnastic swinging even through the Markov property may not be guaranteed in such dynamic motions. As for this, several studies have relied on information from the robotic models or multiple controllers, but very few studies have tried Q-learning of human-like swing motion without preliminary knowledge. We avoid this Markov property problem by embedding the dynamic information in a robotic state space and averaging action-value functions. In this study, Q-learning is executed with a dynamic simulator based on a real humanoid robot with 5 degrees of freedom (DOF) and we verify the learning effectiveness by actually applying the learning results to the real robot. The particularly significant point in our Q-learning is that preliminary information is eliminated as far as possible; only the reward and current robotic state are available. The key factor in robotic giant-swing motion is discussed by examining effects of various rewards on the robotic performance. In addition, we argue a method for improving the repeatability and duration until reaching giant-swing motion. Finally, this paper demonstrates an attractive robotic giant-swing motion generated by only the environmental interaction.

Giant Swing Motion Control of 3-link Gymnastic Robot with Friction around an Underactuated Joint

This paper discusses a control method which can stabilize the giant swing motions of 3-link horizontal bar gymnastic robot. A method called Multiple-prediction Delayed Feedback Control(MDFC) proposed by the authors, which has been shown to be effective in control such chaos system, is extended in this paper to experimental gymnastic robot system, taking in account the effect of friction around the link joint. The dynamic of underactuated systems like gymnastic robot shows significant differences in cases of with friction and without friction. However, MDFC considers only the ideal situation without friction. Therefore, a stabilization control method consisting of three kind of control inputs which the one is MDFC for guaranteeing asymptotic stability and the others are used for friction compensation is derived. Numerical simulations and experimental results prove its effectiveness of our proposed method.

1A1-M06 強化学習を用いた大車輪ロボットの学習の汎化性に関する考察(進化・学習とロボティクス)

We have studied a giant swing robot whose giant swing motion can be obtained simply by applying Q learning method as one of reinforcement learning methods. In this study, we focused on learning generalization that is a problem whether a robot can choose one rotational direction. In our previous studies, it was successful that the real robot can't acquire backward rotation but forward rotation. This paper clarifies whether learning generalization can apply both forward and backward rotation on the simulation base. Learning simulation experiments take place in the various conditions like rewards, actions after rotations and learning system. As a results, it becomes obvious that trend of rotational direction can be controlled to some extent.

遅延フィードバック制御を用いた2リンク鉄棒ロボットの大車輪運動の実現(機械力学,計測,自動制御)

In this paper, a method of the giant swing motion of a 2-DOF link horizontal bar mechanism using Delayed Feedback Control is discussed. Firstly, a kind of Poincare section to regard the system as a discrete-time system for evaluating its stability is defined. Meanwhile, a way to calculate numerically the transitional error matrix and the input control matrix is shown. Secondly, we investigate how we can obtain the gain of feedback on the basis of analysis about the stabilization of system under the control of delayed feedback. Finally, a simulation result to show the effectiveness of this method is represented.

Acquisition of Robotic Giant-swing Motion Using Reinforcement Learning and Its Consideration of Motion Forms

Acquisition of Robotic Giant-swing Motion Using Reinforcement Learning and Its Consideration of Motion Forms

A11 遅延フィードバック制御を用いた2リンク鉄棒ロボットの大車輪運動の実現(OS3 ロボットのダイナミクスと制御1)

In this paper we discuss a method of the giant swing motion of a 2-DOF link horizontal bar mechanism using Delayed Feedback Control. Firstly we define a kind of Poincare section to regard the system as a discrete-time system for evaluating its stability. On the other hand, a way to calculate numerically the transitional error matrix is shown. Secondly, we investigate how we can obtain the gain of feedback on the basis of analysis about the stabilization of system under the control of delayed feedback. Finally, we present a simulation result to show the effectiveness of this method.

2A2-D17 強化学習を用いたロボットの大車輪運動の獲得に関する研究

2A2-D17 強化学習を用いたロボットの大車輪運動の獲得に関する研究

3605 強化学習によるスポーツロボットの大車輪運動に関する研究(G15-1 ロボティクス・メカトロニクス(1) 歩行ロボット・アクロボット,21世紀地球環境革命の機械工学:人・マイクロナノ・エネルギー・環境)

In this paper, we attempt to make a compact humanoid robot acquire the giant swing motion without any robotic models by using reinforcement learning; in particular, this paper applies Q-learning algorithm. The reinforcement learning method is said to be not suitable for dynamic tasks because Markov property is not necessarily guaranteed. However, we tried to avoid this problem by considering angular velocity in state definition and averaging Q-value at every step to reduce the dynamic noise. The ODE-based simulator is also used to find the best reward for acquiring the giant motion. Further, learning process is divided into two stages, and finally they enable attractive giant swing motion like gymnast.

A Realization Method of Giant-swing Motion of 3-DOF Link Mechanism

This paper discusses a control method of the giant swing motion of a 3-DOF link horizontal bar mechanism that is controlled by a discrete-time controller. The stability of a periodic motion that is controlled by a discrete-time controller is firstly defined. Secondly, in order to increase stability of the giant swing motion of a 3 DOF link horizontal bar mechanism, a control technique that is named the discrete-time variable gain configuration control is proposed. Moreover, an acceleration control method that accelerates swing motion of the mechanism to a certain free giant swing motion is derived. Experimental result proves an effectiveness of our control methods.

Giant-swing Motions and Control of a 3-DOF Link Mechanism

This paper investigates the giant swing motion and control of a 3-DOF link horizontal bar mechanism. Five types of free gaint swing motion are derived for an experimental setup by the shooting method. To evaluate the stability of the giant swing motion, a new criterion is proposed. It is shown that the continuous-time configuration based control can stabilize some types of giant swing motion. Sets of six feedback gains in the continuous-time configuration based control are successfully determined with Genetic Algorithm for various giant swing motions.

218 可変ゲインコンフィギュレーション制御による鉄棒ロボットの大車輪運動

This paper describes a new control method that realizes a giant swing motion of a 3-DOF link horizontal bar mechanism. Trajectory of free giant swing motion represented with respect to the wrist joint-angle is utilized as a reference-trajectory in the configuration based control that was proposed in our previous study. Tracking performance is investigated minutely in cases of the continuous- and discrete-time configuration based controls. Additionally,we introduce a set of wrist joint angle-variant feedback gains to decrease the tracking error in case of the discrete-time configuration based control using a digital controller. The validity of proposed control method was shown by computer simulations.

Control of giant swing motion of a two-link horizontal bar gymnastic robot

We investigated a control method to realize the three different types of free giant swing motions produced by a two-link horizontal bar gymnastic robot. By evaluating the eigenvalues of the transitional error matrix on the Poincare plane, it was found that the stable giant swing motions could be obtained by a proposed configuration control, in which the actuated joint torque is controlled such that the measured state variables follow the reference configuration with respect to the angular position of the passive joint. We also demonstrated that the two types of stable giant swing motions could be accomplished by the configuration method.