Small implementation of decision-making policy for the height task of the Acrobot

Abstract

Since decision-making algorithms on high-performance computing yield large-size policies, compression methods are necessary for utilizing them on small robots or on robots that must store a number of policies for various tasks. A height task of the Acrobot, which is a well-known problem, is solved by value iteration and its decision-making policy is compressed to the utmost limit in this paper. From the result, we discuss the availability of brute force approach under severe limitations of memory available on robots. We have obtained a 7060 bit policy using vector quantization and run-length encoding from a policy on a 60,484,176 bit look-up table though the table is complicated by the chaotic nature of the Acrobot. Moreover, we have evaluated the size of software for using the compressed policy, and have created a 15,424 bit object code for an ARM processor. The total size 22,752 bit is in the same range with those of policies implemented as several if-then rules or equations coded by hand.