A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression

Abstract

Truss layout design aims to find the optimal layout, considering node locations, connection topology between nodes, and cross-sectional areas of connecting bars. The design process of trusses can be represented as a reinforcement learning problem by formulating the optimization task into a Markov Decision Process (MDP). The optimization variables such as node positions need to be transformed into discrete actions in this MDP; however, the common method is to uniformly discretize the design domain by generating a set of candidate actions, which brings dimension explosion problems in spatial truss design. In this paper, a reinforcement learning algorithm is proposed to deal with continuous action spaces in truss layout design problems by using kernel regression. It is a nonparametric regression way to sample the continuous action space and generalize the information about action value between sampled actions and unexplored parts of the action space. As the number of searches increases, the algorithm can gradually increase the candidate action set by appending actions of high confidence value from the continuous action space. The value correlation between actions is mapped by the Gaussian function and Euclidean distance. In this sampling strategy, a modified Confidence Upper Bound formula is proposed to evaluate the heuristics of sampled actions, including both 2D and 3D cases. The proposed algorithm was tested in various layout design problems of planar and spatial trusses. The results indicate that the proposed algorithm has a good performance in finding the truss layout with minimum weight. This implies the validity and efficiency of the established algorithm.