Multi-objective symbolic regression using long-term artificial neural network memory (LTANN-MEM) and neural symbolization algorithm (NSA)

Abstract

Symbolic regression is commonly performed using evolutionary algorithms like genetic programming (GP). The goal of this research work is to construct symbolic models from examples where a new symbolic regression approach based on artificial neural networks is proposed. This approach is composed of a long-term artificial neural network memory (LTANN-MEM), a working memory (WM) in addition to a proposed neural symbolization algorithm (NSA) which uses LTANN-MEM and WM for synthesizing symbolic models equivalent to learning examples. The proposed LTANN-MEM is composed of two separate multilayer perceptron (MLP) feed-forward neural networks as well as the working memory which is composed of a single MLP. The core idea of the proposed approach is based on memorizing the learning experience of individual perceptrons in long-term memory (LTM), so they become available to be reused in generating and developing hypotheses about the learning examples. Although this idea is generic and could be used for the purpose of symbolization in general, it is applied here in symbolic regression for Boolean domain only. The obtained results show the ability of the proposed approach to search the solutions space using learning experience stored previously in LTM to guide the search process. A comparison is done with GP and found that the proposed NSA algorithm outperforms GP in its performance when increasing the number of inputs and outputs in the same problem by comparing the number of emerged candidate solutions in both approaches.