Forecasting daily stock trend using multi-filter feature selection and deep learning

Abstract

Stock market forecasting has attracted significant attention mainly due to the potential monetary benefits. Predicting these markets is a challenging task due to numerous interrelated factors, and needs a complete and efficient feature selection process to identify the most informative factors. As a time series problem, stock price movements are also dependent on movements on its previous trading days. Feature selection techniques have been widely applied in stock forecasting, but existing approaches usually use a single feature selection technique, which may overlook some important assumptions about the underlying regression function linking the input and output variables. In this study, we combine features selected by multiple feature selection techniques to generate an optimal feature subset and then use a deep generative model to predict future price movements. First, we compute an extended set of forty-four technical indicators from daily stock data of eighty-eight stocks and then compute their importance by independently training logistic regression model, support vector machine and random forests. Based on a prespecified threshold, the lowest ranked features are dropped and the rest are grouped into clusters. The variable importance measure is reused to select the most important feature from each cluster to generate the final subset. The input is then fed to a deep generative model comprising of a market signal extractor and an attention mechanism. The market signal extractor recurrently decodes market movement from the latent variables to deal with stochastic nature of the stock data and the attention mechanism discriminates between predictive dependencies of different temporal auxiliary outputs. The results demonstrate that combining features selected by multiple feature selection approaches and using them as input into a deep generative model outperforms state-of-the-art approaches.