Stock Classification Prediction Based on Spark

Abstract

The fluctuation of stock price is a major concern for stock investors. In order to provide investors with decision-making suggestions for buying or selling stocks, we use big data computing platform-Spark to perform binary classification prediction of stocks’ rise and fall according to the price and related information in stock trading day, and compare four classification algorithms: naïve bayes, random forest, decision tree, and logistic regression from four metrics: run time, classification accuracy, AUC(Area under the receiver operating characteristic curve) and PR(Area under the Precision-Recall curve). Experimental results show that: (1) the run time is negatively correlated with the scale of the cluster. When the number of node in the cluster is increased by 3, the run time is reduced to about two-fifths; 2. Naïve Bayes has the lowest accuracy 0.7436, and the accuracy of random forests, decision trees, and logistic regression are 0.8297, 0.8318, 0.8085 respectively; (3) both the random forest and decision tree algorithm have a higher AUC and PR value, the values of the random forest are 0.6891,0.7497 and the value of the decision tree are 0.7165,0.7485. The evaluation results show that the random forest and decision tree algorithms are more suitable for stock classification prediction, because they pay more attention to the features with large information gain values and can reduce the error of the classification.