A data mining framework for financial prediction

Abstract

In the financial markets, because real-time transactions directly relate to profit, it is important to process and analyze data on a real-time basis. In practice, decisions influenced by experts’ experiences from fundamental and technical analysis occur frequently compared to decisions using prediction algorithms. A domain-specific data mining framework was proposed recently to reduce related cost. Therefore, this study proposes a novel data mining framework suitable for financial markets according to expert knowledge. The proposed framework predominantly considers the following three perspectives as the standards for the effectiveness of research: interpretability, proper prediction metrics, and reporting methods. We applied our framework to the real-world financial prediction problems, such as the 3–10 year treasury spread forecasts. Consequently, we achieved an 84% prediction performance on the spread prediction and used hierarchical information to provide additional insight. In addition, we obtained practical knowledge and synergies through extraction of critical variables that can be used as a quick and accurate data-driven decision making support tool by active agents in the real world.