Limit Order Book Data Research Articles

Lob-based deep learning models for stock price trend prediction: a benchmark study

The recent advancements in Deep Learning (DL) research have notably influenced the finance sector. We examine the robustness and generalizability of fifteen state-of-the-art DL models focusing on Stock Price Trend Prediction (SPTP) based on Limit Order Book (LOB) data. To carry out this study, we developed LOBCAST, an open-source framework that incorporates data preprocessing, DL model training, evaluation, and profit analysis. Our extensive experiments reveal that all models exhibit a significant performance drop when exposed to new data, thereby raising questions about their real-world market applicability. Our work serves as a benchmark, illuminating the potential and the limitations of current approaches and providing insight for innovative solutions.

Neural stochastic agent‐based limit order book simulation with neural point process and diffusion probabilistic model

SummaryModern financial exchanges use an electronic limit order book (LOB) to store bid and ask orders for a specific financial asset. As the most fine‐grained information depicting the demand and supply of an asset, LOB data are essential in understanding market dynamics. Therefore, realistic LOB simulations offer a valuable methodology for explaining the empirical properties of markets. Mainstream simulation models include agent‐based models (ABMs) and stochastic models (SMs). However, ABMs tend not to be grounded on real historical data, whereas SMs tend not to enable dynamic agent‐interaction. More recently, deep generative approaches have been successfully implemented to tackle these issues, whereas its black‐box essence hampers the explainability and transparency of the system. To overcome these limitations, we propose a novel hybrid neural stochastic agent‐based model (NS‐ABM) for LOB simulation that incorporates a neural stochastic trader in agent‐based simulation, characterised by (1) representing the aggregation of market events' logic by a neural stochastic background trader that is pre‐trained on historical LOB data through a neural point process model; (2) learning the complex distribution for order‐related attributes conditioned on various market indicators through a non‐parametric diffusion probabilistic model; and (3) embedding the background trader in a multi‐agent simulation platform to enable interaction with other strategic trading agents. We instantiate this hybrid NS‐ABM model using the ABIDES platform. We first run the background trader in isolation and show that the simulated LOB can recreate a comprehensive list of stylised facts that demonstrate realistic market behaviour. We then introduce a population of ‘trend’ and ‘value’ trading agents, which interact with the background trader. We show that the stylised facts remain and we demonstrate order flow impact and financial herding behaviours that are in accordance with empirical observations of real markets.

Modeling of limit order book data with ordered fuzzy numbers

This paper presents a novel approach to representing the Limit Order Book data at a given timestamp using the Ordered Fuzzy Numbers concept. The limit order book contains all buy and sell orders placed by investors, updated in real-time, for the most liquid securities, even several hundred times a minute. Due to its irregular nature (different and dynamic changes in the number of buy and sell orders), direct calculations on the order book data are not feasible without transforming it into feature vectors. Currently, most studies use a price level-based data representation scheme when applying deep learning models on limit order book data. However, this scheme has limitations, particularly its sensitivity to subtle perturbations that can negatively impact model performance. On the other hand, the ordered fuzzy number is a mathematical object (a pair of two functions) used to process imprecise and uncertain data. Ordered Fuzzy Numbers possess well-defined arithmetic properties. Converting the limit order book data to ordered fuzzy numbers allows the creation of a time series of ordered fuzzy numbers (order books) and use them for further calculations, e.g., to represent input data for deep learning models or employing the concept of fuzzy time series in various domains, such as defining liquidity measures based on limit order book data. In this paper, the proposed approach is tested using one-year market data from the Polish Stock Exchange for the five biggest companies. The DeepLOB model is employed to predict mid-price movement using different input data representations. The proposed representation of Limit Order Book data demonstrated remarkably stable out-of-sample prediction accuracy, even when subjected to data perturbation.

Short-term stock price trend prediction with imaging high frequency limit order book data

Predicting price movements over a short period is a challenging problem in high-frequency trading. Deep learning methods have recently been used to forecast short-term prices via limit order book (LOB) data. In this paper, we propose a framework to convert LOB data into a series of standard images in 2D matrices and predict the mid-price movements via an image-based convolutional neural network (CNN). The empirical study shows that the image-based CNN model outperforms other traditional machine learning and deep learning methods based on raw LOB data. Our findings suggest that the additional information implicit in LOB images contributes to short-term price forecasting.

Tick size, liquid stocks and market quality: Evidence from a natural experiment in a unique setting

We study the causal impacts of a tick size reduction policy in highly liquid stocks, exploiting a unique experiment in Borsa Istanbul leading to substantial exogenous variation in the tick size. Adapting a differences-in-differences strategy with a novel limit order and trade book data with intraday calculations, we find evidence that the tick size reduction improves market quality for the liquid stocks by significantly lowering quoted, effective, and realized spread yet reducing the depth at the best bid offer and aggregate depth. Moreover, results indicate a lower price impact for the treated stocks while there is no significant impact on trading volume, however, the number of trades increases but trade and order sizes get smaller significantly. Estimates show null effects on the submission of market orders, volatility, and returns. Our study departs from the existing literature with no dark pool trading and internalization of orders in the Turkish Stock Markets.

Online Hybrid Neural Network for Stock Price Prediction: A Case Study of High-Frequency Stock Trading in the Chinese Market

Time-series data, which exhibit a low signal-to-noise ratio, non-stationarity, and non-linearity, are commonly seen in high-frequency stock trading, where the objective is to increase the likelihood of profit by taking advantage of tiny discrepancies in prices and trading on them quickly and in huge quantities. For this purpose, it is essential to apply a trading method that is capable of fast and accurate prediction from such time-series data. In this paper, we developed an online time series forecasting method for high-frequency trading (HFT) by integrating three neural network deep learning models, i.e., long short-term memory (LSTM), gated recurrent unit (GRU), and transformer; and we abbreviate the new method to online LGT or O-LGT. The key innovation underlying our method is its efficient storage management, which enables super-fast computing. Specifically, when computing the forecast for the immediate future, we only use the output calculated from the previous trading data (rather than the previous trading data themselves) together with the current trading data. Thus, the computing only involves updating the current data into the process. We evaluated the performance of O-LGT by analyzing high-frequency limit order book (LOB) data from the Chinese market. It shows that, in most cases, our model achieves a similar speed with a much higher accuracy than the conventional fast supervised learning models for HFT. However, with a slight sacrifice in accuracy, O-LGT is approximately 12 to 64 times faster than the existing high-accuracy neural network models for LOB data from the Chinese market.

Continuous-time modeling of financial returns based on Limit Order Book data

The Limit Order Book (LOB) of a financial security contains the list of buy and sell orders placed by traders on a single or across many exchanges for a given stock. As such, the amount of information contained in the LOB is much richer than that contained in the time history of the mean price, so it can potentially be exploited to model and predict more accurately the future returns. As far as we are aware, this work offers - for the first time - a dynamical systems perspective on the latter modeling problem, leading to a two-fold contribution: (i) we will define a new regressor, called Base Imbalance, and we will show that its predictive potential is more significant than the ones usually employed in the literature; (ii) we will discuss that the most suitable way to derive a model of the LOB dynamics is via continuous-time system identification, due to the event-based nature of the phenomenon at hand. The effectiveness of the proposed modeling strategy to be used in trading policies will be illustrated by means of a benchmark experimental data.

A multifactor regime-switching model for inter-trade durations in the high-frequency limit order market

Modeling behaviors of inter-trade durations has been an important step to understand the market microstructure. Given that most existing models focused on intraday transaction data, we analyze limit order book (LOB) data in a high-frequency (HF) market and find that stocks’ inter-trade durations follow bimodal distributions peaking at levels of second and millisecond. To characterize this empirical property, we develop a multifactor regime-switching duration (MF-RSD) model, in which the modes of inter-trade durations are characterized as regimes and the transitions of regimes are determined by LOB-factor-dependent transition probability matrices. Using the model, we analyze the impact of LOB factors on inter-trade duration dynamics and find that the bimodal phenomenon is caused by the switchings of traders’ trading strategies between market making and price speculation. Our empirical studies show that the MF-RSD model not only produces good in-sample fit but also outperforms some benchmark models in out-of-sample predictions of inter-trade durations. • Inter-trade durations in high-frequency LOB exhibit a bimodal distribution. • A multifactor regime-switching model is built for inter-trade durations in HF. • The regimes of inter-trade durations are associated with the trader’s behaviors. • Our model has better out-of-sample performance compared with some benchmarks. • The summarized effects of LOB factors reveal principle of market microstructure.

Stock Price Prediction Using a Frequency Decomposition Based GRU Transformer Neural Network

Stock price prediction is crucial but also challenging in any trading system in stock markets. Currently, family of recurrent neural networks (RNNs) have been widely used for stock prediction with many successes. However, difficulties still remain to make RNNs more successful in a cluttered stock market. Specifically, RNNs lack power to retrieve discerning features from a clutter of signals in stock information flow. Making it worse, by RNN a single long time cell from the market is often fused into a single feature, losing all the information about time which is essential for temporal stock prediction. To tackle these two issues, we develop in this paper a novel hybrid neural network for price prediction, which is named frequency decomposition induced gate recurrent unit (GRU) transformer, abbreviated to FDGRU-transformer or FDG-trans). Inspired by the success of frequency decomposition, in FDG-transformer we apply empirical model decomposition to decompose the complete ensemble of cluttered data into a trend component plus several informative and independent mode components. Equipped with the decomposition, FDG-transformer has the capacity to extract the discriminative insights from the cluttered signals. To retain the temporal information in the observed cluttered data, FDG-transformer utilizes hybrid neural network of GRU, long short term memory (LSTM) and multi-head attention (MHA) transformers. The integrated transformer network is capable of encoding the impact of different weights from each past time step to the current one, resulting in the establishment of a time series model from a deeper fine-grained level. We appy the developed FDG-transformer model to analyze Limit Order Book data and compare the results with that obtained from other state-of-the-art methods. The comparison shows that our model delivers effective price forecasting. Moreover, an ablation study is conducted to validate the importance and necessity of each component in the proposed model.

A machine learning approach to classification for traders in financial markets

We introduce new machine learning methods for clustering traders who are actively trading in a modern electronic exchange which uses a matching engine to track aggregate and individual‐level limit order books. Each trader's individual limit order book is centered (with the current best bid and ask prices acting as a central reference), and the patterns in the individual limit order books are identified using a Wasserstein distance. The method is illustrated using simulated limit order book data and limit order book data from a stock exchange in Canada.

Carry Forward Modeling for High-Frequency Limit-Order Executions: An Emerging Market Perspective

In this study, we estimate the order execution probability of a limit-order book (LOB) and analyze its determinants using high-frequency LOB data from the National Stock Exchange (NSE) of India. For this purpose, we propose an algorithm that estimates the LOB execution time. Using a survival function with log-normal distribution, this study analyzes the significant determinants of the limit-order execution times. The average execution probability is found to be higher for stocks belonging to the information technology and telecom sectors. The limit-order execution probability increases with a larger bid–ask spread, lower limit-order size, and deeper opposite order book. On the other hand, multiple factors, including price aggressiveness, inferior price, limit-order size, and spread, have a direct impact on execution times. The findings could help traders understand various factors influencing the probability of execution and execution time of LOBs. This study is unique in that it models limit-order execution using high-frequency tick-by-tick trading data for emerging markets, such as the NSE of India.

Predicting Stock Price Changes Based on the Limit Order Book: A Survey

This survey starts with a general overview of the strategies for stock price change predictions based on market data and in particular Limit Order Book (LOB) data. The main discussion is devoted to the systematic analysis, comparison, and critical evaluation of the state-of-the-art studies in the research area of stock price movement predictions based on LOB data. LOB and Order Flow data are two of the most valuable information sources available to traders on the stock markets. Academic researchers are actively exploring the application of different quantitative methods and algorithms for this type of data to predict stock price movements. With the advancements in machine learning and subsequently in deep learning, the complexity and computational intensity of these models was growing, as well as the claimed predictive power. Some researchers claim accuracy of stock price movement prediction well in excess of 80%. These models are now commonly employed by automated market-making programs to set bids and ask quotes. If these results were also applicable to arbitrage trading strategies, then those algorithms could make a fortune for their developers. Thus, the open question is whether these results could be used to generate buy and sell signals that could be exploited with active trading. Therefore, this survey paper is intended to answer this question by reviewing these results and scrutinising their reliability. The ultimate conclusion from this analysis is that although considerable progress was achieved in this direction, even the state-of-art models can not guarantee a consistent profit in active trading. Taking this into account several suggestions for future research in this area were formulated along the three dimensions: input data, model’s architecture, and experimental setup. In particular, from the input data perspective, it is critical that the dataset is properly processed, up-to-date, and its size is sufficient for the particular model training. From the model architecture perspective, even though deep learning models are demonstrating a stronger performance than classical models, they are also more prone to over-fitting. To avoid over-fitting it is suggested to optimize the feature space, as well as a number of layers and neurons, and apply dropout functionality. The over-fitting problem can be also addressed by optimising the experimental setup in several ways: Introducing the early stopping mechanism; Saving the best weights of the model achieved during the training; Testing the model on the out-of-sample data, which should be separated from the validation and training samples. Finally, it is suggested to always conduct the trading simulation under realistic market conditions considering transactions costs, bid–ask spreads, and market impact.

High-Frequency Trading with Machine Learning Algorithms and Limit Order Book Data

<abstract><p>In this paper, we examine the usefulness of machine learning methods such as support vector machines, random forests and bagging for the extraction of information from the limit order book that can be used for intraday trading. For our empirical analysis, we first get 50 raw features from the LOBSTER message file and order book file of the iShares Core S & P 500 ETF for the time period from 27.06.2007 to 30.04.2019 and then construct 18 higher-level features (aggregated to 5 minutes frequency) which serve as predictors. Using straightforward specifications for the machine learning procedures and thereby avoiding excessive data snooping, we find that these procedures are unable to find high dimensional patterns in the order book that could be used for trading purposes. The observed significant predictability is mainly due to the inclusion of only one variable, namely the last price change, and is probably too small to ensure profitability once transaction costs are taken into account.</p></abstract>

HFTS AND NON-HFTS ALONG THE LIMIT ORDER BOOK

Using NASDAQ high frequency trading and limit order book data over 120 stocks between 2008 and 2010, we document that HFTs are active in order placement along the book with an average placement at the 5th step, slightly ahead of NHFTs. HFT orders are further ahead during market crisis, though price-wise they retreat. Liquidity provider in trades, not consumer, matters in order placement along the book. Traders becomes more (less) aggressive placing orders when their own (the other) kind provides liquidity in trading. Trades between HFTs (NHFTs) are significantly related with less aggressive orders from NHFTs (HFTs) along the book.

State-dependent Hawkes processes and their application to limit order book modelling

We study statistical aspects of state-dependent Hawkes processes, which are an extension of Hawkes processes where a self- and cross-exciting counting process and a state process are fully coupled, interacting with each other. The excitation kernel of the counting process depends on the state process that, reciprocally, switches state when there is an event in the counting process. We first establish the existence and uniqueness of state-dependent Hawkes processes and explain how they can be simulated. Then we develop maximum likelihood estimation methodology for parametric specifications of the process. We apply state-dependent Hawkes processes to high-frequency limit order book data, allowing us to build a novel model that captures the feedback loop between the order flow and the shape of the limit order book. We estimate two specifications of the model, using the bid–ask spread and the queue imbalance as state variables, and find that excitation effects in the order flow are strongly state-dependent. Additionally, we find that the endogeneity of the order flow, measured by the magnitude of excitation, is also state-dependent, being more pronounced in disequilibrium states of the limit order book.

Order flow in the financial markets from the perspective of the Fractional Lévy stable motion

It is a challenging task to identify the best possible models based on given empirical data of observed time series. Though the financial markets provide us with a vast amount of empirical data, the best model selection is still a big challenge for researchers. The widely used long-range memory and self-similarity estimators give varying values of the parameters as these estimators themselves are developed for the specific models of time series. Here we investigate from the general fractional Lévy stable motion perspective the order disbalance time series constructed from the limit order book data of the financial markets. Our results suggest that previous findings of persistence in order flow could be related to the power-law distribution of order sizes and other deviations from the normal distribution. Still, orders have stable estimates of anti-correlation for the 18 randomly selected stocks when Absolute value and Higuchi’s estimators are implemented. Though the burst duration analysis based on the first passage problem of time series and implemented in this research gives slightly higher estimates of the Hurst and memory parameters, it qualitatively supports the importance of the power-law distribution of order sizes.

Quantifying the High-Frequency Trading “Arms Race”

Abstract We use stock exchange message data to quantify the negative aspect of high-frequency trading, known as “latency arbitrage.” The key difference between message data and widely familiar limit order book data is that message data contain attempts to trade or cancel that fail. This allows the researcher to observe both winners and losers in a race, whereas in limit order book data you cannot see the losers, so you cannot directly see the races. We find that latency arbitrage races are very frequent (about one per minute per symbol for FTSE 100 stocks), extremely fast (the modal race lasts 5–10 millionths of a second), and account for a remarkably large portion of overall trading volume (about 20%). Race participation is concentrated, with the top six firms accounting for over 80% of all race wins and losses. The average race is worth just a small amount (about half a price tick), but because of the large volumes the stakes add up. Our main estimates suggest that races constitute roughly one-third of price impact and the effective spread (key microstructure measures of the cost of liquidity), that latency arbitrage imposes a roughly 0.5 basis point tax on trading, that market designs that eliminate latency arbitrage would reduce the market’s cost of liquidity by 17%, and that the total sums at stake are on the order of $5 billion per year in global equity markets alone.

Optimal trade execution for Gaussian signals with power-law resilience

We characterize the optimal signal-adaptive liquidation strategy for an agent subject to power-law resilience and zero temporary price impact with a Gaussian signal, which can include e.g an OU process or fractional Brownian motion. We show that the optimal selling speed is a Gaussian Volterra process of the form on , where and satisfy a family of (linear) Fredholm integral equations of the first kind which can be solved in terms of fractional derivatives. The term is the (deterministic) solution for the no-signal case given in Gatheral et al. [Transient linear price impact and Fredholm integral equations. Math. Finance, 2012, 22, 445–474], and we give an explicit formula for for the case of a Riemann-Liouville price process as a canonical example of a rough signal. With non-zero linear temporary price impact, the integral equation for becomes a Fredholm equation of the second kind. These results build on the earlier work of Gatheral et al. [Transient linear price impact and Fredholm integral equations. Math. Finance, 2012, 22, 445–474] for the no-signal case, and complement the recent work of Neuman and Voß[Optimal signal-adaptive trading with temporary and transient price impact. Preprint, 2020]. Finally we show how to re-express the trading speed in terms of the price history using a new inversion formula for Gaussian Volterra processes of the form , and we calibrate the model to high frequency limit order book data for various NASDAQ stocks.

The LOB Recreation Model: Predicting the Limit Order Book from TAQ History Using an Ordinary Differential Equation Recurrent Neural Network

In an order-driven financial market, the price of a financial asset is discovered through the interaction of orders - requests to buy or sell at a particular price - that are posted to the public limit order book (LOB). Therefore, LOB data is extremely valuable for modelling market dynamics. However, LOB data is not freely accessible, which poses a challenge to market participants and researchers wishing to exploit this information. Fortunately, trades and quotes (TAQ) data - orders arriving at the top of the LOB, and trades executing in the market - are more readily available. In this paper, we present the LOB recreation model, a first attempt from a deep learning perspective to recreate the top five price levels of the LOB for small-tick stocks using only TAQ data. Volumes of orders sitting deep in the LOB are predicted by combining outputs from: (1) a history compiler that uses a Gated Recurrent Unit (GRU) module to selectively compile prediction relevant quote history; (2) a market events simulator, which uses an Ordinary Differential Equation Recurrent Neural Network (ODE-RNN) to simulate the accumulation of net order arrivals; and (3) a weighting scheme to adaptively combine the predictions generated by (1) and (2). By the paradigm of transfer learning, the core encoder trained on one stock can be fine-tuned to enable application to other financial assets of the same class with much lower demand on additional data. Comprehensive experiments conducted on two real world intraday LOB datasets demonstrate that the proposed model can efficiently recreate the LOB with high accuracy using only TAQ data as input.

Investment Sizing with Deep Learning Prediction Uncertainties for High-Frequency Eurodollar Futures Trading

In this article, the authors show that prediction uncertainty estimates gleaned from deep learning models can be useful inputs for influencing the relative allocation of risk capital across trades. In this way, consideration of uncertainty is important because it permits the scaling of investment size across trade opportunities in a principled and data-driven way. The authors showcase this insight with a prediction model and, based on a Sharpe ratio metric, find clear outperformance relative to trading strategies that either do not take uncertainty into account or use an alternative market-based statistic as a proxy for uncertainty. Of added novelty is their modeling of high-frequency data at the top level of the Eurodollar futures limit order book for each trading day of 2018, whereby they predict interest rate curve changes on small time horizons. The authors are motivated to study the market for these popularly traded interest rate derivatives because it is deep and liquid and contributes to the efficient functioning of global finance—though there is relatively little by way of its modeling contained in the academic literature. Hence, they verify the utility of prediction models and uncertainty estimates for trading applications in this complex and multidimensional asset price space. <b>TOPICS:</b>Big data/machine learning, derivatives, simulations, statistical methods <b>Key Findings</b> ▪ The authors model high-frequency Eurodollar Futures limit order book data using state-of-the-art deep learning to predict interest rate curve changes on small time horizons. ▪ They further augment their models to yield estimates of prediction uncertainty. ▪ In certain settings, the uncertainty estimates can be used in conjunction with return predictions for scaling bankroll allocation between trades. This can lead to clear trading outperformance relative to the case that uncertainty is not taken into account.