Anomaly Detection in Market Data Structures Via Machine Learning Algorithms

Abstract

We made use of a stacked autoencoder machine learning algorithm to detect injected anomalies in swaption volatility offset cubes, a financial market data entity. These consist of 240 datapoints each. Injected cubes contained a modification at one single point, five modi-fied cubes with modifications of different severity had been constructed as our test cases. We investigated the detection results of four slightly different autoencoder architectures with SELU activation, ADAM optimizer and five symmetric hidden layers, differing in number of nodes in the bottleneck layer and whether input noise was applied or not. Implementation was established using the tensorflow library. Unsupervised training was performed on 638 historical swaption volatility offset cubes, ten percent of which had been randomly selected and set aside to constitute the test dataset. The trained algorithms detected the three largest injections reliably and marked a total of 25 da-tasets out of 638 historical and 5 injected ones as ‘spurious’, including all five cubes containing an injection. Using this technology in quality assurance to pre-select for conspicuous data, we could – in our case – reduce the data requiring human attention by 96%, making it a useful tool for the market data quality manager which can easily be applied to other problems. Code and example data can be found on our GitHub repo (Roeder Dirk, Mueller Henning, 2019).