Real Financial Datasets Research Articles

A Characterization of Quantum Generative Models

Quantum generative modeling is a growing area of interest for industry-relevant applications. This work systematically compares a broad range of techniques to guide quantum computing practitioners when deciding which models and methods to use in their applications. We compare fundamentally different architectural ansatzes of parametric quantum circuits: (1) A continuous architecture, which produces continuous-valued data samples, and (2) a discrete architecture, which samples on a discrete grid. We also compare the performance of different data transformations: the min-max and the probability integral transforms. We use two popular training methods: (1) quantum circuit Born machines (QCBM), and (2) quantum generative adversarial networks (QGAN). We study their performance and tradeoffs as the number of model parameters increases, with a baseline comparison of similarly trained classical neural networks. The study is performed on six low-dimensional synthetic and two real financial data sets. Our two key findings are that: (1) For all data sets, our quantum models require similar or fewer parameters than their classical counterparts. In the extreme case, the quantum models require two orders of magnitude less parameters. (2) We empirically find that a variant of the discrete architecture, which learns the copula of the probability distribution, outperforms all other methods.

MDGAD: Meta domain generalization for distribution drift in anomaly detection

Anomalies often experience distribution drift over time in anomaly detection. Traditional anomaly detection methods detect anomalies with the same distribution effectively, but it is difficult to detect anomalies with changing distributions. In this paper, we propose a meta domain generalization based anomaly detection (MDGAD) framework to detect anomalies when distribution drifts. The framework first divides the data into a series of subsets. We measure the domain shift between sets with a class-sensitive distance metric, and merge similar sets, thereby generating different source domains to enhance domain diversity. We utilize the distribution shifts that existed on these source domains to simulate the distribution shifts that would be encountered when the model was applied. Meta learning is employed during training. First, the network is updated once on the meta-training set, and then another update is performed on the meta-verification set. These two gradients together determine the update direction of the network. At the same time, our framework will complete the domain alignment task on multiple source domains to extract domain invariant features and enhance the generalization of feature learning. We use Devnet as the base model and test our framework on 5 simulated data sets. The results confirm that our MDGAD outperforms current popular algorithms in detecting unknown anomalies and has better robustness. In addition, we also test our framework on real financial datasets to demonstrate the effectiveness.

Financial trading decisions based on deep fuzzy self-organizing map

The volatility features of financial data would considerably change in different periods, that is one of the main factors affecting the applications of machine learning in quantitative trading. Therefore, to effectively distinguish fluctuation patterns of financial markets can provide meaningful information for the trading decision. In this article, a novel intelligent trading system based on deep fuzzy self-organizing map (DFSOM) companied with GRU networks is proposed, where DFSOM is utilized for the clustering of financial data to acquire multiple fluctuation patterns in an unsupervised way. Firstly, in order to capture the trend features and evade the effect of high noises in financial data, the images of extended candlestick charts instead of raw data are processed and the obtained features are applied for the following unsupervised learning, where candlestick charts are produced with both price and volume information. Secondly, by using the candlestick features, a two-layer deep fuzzy self-organizing map is constructed to carry out the clustering, where two-layer models carry out the clustering in multiple time scales to improve the processing of time-dependent information. Thirdly, GRU networks are used to implement the prediction task, based on which an intelligent trading model is constructed. The feasibility and effectiveness of the proposed method are verified by using various real financial datasets.

On a quantile autoregressive conditional duration model

Autoregressive conditional duration (ACD) models are primarily used to deal with data arising from times between two successive events. These models are usually specified in terms of a time-varying conditional mean or median duration. In this work, we relax this assumption and consider a conditional quantile approach to facilitate the modeling of different percentiles. The proposed ACD quantile model is based on a skewed version of Birnbaum–Saunders distribution, which yields better fit of the tails than the traditional Birnbaum–Saunders distribution, in addition to facilitating the implementation of an expectation conditional maximization (ECM) algorithm. A Monte Carlo simulation study is performed to assess the behavior of the model as well as the parameter estimation method and the evaluation of a form of residuals. Two real financial transaction data sets are finally analyzed to illustrate the proposed approach.

Optimal Admission Control for Multiclass Queues with Time-Varying Arrival Rates via State Abstraction

We consider a novel queuing problem where the decision-maker must choose to accept or reject randomly arriving tasks into a no buffer queue which are processed by N identical servers. Each task has a price, which is a positive real number, and a class. Each class of task has a different price distribution, service rate, and arrives according to an inhomogenous Poisson process. The objective is to decide which tasks to accept so that the total price of tasks processed is maximised over a finite horizon. We formulate the problem using a discrete time Markov Decision Process (MDP) with a hybrid state space. We show that the optimal value function has a specific structure, which enables us to solve the hybrid MDP exactly. Moreover, we rigorously prove that as the gap between successive decision epochs grows smaller, the discrete time solution approaches the optimal solution to the original continuous time problem. To improve the scalability of our approach to a greater number of servers and task classes, we present an approximation based on state abstraction. We validate our approach on synthetic data, as well as a real financial fraud data set, which is the motivating application for this work.

A combinatorial optimization approach to scenario filtering in portfolio selection

Recent studies stressed the fact that covariance matrices computed from empirical financial time series appear to contain a high amount of noise. This makes the classical Markowitz Mean–Variance Optimization model unable to correctly evaluate the performance associated to selected portfolios. Since the Markowitz model is still one of the most used practitioner-oriented tool, several filtering methods have been proposed in the literature to overcome the problem. Among them, the two most promising ones refer to the Random Matrix Theory and to the Power Mapping strategy. The basic idea of these methods is to transform the estimated correlation matrix before applying the Mean–Variance Optimization model. However, experimental analysis shows that these two strategies are not always effective when applied to real financial datasets.In this paper we propose a new filtering method based on Quadratic Programming. We develop a Mixed Integer Quadratic Programming model, which is able to filter those observations that may affect the performance of the selected portfolio. We discuss the properties of this new model and test it on some real financial datasets. We compare the out-of-sample performance of our portfolios with the one of the portfolios provided by the two above mentioned alternative filtering methods giving evidence that our method outperforms them. Although our model can be solved efficiently with standard optimization solvers, the computational burden increases for large datasets. To solve also these problems, we propose a heuristic procedure, which, on the basis of our empirical results, shows to be both efficient and effective.

Semi-Supervised Clustering for Financial Risk Analysis.

Many methods have been developed for financial risk analysis. In general, the conventional unsupervised approaches lack sufficient accuracy and semantics for the clustering, and the supervised approaches rely on large amount of training data for the classification. This paper explores the semi-supervised scheme for the financial data prediction, in which accurate predictions are expected with a small amount of labeled data. Due to lack of sufficient distinguishability in financial data, it is hard for the existing semi-supervised approaches to obtain satisfactory results. In order to improve the performance, we first convert the input labeled clues to the global prior probability, and propagate the’soft’ prior probability to learn the posterior probability instead of directly propagating the’hard’ labeled data. A label diffusion model is then constructed to adaptively fuse the information at feature space and label space, which makes the structures of data affinity and labeling more consistent. Experiments on two public real financial datasets validate the effectiveness of the proposed method.

Unified Tests for a Dynamic Predictive Regression

Testing for predictability of asset returns has been a long history in economics and finance. Recently, based on a simple predictive regression, Kostakis, Magdalinos, and Stamatogiannis derived a Wald type test based on the context of the extended instrumental variable (IVX) methodology for testing predictability of stock returns, and Demetrescu showed that the local power of the standard IVX-based test could be improved for some range of alternative hypotheses and the tuning parameter when a lagged predicted variable is added to the predictive regression on purpose, which poses an important question on whether the predictive model should include a lagged predicted variable. This article proposes novel robust procedures for testing both the existence of a lagged predicted variable and the predictability of asset returns regardless of regressors being stationary or nearly integrated or unit root and the AR model for regressors with or without an intercept. A simulation study confirms the good finite sample performance of the proposed tests before illustrating their practical usefulness in analyzing real financial datasets.

Clustering and portfolio selection problems: A unified framework

Given a set of assets and an investment capital, the classical portfolio selection problem consists in determining the amount of capital to be invested in each asset in order to build the most profitable portfolio. The portfolio optimization problem is naturally modeled as a mean-risk bi-criteria optimization problem where the mean rate of return of the portfolio must be maximized whereas a given risk measure must be minimized. Several mathematical programming models and techniques have been presented in the literature in order to efficiently solve the portfolio problem. A relatively recent promising line of research is to exploit clustering information of an assets network in order to develop new portfolio optimization paradigms. In this paper we endow the assets network with a metric based on correlation coefficients between assets’ returns, and show how classical location problems on networks can be used for clustering assets. In particular, by adding a new criterion to the portfolio selection problem based on an objective function of a classical location problem, we are able to measure the effect of clustering on the selected assets with respect to the non-selected ones. Most papers dealing with clustering and portfolio selection models solve these problems in two distinct steps: cluster first and then selection. The innovative contribution of this paper is that we propose a Mixed-Integer Linear Programming formulation for dealing with this problem in a unified phase. The effectiveness of our approach is validated reporting some computational experiments on some real financial datasets.

An Universal, Simple, Circular Statistics-Based Estimator of α for Symmetric Stable Family

The aim of this article is to obtain a simple and efficient estimator of the index parameter of symmetric stable distribution that holds universally, i.e., over the entire range of the parameter. We appeal to directional statistics on the classical result on wrapping of a distribution in obtaining the wrapped stable family of distributions. The performance of the estimator obtained is better than the existing estimators in the literature in terms of both consistency and efficiency. The estimator is applied to model some real life financial datasets. A mixture of normal and Cauchy distributions is compared with the stable family of distributions when the estimate of the parameter α lies between 1 and 2. A similar approach can be adopted when α (or its estimate) belongs to (0.5,1). In this case, one may compare with a mixture of Laplace and Cauchy distributions. A new measure of goodness of fit is proposed for the above family of distributions.

Stieltjes Sample Characteristic Function: Singular Continuous Distributions Parameters Estimation

In this paper, the Steiltjes sample characteristic function is introduced and studied. It is applied to establish a new parameter estimation method. The method is useful to estimate parameters of singular continuous distributions, where other estimation procedures such as maximum likelihood and empirical characteristic function fail to be applied. We fit well the Cantor's type distribution with the parameters estimated by the proposed method on the log returns of the daily price of Brazilian coffee. Effectiveness of the estimation approach is also numerically demonstrated in estimating the parameters of the standard two-sided power and the asymmetric Laplace distributions, using simulated and a real financial data set. Certain theoretical derivations concerning the Stieltjes sample characteristic function are given as well.

An Effective Approach for the Diverse Group Stock Portfolio Optimization Using Grouping Genetic Algorithm

Finding useful portfolios that could be a portfolio of trading strategy or a stock portfolio from financial datasets is always an attractive research topic due to the nature of financial markets. Because investors always want an approach that can continually provide various portfolios, the issue of group stock portfolio optimization (GSPO) has been raised and the algorithms to obtain a group stock portfolio (GSP) have also been described in the past. A GSP divides the whole set of stocks into several stock groups, and the stocks in a group are exchangeable in investment. Thus, when investors are not satisfied with a suggested stock, they can select another stock from the same group to replace the original one. However, the industry diversity of stocks within a group is not regarded in the existing literatures. In this paper, an algorithm for dealing with the diverse group stock portfolio optimization (DGSPO) is proposed to obtain a diverse group stock portfolio (DGSP). The proposed algorithm is based on the group genetic algorithm with the chromosome representation and the fitness function designed for the purpose of finding a good DGSP. Especially, a factor called group diversity is designed to diversify stocks from different industries and is considered in the fitness evaluation. Another factor considers cash dividend is also applied to keep companies with good quality in the portfolio for increasing the profit of a DGSP. Two real financial datasets are used in the experiments to verify the effectiveness of the proposed approach.

A Data Masking Utility to Secure Sensitive Production Data in Non-Production Environment

The right to privacy is a fundamental right. However, despite increasing recognition and awareness of data protection, there is still a lack of legal and institutional frameworks, processes, and infrastructure support to protect data and privacy rights. The increasing use of personal data, with the emergence of technologies, enabling new ways of processing and using it, means that regulating an effective data protection framework is more important than ever. Researchers developed a non-deterministic masking technique that provides security to sensitive data, without modifying the appearance of original values of confidential variables. In this study, we develop such a new procedure and discuss its underlying theory. Referred to as non-deterministic masking, this procedure will allow organisations to publish and share data for non-production environment, with minimal disclosure risk. The proposed model is evaluated on a real financial dataset and the results showed masked dataset as similar as the original dataset.

Lack of Fit Test for Infinite Variation Jumps at High Frequencies

In this paper, we are concerned with testing for infinite variation jumps in addition to a continuous local martingale component driven by Brownian motion using high-frequency data. We developed a lack of fit type test based on the empirical distribution of the devolatized increments. Under the null that the jump component is of finite variation, the empirical process associated with the devolatized increments converges to a Gaussian process in the Skorohod topology. Under the alternative hypothesis that the jumps are of infinite variation, that empirical process explodes instead. Theoretical results show good performance of the size and power. Simulation studies justify the theory. A real financial data set is analyzed.

Poisson-Driven Stationary Markov Models

ABSTRACT We propose a simple yet powerful method to construct strictly stationary Markovian models with given but arbitrary invariant distributions. The idea is based on a Poisson-type transform modulating the dependence structure in the model. An appealing feature of our approach is the possibility to control the underlying transition probabilities and, therefore, incorporate them within standard estimation methods. Given the resulting representation of the transition density, a Gibbs sampler algorithm based on the slice method is proposed and implemented. In the discrete-time case, special attention is placed to the class of generalized inverse Gaussian distributions. In the continuous case, we first provide a brief treatment of the class of gamma distributions, and then extend it to cover other invariant distributions, such as the generalized extreme value class. The proposed approach and estimation algorithm are illustrated with real financial datasets. Supplementary materials for this article are available online.

On the number of common factors with high-frequency data

SummaryIn this paper, we introduce a local principal component analysis approach to determining the number of common factors of a continuous-time factor model with time-varying factor loadings using high-frequency data. The model is approximated locally on shrinking blocks using discrete-time factor models. The number of common factors is estimated by minimizing the penalized aggregated mean squared residual error over all shrinking blocks. While the local mean squared residual error on each block converges at rate $\min(n^{1/4}, p)$, where $n$ is the sample size and $p$ is the dimension, the aggregated mean squared residual error converges at rate $\min(n^{1/2}, p)$; this achieves the convergence rate of the penalized criterion function of the global principal component analysis method, assuming restrictive constant factor loading. An estimator of the number of factors based on the local principal component analysis is consistent. Simulation results justify the performance of our estimator. A real financial dataset is analysed.

Network reconstruction via density sampling

Reconstructing weighted networks from partial information is necessary in many important circumstances, e.g. for a correct estimation of systemic risk. It has been shown that, in order to achieve an accurate reconstruction, it is crucial to reliably replicate the empirical degree sequence, which is however unknown in many realistic situations. More recently, it has been found that the knowledge of the degree sequence can be replaced by the knowledge of the strength sequence, which is typically accessible, complemented by that of the total number of links, thus considerably relaxing the observational requirements. Here we further relax these requirements and devise a procedure valid when even the the total number of links is unavailable. We assume that, apart from the heterogeneity induced by the degree sequence itself, the network is homogeneous, so that its (global) link density can be estimated by sampling subsets of nodes with representative density. We show that the best way of sampling nodes is the random selection scheme, any other procedure being biased towards unrealistically large, or small, link densities. We then introduce our core technique for reconstructing both the topology and the link weights of the unknown network in detail. When tested on real economic and financial data sets, our method achieves a remarkable accuracy and is very robust with respect to the sampled subsets, thus representing a reliable practical tool whenever the available topological information is restricted to small portions of nodes.

On weighted and locally polynomial directional quantile regression

The article deals with certain quantile regression methods for vector responses. In particular, it describes weighted and locally polynomial extensions to the projectional quantile regression, discusses their properties, addresses their computational side, compares their outcome with recent analogous generalizations of the competing multiple-output directional quantile regression, demonstrates a link between the two competing methodologies, complements the results already available in the literature, illustrates the concepts with a few simulated and insightful examples illustrating some of their features, and shows their application to a real financial data set, namely to Forex 1M exchange rates. The real-data example strongly indicates that the presented methods might have a huge impact on the analysis of multivariate time series consisting of two to four dimensional observations.

Enhanced indexing for risk averse investors using relaxed second order stochastic dominance

The less efficient markets offer scope for enhanced indexing (EI), an investment strategy of portfolio selection which seeks to earn more return than the benchmark index. In this context, we examine the use of relaxed second order stochastic dominance (RSSD) by introducing underachievement and overachievement variables in the second order stochastic dominance (SSD), for EI. We propose a linear optimization model that maximizes the mean return subject to the constraints formed using RSSD. We impose bounds on the ratio of the total underachievement to the sum of total underachievement and total overachievement variables depicting the risk-return tradeoff in the model. The proposed model for EI is inspired from many applications of SSD and almost SSD (ASSD). We examine the performance of the proposed model with the SSD model, EI model of maximizing mean return and minimizing the underperformance (MM) from the benchmark index, \(\epsilon \)-almost second order stochastic dominance (\(\epsilon \)-ASSD) model, and the naive 1/N portfolio, on two Indian stock indices, CNX 100 and CNX 200, through a rolling window strategy. To widen the empirical analysis, we also compare all models on the eight publicly available real financial data sets from Beasley OR library through a single window strategy. The portfolios from the proposed model are shown to produce statistically significant mean excess return and excess Sharpe ratio (both from the benchmark indices) more often than the MM and \(\epsilon \)-ASSD models. Also, the portfolios from the proposed model always have smaller violation area in SSD constraints from benchmark indices than the MM and \(\epsilon \)-ASSD models.

Lack of Fit Test for Infinite Variation Jumps at High Frequencies

In this paper, we are concerned with testing for infinite variation jumps in addition to a continuous local martingale component driven by Brownian motion using high-frequency data. We developed a lack of fit type test based on the empirical distribution of the devolatized increments. Under the null that the jump component is of finite variation, the empirical process associated with the devolatized increments converges to a Gaussian process in the Skorohod topology. Under the alternative hypothesis that the jumps are of infinite variation, that empirical process explodes instead. Theoretical results show good performance of the size and power. Simulation studies justify the theory. A real financial data set is analyzed.