Real-world Financial Dataset Research Articles

Adaptive location and scale estimation with kernel-weighted averages

A wide variety of location and scale estimators have been developed for light-tailed distributions. Despite indisputable importance in business, finance, cybersecurity, etc., statistical estimation and inference in the presence of heavy tails have received less attention in the literature. We adopt the Kernel-Weighted Average (KWA) approach to location and scale estimation and present a set of extensive comparisons with five prominent competitors. Unlike nonparametric kernel density estimation, the optimally tuned bandwidth for KWA estimators does not necessarily converge to zero as sample size grows. We also perform a large-scale Monte Carlo simulation to search for the optimal bandwidth that minimizes the mean squared error (MSE) of KWA location and scale estimators with simulated samples from Student’s t-distribution with degrees of freedom (df) 1 , 2 , … , 30 . We further develop an adaptive technique to estimate the df that best match the observed samples using Cramér-von Mises test of goodness-of-fit. Unlike many existing methodologies, our approach is data-driven and exhibits excellent statistical performance. To illustrate this, we apply it to three real-world financial datasets containing daily closing prices of AMC Entertainment (AMC), GameStop (GME) and Meta Platforms (META) stocks to calibrate a geometric random walk model with Student’s t log-increments.

Managing ESG ratings disagreement in sustainable portfolio selection

In sustainable portfolio selection, investors have a twofold objective: they want to achieve the best compromise between portfolio risk and return, but they also want to take into account the sustainability of their investment, assessed through some Environmental, Social, and Governance (ESG) evaluation criteria. The inclusion of sustainable goals in the portfolio selection process may have an impact on the portfolio financial performance. ESG scores provided by the rating agencies are generally considered good proxies for the sustainability performance of an investment, as well as appropriate measures for Socially Responsible Investments (SRI). In this framework, the lack of alignment between ratings provided by different agencies is a crucial issue that inevitably undermines the robustness and reliability of these evaluation measures. In fact, the ESG rating disagreement may produce conflicting information, implying difficulty for the investors in the ESG evaluation of their portfolios. This may cause underestimation or overestimation of the market opportunities for a sustainable investment.In this paper, we deal with a multi-criteria portfolio selection problem, taking into account risk, return, and ESG criteria. For the ESG evaluation of the securities, we consider more than one agency and propose a new approach to overcome the problem related to the disagreement between the ESG ratings given by different agencies. For our three-criteria portfolio selection problem, we present an optimization model which adopts the so-called k−sum operator to formulate a concise ESG evaluation measure. The natural formulation of the model, in which the portfolio risk is measured by the variance of its returns, leads to a nonlinear program, but we show that it can be reformulated as an equivalent convex quadratic model. We also show that the model can be generalized to include any convex portfolio risk measures. An extensive empirical analysis of the out-of-sample performance of this model is provided on real-world financial data sets. From a theoretical viewpoint, our work extends the use of k−sum methodology to quadratic programming.

FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design

Recent advancements in Large Language Models (LLMs) have exhibited notable efficacy in question-answering (QA) tasks across diverse domains. Their prowess in integrating extensive web knowledge has fueled interest in developing LLM-based autonomous agents. While LLMs are efficient in decoding human instructions and deriving solutions by holistically processing historical inputs, transitioning to purpose-driven agents requires a supplementary rational architecture to process multi-source information, establish reasoning chains, and prioritize critical tasks. Addressing this, we introduce FinMem, a novel LLM-based agent framework devised for financial decision-making. It encompasses three core modules: Profiling, to customize the agent's characteristics; Memory, with layered message processing, to aid the agent in assimilating hierarchical financial data; and Decision-making, to convert insights gained from memories into investment decisions. Notably, FinMem's memory module aligns closely with the cognitive structure of human traders, offering robust interpretability and real-time tuning. Its adjustable cognitive span allows for the retention of critical information beyond human perceptual limits, thereby enhancing trading outcomes. This framework enables the agent to self-evolve its professional knowledge, react agilely to new investment cues, and continuously refine trading decisions in the volatile financial environment. We first compare FinMem with various algorithmic agents on a scalable real-world financial dataset, underscoring its leading trading performance in stocks. We then fine-tuned the agent's perceptual span and character setting to achieve a significantly enhanced trading performance. Collectively, FinMem presents a cutting-edge LLM agent framework for automated trading, boosting cumulative investment returns.

Learning sub-patterns in piecewise continuous functions

Most stochastic gradient descent algorithms can optimize neural networks that are sub-differentiable in their parameters; however, this implies that the neural network’s activation function must exhibit a degree of continuity which limits the neural network model’s uniform approximation capacity to continuous functions. This paper focuses on the case where the discontinuities arise from distinct sub-patterns, each defined on different parts of the input space. We propose a new discontinuous deep neural network model trainable via a decoupled two-step procedure that avoids passing gradient updates through the network’s only and strategically placed, discontinuous unit. We provide approximation guarantees for our architecture in the space of bounded continuous functions and universal approximation guarantees in the space of piecewise continuous functions which we introduced herein. We present a novel semi-supervised two-step training procedure for our discontinuous deep learning model, tailored to its structure, and we provide theoretical support for its effectiveness. The performance of our model and trained with the propose procedure is evaluated experimentally on both real-world financial datasets and synthetic datasets.

Spatiotemporal blocking of the bouncy particle sampler for efficient inference in state-space models

We propose a novel blocked version of the continuous-time bouncy particle sampler of Bouchard-Côté et al. (J Am Stat Assoc 113(522):855–867, 2018) which is applicable to any differentiable probability density. This alternative implementation is motivated by blocked Gibbs sampling for state-space models (Singh et al. in Biometrika 104(4):953–969, 2017) and leads to significant improvement in terms of effective sample size per second, and furthermore, allows for significant parallelization of the resulting algorithm. The new algorithms are particularly efficient for latent state inference in high-dimensional state-space models, where blocking in both space and time is necessary to avoid degeneracy of MCMC. The efficiency of our blocked bouncy particle sampler, in comparison with both the standard implementation of the bouncy particle sampler and the particle Gibbs algorithm of Andrieu et al. (J R Stat Soc Ser B Stat Methodol 72(3):269–342, 2010), is illustrated numerically for both simulated data and a challenging real-world financial dataset.

Quantum versus classical generative modelling in finance

Finding a concrete use case for quantum computers in the near term is still an open question, with machine learning typically touted as one of the first fields which will be impacted by quantum technologies. In this work, we investigate and compare the capabilities of quantum versus classical models for the task of generative modelling in machine learning. We use a real world financial dataset consisting of correlated currency pairs and compare two models in their ability to learn the resulting distribution—a restricted Boltzmann machine, and a quantum circuit Born machine. We provide extensive numerical results indicating that the simulated Born machine always at least matches the performance of the Boltzmann machine in this task, and demonstrates superior performance as the model scales. We perform experiments on both simulated and physical quantum chips using the Rigetti QCSTM platform, and also are able to partially train the largest instance to date of a quantum circuit Born machine on quantum hardware. Finally, by studying the entanglement capacity of the training Born machines, we find that entanglement typically plays a role in the problem instances which demonstrate an advantage over the Boltzmann machine.

A2GAN: A Deep Reinforcement-Based Learning Algorithm for Risk-Aware in Finance

Financial data suffer from missing, unlabeled and unbalanced data, thus weakening the performance of decision-making systems. In addition, the aim of financial institutions is not only to find decision-making models that achieve high scores for the standard metrics (e.g., AUC, accuracy, F-score) but to reduce the risk from miss-classification cases. This paper addresses these problems by proposing a novel framework inspired by reinforcement learning, specifically actor-critic, for decision-making and implementing generative adversarial networks for imputing missing data, as well as utilizing the unlabeled dataset. Moreover, by taking advantage of reinforcement learning, the trained model is calibrated using a customizable reward function, which can be designed for different purposes of financial institutions. We evaluate the framework via real-world financial datasets that only have a small amount of labeled data and exhibit missing data. Our experiment shows promising results where the financial risk is dramatically reduced without too much sacrifice on standard metrics.

On Generalizations of Mean–Variance Model

Mean-variance analysis is a mathematical framework for assembling a portfolio of assets such that the expected return is maximized for a given level of risk. This paper is closely related to the mean-variance model, considers the variation of market parameters and the correspond influence on efficient frontier and generalizes the mean-variance model to separable utility function, then compares the efficient frontier with the original mean-variance model and tests the conclusions on real world financial dataset.

A new dynamic modeling framework for credit risk assessment

We propose a new dynamic modeling framework for credit risk assessment that extends the prevailing credit scoring models built upon historical data static settings. The driving idea mimics the principle of films, by composing the model with a sequence of snapshots, rather than a single photograph. In doing so, the dynamic modeling consists of sequential learning from the new incoming data. A key contribution is provided by the insight that different amounts of memory can be explored concurrently. Memory refers to the amount of historic data being used for estimation. This is important in the credit risk area, which often seems to undergo shocks. During a shock, limited memory is important. Other times, a larger memory has merit. An application to a real-world financial dataset of credit cards from a financial institution in Brazil illustrates our methodology, which is able to consistently outperform the static modeling schema.

A Compact Evolutionary Interval-Valued Fuzzy Rule-Based Classification System for the Modeling and Prediction of Real-World Financial Applications With Imbalanced Data

The current financial crisis has stressed the need to obtain more accurate prediction models in order to decrease risk when investing money on economic opportunities. In addition, the transparency of the process followed to make the decisions in financial applications is becoming an important issue. Furthermore, there is a need to handle real-world imbalanced financial datasets without using sampling techniques that might introduce noise in the used data. In this paper, we present a compact evolutionary interval-valued fuzzy rule-based classification system, which is based on interval-valued fuzzy rule-based classification system with tuning and rule selection (IVTURS $_{\rm FARC{\hbox{-}}HD}$ ) for the modeling and prediction of real-world financial applications. This proposed system allows obtaining good prediction accuracies using a small set of short fuzzy rules implying a high degree of interpretability of the generated linguistic model. Furthermore, the proposed system deals with the financial imbalanced datasets with no need for any preprocessing or sampling method and, thus, avoiding the accidental introduction of noise in the data used in the learning process. The system is also provided with a mechanism to handle examples that are not covered by any fuzzy rule in the generated rule base. To test the quality of our proposal, we will present an experimental study including 11 real-world financial datasets. We will show that the proposed system outperforms the original C4.5 decision tree, type-1, and interval-valued fuzzy counterparts that use the synthetic minority oversampling technique (SMOTE) to preprocess data and the original FURIA, which is a fuzzy approximative classifier. Furthermore, the proposed method enhances the results achieved by the cost-sensitive C4.5, and it gives competitive results when compared with FURIA using SMOTE, while our proposal avoids preprocessing techniques, and it provides interpretable models that allow obtaining more accurate results.

Selection of Support Vector Machines based classifiers for credit risk domain

This paper describes an approach for credit risk evaluation based on linear Support Vector Machines classifiers, combined with external evaluation and sliding window testing, with focus on application on larger datasets. It presents a technique for optimal linear SVM classifier selection based on particle swarm optimization technique, providing significant amount of focus on imbalanced learning issue. It is compared to other classifiers in terms of accuracy and identification of each class. Experimental classification performance results, obtained using real world financial dataset from SEC EDGAR database, lead to conclusion that proposed technique is capable to produce results, comparable to other classifiers, such as logistic regression and RBF network, and thus be can be an appealing option for future development of real credit risk evaluation models.

A linear risk-return model for enhanced indexation in portfolio optimization

Enhanced indexation (EI) is the problem of selecting a portfolio that should produce excess return with respect to a given benchmark index. In this work, we propose a linear bi-objective optimization approach to EI that maximizes average excess return and minimizes underperformance over a learning period. Our model can be efficiently solved to optimality by means of standard linear programming techniques. On the theoretical side, we investigate conditions that guarantee or forbid the existence of a portfolio strictly outperforming the index. On the practical side, we support our model with extensive empirical analysis on publicly available real-world financial datasets, including comparison with previous studies, performance and diversification analysis, and verification of some of the proposed theoretical results on real data.

A Linear Risk-Return Model for Enhanced Indexation

Enhanced Indexation is the problem of selecting a portfolio that should produce excess return with respect to a given benchmark index. In this work we propose a linear bi-objective optimization approach to Enhanced Indexation that maximizes average excess return and minimizes underperformance over a learning period. Our model can be efficiently solved to optimality by means of standard Linear Programming techniques. On the theoretical side, we investigate conditions that guarantee or forbid the existence of a portfolio strictly outperforming the index. We also support our model with extensive empirical analysis on publicly available real-world financial datasets, including comparison with previous studies, performance and diversification analysis, and verification of some of the proposed theoretical results on real data.

Decision trees for ordinal classification

In many classification problems the domains of the attributes and the classes are linearly ordered. For such problems the classification rule often needs to be order-preserving or monotonic as we call it. Since the known decision tree methods generate non-monotonic trees, these methods are not suitable for monotonic classification problems. We provide an order-preserving tree-generation algorithm for multi-attribute classification problems with $k$ linearly ordered classes, and an algorithm for repairing non-monotonic decision trees. The performance of these algorithms is tested on a real-world financial dataset and on random monotonic datasets.