High-dimensional Portfolios Research Articles

Teaching–Learning-Based Optimization Algorithm with Stochastic Crossover Self-Learning and Blended Learning Model and Its Application

This paper presents a novel variant of the teaching–learning-based optimization algorithm, termed BLTLBO, which draws inspiration from the blended learning model, specifically designed to tackle high-dimensional multimodal complex optimization problems. Firstly, the perturbation conditions in the “teaching” and “learning” stages of the original TLBO algorithm are interpreted geometrically, based on which the search capability of the TLBO is enhanced by adjusting the range of values of random numbers. Second, a strategic restructuring has been ingeniously implemented, dividing the algorithm into three distinct phases: pre-course self-study, classroom blended learning, and post-course consolidation; this structural reorganization and the random crossover strategy in the self-learning phase effectively enhance the global optimization capability of TLBO. To evaluate its performance, the BLTLBO algorithm was tested alongside seven distinguished variants of the TLBO algorithm on thirteen multimodal functions from the CEC2014 suite. Furthermore, two excellent high-dimensional optimization algorithms were added to the comparison algorithm and tested in high-dimensional mode on five scalable multimodal functions from the CEC2008 suite. The empirical results illustrate the BLTLBO algorithm’s superior efficacy in handling high-dimensional multimodal challenges. Finally, a high-dimensional portfolio optimization problem was successfully addressed using the BLTLBO algorithm, thereby validating the practicality and effectiveness of the proposed method.

Random Matrix Theory and Nested Clustered Optimization on high-dimensional portfolios

This work aims to address the optimal allocation instability problem of Markowitz’s modern portfolio theory in high dimensionality. We propose a combined strategy that considers covariance matrix estimators from Random Matrix Theory (RMT) and the machine learning allocation methodology known as Nested Clustered Optimization (NCO). The latter methodology is modified and reformulated in terms of the spectral clustering algorithm and Minimum Spanning Tree (MST) to solve internal problems inherent in the original proposal. Markowitz’s classical mean-variance allocation and the modified NCO machine learning approach are tested on the financial instruments that compose the S&P 500 index listed on the New York Stock Exchange (NYSE) from 2012 to 2021. The modified NCO algorithm achieves stable allocations by incorporating RMT covariance estimators. A particular combination of covariance estimators, known as two-step estimators involving a hierarchical clustering technique, achieves the best portfolio performance. Surprisingly, a common pattern suggests that the improvement in performance is agnostic to the allocation strategy and, in fact, relies heavily on the covariance estimator selection. Our results suggest that the amount of leverage can be drastically reduced in investment strategies through RMT inference and statistical learning techniques.

Dynamic factor copula-based modeling for market risk optimization with an application to the real industry in China

As finance returns to its fundamental purpose of serving the real economy, its connections with various industries are strengthening. Accurately depicting the interdependence among these industries and mitigating financial risks has become increasingly critical. The dependence among China's real industries is dynamic rather than static, which is particularly pronounced during the COVID-19 pandemic. In this paper, we propose a dynamic factor model to optimize the risk of high-dimensional portfolios. To describe the dependence structure, we employ the factor copula model, driven by a GAS (Generalized Autoregressive Score) model. By combining the dynamic factor model with a mean-ES (Expected Shortfall) model, we construct a dynamic factor copula-mean-ES model. Our empirical findings, based on an analysis of 24 industries in China, suggest that the dynamic heterogeneous factor copula model is the most suitable for describing portfolio risk. Furthermore, the mean-ES model ensures the lowest portfolio risk for a given expected return. Accurate return predictions enable leveraging market information to develop a "good knowledge" of dynamic copula and risk optimization. This "good knowledge" of dynamic copulas facilitates precise return prediction and effective risk optimization of portfolios, thereby addressing the relationship between risk prevention and sustainability. Moreover, it reveals the internal connection between China's real industry and the risk landscape of the financial market.

Large portfolio allocation based on high-dimensional regression and Kendall’s Tau

In financial investments, portfolio allocation is always one of the most fundamental and challenging tasks. This paper proposes a robust portfolio optimization approach, extending the application of classical mean-variance (M-VAR) method for high-dimensional situations. The yielded assets return of the proposed method can be enhanced to some extent. It is called Kendall’s tau unconstrained shrinkage regression for M-VAR method (KUSR-MV). By some representative empirical studies, it is shown to have a more robust estimation of high-dimensional portfolio allocation compared to its competitors. Besides, its Sharpe ratio can be improved while the risk constraint can be well remained.

Is the empirical out-of-sample variance an informative risk measure for the high-dimensional portfolios?

The main contribution of this paper is the derivation of the asymptotic behavior of the out-of-sample variance, the out-of-sample relative loss, and of their empirical counterparts in the high-dimensional setting, i.e., when both ratios p/n and p/m tend to some positive constants as m→∞ and n→∞, where p is the portfolio dimension, while n and m are the sample sizes from the in-sample and out-of-sample periods, respectively. The results are obtained for the traditional estimator of the global minimum variance (GMV) portfolio and for the two shrinkage estimators introduced by Frahm and Memmel (2010) and Bodnar et al. (2018). We show that the behavior of the empirical out-of-sample variance may be misleading in many practical situations, leading, for example, to a comparison of zeros. On the other hand, this will never happen with the empirical out-of-sample relative loss, which seems to provide a natural normalization of the out-of-sample variance in the high-dimensional setup. As a result, an important question arises if the out-of-sample variance can safely be used in practice for portfolios constructed from a large asset universe.

Bayesian Estimation and Optimization for Learning Sequential Regularized Portfolios

This paper incorporates Bayesian estimation and optimization into a portfolio selection framework, particularly for high-dimensional portfolios in which the number of assets is larger than the number of observations. We leverage a constrained minimization approach, called the linear programming optimal (LPO) portfolio, to directly estimate effective parameters appearing in the optimal portfolio. We propose two refinements for the LPO strategy. First, we explore improved Bayesian estimates, instead of sample estimates, of the covariance matrix of asset returns. Second, we introduce Bayesian optimization (BO) to replace traditional grid-search cross-validation (CV) in tuning hyperparameters of the LPO strategy. We further propose modifications in the BO algorithm by (1) taking into account the time-dependent nature of financial problems and (2) extending the commonly used expected improvement acquisition function to include a tunable trade-off with the improvement’s variance. Allowing a general case of noisy observations, we theoretically derive the sublinear convergence rate of BO under the newly proposed EIVar and thus our algorithm has no regret. Our empirical studies confirm that the adjusted BO results in portfolios with higher out-of-sample Sharpe ratio, certainty equivalent, and lower turnover compared to those tuned with CV. This superior performance is achieved with a significant reduction in time elapsed, thus also addressing time-consuming issues of CV. Furthermore, LPO with Bayesian estimates outperforms the original proposal of LPO, as well as the benchmark equally weighted and plugin strategies.

Shrinkage estimation with reinforcement learning of large variance matrices for portfolio selection

A large amount of assets characterizes high-dimensional portfolio selection problems compared to temporal observation. In such a high-dimensional framework, the asset allocation is unfeasible because the covariance matrix obtained with the usual sample estimators cannot be inverted. This paper proposes a new shrinkage estimator based on reinforcement learning for large covariance matrices that is optimal in the context of portfolio selection. The resulting estimator is entirely data-driven since the optimal shrinkage intensity is given by optimizing neural network weights. This paper presents two different architectures: a standard fully connected network for a classical Policy Gradient Agent (PGA) and a Gated Recurrent Unit for a Recurrent Policy Gradient Agent (RPGA). To show the validity of the proposal, an application to asset allocation with Industry portfolios is provided. The results indicate that the RPGA-based approach in shrinkage estimation provides the best performance in out-of-sample comparison.

High-Dimensional Portfolio Selection with Cardinality Constraints

The expanding number of assets offers more opportunities for investors but poses new challenges for modern portfolio management (PM). As a central plank of PM, portfolio selection by expected utility maximization (EUM) faces uncontrollable estimation and optimization errors in ultrahigh-dimensional scenarios. Past strategies for high-dimensional PM mainly concern only large-cap companies and select many stocks, making PM impractical. We propose a sample-average-approximation-based portfolio strategy to tackle the difficulties above with cardinality constraints. Our strategy bypasses the estimation of mean and covariance, the Chinese walls in high-dimensional scenarios. Empirical results on S&P 500 and Russell 2000 show that an appropriate number of carefully chosen assets leads to better out-of-sample mean-variance efficiency. On Russell 2000, our best portfolio profits as much as the equally weighted portfolio but reduces the maximum drawdown and the average number of assets by 10% and 90%, respectively. The flexibility and the stability of incorporating factor signals for augmenting out-of-sample performances are also demonstrated. Our strategy balances the tradeoff among the return, the risk, and the number of assets with cardinality constraints. Therefore, we provide a theoretically sound and computationally efficient strategy to make PM practical in the growing global financial market. Supplementary materials for this article are available online.

Asset splitting algorithm for ultrahigh dimensional portfolio selection and its theoretical property

The presence of a huge number of assets poses challenges to classical portfolio selection algorithms. Constrained l1 minimization approaches have been proposed to directly estimate effective parameters in the optimal portfolio. Linear programming method and alternating direction method of multiplier (ADMM) algorithm is used to solve the corresponding minimization problems. However, these two algorithms may fail due to the limitations of computing time and computing memory when a huge number of assets are considered in the portfolio optimization. This article proposes an asset splitting ADMM (AS-ADMM for short), a parallel computing algorithm, to tackle such challenges, and establishes the convergence property of the new algorithm. Furthermore, we develop a new regularization method for estimating the effective parameters with the folded-concave penalty and establish its oracle property. The local linear approximation (LLA) algorithm is used to redirect the new method to a weighted l1 regularization method. We conduct simulation studies to investigate the advantage of the proposed algorithm and regularized model in solving the high dimensional portfolio selection problems. A real data example is also included to demonstrate the applicability of the proposed algorithms and regularization methods.

Smart network based portfolios

In this article we deal with the problem of portfolio allocation by enhancing network theory tools. We propose the use of the correlation network dependence structure in constructing some well-known risk-based models in which the estimation of the correlation matrix is a building block in the portfolio optimization. We formulate and solve all these portfolio allocation problems using both the standard approach and the network-based approach. Moreover, in constructing the network-based portfolios we propose the use of three different estimators for the covariance matrix: the sample, the shrinkage toward constant correlation and the depth-based estimators . All the strategies under analysis are implemented on three high-dimensional portfolios having different characteristics. We find that the network-based portfolio consistently performs better and has lower risk compared to the corresponding standard portfolio in an out-of-sample perspective.

High-Dimensional Portfolio Optimization Based on Tree-Structured Factor Model

High-Dimensional Portfolio Optimization Based on Tree-Structured Factor Model

Optimal Shrinkage-Based Portfolio Selection in High Dimensions

ABSTRACT In this article, we estimate the mean-variance portfolio in the high-dimensional case using the recent results from the theory of random matrices. We construct a linear shrinkage estimator which is distribution-free and is optimal in the sense of maximizing with probability 1 the asymptotic out-of-sample expected utility, that is, mean-variance objective function for different values of risk aversion coefficient which in particular leads to the maximization of the out-of-sample expected utility and to the minimization of the out-of-sample variance. One of the main features of our estimator is the inclusion of the estimation risk related to the sample mean vector into the high-dimensional portfolio optimization. The asymptotic properties of the new estimator are investigated when the number of assets p and the sample size n tend simultaneously to infinity such that . The results are obtained under weak assumptions imposed on the distribution of the asset returns, namely the existence of the moments is only required. Thereafter we perform numerical and empirical studies where the small- and large-sample behavior of the derived estimator is investigated. The suggested estimator shows significant improvements over the existent approaches including the nonlinear shrinkage estimator and the three-fund portfolio rule, especially when the portfolio dimension is larger than the sample size. Moreover, it is robust to deviations from normality.

Correction to: Solving High-Dimensional Dynamic Portfolio Choice Models with Hierarchical B-Splines on Sparse Grids

The original publication has been updated. In the original publication of this article, under the Introduction heading section, the corrections to the second paragraph’s inline equation were not incorporated. The author’s additional corrections have also been incorporated. The publisher apologizes for the error made during production.

Bayesian Estimation and Optimization for Learning Sequential Regularized Portfolios

This paper incorporates Bayesian estimation and optimization into portfolio selection framework, particularly for high-dimensional portfolio in which the number of assets is larger than the number of observations. We leverage a constrained 𝓁1 minimization approach, called linear programming optimal (LPO) portfolio, to directly estimate effective parameters appearing in the optimal portfolio. We propose two refinements for the LPO strategy. First, we explore improved Bayesian estimates, instead of sample estimates, of the covariance matrix of asset returns. Second, we introduce Bayesian optimization (BO) to replace traditional grid-search cross-validation (CV) in tuning hyperparameters of the LPO strategy. We further propose modifications in the BO algorithm by (1) taking into account time-dependent nature of financial problems and (2) extending commonly used expected improvement (EI) acquisition function to include a tunable trade-off with the improvement's variance (EIVar). Allowing a general case of noisy observations, we theoretically derive the sub-linear convergence rate of BO under the newly proposed EIVar and thus our algorithm has no regret. Our empirical studies confirm that the adjusted BO result in portfolios with higher out-of-sample Sharpe ratio, certainty equivalent, and lower turnover compared to those tuned with CV. This superior performance is achieved with significant reduction in time elapsed, thus also addressing time-consuming issues of CV. Furthermore, LPO with Bayesian estimates outperform original proposal of LPO, as well as the benchmark equally weighted and plug-in strategies.

Deep-Learning Solution to Portfolio Selection with Serially-Dependent Returns

This paper investigates a deep-learning solution to high-dimensional multi-period portfolio optimization problems with bounding constraints on the control. We propose a deep neural network (DNN) architecture to describe the underlying control process. The DNN consists of $K$ subnetworks, where $K$ is the total number of decision steps. The feedback control function is determined solely by the network parameters. In this way, the multi-period portfolio optimization problem is linked to a training problem of the DNN, that can be efficiently computed by the standard optimization techniques for network training. We offer a sufficient condition for the algorithm to converge for a general utility function and general asset return dynamics including serially-dependent returns. Specifically, under the condition that the global minimum of the DNN training problem is attained, we prove that the algorithm converges with the quadratic utility function when the risky asset returns jointly follow multivariate AR(1) models and/or multivariate GARCH(1,1) models. Numerical examples demonstrate the superior performance of the DNN algorithm in various return dynamics for a high-dimensional portfolio (up to 100 dimensions).

G-Learner and GIRL: Goal Based Wealth Management with Reinforcement Learning

We present a reinforcement learning approach to goal based wealth management problems such as optimization of retirement plans or target dated funds. In such problems, an investor seeks to achieve a financial goal by making periodic investments in the portfolio while being employed, and periodically draws from the account when in retirement, in addition to the ability to re-balance the portfolio by selling and buying different assets (e.g. stocks). Instead of relying on a utility of consumption, we present G-Learner: a reinforcement learning algorithm that operates with explicitly defined one-step rewards, does not assume a data generation process, and is suitable for noisy data. Our approach is based on G-learning (Fox et al., 2015) - a probabilistic extension of the Q-learning method of reinforcement learning. In this paper, we demonstrate how G-learning, when applied to a quadratic reward and Gaussian reference policy, gives an entropy-regulated Linear Quadratic Regulator (LQR). This critical insight provides a novel and computationally tractable tool for wealth management tasks which scales to high dimensional portfolios. In addition to the solution of the direct problem of G-learning, we also present a new algorithm, GIRL, that extends our goal-based G-learning approach to the setting of Inverse Reinforcement Learning (IRL) where rewards collected by the agent are not observed, and should instead be inferred. We demonstrate that GIRL can successfully learn the reward parameters of a G-Learner agent and thus imitate its behavior. Finally, we discuss potential applications of the G-Learner and GIRL algorithms for wealth management and robo-advising.

Multiplicative Conditional Correlation Models for Realized Covariance Matrices

We introduce a class of multiplicative dynamic models for realized covariance matrices assumed to be conditionally Wishart distributed. The multiplicative structure enables consistent three-step estimation of the parameters, starting by covariance targeting of a scale matrix. The dynamics of conditional variances and correlations are inspired by specifications akin to the consistent dynamic conditional correlation model of the multivariate GARCH literature, and estimation is performed by quasi maximum likelihood. Simulations show that in finite samples the three-step estimator has smaller bias and root mean squared error than the full estimator when the cross-sectional dimension increases. An empirical application illustrates the flexibility of these models in a low-dimensional setting, and another one illustrates their effectiveness and practical usefulness in high dimensional portfolio allocation strategies.

A Nodewise Regression Approach to Estimating Large Portfolios

This article investigates the large sample properties of the variance, weights, and risk of high-dimensional portfolios where the inverse of the covariance matrix of excess asset returns is estimated using a technique called nodewise regression. Nodewise regression provides a direct estimator for the inverse covariance matrix using the least absolute shrinkage and selection operator to estimate the entries of a sparse precision matrix. We show that the variance, weights, and risk of the global minimum variance portfolios and the Markowitz mean-variance portfolios are consistently estimated with more assets than observations. We show, empirically, that the nodewise regression-based approach performs well in comparison to factor models and shrinkage methods. Supplementary materials for this article are available online.

Lasso-based simulation for high-dimensional multi-period portfolio optimization

AbstractThis paper proposes a regression-based simulation algorithm for multi-period mean-variance portfolio optimization problems with constraints under a high-dimensional setting. For a high-dimensional portfolio, the least squares Monte Carlo algorithm for portfolio optimization can perform less satisfactorily with finite sample paths due to the estimation error from the ordinary least squares (OLS) in the regression steps. Our algorithm, which resolves this problem e, that demonstrates significant improvements in numerical performance for the case of finite sample path and high dimensionality. Specifically, we replace the OLS by the least absolute shrinkage and selection operator (lasso). Our major contribution is the proof of the asymptotic convergence of the novel lasso-based simulation in a recursive regression setting. Numerical experiments suggest that our algorithm achieves good stability in both low- and higher-dimensional cases.

LASSO-Based Simulation for High-Dimensional Multi-Period Portfolio Optimization

This paper proposes a regression-based simulation algorithm for multi-period mean-variance portfolio (MVP) optimization problems with constraints under a high-dimensional setting. We show that the least-squares Monte Carlo (LSMC) algorithm for portfolio optimization is either unstable or degenerates in a high-dimensional portfolio. The problem stems from the estimation error of the ordinary least-squares (OLS) in the regression steps. By introducing l1-regularization technique, we greatly extend the LSMC algorithm to a high-dimensional setting. Specifically, we propose to replace the OLS by the least absolute shrinkage and selection operator (LASSO). We prove the asymptotic convergence of the novel LASSO-based simulation under such a recursive regression setting. Numerical experiments suggest our algorithm achieves great stability under both low- and higher-dimensional cases.