Beating a Benchmark: Dynamic Programming May Not Be the Right Numerical Approach

Abstract

We analyze dynamic investment strategies for benchmark outperformance using two widely used objectives of practical interest to investors: (i) maximizing the information ratio (IR), and (ii) obtaining a favorable tracking difference (cumulative outperformance) relative to the benchmark. In the case of the tracking difference, we propose a simple and intuitive objective function based on the quadratic deviation (QD) from an elevated benchmark. In order to gain some intuition about these strategies, we provide closed-form solutions for the controls under idealized assumptions. For more realistic cases, we represent the control using a neural network (NN) and directly solve a sampled optimization problem, which approximates the original optimal stochastic control formulation. Unlike the typical approach based on dynamic programming (DP), e.g., reinforcement learning, solving the sampled optimization with an NN as a control avoids computing conditional expectations and leads to an optimization problem with a small number of variables. In addition, our NN parameter size is independent of the number of portfolio rebalancing times. Under some assumptions, we prove that a traditional DP approach results in a high-dimensional problem, whereas directly solving for the control without using DP yields a low-dimensional problem. Our analytical and numerical results illustrate that, compared with IR-optimal strategies with the same expected value of terminal wealth, the QD-optimal investment strategies result in comparatively more diversified asset allocations during certain periods of the investment time horizon.