Relatedness in the era of machine learning

Abstract

Relatedness is a quantification of how much two human activities are similar in terms of the inputs and contexts needed for their development. Under the idea that it is easier to move between related activities than towards unrelated ones, empirical approaches to quantify relatedness are currently used as predictive tools to inform policies and development strategies in governments, international organizations, and firms. Here we show that the standard, widespread approach of estimating Relatedness through the co-location of activities (e.g. Product Space) generates a measure of relatedness that performs worse than trivial auto-correlation prediction strategies. In this paper, working on data about countries’ trade, technologies, and scientific production, we show two main findings. First, we find that a shift from two-product correlations (network-density based) to many-product correlations (decision trees) can dramatically improve the quality of forecasts, allowing the possibility to assist policymakers in optimizing decisions to promote growth. Then, we propose a new methodology to empirically estimate Relatedness that we call Continuous Projection Space (CPS). CPS, which represents a general network embedding technique, vastly outperforms all the co-location, network-based approaches, while retaining similar interpretability in terms of pairwise distances. Depending on the dataset the best approach is always either CPS or machine learning algorithms based on decision trees.