Deep hierarchical reinforcement learning to manage the trade-off between sustainability and profitability in common pool resources systems

Abstract

Natural resource management refers, in theory, to a sustainable utilization of fundamental natural resources. For the different stakeholders, there is a trade-off between profitability and a sustainable exploitation that allows ecosystems and resources to be preserved. These issues can be managed mathematically or with optimization via simulation, but it requires perfect knowledge and robust data to be valid, which often represents a lack in marine sciences, for various reasons (lack of sampling, local policies, difficulties in accessing quantitative data etc.). In this paper, we focus on deep hierarchical reinforcement learning based approach to help decision making in a common pool resource system. This approach allows us to i/ model decisionmakers behavior, ii/ determine sustainable exploitation quotas, iii/ model owners wishing to maximize their profitability on an individual scale, taking into account the uncertainties and partial knowledge concerning the resources, in a partially observable Markov game. By comparing two types of fishing gears, selective and non-selective, we show the limits of the trade-off between profitability and sustainability. Indeed, the latter depends on the amount of greedy behavior, the power of exploitation and the need for awareness and a stricter regulation of the exploitation of the most vulnerable resources.