“WaterMan”: an on-farm water management game with heuristic capabilities

Abstract

A modern computer-based simulation tool (WaterMan) in the form of a game for on-farm water management was developed for application in training events for farmers, students, and irrigators. The WaterMan game utilizes an interactive framework, thereby allowing the user to develop scenarios and test alternatives in a convenient, risk-free environment. It includes a comprehensive soil water and salt balance calculation algorithm. It also employs heuristic capabilities for modeling all of the important aspects of on-farm water management, and to provide quantitative performance evaluations and practical water management advice to the trainees. Random events (both favorable and unfavorable) and different strategic decisions are included in the game for more realism and to provide an appropriate level of challenge according to player performance. Thus, the ability to anticipate the player skill level, and to reply with random events appropriate to the anticipated level, is provided by the heuristic capabilities used in the software. These heuristic features were developed based on a combination of two artificial intelligence approaches: (1) a pattern recognition approach and (2) reinforcement learning based on a Markov decision processes approach, specifically the Q-learning method. These two approaches were combined in a new way to account for the difference in the effect of actions taken by the player and action taken by the system in the game world. The reward function for the Q-learning method was modified to reflect the suggested classification of the WaterMan game as what is referred to as a partially competitive and partially cooperative game.