Mission Planning in Time-Invariant Domains with MDPs and Gaussian Distribution

Abstract

This thesis uses the Markov decision process (MDP) to investigate how the optimal path is decided by the underwater robot in the presence of different currents. The use of this MDP decision allows the machine to reduce the unwanted travel energy and travel time in different environments, enhancing the cost of using the machine. The scope of the study is a specific problem, and the paper describes how to use a python planning framework to solve path selection in the case where the direction and strength of the currents are determined, while incorporating a probability distribution of Gaussian functions in a finite space. The dynamical ocean current problem is used as a basis to use the MDP decision process. The final results show that the present study meets the hypotheses presented in the expectations. The results are able to simulate that the robot can find the optimal path and reach the target point when facing different currents underwater. By expanding the spatial extent of the observation and changing the parameter settings, it is possible to simulate the robot’s path in other situations. The research method can be extended from specificity to generality, and the same research method can be used to obtain optimal path decisions for other different situations.