Abstract
We present a reinforcement learning (RL) model that is based on Q-learning for the autonomous control of ship auxiliary power networks. The development and application of the proposed model is demonstrated using a case-study ship as the platform. The auxiliary power network of the ship is represented as a Markov Decision Process (MDP). Q-learning is then used to teach an agent to operate in this MDP by choosing actions in each operating state which would minimize fuel consumption while also respecting the boundary conditions of the network. The presented work is based on an extensive data set received from one of the cruise-line operators on the Baltic Sea. This data set was preprocessed to extract information for the state representation of the auxiliary network, which was used for training and validating the model. As a result, it is shown that the developed method produces an autonomous control policy for the auxiliary power network that outperforms the current human operated manual control of the case-study ship. An average of 0.9 % fuel oil savings are attained over the analyzed round-trips with control that displayed similar robustness against blackouts as the current operation of the ship. This amounts to 32 tons of fuel oil saved annually. In addition, it is shown that the developed model can be reconfigured for different levels of robustness, depending on the preferred trade-off between maintained reserve power and fuel savings.
Highlights
Increasing energy efficiency and autonomous operation are currently two major trends in the shipping industry
Real operational data was gathered from the automation system of a passenger vessel and used to train the reinforcement learning (RL) model which was based on Q-Learning
The focus of the work was on modelling the auxiliary power network of the vessel as a Markov Decision Process (MDP), respecting the real-life restrictions of the network, such as the need to retain a certain amount of reserve power
Summary
Increasing energy efficiency and autonomous operation are currently two major trends in the shipping industry. The Paris Agreement does not apply to the shipping sector, the International Maritime Organization has set its own strategy on the best ways to reach similar goals [2]. Reaching these goals depends on the development of various measures that would reduce the amount of air pollutants generated by ships. The transition between states occurs through actions, one time-step increment at a time. It can be defined as a 4-tuple (S, A, Pt , Rt ) where
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
