MURE: Multi-layer real-time livestock management architecture with unmanned aerial vehicles using deep reinforcement learning

Abstract

In recent years, the combination of unmanned aerial vehicles (UAVs) and wireless sensor networks (WSNs) has gained popularity in livestock management (LM) due to energy constraints and network instability. Limited energy storage of sensor nodes (SNs) and the possibility of packet loss contribute to fast energy consumption and unstable networks, respectively. UAVs serve as relay nodes and data sinks, addressing these issues by temporarily storing data to reduce SN workload and establishing mobile nodes for network stability. We propose two innovations based on previous work: 1) We introduce a multi-layer wireless network architecture, categorizing UAVs into two layers based on their functions including data collection and data processing. This enhances task parallelization, bridging performance gaps among multiple UAVs; 2) We overcome the mobility limitation of SNs, considering their real-time movement in the network. Through deep reinforcement learning, UAVs learn to cooperatively locate moving SNs. This accounts for the inevitable mobility of livestock in the industry. Additionally, we simulate the environment and compare our approach to traditional methods, evaluating metrics such as collected data per timestep (DCPS), energy consumed per timestep (ECPS), and network stability (NS). Experimental results demonstrate that our method outperforms traditional approaches, achieving a data collecting gain of 4.84% and 8.20% compared to the methods without considering SN mobility or the multi-layer characteristics of WSNs, respectively. Under energy consumption limits, our method yields energy savings of 3.00% and 1.35% respectively. Furthermore, we extensively study and validate our method against other path planning algorithms, including genetic particle swarm optimization (GPSO), modified central force optimization (MCFO), and rapidly-exploring random trees (RRT). Our approach surpasses these methods in terms of data collecting efficiency and network stability.