Hardware‐in‐the‐loop simulation for visual target tracking of octorotor UAV

Abstract

PurposeThe purpose of this paper is to present the development of hardware‐in‐the‐loop simulation (HILS) for visual target tracking of an octorotor unmanned aerial vehicle (UAV) with onboard computer vision.Design/methodology/approachHILS for visual target tracking of an octorotor UAV is developed by integrating real embedded computer vision hardware and camera to software simulation of the UAV dynamics, flight control and navigation systems run on Simulink. Visualization of the visual target tracking is developed using FlightGear. The computer vision system is used to recognize and track a moving target using feature correlation between captured scene images and object images stored in the database. Features of the captured images are extracted using speed‐up robust feature (SURF) algorithm, and subsequently matched with features extracted from object image using fast library for approximate nearest neighbor (FLANN) algorithm. Kalman filter is applied to predict the position of the moving target on image plane. The integrated HILS environment is developed to allow real‐time testing and evaluation of onboard embedded computer vision for UAV's visual target tracking.FindingsUtilization of HILS is found to be useful in evaluating functionality and performance of the real machine vision software and hardware prior to its operation in a flight test. Integrating computer vision with UAV enables the construction of an unmanned system with the capability of tracking a moving object.Practical implicationsHILS for visual target tracking of UAV described in this paper could be applied in practice to minimize trial and error in various parameters tuning of the machine vision algorithm as well as of the autopilot and navigation system. It also could reduce development costs, in addition to reducing the risk of crashing the UAV in a flight test.Originality/valueA HILS integrated environment for octorotor UAV's visual target tracking for real‐time testing and evaluation of onboard computer vision is proposed. Another contribution involves implementation of SURF, FLANN, and Kalman filter algorithms on an onboard embedded PC and its integration with navigation and flight control systems which enables the UAV to track a moving object.