A photogrammetry-based image registration method for multi-camera systems – With applications in images of a tree crop

Abstract

In precision agriculture, estimating crop yield using remote sensing techniques is an active research field. To achieve high accuracies, researchers frequently combined different imaging sources, such as colour (Red, Green, Blue [RGB]) images, thermal images, and near-infrared images. However, fusing information from those images has been a difficult task. Therefore, accurate image registration methods are necessary. This study aimed to develop a thermal-colour camera system which will register thermal images with colour images of tree canopies in preparation of information fusion and fruit detection. The registration method created in this study was based on photogrammetry. In preparation of registration, a camera system was built, consisting of a thermal camera and two colour cameras. Camera calibration, image intersection, and space resection were combined in a single step named ‘stereo-calibration’, to compute cameras' parameters and poses. Speeded-up robust features (SURF) were used to find points of interest from colour images. Random sample consensus (RANSAC) was utilised to search for optimal homography transforms between thermal and colour images. In addition, this study created a procedure for accurate registrations of regions of interest in thermal-colour image pairs, utilising structural similarity (SSIM) index. The proposed method offered pixel-level registration accuracy and achieved an average accuracy of 3 pixels in 640 × 480 – pixel citrus canopy images.