Registration of multispectral 3D points for plant inspection

Abstract

Machine vision technologies have shown advantages for efficient and accurate plant inspection in precision agriculture. Regarding the balance between accuracy of inspection and compactness for infield applications, multispectral imaging systems would be more suitable than RGB colour cameras or hyperspectral imaging systems. Multispectral image registration (MIR) is a key issue for multispectral imaging systems, however, this task is challenging. First of all, in many cases, two images needing registration do not have a one-to-one linear mapping in 2D space and therefore they cannot be aligned in 2D images. Furthermore, the general MIR algorithms are limited to images with uniform intensity and are incapable of registering images with rich features. This study developed a machine vision system (MVS) and a MIR method which replaces 2D-2D image registration by 3D-3D point cloud registration. The system can register 3D point clouds of ultraviolet (UV), blue, green, red and near-infrared (NIR) spectra in 3D space. It was found that the point clouds of general plants created by images of different spectral bands have a complementary property, and therefore a combined point cloud, called multispectral 3D point cloud, is denser than any cloud created by a single spectral band. Intensity information of each spectral band is available in a multispectral 3D point cloud and therefore image fusion and 3D morphological analysis can be conducted in the cloud. The MVS could be used as a sensor of a robotic system to fulfil on-the-go infield plant inspection tasks.