A 3D white referencing method for soybean leaves based on fusion of hyperspectral images and 3D point clouds

Abstract

In recent years, plant phenotyping technologies have been widely applied to evaluate complex plant traits such as morphology, physiology, ecology, biochemistry, tolerance, growth and yield. Hyperspectral/multispectral cameras, artificial lighting sources, mechanisms and computers together capture images of different species of plants. Due to the non-uniform intensity of lighting sources in different wavelengths, raw images need to be calibrated using white references. Flat white panels are typically scanned as a white reference. However, geometrical factors such as leaf tilt angles cannot be calibrated by flat white references. In this publication, the effectiveness of using angled white reference to calibrate corresponding raw images was first demonstrated. Furthermore, a 3D white referencing library integrating different angles and spatial positions in the system of a hyperspectral camera and a Kinect V2 depth sensor was created. Thus, a pixel on the leaf surface can be calibrated by a point with the nearest tilt angle and spatial position in the 3D referencing library. The validating samples for this referencing library were soybean leaves grown in a greenhouse. The results showed that the reflectance spectra after 3D calibration were closer to the standard calibration (flat leaf calibrated by flat white reference) than the conventional flat white referencing calibration. Furthermore, the pixel-level normalized difference vegetation index (NDVI) distribution over the soybean leaf surface after 3D calibration was also closer to the standard calibration. This proposed 3D white referencing method had the potential to improve calibration quality of plant images. Integrating with LiDAR sensors, this new approach has an opportunity to be applied in field environments.