Polarimetric monocular leaf normal estimation model for plant phenotyping

Abstract

We develop in this paper an accurate, pixel-level, plant leaf normal estimation model by a single polarization image. Though traditional sensing can generate leaf 3-D surface information, these techniques are limited in application because of sensor cost and acquisition-time-related considerations. To achieve the detailed and monocular estimation capacity, we propose a novel surface polarization reflection model that considers a mixture of diffuse or specular reflections and describes the actual reflection process more accurately than prevailing models. Our model also directly corresponds with the recorded polarization states, allowing for direct implementation, no additional computational cost, and no requirement for prior knowledge. We also propose a new strategy to disambiguate the normal solution associated with polarization-based imaging. In contrast to existing methods, which use auxiliary sensory information for the disambiguation, we derive a coarse normal map directly from our image data using an off-the-shelf convolutional neural network. Consequently, we facilitate instantaneous data acquisition, which is essential when modeling dynamic non-rigid objects. Using the coarse normal map as a constraint and optimizing optical smoothness properties makes our estimated outcome more accurate than state-of-the-art results. Experiments show that our median normal angular error is 5.6°, offering a threefold improvement to current polarimetric methods and equivalent or better than what SfM-MVS methods provide, yet using only a single image. Our leaf orientation map is also more detailed than existing methods while exhibiting robustness to polarization image noise and the guiding depth map quality. Hence, by a single polarization image, we obtain high-quality surface normal data with no additional aid.