Multi-view triangulation without correspondences

Abstract

We introduce a novel method to solve the joint problem of correspondence and triangulation of points from multiple calibrated perspective views and show its application to counting the number of leaves present on plants photographed from multiple angles in phenotyping facilities. Assuming there is a set of n calibrated views of an object, the input to our algorithm is a set of 2D points (e.g., leaf tips, fruits, flowers) detected in each of the n views, and the output is a set of 3D points corresponding to the 2D points when re-projected on views. Our approach is robust to noise and occlusion. In particular, it is not required for all points to be visible from one of the views, as our algorithm can infer that a point is occluded by reasoning on the 3D geometry of the scene. Our algorithm is suitable for many points (approx. thousands) reconstructed from a reduced set of views (up to 15). For example, our implementation finds the correspondence of 20 points captured by six cameras in about one second on consumer hardware. We evaluate the performance on synthetic data as well as real examples. We show that the accuracy of leaf counting from multi-view images is drastically improved by our algorithm.