Measuring food volume from RGB-Depth image with point cloud conversion method using geometrical approach and robust ellipsoid fitting algorithm

Abstract

Volume measurement is an essential feature for food analysis and recognition systems to estimate nutritional intake from patients. Understanding the volume of consumed food is one approach for calculating how much food is consumed. This research provides a novel method for estimating food volume from RGB-Depth images using a combined point cloud conversion method and fitting points into ellipsoids using a geometrical approach. Point cloud segmentation is also utilized to separate the main object from the background. After the point cloud is downsampled, it is converted into real-world coordinates using the conversion method. The volume prediction is generated via numerical approaches, that is fitting to an ellipsoid using the least square approach. The objects in this study were tomatoes, oranges, kiwis, lemons, and potatoes, which were examined through multiple experiments. The evaluation results show that the predicted volume with the proposed method produces an Absolute Relative Error (ARE) of 2.6% for objects with a spherical shape, ARE of 3.1% for objects with an ellipsoid shape, and ARE of 3.0% for objects with an irregular surface, such as potatoes. When items are put on a white or blue background, volume predictions reach the ARE of 3.0% and 4.1%, respectively. As a result, white backgrounds were used more frequently in this study. Furthermore, experiments were carried out with various object orientations, camera distance locations, and object centers to evaluate the point cloud conversion method, yielding ARE of 3.4% and 4.4%, respectively. The obtained ARE indicates that the proposed approach makes reliable volume estimation.