Non-destructive fruit firmness evaluation using a soft gripper and vision-based tactile sensing

Abstract

As fruit firmness is a crucial characteristic associated with the maturity level, its accurate estimation is of great importance to post-harvest processing and wholesale in the industry. Benefiting from the advances of soft robotics, a soft gripper with simultaneous compliant deformation and tactile sensing is proposed in this study for the fruit firmness classification. The gripper design inspired by the fin ray effect can achieve active deformation, which helps simplify the actuation system and improve the delicate manipulation capability. Finite element modelling, along with experimental tests, is first utilized to validate the gripper’s feasibility in compliant and safe fruit grasping, and respiratory tests are then conducted to further demonstrate the non-destructive nature. Moreover, fruit–gripper interaction is captured by visual sensors and then processed using an attention-based CNN–LSTM algorithm to predict firmness information. Tomatoes and nectarines are chosen as the sample fruit for experimental validation. R2 values of their firmness prediction are 0.795 and 0.753, and the accuracy of maturity grading is 84.6% and 81.5%, respectively. In general, the soft gripper provides a promising solution for both safe grasping and non-destructive firmness evaluation, and it is expected to be integrated into automated production lines to pack fruit based on different firmness levels.