Many studies have investigated the transfer of skills between laparoscopic and robot-assisted surgery (RAS). These studies have considered time, error, and clinical outcomes in the assessment of skill transfer. However, little is known about the specific operations of the surgeon. Clutch control use is an important skill in RAS. Therefore, the present study aimed to propose a novel objective algorithm based on computer vision that can automatically evaluate a surgeon's clutch use. Additionally, the study aimed to evaluate the correlation between clutch metrics and surgical skill on different surgical robot platforms. The robotic surgery training center of Wuhan University trained 30 laparoscopic surgeons as the study group between 2023 and 2024. Laparoscopic surgeons were trained by combining robotic simulator exercises and RAS animal experiments. During the training, video and hand movement data were collected. Hand movements identified by a skin-color model were combined with labeling information to classify clutch use. The metrics were validated on different robotic platforms (dv-Trainer, EDGE MP1000, Toumai™ MT1000, and DaVinci Xi system) and among surgeons with different surgical skill levels. On the robotic simulator, clutch accuracy in the expert group was significantly higher than in the study group for all tasks. No significant differences were observed in the number of clutches between the expert and study groups. In the RAS experiment, the number of clutches decreased significantly for both study and expert groups. The accuracy was maintained at a high level in the expert group but decreased rapidly in the study group. We proposed a new objective assessment of surgical skills, clutch use metrics, in cross-platform RAS. Additionally, we verified that the metrics significantly correlated with the surgical skill levels of the surgeons.