Preliminary stage in the development of an artificial intelligence algorithm: Variations between 100 surgeons in phase annotation in a video of internal fixation of distal radius fracture

Abstract

IntroductionIn order to be used naturally and widely, an artificial intelligence algorithm of phase detection in surgical videos presupposes an expert consensus defining phases. ObjectivesThe aim of the present study was to seek consensus in defining the various phases of a surgical technique in wrist traumatology. MethodsThree thousand two hundred and twenty-nine surgeons were sent a video showing anterior plate fixation of the distal radius and a questionnaire on the number of phases they distinguished and the visual cues signaling the beginning of each phase. Three experimenters predefined the number of phases (5: installation, approach, fixation, verification, closure) and sub-phases (3a: introduction of plate; 3b: positioning distal screws; 3c: positioning proximal screws) and the cues signaling the beginning of each. The numbers of the responses per item were collected. ResultsOnly 216 (6.7%) surgeons opened the questionnaire, and 100 answered all questions (3.1%). Most respondents claimed 5/5 expertise. Number of phases identified ranged between 3 and 10. More than two-thirds of respondents identified the same phase cue as defined by the 3 experimenters in most cases, except for “verification” and “positioning proximal screws”. DiscussionSurgical procedures comprise a succession of phases, the beginning or end of which can be defined by a precise visual cue on video, either beginning with the appearance of the cue or the disappearance of the cue defining the preceding phase. ConclusionThese cues need to be defined very precisely before attempting manual annotation of surgical videos in order to develop an artificial intelligence algorithm. Level of evidenceII.