Inter- and intra-operator variations in manual segmentation of hippocampus from MRI

Abstract

PurposeWe evaluate inter- and intra-operator variations in manual segmentation of hippocampus and present their potential sources. Hippocampal atrophy is common in mesial temporal lobe epilepsy (mTLE). Effective diagnosis and treatment of mTLE depends on accurate and efficient segmentation of hippocampus from magnetic resonance imaging (MRI) data. Manual segmentation by expert radiologists remains the gold standard, although automated segmentation methods exist. MethodsHippocampus was segmented in MRI of 118 unilateral mTLE patients and 25 non-epileptic subjects (65 males, 78 females; mean age 39 years) by three operators (M1, M2, M3) manually and by three software tools (FreeSurfer, LocalInfo, ABSS) automatically. Segmentation results were evaluated using 7 volume-, voxel-, and distance-based performance measures. Inter-operator variation was evaluated by comparing the segmentation results of the three operators. Intra-operator variation was evaluated by comparing manual and automatic segmentation results. Segmentation results were used to lateralize epileptogenicity in mTLE patients. ResultsRanking of performance measures differed when using M3 segmentations as ground truth rather than M1 or M2 segmentations. The standard deviation tended to be higher when using M3 as ground truth and M1 or M2 as test segmentation. Variation in performance measures and increased standard deviation when using M3 as ground truth are indicative of inter-operator variability. Standard deviations were larger when using M2 segmentations as ground truth and automated segmentations as test segmentation, rather than M1 or M3. Large standard deviations between M2 segmentations and automated segmentations are indicative of intra-operator variation. Among automated segmentation methods, ABSS produced segmentations most similar to manual segmentations, but FreeSurfer and LocalInfo lateralized epileptogenicity more accurately. ConclusionsPart of inter-operator variation might be due to temporal separation of M3 segmentations from M1 and M2 segmentations (M3 performed segmentations a few years before operators M1 and M2). Inter-operator variation will likely reduce if all operators segment within the same time frame, reducing discrepancies in training of operators. Intra-operator variation can likely be mitigated with additional oversight by a neuroradiologist. Inter- and intra-operator variability may generate inconsistencies in outcomes. Future automated segmentation techniques may integrate neural-network-based (ABSS) and atlas-based (FreeSurfer and LocalInfo) segmentation techniques for optimal performance.