Comparison of accuracy between FSL\u2019s FIRST and Freesurfer for caudate nucleus and putamen segmentation

Gabor Perlaki,Reka Horvath,Peter Bogner,Gergely Orsi,Szilvia Anett Nagy,Jozsef Janszky,Tamas Doczi

doi:10.1038/s41598-017-02584-5

Abstract

Although several methods have been developed to automatically delineate subcortical gray matter structures from MR images, the accuracy of these algorithms has not been comprehensively examined. Most of earlier studies focused primarily on the hippocampus. Here, we assessed the accuracy of two widely used non-commercial programs (FSL-FIRST and Freesurfer) for segmenting the caudate and putamen. T1-weighted 1 mm3 isotropic resolution MR images were acquired for thirty healthy subjects (15 females). Caudate nucleus and putamen were segmented manually by two independent observers and automatically by FIRST and Freesurfer (v4.5 and v5.3). Utilizing manual labels as reference standard the following measures were studied: Dice coefficient (D), percentage volume difference (PVD), absolute volume difference as well as intraclass correlation coefficient (ICC) for consistency and absolute agreement. For putamen segmentation, FIRST achieved higher D, lower PVD and higher ICC for absolute agreement with manual tracing than either version of Freesurfer. Freesurfer overestimated the putamen, while FIRST was not statistically different from manual tracing. The ICC for consistency with manual tracing was similar between the two methods. For caudate segmentation, FIRST and Freesurfer performed more similarly. In conclusion, Freesurfer and FIRST are not equivalent when comparing to manual tracing. FIRST was superior for putaminal segmentation.

Highlights

Several methods have been developed to automatically delineate subcortical gray matter structures from MR images, the accuracy of these algorithms has not been comprehensively examined
This study examined the reliability of two popular non-commercial automatic programs (FSL-FIRST and Freesurfer) for segmenting the caudate and putamen in a group of normal subjects, using manual segmentations by two independent observers as reference
To our knowledge, attempted to compare the reliability of Freesurfer and FIRST relative to manual tracing in the caudate and putamen[37], but it did not report spatial overlap of the automated segmentations with manual tracing and their results may be suboptimal due to including both healthy subjects and psychiatric patients in the same statistics and using Magnetic resonance imaging (MRI) measurements which were not standardized across the small number of subjects (N = 20)

Summary

Introduction

Several methods have been developed to automatically delineate subcortical gray matter structures from MR images, the accuracy of these algorithms has not been comprehensively examined. Brain regions are segmented manually and manual segmentation is considered the gold standard approach even today This simple method is subjective, extremely time-consuming, laborious and human resource intensive and unfeasible for large MRI data sets[18, 19]. Several automated segmentation tools were proposed, including the widely used non-commercial FSL-FIRST and Freesurfer methods Despite both of these methods are well-published and validated by their developers[20, 21], relatively few studies discussed the comparison of their accuracy and most of these studies focused primarily on the segmentation of hippocampus[22,23,24,25,26,27]. Importance in a variety of diseases including Parkinson’s disease[28], Huntington’s disease[29], obsessive-compulsive disorder[30], Alzheimer’s disease[31, 32], primary focal dystonia[33], attention-deficit hyperactivity disorder[34], depression[35] and schizophrenia[36] and because their segmentation is challenging due to the fact that MR image intensities alone cannot be used to successfully distinguish them from adjacent brain structures[21]

Objectives

Methods

Results

Conclusion