Two-stage meniscus segmentation framework integrating multiclass localization network and adversarial learning-based segmentation network in knee MR images

Abstract

Meniscus segmentation from knee MR images is an essential step in finding the most suitable implant prototype for meniscus allograft transplantation using a 3D reconstruction model from the patient’s normal meniscus. However, the segmentation of the meniscus is challenging due to its thin shape, similar intensities with nearby structures such as cruciate and collateral ligaments in the knee MR images, large shape variations among patients, and inhomogeneous intensity within the meniscus. In addition, conventional deep convolutional neural network (DCNN)-based meniscus segmentation method mainly uses a pixel-wise objective function, and thus produces rather under-segmentation results due to the small shape of the meniscus or suffers from false positives that occur around the meniscus. To overcome these limitations, we propose a two-stage DCNN that combines a 2D U-Net-based meniscus localization network with a conditional generative adversarial network-based segmentation network. To efficiently localize the meniscus region to medial and lateral meniscus and feed the localized ROIs to the segmentation network, 2D U-Net-based DCNN segments knee MR images into six classes. To segment the medial and lateral meniscus while preventing under-segmentation due to intensity inhomogeneity within the meniscus and over-segmentation due to intensity similarity with surrounding structures, adversarial learning is performed repeatedly on the localized meniscus ROIs. The average DSC of the meniscus was 84.06% at the medial meniscus, and 83.19% at the lateral meniscus, respectively. These results showed that the proposed method prevented the meniscus from being over- and under-segmented by repeatedly judging and complementing the quality of segmentation results through adversarial learning.