Performance evaluation of three versions of a convolutional neural network for object detection and segmentation using a multiclass and reduced panoramic radiograph dataset

Abstract

ObjectivesTo evaluate the diagnostic performance of three versions of a deep-learning convolutional neural network in terms of object detection and segmentation using a multiclass panoramic radiograph dataset. MethodsA total of 600 orthopantomographies were randomly selected for this study and manually annotated by a single operator using an image annotation tool (COCO Annotator v.11.0.1) to establish ground truth. The annotation classes included teeth, maxilla, mandible, inferior alveolar nerve, dento- and implant-supported crowns/pontics, endodontic treatment, resin-based restorations, metallic restorations, and implants. The dataset was then divided into training, validation, and testing subsets, which were used to train versions 5, 7, and 8 of You Only Look Once (YOLO) Neural Network. Results were stored, and a posterior performance analysis was carried out by calculating the precision (P), recall (R), F1 Score, Intersection over Union (IoU), and mean average precision (mAP) at 0.5 and 0.5–0.95 thresholds. The confusion matrix and recall precision graphs were also sketched. ResultsYOLOv5s showed an improvement in object detection results with an average R = 0.634, P = 0.781, mAP0.5 = 0.631, and mAP0.5–0.95 = 0.392. YOLOv7m achieved the best object detection results with average R = 0.793, P = 0.779, mAP0.5 = 0.740, and mAP0.5–0.95 = 0,481. For object segmentation, YOLOv8m obtained the best average results (R = 0.589, P = 0.755, mAP0.5 = 0.591, and mAP0.5–0.95 = 0.272). ConclusionsYOLOv7m was better suited for object detection, while YOLOv8m demonstrated superior performance in object segmentation. The most frequent error in object detection was related to background classification. Conversely, in object segmentation, there is a tendency to misclassify True Positives across different dental treatment categories. Clinical SignificanceGeneral diagnostic and treatment decisions based on panoramic radiographs can be enhanced using new artificial intelligence-based tools. Nevertheless, the reliability of these neural networks should be subjected to training and validation to ensure their generalizability.