Standard Plane Detection Research Articles

LPC-SonoNet: A Lightweight Network Based on SonoNet and Light Pyramid Convolution for Fetal Ultrasound Standard Plane Detection

The detection of fetal ultrasound standard planes (FUSPs) is important for the diagnosis of fetal malformation and the prevention of perinatal death. As a promising deep-learning technique in FUSP detection, SonoNet’s network parameters have a large size. In this paper, we introduced a light pyramid convolution (LPC) block into SonoNet and proposed LPC-SonoNet with reduced network parameters for FUSP detection. The LPC block used pyramid convolution architecture inspired by SimSPPF from YOLOv6 and was able to extract features from various scales with a small parameter size. Using SonoNet64 as the backbone, the proposed network removed one of the convolutional blocks in SonoNet64 and replaced the others with LPC blocks. The proposed LPC-SonoNet model was trained and tested on a publicly available dataset with 12,400 ultrasound images. The dataset with six categories was further divided into nine categories. The images were randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1. Data augmentation was conducted on the training set to address the data imbalance issue. In the classification of six categories and nine categories, LPC-SonoNet obtained the accuracy of 97.0% and 91.9% on the test set, respectively, slightly higher than the accuracy of 96.60% and 91.70% by SonoNet64. Compared with SonoNet64 with 14.9 million parameters, LPC-SonoNet had a much smaller parameter size (4.3 million). This study pioneered the deep-learning classification of nine categories of FUSPs. The proposed LPC-SonoNet may be used as a lightweight network for FUSP detection.

Automatic Detection of Standard Planes in Fetal Ultrasound Images based on Convolutional Neural Networks and Ensemble Learning

aims: This study aims to leverage artificial intelligence for enhancing medical diagnosis, focusing on ultrasound evaluation of fetal development and detection of fetal diseases. background: Traditional diagnostic methods in ultrasound are known for being time-consuming and laborious, prompting the need for more efficient approaches. objective: The objective of this research is to develop an end-to-end automatic diagnosis system using convolutional neural networks with ensemble learning to enhance robustness and accuracy in classifying ultrasound images. method: The study involves constructing and implementing the automatic diagnosis system, training it on a diverse dataset encompassing six categories: abdomen, brain, femur, thorax, maternal cervix, and other planes. result: Experimental results demonstrate that the proposed end-to-end system significantly improves the detection accuracy of the standard plane in ultrasound images. conclusion: The application of artificial intelligence through an ensemble learning-based automatic diagnosis system shows promise in advancing ultrasound-based medical diagnosis, particularly in fetal development assessment. other: This research contributes to the ongoing efforts in leveraging technology for more efficient and accurate medical diagnostic processes.

Automating the Human Action of First-Trimester Biometry Measurement from Real-World Freehand Ultrasound

Objective: Automated medical image analysis solutions should closely mimic complete human actions to be useful in clinical practice. However, more often an automated image analysis solution only represents part of a human task which restricts its practical utility. In the case of ultrasound-based fetal biometry, an automated solution should ideally recognize key fetal structures in free-hand video guidance, select a standard plane from a video stream and perform biometry. A complete automated solution should automate all three sub-actions.Methods: This paper considers how to automate the complete human action of first-trimester biometry measurement from real-world freehand ultrasound. In the proposed hybrid Convolutional Neural Network (CNN) architecture design, a classification-regression-based guidance model detects and tracks fetal anatomical structures (using visual cues) in the ultrasound video. Several high-quality standard planes which contain the mid-sagittal view of the fetus are sampled at multiple timestamps (using a custom-designed confident-frame detector) based on the estimated probability values associated with predicted anatomical structures that define the biometry plane. Automated semantic segmentation is performed on the selected frames to extract fetal anatomical landmarks. A crown-rump length (CRL) estimate is calculated as the mean CRL from these multiple frames.Results: Our fully automated method shows a high correlation with clinical expert CRL measurement (Pearson ρ=0.92, R-Squared (R2)=0.84) and a low mean absolute error of 0.834 (weeks) for fetal age estimation on a test data set of 42 videos.Conclusion: A novel algorithm for standard plane detection employs a quality detection mechanism defined by clinical standards and ensuring precise biometric measurements.

Fetal cardiac ultrasound standard section detection model based on multitask learning and mixed attention mechanism

Fetal cardiac ultrasound is a valuable tool for screening fetal cardiac health during pregnancy. Ultrasound standard section testing is an essential part of fetal heart ultrasound diagnosis. Due to the many scattered and partially similar anatomical structures in standard ultrasound views of the fetal heart, the results strongly depend on the clinical experience and knowledge of the sonographer. To improve detection efficiency and reduce misdiagnosis and omission, we propose a single-stage fetal cardiac ultrasound standard plane detection model (FCUM) based on multitask learning and a hybrid attention mechanism to assist sonographers in diagnosis. The feature fusion pyramids of the backbone and detection networks of this model are each embedded with a hybrid attention mechanism module of our design, which enables the multitasking network to extract shared features more accurately and efficiently and improves the accuracy of the detection and classification networks. We designed a classification module for multilayer residual network feature fusion that leads to better classification and faster convergence time. We conducted comprehensive experiments on a dataset of fetal cardiac ultrasound images acquired with several types of devices and in different geographic regions. The experimental results show that our model outperforms baseline models such as YOLOv8 and ResNet-50 in terms of detection precision and classification accuracy.

Fetal Ultrasound Standard Plane Detection With Coarse-to-Fine Multi-Task Learning.

The ultrasound standard plane plays an important role in prenatal fetal growth parameter measurement and disease diagnosis in prenatal screening. However, obtaining standard planes in a fetal ultrasound video is not only laborious and time-consuming but also depends on the clinical experience of sonographers to a certain extent. To improve the acquisition efficiency and accuracy of the ultrasound standard plane, we propose a novel detection framework that utilizes both the coarse-to-fine detection strategy and multi-task learning mechanism for feature-fused images. First, traditional manually-designed features and deep learning-based features are fused to obtain low-level shared features, which can enhance the model's feature expression ability. Inspired by the process of human recognition, ultrasound standard plane detection is divided into a coarse process of plane type classification and a fine process of standard-or-not detection, which is implemented via an end-to-end multi-task learning network. The region-of-interest area is also recognised in our detection framework to suppress the influence of a variable maternal background. Extensive experiments are conducted on three ultrasound planes of the first-class fetal examination, i.e., the femur, thalamus, and abdomen ultrasound images. The experiment results show that our method outperforms competing methods in terms of accuracy, which demonstrates the efficacy of the proposed method and can reduce the workload of sonographers in prenatal screening.

EP02.07: Novel algorithm of standard plane detection in second trimester fetal neurosonography using three‐dimensional automatic segmentation

EP02.07: Novel algorithm of standard plane detection in second trimester fetal neurosonography using three‐dimensional automatic segmentation

Locating Multiple Standard Planes in First-Trimester Ultrasound Videos via the Detection and Scoring of Key Anatomical Structures

This study was aimed at developing a first-trimester standard plane detection (FTSPD) system that can automatically locate nine standard planes in ultrasound videos and investigating its utility in clinical practice. The FTSPD system, based on the YOLOv3 network, was developed to detect structures and evaluate the quality of plane images by using a pre-defined scoring system. A total of 220 videos from two different ultrasound scanners were collected to compare detection performance between our FTSPD system and sonographers with different levels of experience. The quality of the detected standard planes was quantitatively rated by an expert according to a scoring protocol. Kolmogorov-Smirnov analysis was used to compare the distributions of scores across all nine standard planes. The expert-rated scores indicated that the quality of the standard planes detected by the FTSPD system was on par with that of the planes detected by senior sonographers. There were no significant differences in the distributions of the scores across all nine standard planes. The FTSPD system performed significantly better than junior sonographers in five standard plane types. The results of this study suggest that our FTSPD system has significant potential for detecting standard planes in first-trimester ultrasound screening, which may help to improve the accuracy of fetal ultrasound screening and facilitate early diagnosis of abnormalities. The quality of the standard planes selected by junior sonographers can be significantly improved with the assistance of our FTSPD system.

Application and Progress of Artificial Intelligence in Fetal Ultrasound.

Prenatal ultrasonography is the most crucial imaging modality during pregnancy. However, problems such as high fetal mobility, excessive maternal abdominal wall thickness, and inter-observer variability limit the development of traditional ultrasound in clinical applications. The combination of artificial intelligence (AI) and obstetric ultrasound may help optimize fetal ultrasound examination by shortening the examination time, reducing the physician's workload, and improving diagnostic accuracy. AI has been successfully applied to automatic fetal ultrasound standard plane detection, biometric parameter measurement, and disease diagnosis to facilitate conventional imaging approaches. In this review, we attempt to thoroughly review the applications and advantages of AI in prenatal fetal ultrasound and discuss the challenges and promises of this new field.

Development of a Fully Automated Graf Standard Plane and Angle Evaluation Method for Infant Hip Ultrasound Scans.

Background: Graf’s method is currently the most commonly used ultrasound-based technique for the diagnosis of developmental dysplasia of the hip (DDH). However, the efficiency and accuracy of diagnosis are highly affected by the sonographers’ qualification and the time and effort expended, which has a significant intra- and inter-observer variability. Methods: Aiming to minimize the manual intervention in the diagnosis process, we developed a deep learning-based computer-aided framework for the DDH diagnosis, which can perform fully automated standard plane detection and angle measurement for Graf type I and type II hips. The proposed framework is composed of three modules: an anatomical structure detection module, a standard plane scoring module, and an angle measurement module. This framework can be applied to two common clinical scenarios. The first is the static mode, measurement and classification are performed directly based on the given standard plane. The second is the dynamic mode, where a standard plane from ultrasound video is first determined, and measurement and classification are then completed. To the best of our knowledge, our proposed framework is the first CAD method that can automatically perform the entire measurement process of Graf’s method. Results: In our experiments, 1051 US images and 289 US videos of Graf type I and type II hips were used to evaluate the performance of the proposed framework. In static mode, the mean absolute error of α, β angles are 1.71° and 2.40°, and the classification accuracy is 94.71%. In dynamic mode, the mean absolute error of α, β angles are 1.97° and 2.53°, the classification accuracy is 89.51%, and the running speed is 31 fps. Conclusions: Experimental results demonstrate that our fully automated framework can accurately perform standard plane detection and angle measurement of an infant’s hip at a fast speed, showing great potential for clinical application.

An ultrasound standard plane detection model of fetal head based on multi-task learning and hybrid knowledge graph

Prenatal ultrasound examination is a powerful tool to prevent birth defects and assess fetal health. Obtaining ultrasound standard planes is a prerequisite for prenatal ultrasound diagnosis. However, ultrasound standard plane detection depends heavily on the sonographer’s sufficient clinical experience and solid knowledge of fetal anatomy. In this study, to lighten the workload of the sonographer and promote the accuracy, efficiency, and interpretability of ultrasound standard plane detection, we propose an ultrasound standard plane detection (USPD) model based on multi-task learning and a hybrid knowledge graph. We first design a multi-task learning strategy to learn the shared features of fetal ultrasound images through convolutional blocks. Then, we optimize the generalization performance by extending the shared features into the task-specific output streams. In addition, USPD integrates clinical prior knowledge graphs to reduce the error rate and missed detection rate. The USPD model can recognize the key anatomical structures of fetal heads and analyze the types of ultrasound planes. Furthermore, unlike most “end-to-end” automatic detection models, the USPD model not only outputs the prediction results but also provides consistent interpretation for professional sonographers, thereby increasing the interpretability of the model without the sonographer’s intervention. We conduct extensive experiments on a fetal head ultrasound image dataset to assess the proposed USPD model via comparison with competitive methods. Experimental results illustrate that the proposed USPD model outperforms the competitive methods with regard to accuracy and performance, and it can meet the clinical requirements in practical application.

Deep learning fetal ultrasound video model match human observers in biometric measurements

Objective. This work investigates the use of deep convolutional neural networks (CNN) to automatically perform measurements of fetal body parts, including head circumference, biparietal diameter, abdominal circumference and femur length, and to estimate gestational age and fetal weight using fetal ultrasound videos. Approach. We developed a novel multi-task CNN-based spatio-temporal fetal US feature extraction and standard plane detection algorithm (called FUVAI) and evaluated the method on 50 freehand fetal US video scans. We compared FUVAI fetal biometric measurements with measurements made by five experienced sonographers at two time points separated by at least two weeks. Intra- and inter-observer variabilities were estimated. Main results. We found that automated fetal biometric measurements obtained by FUVAI were comparable to the measurements performed by experienced sonographers The observed differences in measurement values were within the range of inter- and intra-observer variability. Moreover, analysis has shown that these differences were not statistically significant when comparing any individual medical expert to our model. Significance. We argue that FUVAI has the potential to assist sonographers who perform fetal biometric measurements in clinical settings by providing them with suggestions regarding the best measuring frames, along with automated measurements. Moreover, FUVAI is able perform these tasks in just a few seconds, which is a huge difference compared to the average of six minutes taken by sonographers. This is significant, given the shortage of medical experts capable of interpreting fetal ultrasound images in numerous countries.

NHBS-Net: A Feature Fusion Attention Network for Ultrasound Neonatal Hip Bone Segmentation.

Ultrasound is a widely used technology for diagnosing developmental dysplasia of the hip (DDH) because it does not use radiation. Due to its low cost and convenience, 2-D ultrasound is still the most common examination in DDH diagnosis. In clinical usage, the complexity of both ultrasound image standardization and measurement leads to a high error rate for sonographers. The automatic segmentation results of key structures in the hip joint can be used to develop a standard plane detection method that helps sonographers decrease the error rate. However, current automatic segmentation methods still face challenges in robustness and accuracy. Thus, we propose a neonatal hip bone segmentation network (NHBS-Net) for the first time for the segmentation of seven key structures. We design three improvements, an enhanced dual attention module, a two-class feature fusion module, and a coordinate convolution output head, to help segment different structures. Compared with current state-of-the-art networks, NHBS-Net gains outstanding performance accuracy and generalizability, as shown in the experiments. Additionally, image standardization is a common need in ultrasonography. The ability of segmentation-based standard plane detection is tested on a 50-image standard dataset. The experiments show that our method can help healthcare workers decrease their error rate from 6%-10% to 2%. In addition, the segmentation performance in another ultrasound dataset (fetal heart) demonstrates the ability of our network.

VP18.08: An artificial intelligence approach for fully automatic standard plane detection and volumetric measurements in first trimester three‐dimensional ultrasound datasets

VP18.08: An artificial intelligence approach for fully automatic standard plane detection and volumetric measurements in first trimester three‐dimensional ultrasound datasets

Self-Supervised Representation Learning for Ultrasound Video.

Recent advances in deep learning have achieved promising performance for medical image analysis, while in most cases ground-truth annotations from human experts are necessary to train the deep model. In practice, such annotations are expensive to collect and can be scarce for medical imaging applications. Therefore, there is significant interest in learning representations from unlabelled raw data. In this paper, we propose a self-supervised learning approach to learn meaningful and transferable representations from medical imaging video without any type of human annotation. We assume that in order to learn such a representation, the model should identify anatomical structures from the unlabelled data. Therefore we force the model to address anatomy-aware tasks with free supervision from the data itself. Specifically, the model is designed to correct the order of a reshuffled video clip and at the same time predict the geometric transformation applied to the video clip. Experiments on fetal ultrasound video show that the proposed approach can effectively learn meaningful and strong representations, which transfer well to downstream tasks like standard plane detection and saliency prediction.

Self-supervised Contrastive Video-Speech Representation Learning for Ultrasound.

In medical imaging, manual annotations can be expensive to acquire and sometimes infeasible to access, making conventional deep learning-based models difficult to scale. As a result, it would be beneficial if useful representations could be derived from raw data without the need for manual annotations. In this paper, we propose to address the problem of self-supervised representation learning with multi-modal ultrasound video-speech raw data. For this case, we assume that there is a high correlation between the ultrasound video and the corresponding narrative speech audio of the sonographer. In order to learn meaningful representations, the model needs to identify such correlation and at the same time understand the underlying anatomical features. We designed a framework to model the correspondence between video and audio without any kind of human annotations. Within this framework, we introduce cross-modal contrastive learning and an affinity-aware self-paced learning scheme to enhance correlation modelling. Experimental evaluations on multi-modal fetal ultrasound video and audio show that the proposed approach is able to learn strong representations and transfers well to downstream tasks of standard plane detection and eye-gaze prediction.

Multi-task learning for quality assessment of fetal head ultrasound images.

It is essential to measure anatomical parameters in prenatal ultrasound images for the growth and development of the fetus, which is highly relied on obtaining a standard plane. However, the acquisition of a standard plane is, in turn, highly subjective and depends on the clinical experience of sonographers. In order to deal with this challenge, we propose a new multi-task learning framework using a faster regional convolutional neural network (MF R-CNN) architecture for standard plane detection and quality assessment. MF R-CNN can identify the critical anatomical structure of the fetal head and analyze whether the magnification of the ultrasound image is appropriate, and then performs quality assessment of ultrasound images based on clinical protocols. Specifically, the first five convolution blocks of the MF R-CNN learn the features shared within the input data, which can be associated with the detection and classification tasks, and then extend to the task-specific output streams. In training, in order to speed up the different convergence of different tasks, we devise a section train method based on transfer learning. In addition, our proposed method also uses prior clinical and statistical knowledge to reduce the false detection rate. By identifying the key anatomical structure and magnification of the ultrasound image, we score the ultrasonic plane of fetal head to judge whether it is a standard image or not. Experimental results on our own-collected dataset show that our method can accurately make a quality assessment of an ultrasound plane within half a second. Our method achieves promising performance compared with state-of-the-art methods, which can improve the examination effectiveness and alleviate the measurement error caused by improper ultrasound scanning.

Ultrasound Image Representation Learning by Modeling Sonographer Visual Attention.

Image representations are commonly learned from class labels, which are a simplistic approximation of human image understanding. In this paper we demonstrate that transferable representations of images can be learned without manual annotations by modeling human visual attention. The basis of our analyses is a unique gaze tracking dataset of sonographers performing routine clinical fetal anomaly screenings. Models of sonographer visual attention are learned by training a convolutional neural network (CNN) to predict gaze on ultrasound video frames through visual saliency prediction or gaze-point regression. We evaluate the transferability of the learned representations to the task of ultrasound standard plane detection in two contexts. Firstly, we perform transfer learning by fine-tuning the CNN with a limited number of labeled standard plane images. We find that fine-tuning the saliency predictor is superior to training from random initialization, with an average F1-score improvement of 9.6% overall and 15.3% for the cardiac planes. Secondly, we train a simple softmax regression on the feature activations of each CNN layer in order to evaluate the representations independently of transfer learning hyper-parameters. We find that the attention models derive strong representations, approaching the precision of a fully-supervised baseline model for all but the last layer.

Ultrasound Standard Plane Detection Using a Composite Neural Network Framework.

Ultrasound (US) imaging is a widely used screening tool for obstetric examination and diagnosis. Accurate acquisition of fetal standard planes with key anatomical structures is very crucial for substantial biometric measurement and diagnosis. However, the standard plane acquisition is a labor-intensive task and requires operator equipped with a thorough knowledge of fetal anatomy. Therefore, automatic approaches are highly demanded in clinical practice to alleviate the workload and boost the examination efficiency. The automatic detection of standard planes from US videos remains a challenging problem due to the high intraclass and low interclass variations of standard planes, and the relatively low image quality. Unlike previous studies which were specifically designed for individual anatomical standard planes, respectively, we present a general framework for the automatic identification of different standard planes from US videos. Distinct from conventional way that devises hand-crafted visual features for detection, our framework explores in- and between-plane feature learning with a novel composite framework of the convolutional and recurrent neural networks. To further address the issue of limited training data, a multitask learning framework is implemented to exploit common knowledge across detection tasks of distinctive standard planes for the augmentation of feature learning. Extensive experiments have been conducted on hundreds of US fetus videos to corroborate the better efficacy of the proposed framework on the difficult standard plane detection problem.

A hierarchical model for automated standard sagittal-view detection from 3D ultrasound data in 11–14 weeks

The nuchal translucency thickness is an important parameter for the diagnosis of fetal abnormalities during 11–14 weeks. Currently in clinical practice, it first requires manual scanning operations to determine the fetal standard sagittal-view plane and then the measurements can be performed in the corresponding plane images. Besides the difficulty of such standard plane detection, this also leads to the time-consuming and detection-variability problems. In the paper, a hierarchical model is proposed to automatically detect the standard sagittal-view plane based on 3D ultrasound data. In the model, Hessian-matrix based filtering is first applied for obtaining the plate-structure distribution in the data. Then the sphere distribution is calculated by convolving sphere detectors with the ultrasound data. Based on the two prior distributions, the sampling-based Hough transform is further performed for the plane detection. The performance of the proposed model is verified by the experimental results on 3D synthetic data and 241 clinical 3D ultrasound data in 11–14 weeks.