Localization Of Anatomical Landmarks Research Articles

Enhancing YOLO5 for the Assessment of Irregular Pelvic Radiographs with Multimodal Information.

Developmental dysplasia of the hip (DDH) is one of the most common orthopedic disorders in infants and young children. Accurate identification and localization of anatomical landmarks are prerequisites for the diagnosis of DDH. In recent years, various works have employed deep learning algorithms on radiography images for DDH diagnosis. However, none of these works have considered the incorporation of multimodal information. The pelvis exhibits distinct structures at different developmental stages, and there are also gender-based differences. In light of this, this study proposes a method to enhance the performance of deep learning models in diagnosing DDH by incorporating age and gender information into the channels. The study utilizes YOLO5 to construct a deep learning network for detecting hip joint landmarks. Moreover, a comprehensive dataset of 7750 pelvic X-ray images is established, covering ages from 4months to 16years and encompassing various conditions, such as deformities and post-operative cases, which authentically capture the temporal diversity and pathological complexities of DDH. Experimental results show that the YOLO5 model with integrated multimodal information achieves a mAP0.5-0.95 of 83.1% and a diagnostic accuracy of 86.7% in test dataset. The F1 scores for diagnosing cases of normal (NM), suspected dislocation (SD), mild dislocation (MD), and heavily dislocation (HD) are 90.9%, 79.8%, 63.5%, and 97.4%, respectively. Furthermore, experiments conducted on datasets of different sizes and networks of different sizes demonstrate the beneficial impact of multimodal information in improving the effectiveness of deep learning in diagnosing DDH.

Landmark annotation through feature combinations: a comparative study on cephalometric images with in-depth analysis of model's explainability.

The objectives of this study are to explore and evaluate the automation of anatomical landmark localization in cephalometric images using machine learning techniques, with a focus on feature extraction and combinations, contextual analysis, and model interpretability through Shapley Additive exPlanations (SHAP) values. We conducted extensive experimentation on a private dataset of 300 lateral cephalograms to thoroughly study the annotation results obtained using pixel feature descriptors including raw pixel, gradient magnitude, gradient direction, and histogram-oriented gradient (HOG) values. The study includes evaluation and comparison of these feature descriptions calculated at different contexts namely local, pyramid, and global. The feature descriptor obtained using individual combinations is used to discern between landmark and nonlandmark pixels using classification method. Additionally, this study addresses the opacity of LGBM ensemble tree models across landmarks, introducing SHAP values to enhance interpretability. The performance of feature combinations was assessed using metrics like mean radial error, standard deviation, success detection rate (SDR) (2 mm), and test time. Remarkably, among all the combinations explored, both the HOG and gradient direction operations demonstrated significant performance across all context combinations. At the contextual level, the global texture outperformed the others, although it came with the trade-off of increased test time. The HOG in the local context emerged as the top performer with an SDR of 75.84% compared to others. The presented analysis enhances the understanding of the significance of different features and their combinations in the realm of landmark annotation but also paves the way for further exploration of landmark-specific feature combination methods, facilitated by explainability.

Positioning errors of anatomical landmarks identified by fixed vertices in homologous meshes

BackgroundHuman movement analysis is usually achieved by tracking markers attached to anatomical landmarks with photogrammetry. Such marker-based systems have disadvantages that have led to the development of markerless procedures, although their accuracy is not usually comparable to that of manual palpation procedures. New motion acquisition systems, such as 3D temporal scanners, provide homologous meshes that can be exploited for this purpose. Research questionCan fixed vertices of a homologous mesh be used to identify anatomical landmarks with an accuracy equivalent to that of manual palpation? MethodsWe used 3165 human shape scans from the CAESAR dataset, with labelled locations of anatomical landmarks. First, we fitted a template mesh to the scans, and assigned a vertex of that mesh to 53 anatomical landmarks in all subjects. Then we defined a nominal vertex for each landmark, as the more centred vertex out of the set assigned for that landmark. We calculated the errors of the template-fitting and the nominal vertex determination procedures, and analysed their relationship to subject’s sex, height and body mass index, as well as their size compared to manual palpation errors. ResultsThe template-fitting errors were below 5 mm, and the nominal vertex determination errors reached maximum values of 24 mm. Except for the trochanter, those errors were the same order of magnitude or smaller than inter-examiner errors of lower limb landmarks. Errors increased with height and body mass index, and were smaller for men than for women of the same height and body mass index. SignificanceWe defined a set of vertices for 53 anatomical landmarks in a homologous mesh, which yields location errors comparable to those obtained by manual palpation for the majority of landmarks. We also quantified how the subject’s sex and anthropometric features can affect the size of those errors.

Angular Photogrammetric Analysis of Facial Soft Tissue by Image Processing Algorithms.

The main aim of this study was to present an automatic method based on image processing algorithms for facial anatomical landmark localization and angular photogrammetric analysis applicable for rhinoplasty surgery. We studied and measured color profile photographs of 100 patients before and after rhinoplasty surgery. In facial anthropometry analysis, anatomical landmarks are often defined by specialists, manually. This process is time-consuming and requires training and skill. The Cascade Regression Method (CRM) was utilized for facial landmark detection to overcome the mentioned problem. In this study, 11 anatomical landmarks were used to measure 9 facial angular metrics. Finally, a t-test (with the significance level set at a p-value of 0.05) was applied to analyze before surgery versus after surgery comparisons. Experimental results dedicated that there is a significance difference (p < 0.001) in nasofrontal, nasolabial, mentolabial, nasomental, facial convexity including nose, facial convexity excluding nose, projection of the upper lip to chin, and H angles before and after surgery. Also, results showed that there is not a significance difference in nose tip angle. We believe that the presented system can aim to reduce the personal errors made by manual measurement and to facilitate facial anthropometry analysis before and after surgery with high accuracy. Also, the normative data for Iranian women can be used as a guide for the diagnosis and planning of oral and maxillofacial, ENT, and plastic surgeries. This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors www.springer.com/00266 .

CMF-Net: craniomaxillofacial landmark localization on CBCT images using geometric constraint and transformer

Accurate and robust anatomical landmark localization is a mandatory and crucial step in deformation diagnosis and treatment planning for patients with craniomaxillofacial (CMF) malformations. In this paper, we propose a trainable end-to-end cephalometric landmark localization framework on Cone-beam computed tomography (CBCT) scans, referred to as CMF-Net, which combines the appearance with transformers, geometric constraint, and adaptive wing (AWing) loss. More precisely: (1) we decompose the localization task into two branches: the appearance branch integrates transformers for identifying the exact positions of candidates, while the geometric constraint branch at low resolution allows the implicit spatial relationships to be effectively learned on the reduced training data. (2) We use the AWing loss to leverage the difference between the pixel values of the target heatmaps and the automatic prediction heatmaps. We verify our CMF-Net by identifying the 24 most relevant clinical landmarks on 150 dental CBCT scans with complicated scenarios collected from real-world clinics. Comprehensive experiments show that it performs better than the state-of-the-art deep learning methods, with an average localization error of 1.108 mm (the clinically acceptable precision range being 1.5 mm) and a correct landmark detection rate equal to 79.28%. Our CMF-Net is time-efficient and able to locate skull landmarks with high accuracy and significant robustness. This approach could be applied in 3D cephalometric measurement, analysis, and surgical planning.

Uncertainty Estimation for Heatmap-Based Landmark Localization.

Automatic anatomical landmark localization has made great strides by leveraging deep learning methods in recent years. The ability to quantify the uncertainty of these predictions is a vital component needed for these methods to be adopted in clinical settings, where it is imperative that erroneous predictions are caught and corrected. We propose Quantile Binning, a data-driven method to categorize predictions by uncertainty with estimated error bounds. Our framework can be applied to any continuous uncertainty measure, allowing straightforward identification of the best subset of predictions with accompanying estimated error bounds. We facilitate easy comparison between uncertainty measures by constructing two evaluation metrics derived from Quantile Binning. We compare and contrast three epistemic uncertainty measures (two baselines, and a proposed method combining aspects of the two), derived from two heatmap-based landmark localization model paradigms (U-Net and patch-based). We show results across three datasets, including a publicly available Cephalometric dataset. We illustrate how filtering out gross mispredictions caught in our Quantile Bins significantly improves the proportion of predictions under an acceptable error threshold. Finally, we demonstrate that Quantile Binning remains effective on landmarks with high aleatoric uncertainty caused by inherent landmark ambiguity, and offer recommendations on which uncertainty measure to use and how to use it. The code and data are available at https://github.com/schobs/qbin.

Detection of gastrointestinal tract disorders using deep learning methods from colonoscopy images and videos

Colorectal cancer (CRC) is the world's third most common cancer, with the second highest fatality rate. It is primarily the result of lower gastrointestinal tract (GI) disorders. The prevention of CRC mainly depends on the early detection and treatment of anomalies in the lower GI tract. Colonoscopy is the gold standard device used for diagnosing abnormalities in the lower GI tract as well as identifying anatomical landmarks and bowel preparation scales. However, it is time-consuming, tedious, and prone to error process, especially for those hospitals in low resource settings. Therefore, in this research, a real-time automated detection, classification, and localization of lower GI tract pre-colorectal cancerous abnormalities were done. The proposed system enables real-time detection, classification, and localization of common pathology, anatomical landmarks, and bowel preparation scale from colonoscopy images. To do the research, data was gathered both online (at hyper k-vasir dataset) and locally from the Yanet Internal Specialized Center and the Ethio-Tebib Hospital. Data augmentation techniques were applied to increase the training dataset. The pre-trained transfer learning SSD, YOLOv4, and YOLOv5 object detection model was used to develop the system with minimal fine-tuning of the hyper parameters and their performance was compared. The Yolo v5 model achieves good precision, recall, and mean average precision (mAP), 99.071%, 98.064% and 98.8%, respectively, on the testing data set. The developed artificial intelligence-based module would have the potential to assist gastroenterologists and general practitioners in decision-making. Even though the proposed work achieved the best performance, further improvement is required by increasing the size of the dataset to include other GI tract disease diagnoses.

Ultrasonography of the pediatric elbow.

Ultrasound of the elbow can be incorporated into routine pediatric practice in the radiology department, emergency department, orthopedic clinic, and interventional suite. Ultrasound is complementary to radiography and magnetic resonance imaging for the evaluation of elbow pain in athletes with overhead activities or valgus stress, focusing on the ulnar collateral ligament medially and capitellum laterally. As a primary imaging modality, ultrasound can be used for a variety of indications including inflammatory arthritis, fracture diagnosis, and ulnar neuritis/subluxation. Ultrasound is also well-suited to guide diagnostic and therapeutic elbow joint interventions with precise localization of anatomic landmarks and needle placement. Here, we describe technical aspects of elbow ultrasound and illustrate its application in pediatric patients from infants to teen athletes.

A deep learning approach for fully automated cardiac shape modeling in tetralogy of Fallot

BackgroundCardiac shape modeling is a useful computational tool that has provided quantitative insights into the mechanisms underlying dysfunction in heart disease. The manual input and time required to make cardiac shape models, however, limits their clinical utility. Here we present an end-to-end pipeline that uses deep learning for automated view classification, slice selection, phase selection, anatomical landmark localization, and myocardial image segmentation for the automated generation of three-dimensional, biventricular shape models. With this approach, we aim to make cardiac shape modeling a more robust and broadly applicable tool that has processing times consistent with clinical workflows. MethodsCardiovascular magnetic resonance (CMR) images from a cohort of 123 patients with repaired tetralogy of Fallot (rTOF) from two internal sites were used to train and validate each step in the automated pipeline. The complete automated pipeline was tested using CMR images from a cohort of 12 rTOF patients from an internal site and 18 rTOF patients from an external site. Manually and automatically generated shape models from the test set were compared using Euclidean projection distances, global ventricular measurements, and atlas-based shape mode scores. ResultsThe mean absolute error (MAE) between manually and automatically generated shape models in the test set was similar to the voxel resolution of the original CMR images for end-diastolic models (MAE = 1.9 ± 0.5 mm) and end-systolic models (MAE = 2.1 ± 0.7 mm). Global ventricular measurements computed from automated models were in good agreement with those computed from manual models. The average mean absolute difference in shape mode Z-score between manually and automatically generated models was 0.5 standard deviations for the first 20 modes of a reference statistical shape atlas. ConclusionsUsing deep learning, accurate three-dimensional, biventricular shape models can be reliably created. This fully automated end-to-end approach dramatically reduces the manual input required to create shape models, thereby enabling the rapid analysis of large-scale datasets and the potential to deploy statistical atlas-based analyses in point-of-care clinical settings. Training data and networks are available from cardiacatlas.org.

Feature Aggregation and Refinement Network for 2D Anatomical Landmark Detection.

Localization of anatomical landmarks is essential for clinical diagnosis, treatment planning, and research. This paper proposes a novel deep network named feature aggregation and refinement network (FARNet) for automatically detecting anatomical landmarks. FARNet employs an encoder-decoder structure architecture. To alleviate the problem of limited training data in the medical domain, we adopt a backbone network pre-trained on natural images as the encoder. The decoder includes a multi-scale feature aggregation module for multi-scale feature fusion and a feature refinement module for high-resolution heatmap regression. Coarse-to-fine supervisions are applied to the two modules to facilitate end-to-end training. We further propose a novel loss function named Exponential Weighted Center loss for accurate heatmap regression, which focuses on the losses from the pixels near landmarks and suppresses the ones from far away. We evaluate FARNet on three publicly available anatomical landmark detection datasets, including cephalometric, hand, and spine radiographs. Our network achieves state-of-the-art performances on all three datasets. Code is available at https://github.com/JuvenileInWind/FARNet .

Deep learning solution for medical image localization and orientation detection.

Magnetic Resonance (MR) imaging plays an important role in medical diagnosis and biomedical research. Due to the high in-slice resolution and low through-slice resolution nature of MR imaging, the usefulness of the reconstruction highly depends on the positioning of the slice group. Traditional clinical workflow relies on time-consuming manual adjustment that cannot be easily reproduced. Automation of this task can therefore bring important benefits in terms of accuracy, speed and reproducibility. Current auto-slice-positioning methods rely on automatically detected landmarks to derive the positioning, and previous studies suggest that a large, redundant set of landmarks are required to achieve robust results. However, a costly data curation procedure is needed to generate training labels for those landmarks, and the results can still be highly sensitive to landmark detection errors. More importantly, a set of anatomical landmark locations are not naturally produced during the standard clinical workflow, which makes online learning impossible. To address these limitations, we propose a novel framework for auto-slice-positioning that focuses on localizing the canonical planes within a 3D volume. The proposed framework consists of two major steps. A multi-resolution region proposal network is first used to extract a volume-of-interest, after which a V-net-like segmentation network is applied to segment the orientation planes. Importantly, our algorithm also includes a Performance Measurement Index as an indication of the algorithm's confidence. We evaluate the proposed framework on both knee and shoulder MR scans. Our method outperforms state-of-the-art automatic positioning algorithms in terms of accuracy and robustness.

Deep learning for standardized, MRI-based quantification of subcutaneous and subfascial tissue volume for patients with lipedema and lymphedema

ObjectivesTo contribute to a more in-depth assessment of shape, volume, and asymmetry of the lower extremities in patients with lipedema or lymphedema utilizing volume information from MR imaging.MethodsA deep learning (DL) pipeline was developed including (i) localization of anatomical landmarks (femoral heads, symphysis, knees, ankles) and (ii) quality-assured tissue segmentation to enable standardized quantification of subcutaneous (SCT) and subfascial tissue (SFT) volumes. The retrospectively derived dataset for method development consisted of 45 patients (42 female, 44.2 ± 14.8 years) who underwent clinical 3D DIXON MR-lymphangiography examinations of the lower extremities. Five-fold cross-validated training was performed on 16,573 axial slices from 40 patients and testing on 2187 axial slices from 5 patients. For landmark detection, two EfficientNet-B1 convolutional neural networks (CNNs) were applied in an ensemble. One determines the relative foot-head position of each axial slice with respect to the landmarks by regression, the other identifies all landmarks in coronal reconstructed slices using keypoint detection. After landmark detection, segmentation of SCT and SFT was performed on axial slices employing a U-Net architecture with EfficientNet-B1 as encoder. Finally, the determined landmarks were used for standardized analysis and visualization of tissue volume, distribution, and symmetry, independent of leg length, slice thickness, and patient position.ResultsExcellent test results were observed for landmark detection (z-deviation = 4.5 ± 3.1 mm) and segmentation (Dice score: SCT = 0.989 ± 0.004, SFT = 0.994 ± 0.002).ConclusionsThe proposed DL pipeline allows for standardized analysis of tissue volume and distribution and may assist in diagnosis of lipedema and lymphedema or monitoring of conservative and surgical treatments.Key Points• Efficient use of volume information that MRI inherently provides can be extracted automatically by deep learning and enables in-depth assessment of tissue volumes in lipedema and lymphedema.• The deep learning pipeline consisting of body part regression, keypoint detection, and quality-assured tissue segmentation provides detailed information about the volume, distribution, and asymmetry of lower extremity tissues, independent of leg length, slice thickness, and patient position.

Automatic Landmark Detection of Human Back Surface from Depth Images via Deep Learning

Studying human postural structure is one of the challenging issues among scholars and physicians. The spine is known as the central axis of the body, and due to various genetic and environmental reasons, it could suffer from deformities leading to physical dysfunction and correspondingly affecting people's quality of life. Radiography is the most common method for detecting these deformities; however, it frequently exposes the patient to X-rays and ionization and consequently increases cancer risk in patients particularly children and pregnant women.The purpose of this research is to provide an entirely safe and non-invasive method to examine the spiral structure and its deformities. Hence, it is attempted to find the exact location of anatomical landmarks on the human back surface, which provides useful and functional information about the status of the human postural structure to the physician. In this study, using Microsoft Kinect sensor, the depth images from the human back surface of 105 people were recorded and, our proposed approach – a deep convolution neural network- was used as a model to estimate the locations of anatomical landmarks. In network architecture, two learning processes, including landmark position and affinity between the two associated landmarks, are successively performed in two separate branches. This is a bottom-up approach; thus, the runtime complexity is considerably reduced, and then the resulting anatomical points are evaluated concerning manual landmarks marked by the operator as the benchmark. Our results showed 86.9% of PDJ and 80% of PCK that demonstrate more effectiveness compared to other methods.

Automatic Landmark Detection of Human Back Surface from Depth Images via Deep Learning

Studying human postural structure is one of the challenging issues among scholars and physicians. The spine is known as the central axis of the body, and due to various genetic and environmental reasons, it could suffer from deformities leading to physical dysfunction and correspondingly affecting people's quality of life. Radiography is the most common method for detecting these deformities; however, it frequently exposes the patient to X-rays and ionization and consequently increases cancer risk in patients particularly children and pregnant women.The purpose of this research is to provide an entirely safe and non-invasive method to examine the spiral structure and its deformities. Hence, it is attempted to find the exact location of anatomical landmarks on the human back surface, which provides useful and functional information about the status of the human postural structure to the physician. In this study, using Microsoft Kinect sensor, the depth images from the human back surface of 105 people were recorded and, our proposed approach – a deep convolution neural network- was used as a model to estimate the locations of anatomical landmarks. In network architecture, two learning processes, including landmark position and affinity between the two associated landmarks, are successively performed in two separate branches. This is a bottom-up approach; thus, the runtime complexity is considerably reduced, and then the resulting anatomical points are evaluated concerning manual landmarks marked by the operator as the benchmark. Our results showed 86.9% of PDJ and 80% of PCK that demonstrate more effectiveness compared to other methods.

A novel method for 3D knee anatomical landmark localization by combining global and local features

A novel method for 3D knee anatomical landmark localization by combining global and local features

Bone anatomical landmark localization with cascaded spatial configuration network

Accurate localization of bone anatomical landmarks in medical images is of great significance for decision-making in surgical plans and orthopedic surgery visual navigation systems development. However, due to the high dimensionality and large size of medical images, current automatic anatomical landmarks detection methods still have issues in terms of accuracy, robustness, and efficiency. To improve the accuracy of bone anatomical landmark detection in medical images, a new network model with a two-step strategy (coarse localizing and fine localizing) was proposed in this study. By combining fully convolutional neural networks and the heatmap regression model, a cascaded spatial configuration network was designed to combine the global and local features of 3D image features and localize anatomical landmarks stepwise. The model was evaluated with a collected knee CT image dataset and a published spine CT image dataset. The results were compared to existing state-of-the-art models. The proposed method outperformed other models with an average error of 1.31 mm for knee landmarks and 5.31 mm for the localization of spine landmarks. The outlier rates at error radius 3 mm, 5 mm, and 7 mm are also smaller compared with other models, indicating good robustness of the model. Our proposed method provides a new neural network model with reasonably good accuracy and robustness with a limited computational cost for landmark localization tasks.

Bone anatomical landmark localization with cascaded spatial configuration network

Abstract Accurate localization of bone anatomical landmarks in medical images is of great significance for decision-making in surgical plans and orthopedic surgery visual navigation systems development. However, due to the high dimensionality and large size of medical images, current automatic anatomical landmarks detection methods still have issues in terms of accuracy, robustness, and efficiency. To improve the accuracy of bone anatomical landmark detection in medical images, a new network model with a two-step strategy (coarse localizing and fine localizing) was proposed in this study. By combining fully convolutional neural networks and the heatmap regression model, a cascaded spatial configuration network was designed to combine the global and local features of 3D image features and localize anatomical landmarks stepwise. The model was evaluated with the collected knee CT image dataset and published spine CT image dataset. The results were compared to existing state-of-the-art models. The proposed method outperformed other models with an average error of 1.31 mm for knee landmarks and 5.31 mm for the localization of spine landmarks. The outlier rates at error radius 3mm, 5mm, and 7mm are also smaller compared with other models, indicating good robustness of the model. Our proposed method provides a new neural network model with reasonably good accuracy and robustness with a limited computational cost for landmark localization tasks.

Learning to Localize Cross-Anatomy Landmarks in X-Ray Images with a Universal Model.

Objective and Impact Statement. In this work, we develop a universal anatomical landmark detection model which learns once from multiple datasets corresponding to different anatomical regions. Compared with the conventional model trained on a single dataset, this universal model not only is more light weighted and easier to train but also improves the accuracy of the anatomical landmark location. Introduction. The accurate and automatic localization of anatomical landmarks plays an essential role in medical image analysis. However, recent deep learning-based methods only utilize limited data from a single dataset. It is promising and desirable to build a model learned from different regions which harnesses the power of big data. Methods. Our model consists of a local network and a global network, which capture local features and global features, respectively. The local network is a fully convolutional network built up with depth-wise separable convolutions, and the global network uses dilated convolution to enlarge the receptive field to model global dependencies. Results. We evaluate our model on four 2D X-ray image datasets totaling 1710 images and 72 landmarks in four anatomical regions. Extensive experimental results show that our model improves the detection accuracy compared to the state-of-the-art methods. Conclusion. Our model makes the first attempt to train a single network on multiple datasets for landmark detection. Experimental results qualitatively and quantitatively show that our proposed model performs better than other models trained on multiple datasets and even better than models trained on a single dataset separately.

A reliability study of anatomical landmark localization in computational brain model for transcranial direct current stimulation

A reliability study of anatomical landmark localization in computational brain model for transcranial direct current stimulation

Aortography Keypoint Tracking for Transcatheter Aortic Valve Implantation Based on Multi-Task Learning.

Currently, transcatheter aortic valve implantation (TAVI) represents the most efficient treatment option for patients with aortic stenosis, yet its clinical outcomes largely depend on the accuracy of valve positioning that is frequently complicated when routine imaging modalities are applied. Therefore, existing limitations of perioperative imaging underscore the need for the development of novel visual assistance systems enabling accurate procedures. In this paper, we propose an original multi-task learning-based algorithm for tracking the location of anatomical landmarks and labeling critical keypoints on both aortic valve and delivery system during TAVI. In order to optimize the speed and precision of labeling, we designed nine neural networks and then tested them to predict 11 keypoints of interest. These models were based on a variety of neural network architectures, namely MobileNet V2, ResNet V2, Inception V3, Inception ResNet V2 and EfficientNet B5. During training and validation, ResNet V2 and MobileNet V2 architectures showed the best prediction accuracy/time ratio, predicting keypoint labels and coordinates with 97/96% accuracy and 4.7/5.6% mean absolute error, respectively. Our study provides evidence that neural networks with these architectures are capable to perform real-time predictions of aortic valve and delivery system location, thereby contributing to the proper valve positioning during TAVI.