Landmark Detection Research Articles

An automated framework for pediatric hip surveillance and severity assessment using radiographs.

Hip dysplasia is the second most common orthopedic condition in children with cerebral palsy (CP) and may result in disability and pain. The migration percentage (MP) is a widely used metric in hip surveillance, calculated based on an anterior-posterior pelvis radiograph. However, manual quantification of MP values using hip X-ray scans in current standard practice has challenges including being time-intensive, requiring expert knowledge, and not considering human bias. The purpose of this study is to develop a machine learning algorithm to automatically quantify MP values using a hip X-ray scan, and hence provide an assessment for severity, which then can be used for surveillance, treatment planning, and management. X-ray scans from 210 patients were curated, pre-processed, and manually annotated at our clinical center. Several machine learning models were trained using pre-trained weights from Inception ResNet-V2, VGG-16, and VGG-19, with different strategies (pre-processing, with and without region of interest (ROI) detection, with and without data augmentation) to find an optimal model for automatic hip landmarking. The predicted landmarks were then used by our geometric algorithm to quantify the MP value for the input hip X-ray scan. The pre-trained VGG-19 model, fine-tuned with additional custom layers, outputted the lowest mean squared error values for both train and test data, when ROI cropped images were used along with data augmentation for model training. The MP value calculated by the algorithm was compared to manual ground truth labels from our orthopedic fellows using the hip screen application for benchmarking. The results showed the feasibility of the machine learning model in automatic hip landmark detection for reliably quantifying MP value from hip X-ray scans. The algorithm could be used as an accurate and reliable tool in orthopedic care for diagnosing, severity assessment, and hence treatment and surgical planning for hip displacement.

AnyFace++: Deep Multi-Task, Multi-Domain Learning for Efficient Face AI.

Accurate face detection and subsequent localization of facial landmarks are mandatory steps in many computer vision applications, such as emotion recognition, age estimation, and gender identification. Thanks to advancements in deep learning, numerous facial applications have been developed for human faces. However, most have to employ multiple models to accomplish several tasks simultaneously. As a result, they require more memory usage and increased inference time. Also, less attention is paid to other domains, such as animals and cartoon characters. To address these challenges, we propose an input-agnostic face model, AnyFace++, to perform multiple face-related tasks concurrently. The tasks are face detection and prediction of facial landmarks for human, animal, and cartoon faces, including age estimation, gender classification, and emotion recognition for human faces. We trained the model using deep multi-task, multi-domain learning with a heterogeneous cost function. The experimental results demonstrate that AnyFace++ generates outcomes comparable to cutting-edge models designed for specific domains.

Cone-beam CT landmark detection for measuring basal bone width: a retrospective validation study

BackgroundAccurate assessment of basal bone width is essential for distinguishing individuals with normal occlusion from patients with maxillary transverse deficiency who may require maxillary expansion. Herein, we evaluated the effectiveness of a deep learning (DL) model in measuring landmarks of basal bone width and assessed the consistency of automated measurements compared to manual measurements.MethodsBased on the U-Net algorithm, a coarse-to-fine DL model was developed and trained using 80 cone-beam computed tomography (CBCT) images. The model’s prediction capabilities were validated on 10 CBCT scans and tested on an additional 34. To evaluate the performance of the DL model, its measurements were compared with those taken manually by one junior orthodontist using the concordance correlation coefficient (CCC).ResultsIt took approximately 1.5 s for the DL model to perform the measurement task in only CBCT images. This framework showed a mean radial error of 1.22 ± 1.93 mm and achieved successful detection rates of 71.34%, 81.37%, 86.77%, and 91.18% in the 2.0-, 2.5-, 3.0-, and 4.0-mm ranges, respectively. The CCCs (95% confidence interval) of the maxillary basal bone width and mandibular basal bone width distance between the DL model and manual measurement for the 34 cases were 0.96 (0.94–0.97) and 0.98 (0.97–0.99), respectively.ConclusionThe novel DL framework developed in this study improved the diagnostic accuracy of the individual assessment of maxillary width. These results emphasize the potential applicability of this framework as a computer-aided diagnostic tool in orthodontic practice.

Efficient Detection of Palm and Hand Landmark for Speech Impaired People Using Mediapipe Model

Efficient Detection of Palm and Hand Landmark for Speech Impaired People Using Mediapipe Model

PE096 Automated anatomical recognition: Feasibility of landmark detection and generalization between robotic systems in robot-assisted partial nephrectomy

PE096 Automated anatomical recognition: Feasibility of landmark detection and generalization between robotic systems in robot-assisted partial nephrectomy

DualFaceNet: augmentation consistency for optimal facial landmark detection and face mask classification

<p>In an era where face masks are commonplace, facial recognition faces new challenges and opportunities. This study introduces DualFaceNet (DFN), a cutting-edge neural network that efficiently combines facial landmark detection with mask classification. Benefiting from multi-task learning (MTL) and enhanced with a unique consistency loss, DFN outperforms traditional single-task models. Tests using the reputable 300W dataset and a face mask dataset showcase DFN’s strengths: a significant reduction in landmark error to 5.42 and an increase in mask classification accuracy to 92.59%. These results highlight the potential of integrating MTL and custom loss functions in facial recognition. As face masks continue to be globally essential, DFN’s integrated approach offers a fresh perspective in facial recognition studies. Furthermore, DFN paves the way for adaptive facial recognition systems, emphasizing the adaptability needed in our current era.</p>

1171: Automated paediatric vertebrae landmark detection & growth measurement: AI vs analytical approaches

1171: Automated paediatric vertebrae landmark detection & growth measurement: AI vs analytical approaches

Exploring Attentive User Interface Input via Raspberry Pi, Based on Face Landmark Detection, Eye Open-Closed Detection and Head Movements Detection

Exploring Attentive User Interface Input via Raspberry Pi, Based on Face Landmark Detection, Eye Open-Closed Detection and Head Movements Detection

A Survey Study of the 3D Facial Landmark Detection Techniques Used as a Screening Tool for Diagnosis of the Obstructive Sleep Apnea Syndrome.

Obstructive Sleep Apnea (OSA) is a common disorder affecting both adults and children. It is characterized by repeated episodes of apnea (stopped breathing) and hypopnea (reduced breathing), which result in intermittent hypoxia. We recognize pediatric and adult OSA, and this paper focuses on pediatric OSA. While adults often suffer from daytime sleepiness, children are more likely to develop behavioral abnormalities. Early diagnosis and treatment are important to prevent negative effects on children's development. Without the treatment, children may be at increased risk of developing high blood pressure or other heart problems. The gold standard for OSA diagnosis is the polysomnography (sleep study) PSG performed at a sleep center. Not only is it an expensive procedure, but it can also be very stressful, especially for children. Patients have to stay at the sleep center during the night. Therefore, screening tools are very important. Multiple studies have shown that OSA screening tools can be based on facial anatomical landmarks. Anatomical landmarks are landmarks located at specific anatomical locations. For the purpose of the screening tool, a specific list of anatomical locations needs to be identified. We are presenting a survey study of the automatic identification of these landmarks on 3D scans of the patient's head. We are considering and comparing both knowledge-based and AI-based identification techniques, with a focus on the development of the automatic OSA screening tool.

A Study on Facial Landmark Detection and Image Processing

A Study on Facial Landmark Detection and Image Processing

Heatmap-Based Active Shape Model for Landmark Detection in Lumbar X-ray Images.

Medical staff inspect lumbar X-ray images to diagnose lumbar spine diseases, and the analysis process is currently automated using deep-learning techniques. The detection of landmarks is necessary in the automatic process of localizing the position and identifying the morphological features of the vertebrae. However, detection errors may occur owing to the noise and ambiguity of images, as well as individual variations in the shape of the lumbar vertebrae. This study proposes a method to improve the robustness of landmark detection results. This method assumes that landmarks are detected by a convolutional neural network-based two-step model consisting of Pose-Net and M-Net. The model generates a heatmap response to indicate the probable landmark positions. The proposed method then corrects the landmark positions using the heatmap response and active shape model, which employs statistical information on the landmark distribution. Experiments were conducted using 3600 lumbar X-ray images, and the results showed that the landmark detection error was reduced by the proposed method. The average value of maximum errors decreased by 5.58% after applying the proposed method, which combines the outstanding image analysis capabilities of deep learning with statistical shape constraints on landmark distribution. The proposed method could also be easily integrated with other techniques to increase the robustness of landmark detection results such as CoordConv layers and non-directional part affinity field. This resulted in a further enhancement in the landmark detection performance. These advantages can improve the reliability of automatic systems used to inspect lumbar X-ray images. This will benefit both patients and medical staff by reducing medical expenses and increasing diagnostic efficiency.

Involution Transformer Based U-Net for Landmark Detection in Ultrasound Images for Diagnosis of Infantile DDH.

The B-mode ultrasound based computer-aided diagnosis (CAD) has demonstrated its effectiveness for diagnosis of Developmental Dysplasia of the Hip (DDH) in infants, which can conduct the Graf's method by detecting landmarks in hip ultrasound images. However, it is still necessary to explore more valuable information around these landmarks to enhance feature representation for improving detection performance in the detection model. To this end, a novel Involution Transformer based U-Net (IT-UNet) network is proposed for hip landmark detection. The IT-UNet integrates the efficient involution operation into Transformer to develop an Involution Transformer module (ITM), which consists of an involution attention block and a squeeze-and-excitation involution block. The ITM can capture both the spatial-related information and long-range dependencies from hip ultrasound images to effectively improve feature representation. Moreover, an Involution Downsampling block (IDB) is developed to alleviate the issue of feature loss in the encoder modules, which combines involution and convolution for the purpose of downsampling. The experimental results on two DDH ultrasound datasets indicate that the proposed IT-UNet achieves the best landmark detection performance, indicating its potential applications.

Precise facial landmark detection by Dynamic Semantic Aggregation Transformer

At present, deep neural network methods have played a dominant role in face alignment field. However, they generally use predefined network structures to predict landmarks, which tends to learn general features and leads to mediocre performance, e.g., they perform well on neutral samples but struggle with faces exhibiting large poses or occlusions. Moreover, they cannot effectively deal with semantic gaps and ambiguities among features at different scales, which may hinder them from learning efficient features. To address the above issues, in this paper, we propose a Dynamic Semantic-Aggregation Transformer (DSAT) for more discriminative and representative feature (i.e., specialized feature) learning. Specifically, a Dynamic Semantic-Aware (DSA) model is first proposed to partition samples into subsets and activate the specific pathways for them by estimating the semantic correlations of feature channels, making it possible to learn specialized features from each subset. Then, a novel Dynamic Semantic Specialization (DSS) model is designed to mine the homogeneous information from features at different scales for eliminating the semantic gap and ambiguities and enhancing the representation ability. Finally, by integrating the DSA model and DSS model into our proposed DSAT in both dynamic architecture and dynamic parameter manners, more specialized features can be learned for achieving more precise face alignment. It is interesting to show that harder samples can be handled by activating more feature channels. Extensive experiments on popular face alignment datasets demonstrate that our proposed DSAT outperforms state-of-the-art models in the literature. Our code is available at https://github.com/GERMINO-LiuHe/DSAT.

A two-stage regression framework for automated cephalometric landmark detection incorporating semantically fused anatomical features and multi-head refinement loss

Accurate identification and precise localization of cephalometric landmarks provide clinicians with essential insights into craniofacial deformities, aiding in the assessment of treatment strategies for improved patient outcomes. The current methodologies heavily depend on the utilization of multiple CNNs for predicting landmark coordinates, which makes them computationally burdensome and unsuitable for translation to clinical applications. To overcome this limitation, we propose a novel, end-to-end trainable, two-stage regression framework for cephalometric landmark detection. In the initial stage, a single neural network is employed to estimate the locations of all landmarks simultaneously, enabling the identification of potential landmark regions. In the second stage, a semantic fusion block leverages the in-network multi-resolution feature hierarchy to produce high-level semantically rich features. These feature maps are then cropped based on coarsely detected landmark locations and concurrent refinement loss is used to fine-tune and refine the landmark locations. The proposed framework demonstrates the potential for enhancing clinical workflow and treatment outcomes in orthodontics. This is achieved through the utilization of a single CNN backbone augmented with multi-resolution semantically fused anatomical features, which effectively enhances representation learning in a computationally efficient manner. The performance of the proposed framework is evaluated on two publicly available anatomical landmark datasets. The experimental results demonstrate that our framework achieves a state-of-the-art detection accuracy of 87.17% within the clinically accepted range of 2 mm. The source code and the pre-trained weights are made publicly available at this https://github.com/manwaarkhd/CEPHMark-Net, promoting reproducibility and enabling further advancements.

Bifurcation detection in intravascular optical coherence tomography using vision transformer based deep learning

Objective. Bifurcation detection in intravascular optical coherence tomography (IVOCT) images plays a significant role in guiding optimal revascularization strategies for percutaneous coronary intervention (PCI). We propose a bifurcation detection method using vision transformer (ViT) based deep learning in IVOCT. Approach. Instead of relying on lumen segmentation, the proposed method identifies the bifurcation image using a ViT-based classification model and then estimate bifurcation ostium points by a ViT-based landmark detection model. Main results. By processing 8640 clinical images, the Accuracy and F1-score of bifurcation identification by the proposed ViT-based model are 2.54% and 16.08% higher than that of traditional non-deep learning methods, are similar to the best performance of convolutional neural networks (CNNs) based methods, respectively. The ostium distance error of the ViT-based model is 0.305 mm, which is reduced 68.5% compared with the traditional non-deep learning method and reduced 24.81% compared with the best performance of CNNs based methods. The results also show that the proposed ViT-based method achieves the highest success detection rate are 11.3% and 29.2% higher than the non-deep learning method, and 4.6% and 2.5% higher than the best performance of CNNs based methods when the distance section is 0.1 and 0.2 mm, respectively. Significance. The proposed ViT-based method enhances the performance of bifurcation detection of IVOCT images, which maintains a high correlation and consistency between the automatic detection results and the expert manual results. It is of great significance in guiding the selection of PCI treatment strategies.

Micro-expression recognition based on a novel GCN-transformer cooperation model for IoT-eHealth

If a person is truly healthy, his/her well-being encompasses both physical and psychological health. However, the existing IoT-eHealth system typically focus only on monitoring the user’s physical data through various sensors, neglecting their mental state. To enhance the intelligence level of IoT-eHealth system and enable it to have the psychological monitoring ability, a novel collaborative model based on Graph Convolutional Network (GCN) and Transformer is designed for Micro-Expression (ME) recognition in this paper. Firstly, facial information within each frame is transformed into a Spatial Topological Relationship Graph (STRG) by using facial landmarks detection and psychological relationship of local patches. Then, in order to automatically aggregate the key information on facial patches that contribute to ME recognition from the structured graph data, a Hierarchical Adaptive Graph Pooling (HAGP) module is designed for obtaining discriminative frame-level feature based on GCN utilizing graph structure and vertex global dependencies. Finally, in order to model the long-term dependencies among frames and capture the key frame-level features that are beneficial for ME recognition, a Temporal Sensitive Self-Attention (TSSA) mechanism is designed, and a novel Temporal Sensitive Transformer (TST) encoder is constructed based on TSSA to explore the evolution law of facial patterns and obtain discriminative video-level features that are helpful for ME recognition. In the comparative experiments of standard dataset verification and practical dataset testing, designed collaborative model is superior to other methods and can achieve the highest recognition accuracy, which almost can meet the application requirements of IoT-eHealth system.

UniverDetect: Universal landmark detection method for multidomain X-ray images

Landmark detection using X-ray imaging is vital for disease screening, treatment, and prognosis. It provides a framework for subsequent tasks, including segmentation, classification, and target detection. Existing landmark detection methods exhibit limits with the increasing number of X-ray examiners. Not only they serve specific data types, but also their ability to learn the complete semantic information from this dataset is limited, potentially affecting their widespread adoption. We address these challenges by proposing a universal landmark detection model for multidomain X-ray imaging, UniverDetect. By progressively acquiring local semantic information, global semantic insights, and anatomical structure knowledge at various levels, the model ensured that the detected landmarks were closely aligned with the ground-truth labels. UniverDetect consists of three key components. The landmark detection module (LDM) utilizes a U-net network equipped with our innovative pyramid depthwise separable (PDS) convolution module for initial landmark detection. The landmark refinement module (LRM) integrates a fine-tuning module comprising continuously extended convolution blocks. Finally, the landmark correction module (LCM) incorporates a graph convolutional network (GCN) to rectify offset errors during partial landmark detection. An inherent feature of this model is its domain- generalization capability, which enables continuous learning across diverse domains. This model can concurrently learn from eight domains, covering 118 landmarks within a diverse dataset of 5969 images. Extensive experiments on multiple datasets demonstrate that this method consistently outperforms state-of-the-art approaches. This study presents a versatile and effective solution to reduce doctors’ workload while providing precise quantitative analysis.

Cascaded learning with transformer for simultaneous eye landmark, eye state and gaze estimation

Eye tracking have garnered attention in human–machine interaction, disease monitoring, biometrics, etc. Existing investigations for eye tracking have predominantly concentrated on individual task for pupil detection or gaze estimation, overlooking the implicit relationships that exist among different tasks for eye tracking. In this work, we introduce a cascaded framework with transformer to collaboratively realize eye landmark detection, eye state detection and gaze estimation. Within our framework, we leverage Transformer to capture long dependencies with explicit eye-related structural information and implicit correlation among different tasks. Furthermore, the proposed cascade iteration framework alternatively optimize each task and boost the overall performance for pupil center, eye state and gaze estimation simultaneously. To address the problem of manual annotation, we further introduce the Control-Eye Diffusion Model (CEDM), a controllable eye image generation method conditioned on a simple contour with structure information. The proposed methods are evaluated on challenging datasets such as GI4E, BioID and MPIIGaze, and the results show that our methods outperform state-of-the-art methods in several tasks.

Automatic soft-tissue analysis on orthodontic frontal and lateral facial photographs based on deep learning.

To establish the automatic soft-tissue analysis model based on deep learning that performs landmark detection and measurement calculations on orthodontic facial photographs to achieve a more comprehensive quantitative evaluation of soft tissues. A total of 578 frontal photographs and 450 lateral photographs of orthodontic patients were collected to construct datasets. All images were manually annotated by two orthodontists with 43 frontal-image landmarks and 17 lateral-image landmarks. Automatic landmark detection models were established, which consisted of a high-resolution network, a feature fusion module based on depthwise separable convolution, and a prediction model based on pixel shuffle. Ten measurements for frontal images and eight measurements for lateral images were defined. Test sets were used to evaluate the model performance, respectively. The mean radial error of landmarks and measurement error were calculated and statistically analysed to evaluate their reliability. The mean radial error was 14.44 ± 17.20 pixels for the landmarks in the frontal images and 13.48 ± 17.12 pixels for the landmarks in the lateral images. There was no statistically significant difference between the model prediction and manual annotation measurements except for the mid facial-lower facial height index. A total of 14 measurements had a high consistency. Based on deep learning, we established automatic soft-tissue analysis models for orthodontic facial photographs that can automatically detect 43 frontal-image landmarks and 17 lateral-image landmarks while performing comprehensive soft-tissue measurements. The models can assist orthodontists in efficient and accurate quantitative soft-tissue evaluation for clinical application.

Craniomaxillofacial landmarks detection in CT scans with limited labeled data via semi-supervised learning

BackgroundThree-dimensional cephalometric analysis is crucial in craniomaxillofacial assessment, with landmarks detection in craniomaxillofacial (CMF) CT scans being a key component. However, creating robust deep learning models for this task typically requires extensive CMF CT datasets annotated by experienced medical professionals, a process that is time-consuming and labor-intensive. Conversely, acquiring large volume of unlabeled CMF CT data is relatively straightforward. Thus, semi-supervised learning (SSL), leveraging limited labeled data supplemented by sufficient unlabeled dataset, could be a viable solution to this challenge. MethodWe developed an SSL model, named CephaloMatch, based on a strong-weak perturbation consistency framework. The proposed SSL model incorporates a head position rectification technique through coarse detection to enhance consistency between labeled and unlabeled datasets and a multilayers perturbation method which is employed to expand the perturbation space. The proposed SSL model was assessed using 362 CMF CT scans, divided into a training set (60 scans), a validation set (14 scans), and an unlabeled set (288 scans). ResultThe proposed SSL model attained a detection error of 1.60 ± 0.87 mm, significantly surpassing the performance of conventional fully supervised learning model (1.94 ± 1.12 mm). Notably, the proposed SSL model achieved equivalent detection accuracy (1.91 ± 1.00 mm) with only half the labeled dataset, compared to the fully supervised learning model. ConclusionsThe proposed SSL model demonstrated exceptional performance in landmarks detection using a limited labeled CMF CT dataset, significantly reducing the workload of medical professionals and enhances the accuracy of 3D cephalometric analysis.