Landmark Detection Research Articles

Feasibility of real-time compression frequency and compression depth assessment in CPR using a “machine-learning” artificial intelligence tool

BackgroundVideo assisted cardiopulmonary resuscitation (V-CPR) has demonstrated to be efficient in improving CPR quality and patient outcomes, as Emergency Medical Service (EMS) dispatchers can use the video stream of a caller for diagnostic purposes, give instructions and corrections in a CPR scenario. However, the new challenges faced by EMS dispatchers during video-guided CPR (V-CPR)—such as analyzing the video stream, providing feedback to the caller, and managing stress—demand innovative solutions. This study explores the feasibility of incorporating an open-source “machine-learning” tool (artificial intelligence – AI), to evaluate the feasibility and accuracy in correctly detecting the actual compression frequency and compression depth in video footage of a simulated CPR. DesignMediaPipe Pose Landmark Detection (Google LLC, Mountain View, CA, USA), an open-source AI software using “machine-learning” models to detect human bodies in images and videos, was programmed to assess compression frequency an depth in nine videos, showing CPR on a resuscitation manikin. Compression frequency and depth were assessed from compression to compression with AI software and the manikin’s internal software (QCPR, Laerdal, Stavanger, Norway). After testing for Gaussian distribution, means of non-gaussian data were compared using Wilcoxon matched-pairs signed rank test and the Bland Altman method. Main resultsMediaPipe Pose Landmark Detection successfully identified and tracked the person performing CPR in all nine video sequences. There were high levels of agreement between compression frequencies derived from AI and manikin’s software. However, the precision of compression depth showed major inaccuracies and was overall not accurate. ConclusionsThis feasibility study demonstrates the potential of open-source “machine-learning” tools in providing real-time feedback on V-CPR video sequences. In this pilot study, an open-source landmark detection AI software was able to assess CPR compression frequency with high agreement to actual frequency derived from the CPR manikin. For compression depth, its performance regarding compression depth was not accurate yet, suggesting the need for adjustment. Since the software used is currently not intended for medical use, further development is necessary before the technology can be evaluated in realCPR.

Reduced dose helical CT scout imaging on next generation wide volume CT system decreases scan length and overall radiation exposure

PurposeTraditional CT acquisition planning is based on scout projection images from planar anterior-posterior and lateral projections where the radiographer estimates organ locations. Alternatively, a new scout method utilizing ultra-low dose helical CT (3D Landmark Scan) offers cross-sectional imaging to identify anatomic structures in conjunction with artificial intelligence based Anatomic Landmark Detection (ALD) for automatic CT acquisition planning. The purpose of this study is to quantify changes in scan length and radiation dose of CT examinations planned using 3D Landmark Scan and ALD and performed on next generation wide volume CT versus examinations planned using traditional scout methods. We additionally aim to quantify changes in radiation dose reduction of scans planned with 3D Landmark Scan and performed on next generation wide volume CT. MethodsSingle-center retrospective analysis of consecutive patients with prior CT scan of the same organ who underwent clinical CT using 3D Landmark Scan and automatic scan planning. Acquisition length and dose-length-product (DLP) were collected. Data was analyzed by paired t-tests. Results104 total CT examinations (48.1 % chest, 15.4 % abdomen, 36.5 % chest/abdomen/pelvis) on 61 individual consecutive patients at a single center were retrospectively analyzed. 79.8 % of scans using 3D Landmark Scan had reduction in acquisition length compared to the respective prior acquisition. Median acquisition length using 3D Landmark Scan was 26.7 mm shorter than that using traditional scout methods (p < 0.001) with a 23.3 % median total radiation dose reduction (245.6 (IQR 150.0–400.8) mGy cm vs 320.3 (IQR 184.1–547.9) mGy cm). CT dose index similarly was overall decreased for scans planned with 3D Landmark and ALD and performed on next generation CT versus traditional methods (4.85 (IQR 3.8–7) mGy vs. 6.70 (IQR 4.43–9.18) mGy, respectively, p < 0.001). ConclusionScout imaging using reduced dose 3D Landmark Scan images and Anatomic Landmark Detection reduces acquisition range in chest, abdomen, and chest/abdomen/pelvis CT scans. This technology, in combination with next generation wide volume CT reduces total radiation dose.

Abstract Su802: Feasibility of detecting CPR compression frequency using machine-learning software in video footage of CPR

Background: Several studies have demonstrated that video assisted cardiopulmonary resuscitation (V-CPR) is efficient in improving cardiopulmonary resuscitation (CPR) quality and patient outcome. However, currently the technique depends on 911 dispatchers who evaluate the video footage and may thus be subject to inter-operator variations. Research Question: We hypothesized that using an open-source machine learning supported software (“artificial intelligence” – AI) would be able to provide real-time feedback on CPR frequency in a video of CPR. Aims: To evaluate the feasibility and accuracy of a selected AI software to correctly detect the actual compression frequency in video footage of CPR performed in a CPR manikin. Methods: MediaPipe Pose Landmark Detection (Google LLC, Mountain View, CA, USA), an open-source AI software using machine-learning models to detect human body postures in images and videos, was programmed to assess compression frequency of nine videos showing CPR on a manikin, each with a different fixed compression frequency and compression depth. Camera angle and distance were standardized. Frequency was assessed from compression to compression with AI software and the manikin’s internal software (QCPR, Laerdal, Stavanger, Norway). After testing for Gaussian distribution, means were compared using Wilcoxon matched-pairs signed rank test. Results: There were high levels of agreement between compression frequencies derived from AI and manikin’s software (Statistic comparisons are presented in Figure 1, correlations of measurements are presented in Figure 2 ). Conclusions: In this pilot study, an open-source landmark detection AI software was able to assess CPR compression frequency with high agreement to actual frequency derived from a CPR manikin. Since the software used is currently not intended for medical use, further development is necessary before the technology can be evaluated for real-life CPR.

Real-time location of acupuncture points based on anatomical landmarks and pose estimation models

IntroductionPrecise identification of acupuncture points (acupoints) is essential for effective treatment, but manual location by untrained individuals can often lack accuracy and consistency. This study proposes two approaches that use artificial intelligence (AI) specifically computer vision to automatically and accurately identify acupoints on the face and hand in real-time, enhancing both precision and accessibility in acupuncture practices.MethodsThe first approach applies a real-time landmark detection system to locate 38 specific acupoints on the face and hand by translating anatomical landmarks from image data into acupoint coordinates. The second approach uses a convolutional neural network (CNN) specifically optimized for pose estimation to detect five key acupoints on the arm and hand (LI11, LI10, TE5, TE3, LI4), drawing on constrained medical imaging data for training. To validate these methods, we compared the predicted acupoint locations with those annotated by experts.ResultsBoth approaches demonstrated high accuracy, with mean localization errors of less than 5 mm when compared to expert annotations. The landmark detection system successfully mapped multiple acupoints across the face and hand even in complex imaging scenarios. The data-driven approach accurately detected five arm and hand acupoints with a mean Average Precision (mAP) of 0.99 at OKS 50%.DiscussionThese AI-driven methods establish a solid foundation for the automated localization of acupoints, enhancing both self-guided and professional acupuncture practices. By enabling precise, real-time localization of acupoints, these technologies could improve the accuracy of treatments, facilitate self-training, and increase the accessibility of acupuncture. Future developments could expand these models to include additional acupoints and incorporate them into intuitive applications for broader use.

Automatic Landmark Detection for Preoperative Planning of High Tibial Osteotomy Using Traditional Feature Extraction and Deep Learning Methods.

Automatic High Tibial Osteotomy (HTO) landmark detection methods promise to improve the effectiveness and standardisation of HTO preoperative planning. Unfortunately, due to the limited number of HTO datasets, existing methods are less robust when dealing with patients with varied deformities than traditional manual planning, severely limiting their clinical viability and application in practical surgical settings. Here, we present a new HTO landmark detection framework using an integration of optimised heatmap-offset aggregation method and traditional feature extraction. Subjective and objective approaches were employed to reflect the final clinical acceptance of our model. Average Mean Absolute Error of prediction results compared to the surgeon's gold standard was 0.35° for the hip-knee-ankle angle. The objective score rated by surgeons reached 4.4 on a scale of 5. The study demonstrated that the automatic detection method has great potential serving as an alternative to manual radiological analysis in practical surgical pre-operative planning.

Anatomical prior-based vertebral landmark detection for spinal disorder diagnosis

As one of fundamental ways to interpret spine images, detection of vertebral landmarks is an informative prerequisite for further diagnosis and management of spine disorders such as scoliosis and fractures. Most existing machine learning-based methods for automatic vertebral landmark detection suffer from overlapping landmarks or abnormally long distances between nearby landmarks against anatomical priors, and thus lack sufficient reliability and interpretability. To tackle the problem, this paper systematically utilizes anatomical prior knowledge in vertebral landmark detection. We explicitly formulate anatomical priors of the spine, related to distances among vertebrae and spatial order within the spine, and integrate these geometrical constraints within training loss, inference procedure, and evaluation metrics. First, we introduce an anatomy-constraint loss to regularize the training process with the aforementioned contextual priors explicitly. Second, we propose a simple-yet-effective anatomy-aided inference procedure by employing sequential prediction rather than a parallel counterpart. Third, we provide novel anatomy-related metrics to quantitatively evaluate to which extent landmark predictions follow the anatomical priors, as is not reflected within the widely-used landmark localization error metric. We employ the localization framework on 1410 anterior-posterior radiographic images. Compared with competitive baseline models, we achieve superior landmark localization accuracy and comparable Cobb angle estimation for scoliosis assessment. Ablation studies demonstrate the effectiveness of designed components on the decrease of localization error and improvement of anatomical plausibility. Additionally, we exhibit effective generalization performance by transferring our detection method onto sagittal 2-D slices of CT scans and boost the performance of downstream compression fracture classification at vertebra-level.

2S-SGCN: A two-stage stratified graph convolutional network model for facial landmark detection on 3D data

Facial Landmark Detection (FLD) algorithms play a crucial role in numerous computer vision applications, particularly in tasks such as face recognition, head pose estimation, and facial expression analysis. While FLD on images has long been the focus, the emergence of 3D data has led to a surge of interest in FLD on it due to its potential applications in various fields, including medical research. However, automating FLD in this context presents significant challenges, such as selecting suitable network architectures, refining outputs for precise landmark localization and optimizing computational efficiency. In response, this paper presents a novel approach, the 2-Stage Stratified Graph Convolutional Network (2S-SGCN), which addresses these challenges comprehensively. The first stage aims to detect landmark regions using heatmap regression, which leverages both local and long-range dependencies through a stratified approach. In the second stage, 3D landmarks are precisely determined using a new post-processing technique, namely MSE-over-mesh. 2S-SGCN ensures both efficiency and suitability for resource-constrained devices. Experimental results on 3D scans from the public Facescape and Headspace datasets, as well as on point clouds derived from FLAME meshes collected in the DAD-3DHeads dataset, demonstrate that the proposed method achieves state-of-the-art performance across various conditions. Source code is accessible at https://github.com/gfacchi-dev/CVIU-2S-SGCN.

Superior performance of a center-point AI model over VFLDNet in automated cobb angle estimation for scoliosis assessment.

Aims to establish the superiority of our proposed model over the state-of-the-art vertebra-focused landmark detection network (VFLDNet) in automating Cobb angle estimation from spinal radiographs. Utilizing a private dataset for external validation, we compared the performance of our center-point detection-based vertebra localization and tilt estimation network (VLTENet) with the key-point detection-based VFLDNet. Both models' Cobb angle predictions were rigorously evaluated against manual consensus score using metrics such as mean absolute error (MAE), correlation coefficient, intraclass correlation coefficient (ICC), Fleiss' kappa, Bland-Altman analysis, and classification metrics [sensitivity (SN), specificity, accuracy] focusing on major curve estimation and scoliosis severity classification. A retrospective analysis of 118 cases with 342 Cobb angle measurements revealed that our model achieved a MAE of 2.15° for total Cobb angles and 1.89° for the major curve, significantly outperforming VFLDNet's MAE of 2.80°and 2.57°, respectively. Both models demonstrated robust correlation and ICC, but our model excelled in classification consistency, particularly in predicting major curve magnitude (ours: kappa = 0.83; VFLDNet: kappa = 0.67). In subgroup analyses by scoliosis severity, our model consistently surpassed VFLDNet, displaying superior mean (SD) differences, narrower limits of agreement, and higher SN, specificity, and accuracy, most notably in moderate (ours: SN = 86.84%; VFLDNet: SN = 83.16%) to severe (ours: SN = 92.86%; VFLDNet: SN = 85.71%) scoliosis. Our model emerges as the superior choice for automated Cobb angle estimation, particularly in assessing major curve and moderate to severe scoliosis, underscoring its potential to revolutionize clinical workflows and enhance patient care.

An artificial intelligence-based landmark model for early detection of aortic stenosis in Cardiac MRI

Abstract Background The utilisation of ratiometric intensity measures from cardiac magnetic resonance imaging (CMR) offers a promising biomarker for the diagnosis of aortic stenosis (AS)[1]. The Ao:LV ratio, an approximation of AS severity, is determined by comparing the relative blood signal intensity in the aorta and left ventricle (LV) within three-chamber (3Ch) cine sequences[1]. In patients with AS, the blood signal intensity of aortic blood above the valve is diminished compared to that within the LV in steady-state free precession (SSFP) 3Ch CMR cine. Although the Ao:LV ratio serves as a reliable indicator of AS severity, manual computation of this ratio may encounter difficulties due to visual obstructions caused by small LV cavities or prominent papillary muscles. Purpose This study aims to leverage artificial intelligence (AI) to automate the Ao:LV ratio analysis from 3Ch SSFP cine images and introduce the Ao:LA (aorta to left atrium) ratio, thus enhancing the diagnostic accuracy for AS severity. Methods Our methodology extends the ratiometric analysis to include both Ao:LV and Ao:LA ratios in three main steps: first, developing an AI-based algorithm for automated detection and tracking of cardiac landmarks; second, using these landmarks to automatically compute regions of interest (ROIs) in the cardiac chambers; and third, training and validating classification models with 5-fold cross-validation to select the most effective for AS severity classification. The final model's performance is assessed on a holdout test set. Data for model training and evaluation were sourced from CMR scans across three hospitals and two scanner types. Results We analysed 220 patients of which 78 patients had no AS and 122 had AS (mild AS, n = 29; moderate AS; n = 35, severe AS; n = 78). Our model yielded high efficacy in classifying AS, particularly in differentiating any grade of AS from a normal classification, which is clinically relevant. The Ao:LA ratio demonstrated a stronger correlation with AS severity class than Ao:LV, suggesting a more reliable biomarker. The proposed model demonstrated robust performance in AS classification, achieving an Area Under the Curve (AUC) of 0.845 in the automated binary classification of AS at any grade. The model exhibited a precision of 0.889, recall of 0.865, and an F1 score of 0.877 on a holdout set. Conclusion By the integration of AI in automating the analysis of CMR images through a novel ratiometric approach that includes Ao:LV and Ao:LA ratios, our model significantly improves the accuracy of early AS detection. This advancement enhances imaging strategies for AS evaluation, promoting more efficient and effective use of cardiac MRI sequences as a diagnostic tool for aortic stenosis. It has the potential to be implemented in real-time scanning protocols as a clinical decision support system.AI-inferred cardiac landmarks.ROC curve delineating AS prediction.

Anatomical landmark detection on bi-planar radiographs for predicting spinopelvic parameters.

Accurate landmark detection is essential for precise analysis of anatomical structures, supporting diagnosis, treatment planning, and monitoring in patients with spinal deformities. Conventional methods rely on laborious landmark identification by medical experts, which motivates automation. The proposed deep learning pipeline processes bi-planar radiographs to determine spinopelvic parameters and Cobb angles without manual supervision. The dataset used for training and evaluation consisted of 555 bi-planar radiographs from un-instrumented patients, which were manually annotated by medical professionals. The pipeline performed a pre-processing step to determine regions of interest, including the cervical spine, thoracolumbar spine, sacrum, and pelvis. For each ROI, a segmentation network was trained to identify vertebral bodies and pelvic landmarks. The U-Net architecture was trained on 455 bi-planar radiographs using binary cross-entropy loss. The post-processing algorithm determined spinal alignment and angular parameters based on the segmentation output. We evaluated the pipeline on a test set of 100 previously unseen bi-planar radiographs, using the mean absolute difference between annotated and predicted landmarks as the performance metric. The spinopelvic parameter predictions of the pipeline were compared to the measurements of two experienced medical professionals using intraclass correlation coefficient(ICC) and mean absolute deviation (MAD). The pipeline was able to successfully predict the Cobb angles in 61% of all test cases and achieved mean absolute differences of 3.3° (3.6°) and averaged ICC of 0.88. For thoracic kyphosis, lumbar lordosis, sagittal vertical axis, sacral slope, pelvic tilt, and pelvic incidence, the pipeline produced reasonable outputs in 69%, 58%, 86%, 85%, 84%, and 84% of the cases. The MAD was 5.6° (7.8°), 4.7° (4.3°), 2.8mm (3.0mm), 4.5° (7.2°), 1.8° (1.8°), and 5.3° (7.7°), while the ICC was measured at 0.69, 0.82, 0.99, 0.61, 0.96, and 0.70, respectively. Despite limitations in patients with severe pathologies and high BMI, the pipeline automatically predicted coronal and sagittal spinopelvic parameters, which has the potential to simplify clinical routines and large-scale retrospective data analysis.

RP-Net: A Robust Polar Transformation Network for rotation-invariant face detection

Face detection is challenging in unconstrained environments, where it encounters various challenges such as orientation, pose, and occlusion. Deep convolutional neural networks, particularly cascaded ones, have greatly improved detection performance but still struggle with rotating objects due to limitations in the Cartesian coordinate system. Although data augmentation can mitigate this issue, it also increases computational demands. This paper introduces the Robust Polar Transformation Network (RP-Net) for rotation-invariant face detection. RP-Net converts the complex rotational problem into a simpler translational one to enhance feature extraction and computational efficiency. Additionally, the Advanced Spatial-Channel Restoration (ASCR) module optimizes facial landmark detection within polar domains and restores critical details lost during transformation. Experimental results on benchmark datasets show that RP-Net significantly improves rotation invariance over traditional CNNs and surpasses several state-of-the-art rotation-invariant face detection methods.

Detection of hand motion during cadaveric mastoidectomy dissections: a technical note.

Surgical approaches that access the posterior temporal bone require careful drilling motions to achieve adequate exposure while avoiding injury to critical structures. We assessed a deep learning hand motion detector to potentially refine hand motion and precision during power drill use in a cadaveric mastoidectomy procedure. A deep-learning hand motion detector tracked the movement of a surgeon's hands during three cadaveric mastoidectomy procedures. The model provided horizontal and vertical coordinates of 21 landmarks on both hands, which were used to create vertical and horizontal plane tracking plots. Preliminary surgical performance metrics were calculated from the motion detections. 1,948,837 landmark detections were collected, with an overall 85.9% performance. There was similar detection of the dominant hand (48.2%) compared to the non-dominant hand (51.7%). A loss of tracking occurred due to the increased brightness caused by the microscope light at the center of the field and by movements of the hand outside the field of view of the camera. The mean (SD) time spent (seconds) during instrument changes was 21.5 (12.4) and 4.4 (5.7) during adjustments of the microscope. A deep-learning hand motion detector can measure surgical motion without physical sensors attached to the hands during mastoidectomy simulations on cadavers. While preliminary metrics were developed to assess hand motion during mastoidectomy, further studies are needed to expand and validate these metrics for potential use in guiding and evaluating surgical training.

Automated Preoperative Planning Algorithm for Mandibular Angle Osteotomy Based on Anatomical Landmarks Detection: A Retrospective Study.

The aim of the study was to develop and evaluate an automated preoperative planning algorithm based on anatomical landmark point recognition for enhancing the efficiency and intelligence of preoperative planning for mandibular angle osteotomy. A retrospective cohort of 34 patients underwent preoperative planning with this algorithm. The present algorithm was developed using a method based on anatomical marker point recognition. The efficiency, symmetry, and safety of the automated preoperative planning and esthetics were statistically analyzed by paired t test and χ2 test. The results showed that the automated planning algorithm was able to achieve a great improvement in preoperative planning efficiency as well as safety and symmetry. A prospective case report of 2 patients is then reported, illustrating the safety and esthetics of the algorithm with 1-year postoperative follow-up and postoperative esthetic scores. This algorithm can help to improve the efficiency of preoperative planning for surgeons while ensuring safety and esthetics and can be further applied to other craniomaxillofacial personalized design surgeries in the preoperative design in the future.

A real-time eye movement-based computer interface for people with disabilities

It is costly to develop systems that enable individuals exposed to Amyotrophic Lateral Sclerosis and similar diseases that directly affect the neuromotor ability to communicate with the outside world. In this study, a budget friendly, high-accuracy, software-based, gaze-controlled, real-time virtual keyboard approach that can enable these people to communicate effectively is proposed. The proposed application requires only a computer and a webcam and has a user-friendly interface that meets the basic daily needs of individuals with disabilities. Since the proposed system does not require an extra action such as blinking, it makes it possible to use computers in advanced stage patients who cannot blink their eyes. The application which uses a deep learning-based facial landmark detector, determines the letters the user focuses on the screen and converts thoughts into text. The part of the screen that the user focuses on is determined with a new selection approach inspired by the K-Nearest Neighbors algorithm. This approach, which does not require blinking, offers high speed and accuracy. In the tests, a typing speed of 23.33 characters per minute is achieved with an accuracy rate of 95.12%. It is anticipated that the study will increase computer accessibility for disabled individuals with limited mobility and contribute to the development of real-time eye tracking systems.

A lightweight network-based sign language robot with facial mirroring and speech system

Human–robot interaction is an essential capability for humanoid robots to enter the physical world and become companions in people’s lives, learning, and work. While the majority of current research focuses on the voice-based interactions of robots, yet over 60% of communication occurs through nonverbal behaviors, such as facial expressions and hand gestures. Endowing robots with the ability to communicate through nonverbal behavior not only enhances the interactive experience with robots but also provides a potential communication tool for individuals with hearing or speech impairments. Here, we develop a humanoid robot capable of adjusting facial movements by driving servos, and design a novel framework for the robot to integrate sign language recognition and facial landmark detection algorithms. This framework facilitates the robot recognize sign language and translate it into spoken language, while also imitating the facial expressions of the signers. To achieve this, we also propose a lightweight deep learning network called RealTimeSignNet for real-time sign language recognition. Leveraging lightweight 3D convolution modules and time-dependent constraints, this model adapts to various time scales, ensuring efficient processing of sign language recognition tasks. Experimental results demonstrate the outstanding performance of the RealTimeSignNet model on mainstream sign language datasets, achieving an accuracy of 88.1% on the large continuous sign language dataset (continuous SLR), 98.2% on the isolated sign language dataset (SLR 500), and 91.50% on the English sign language dataset (WLAS). The overall assessment demonstrates that our humanoid robot is capable of recognizing sign language and translating it into spoken language, while imitating the facial emotions, providing a comprehensive solution to the communication challenges faced by individuals with hearing and speech impairments.

Reinforcement learning-based anatomical maps for pancreas subregion and duct segmentation.

The pancreas is a complex abdominal organ with many anatomical variations, and therefore automated pancreas segmentation from medical images is a challengingapplication. In this paper, we present a framework for segmenting individual pancreatic subregions and the pancreatic duct from three-dimensional (3D) computed tomography (CT) images. A multiagent reinforcement learning (RL) network was used to detect landmarks of the head, neck, body, and tail of the pancreas, and landmarks along the pancreatic duct in a selected target CT image. Using the landmark detection results, an atlas of pancreases was nonrigidly registered to the target image, resulting in anatomical probability maps for the pancreatic subregions and duct. The probability maps were augmented with multilabel 3D U-Net architectures to obtain the final segmentationresults. To evaluate the performance of our proposed framework, we computed the Dice similarity coefficient (DSC) between the predicted and ground truth manual segmentations on a database of 82 CT images with manually segmented pancreatic subregions and 37 CT images with manually segmented pancreatic ducts. For the four pancreatic subregions, the mean DSC improved from 0.38, 0.44, and 0.39 with standard 3D U-Net, Attention U-Net, and shifted windowing (Swin) U-Net architectures, to 0.51, 0.47, and 0.49, respectively, when utilizing the proposed RL-based framework. For the pancreatic duct, the RL-based framework achieved a mean DSC of 0.70, significantly outperforming the standard approaches and existing methods on different datasets. The resulting accuracy of the proposed RL-based segmentation framework demonstrates an improvement against segmentation with standard U-Netarchitectures.

Eye Based Interactive Communication System

Eye-based communication systems, like the Blink-To-Speak language, are essential tools for individuals with motor neuron disorders, enabling them to articulate their needs and emotions. These systems leverage eye movements as a means of interaction, providing a vital communication channel for those with severe speech impairments. By interpreting specific eye gestures, users can effectively convey their thoughts, fostering a sense of independence and improving their quality of life. Many of the existing eye-tracking technologies tend to be complex and costly, limiting their accessibility in low-income regions. Using a mobile phone camera, the system captures real-time video frames to detect and track the user’s eyes, leveraging advanced facial landmark detection. Four primary eye movements—Left, Right, Up, and Blink—form the basis of the Blink-To-Live communication model, allowing users to communicate a range of more than 60 daily commands. By translating sequences of three eye gestures, the system generates readable sentences that are displayed on a screen and can be vocalized through synthesized speech. Unlike other sensor-dependent systems, Blink-To-Live is designed for affordability, flexibility, and ease of use, making it accessible without the need for specialized software or hardware. A prototype has been tested on a range of participants, showing positive results in terms of ease of use, flexibility, and cost-effectiveness

DetSegDiff: A joint periodontal landmark detection and segmentation in intraoral ultrasound using edge-enhanced diffusion-based network

Individuals with malocclusion require an orthodontic diagnosis and treatment plan based on the severity of their condition. Assessing and monitoring changes in periodontal structures before, during, and after orthodontic procedures is crucial, and intraoral ultrasound (US) imaging has been shown a promising diagnostic tool in imaging periodontium. However, accurately delineating and analyzing periodontal structures in US videos is a challenging task for clinicians, as it is time-consuming and subject to interpretation errors. This paper introduces DetSegDiff, an edge-enhanced diffusion-based network developed to simultaneously detect the cementoenamel junction (CEJ) and segment alveolar bone structure in intraoral US videos. An edge feature encoder is designed to enhance edge and texture information for precise delineation of periodontal structures. Additionally, we employed the spatial squeeze-attention module (SSAM) to extract more representative features to perform both detection and segmentation tasks at global and local levels. This study used 169 videos from 17 orthodontic patients for training purposes and was subsequently tested on 41 videos from 4 additional patients. The proposed method achieved a mean distance difference of 0.17 ± 0.19 mm for the CEJ and an average Dice score of 90.1% for alveolar bone structure. As there is a lack of multi-task benchmark networks, thorough experiments were undertaken to assess and benchmark the proposed method against state-of-the-art (SOTA) detection and segmentation individual networks. The experimental results demonstrated that DetSegDiff outperformed SOTA approaches, confirming the feasibility of using automated diagnostic systems for orthodontists.

An Adaptive and Automatic Power Supply Distribution System with Active Landmarks for Autonomous Mobile Robots.

With the development of automation and intelligent technologies, the demand for autonomous mobile robots in the industry has surged to alleviate labor-intensive tasks and mitigate labor shortages. However, conventional industrial mobile robots' route-tracking algorithms typically rely on passive markers, leading to issues such as inflexibility in changing routes and high deployment costs. To address these challenges, this study proposes a novel approach utilizing active landmarks-battery-powered luminous landmarks that enable robots to recognize and adapt to flexible navigation requirements. However, the reliance on batteries necessitates frequent recharging, prompting the development of an automatic power supply system. This system integrates omnidirectional contact electrodes on mobile robots, allowing to recharge active landmarks without precise positional alignment. Despite these advancements, challenges such as the large size of electrodes and non-adaptive battery charging across landmarks persist, affecting system efficiency. To mitigate these issues, this research focuses on miniaturizing active landmarks and optimizing power distribution among landmarks. The experimental results of this study demonstrated the effectiveness of our automatic power supply method and the high accuracy of landmark detection. Our power distribution calculation method can adaptively manage energy distribution, improving the system's persistence by nearly three times. This study aims to enhance the practicality and efficiency of mobile robot remote control systems utilizing active landmarks by simplifying installation processes and extending operational durations with adaptive and automatic power supply distribution.

Human-AI collaboration improves adults’ oral biomechanical functions: A multi-centre, self-controlled clinical trial

ObjectivesMaintenance of oral muscle functions is important for survival and communication. Utilizing Artificial Intelligence (AI) as a self-health-management material has shown promise. Here we developed a functional and AI-enabled smartphone e-Oral APP that provided real-time feedback features to let humans collaborate with AI, targeting to improve oral biomechanical and speech functions of adults. MethodsA total of 113 neurologically healthy Hong Kong citizens over 50 years old were recruited in this study from June to October 2022. A set of 12 exercises of the tongue, lips, and jaw were conducted with guidance provided in the self-developed e-Oral APP, in which visual-audio demonstrations and immediate feedback on facial movements by an open-source AI-pretrained 68-point Ensemble of Regression Tree (ERT) face landmark detector was given to the users. After 8-week exercises, assessments after 1-week and 4-week post-training were conducted. Primary outcomes were measured on oral functions, including occlusal force, masticatory efficiency, tongue strength, along with dry mouth condition, and oral health-related quality of life (OHQoL), while secondary outcomes were comprised of oral diadochokinesis (DDK) and swallowing ability. The usability of the APP was evaluated by a self-written questionnaire. ResultsAfter exercising a set of 12 exercises on the tongue, lips, and jaw for 8 weeks, 70 out of 113 participants (average age of 67.70±4.93 y.o., 52 female and 18 male) had a significant improvement in the occlusal force (p < 0.001), while masticatory efficiency (p = 0.002), tongue pressure (p < 0.001) and endurance (p = 0.004) were also improved. 80.3 % of respondents rated the APP with an overall rating of 3 out of 5 points, and 68.6 % would recommend the APP to others. ConclusionsAI-based APP can be an effective approach to help healthy adults improve their occlusal force, masticatory efficiency, tongue functions, and oral diadochokinesis after 8-week home oral exercises. Furthermore, these improvements can be sustained for at least four weeks. Clinical significanceThis is the first AI-assisted APP developed for oral muscle training. Our findings demonstrated that a self-administrated human-AI collaboration APP can improve clinically oral muscle biomechanics and functions for healthy adults. AI technology in smartphone gadgets provides a cost-effective, convenient, and reliable means for oral muscles training for adults.