Landmark Features Research Articles

Speeding up echocardiographic examination by automated generation of 2D imaging views from 3D volumetric data using machine learning

Abstract Background 3D echocardiographic data contain infinite 2D echocardiographic planes generated by multiplanar reconstruction, but the process is labor-intensive and time-consuming. Artificial intelligence (AI) utilizing cutting-edge machine learning techniques may improve the accuracy and efficiency of the process. In the study, we developed and validated an AI software that automatically reconstructs 7 2D views recommended by the American Society of Echocardiography from 3D echocardiographic data. Method Convolutional neural network (CNN) architecture utilizing Spatial Configuration Net was used as the machine learning model. We identify a minimum of 3 key feature landmarks on each 2D cross-sectional plane, with the 3D coordinates representing 2D cross-sectional planes accordingly. Subsequently, these landmarks are accurately labeled within a 3D space. 31 landmarks were identified across 7 2D cross-sectional planes and a total of 472 3D volume data were labelled. The performance of the model is evaluated by Root Mean Square Error (RMSE) of the predicted landmark coordinates. Results The mean RMSE over 31 predicted landmarks evaluated in testing is 2.70% for the input 3D volume dimensions. The average time used to generate the 7 2D imaging views was 19.91±3.19 seconds (figure). Conclusion The AI algorithm can efficiently convert 3D echocardiographic data into guideline-recommended 2D images, which shortens the length of echo examination but still preserves the accuracy.AI generation of 2D views from 3D echo

The Wolf Spider Tigrosa helluo Uses Visual Associative and Beacon Landmarks During Water Maze Navigation Tasks

ABSTRACTWolf spiders can learn simple spatial navigation tasks. Previous studies have shown that the wolf spider Tigrosa helluo can use environmental edge features (reference frame landmarks) to learn the location of a dry target in flooded T‐mazes; however, the relative importance of different types or numbers of landmark cues to spatial learning remains unknown. We used a modified open arena water maze and recorded the ability of adult female T. helluo to find a target reward (a dark and dry cup) among cups that were identical to the target but flooded. We measured variation in spatial learning by measuring time to target with no landmark (control), with a beacon (a landmark that is part of the target), with an associative cue (a landmark associated with a specific navigational action), and with both a beacon and an associative cue (N = 92 subjects, n = 23 per landmark cue treatment). For each treatment, we tested females for five trials each on four consecutive days, with the last trial on the fourth day having an altered target location, totaling 19 training trials and one reversal trial (1840 trials). We found that spiders took significantly less time to find the target over subsequent trials within a day and learned more quickly when landmark cues were present, but we found no difference in the type or number of landmark features in the meantime to target entrance. After learning a target location, moving the landmark significantly increased the mean time to target entrance in the combined beacon and associative cue treatment relative to other treatments. Our results indicate that wolf spiders use visual beacons and associative cue landmarks alone or in combination and that performance improves across trials when landmarks are present and deteriorates more when multiple landmarks are moved.

TriCAFFNet: A Tri-Cross-Attention Transformer with a Multi-Feature Fusion Network for Facial Expression Recognition.

In recent years, significant progress has been made in facial expression recognition methods. However, tasks related to facial expression recognition in real environments still require further research. This paper proposes a tri-cross-attention transformer with a multi-feature fusion network (TriCAFFNet) to improve facial expression recognition performance under challenging conditions. By combining LBP (Local Binary Pattern) features, HOG (Histogram of Oriented Gradients) features, landmark features, and CNN (convolutional neural network) features from facial images, the model is provided with a rich input to improve its ability to discern subtle differences between images. Additionally, tri-cross-attention blocks are designed to facilitate information exchange between different features, enabling mutual guidance among different features to capture salient attention. Extensive experiments on several widely used datasets show that our TriCAFFNet achieves the SOTA performance on RAF-DB with 92.17%, AffectNet (7 cls) with 67.40%, and AffectNet (8 cls) with 63.49%, respectively.

Dynamic 4D facial capture pipeline with appearance driven progressive retopology based on optical flow

This paper presents a production-oriented 4D facial reconstruction pipeline designed to produce high-fidelity facial mesh sequences with a consistently structured topology, while preserving the wireframe structure specified by artists. We have designed and developed a compact, efficient, and fast optical capture system based on synchronized camera arrays for high-precision dynamic 3D facial imaging. Unlike prevailing methods that primarily concentrate on single-frame reconstruction, often reliant on labor-intensive manual annotation, our framework exploits the constraint of appearance consistency to autonomously establish feature correspondence and uphold temporal coherence within the mesh. Consequently, our approach eliminates mesh drifting and jitter, enabling full parallelization for dynamic facial expression capture. The proposed pipeline decouples the non-linear deformation of facial expressions from the rigid movements of the skull through a stable external device. Leveraging progressive retopology, our methodology employs artist-guided templates as priors, ensuring the preservation of wireframe structures across the result sequence. Progressive retopology is achieved by constraining different fine-grained features of 3D landmarks, scan surface shapes, and appearance textures. The results of our study showcase facial mesh sequences with production-quality topology, adept at faithfully reproducing character expressions from photographs while achieving artist-friendly stable facial movements.

Flight patterns of monocularly occluded homing pigeons further reveal a left eye/right cerebral hemispheric advantage in the visual processing of familiar landscape/landmark features

Studying pigeons during homing has offered an opportunity to investigate visual–functional brain lateralization in the context of free flying and long distance navigation. In the current study we examine at a high scale of spatial resolution the flight paths of pigeons, which vary with respect to monocular and binocular treatment, as they familiarise themselves with landscape and landmark features during repeated flights home from two locations. The analysis of the flight paths of monocularly occluded pigeons revealed that pigeons using the right eye/left hemisphere visual system were more likely to display more tortuous paths over tracts of 500 m along the flight path home compared to both pigeons using the left eye/right hemisphere visual system and control birds. Accompanying this finding was the observation that pigeons using the right eye/left hemisphere system were more likely to perform flight “loops”, suggesting that, without right hemispheric visual processing, pigeons are more motivated to seek out visual input, behaviourally expanding the now monocularly reduced visual field. Taken together, the data are consistent with a more important role of the left eye/right hemisphere system in processing visual features of the landscape, which may contribute to the construction of the familiar landmark-based map used for navigation. More broadly, our data are consistent with the hypothesis that the right cerebral hemisphere of the avian brain plays a more important role in memorising and using the relational properties of visual stimuli to guide performance on spatial tasks.

Computer Vision Algorithm Design in Image Processing Based on Projective Geometry

Image processing with computer vision, particularly in the realm of projective geometry, offers remarkable potential for various applications. Through the lens of projective geometry, images can be transformed, augmented, and reconstructed with precision, facilitating tasks such as image rectification, 3D reconstruction, and object tracking. Landmark estimation in computer vision is a vital task with broad applications across various domains. This process involves identifying key points or landmarks within images, enabling tasks such as facial recognition, object tracking, and gesture recognition. This paper, proposed a novel approach for landmark estimation in computer vision using Projective Geometry Landmark Estimation (PGLM). The proposed model aims to estimate the landmark features by a projective geometry model. With the estimation of the geometry features landmarks related to the facial, object, and medical images are computed. The PGLM model uses the point features for the location of the landmark features. In order to compare PGLM's performance to that of more conventional classification methods like Random Forest, K-Nearest Neighbors (KNN), and Support Vector Machine (SVM), simulation analysis is carried out. From what we can see, PGLM routinely beats these alternatives when we compare their accuracy, precision, recall, and F1 score. The findings stated the effectiveness of PGLM as a promising approach for landmark estimation in image processing tasks, paving the way for further advancements in this domain.

Deep Adaptation of Adult-Child Facial Expressions by Fusing Landmark Features

Deep Adaptation of Adult-Child Facial Expressions by Fusing Landmark Features

Use of Yolo Detection for 3D Pose Tracking of Cardiac Catheters Using Bi-Plane Fluoroscopy

The increasing rate of minimally invasive procedures and the growing prevalence of cardiovascular disease have led to a demand for higher-quality guidance systems for catheter tracking. Traditional methods for catheter tracking, such as detection based on single points and applying masking techniques, have been limited in their ability to provide accurate pose information. In this paper, we propose a novel deep learning-based method for catheter tracking and pose detection. Our method uses a Yolov5 bounding box neural network with postprocessing to perform landmark detection in four regions of the catheter: the tip, radio-opaque marker, bend, and entry point. This allows us to track the catheter’s position and orientation in real time, without the need for additional masking or segmentation techniques. We evaluated our method on a dataset of fluoroscopic images from two distinct datasets and achieved state-of-the-art results in terms of accuracy and robustness. Our model was able to detect all four landmark features (tip, marker, bend, and entry) used to generate a pose for a catheter with 0.285 ± 0.143 mm, 0.261 ± 0.138 mm, 0.424 ± 0.361 mm, and 0.235 ± 0.085 mm accuracy. We believe that our method has the potential to significantly improve the accuracy and efficiency of catheter tracking in medical procedures that utilize bi-plane fluoroscopy guidance.

DEEPFAKE DETECTION USING TRANSFER LEARNING-BASED XCEPTION MODEL

Justification of the purpose of the research. In recent times, several approaches for face manipulation in videos have been extensively applied and availed to the public which makes editing faces in video easy for everyone effortlessly with realistic efforts. While beneficial in various domains, these methods could significantly harm society if employed to spread misinformation. So, it is also vital to properly detect whether a face has been distorted in a video series. To detect this deepfake, convolutional neural networks can be used in past works. However, it needs a greater number of parameters and more computations. So, to overcome these limitations and to accurately detect deepfakes in videos, a transfer learning-based model named the Improved Xception model is suggested. Obtained results. This model is trained using extracted facial landmark features with robust training. Moreover, the improved Xception model's detection accuracy is evaluated alongside ResNet and Inception, considering model loss, accuracy, ROC, training time, and the Precision-Recall curve. The outcomes confirm the success of the proposed model, which employs transfer learning techniques to identify fraudulent videos. Furthermore, the method demonstrates a noteworthy 5% increase in efficiency compared to current systems.

A Study on the Screening of Children at Risk for Developmental Disabilities Using Facial Landmarks Derived From a Mobile-Based Application

Objective Early detection and intervention of developmental disabilities (DDs) are critical to improving the long-term outcomes of afflicted children. In this study, our objective was to utilize facial landmark features from mobile application to distinguish between children with DDs and typically developing (TD) children.Methods The present study recruited 89 children, including 33 diagnosed with DD, and 56 TD children. The aim was to examine the effectiveness of a deep learning classification model using facial video collected from children through mobile-based application. The study participants underwent comprehensive developmental assessments, which included the child completion of the Korean Psychoeducational Profile-Revised and caregiver completing the Korean versions of Vineland Adaptive Behavior Scale, Korean version of the Childhood Autism Rating Scale, Social Responsiveness Scale, and Child Behavior Checklist. We extracted facial landmarks from recorded videos using mobile application and performed DDs classification using long short-term memory with stratified 5-fold cross-validation.Results The classification model shows an average accuracy of 0.88 (range: 0.78–1.00), an average precision of 0.91 (range: 0.75–1.00), and an average F1-score of 0.80 (range: 0.60–1.00). Upon interpreting prediction results using SHapley Additive exPlanations (SHAP), we verified that the most crucial variable was the nodding head angle variable, with a median SHAP score of 2.6. All the top 10 contributing variables exhibited significant differences in distribution between children with DD and TD (p<0.05).Conclusion The results of this study provide evidence that facial landmarks, utilizing readily available mobile-based video data, can be used to detect DD at an early stage.

Deep neural network-based emotion recognition using facial landmark features and particle swarm optimization

Relating specifically to human–computer interaction (HCI), computer vision research has placed a substantial emphasis on intelligent emotion recognition in recent years. The primary emphasis lies in investigating speech aspects and bodily motions, while the knowledge of recognizing emotions from facial expressions remains relatively unexplored. Automated facial emotion detection allows a machine to assess and understand a person's emotional state, allowing the system to predict intent by analyzing facial expressions. Therefore, this research provides a novel parameter selection strategy using swarm intelligence and a fitness function for intelligent recognition of micro emotions. This paper presents a novel method based on geometric visual representation obtained from facial landmark points. We employ the Deep Neural Networks (DNN) model to analyze the input features from the normalized angle and distance values derived from these landmarks. The results of the experiments show that Particle Swarm Optimization (PSO) worked very well by using only a few carefully chosen features. The method achieved a recognition success rate of 98.76% on the MUG dataset and 97.79% on the GEMEP datasets.

Optimal Pose Estimation and Covariance Analysis with Simultaneous Localization and Mapping Applications

This work provides a theoretical analysis for optimally solving the pose estimation problem using total-least-squares for vector observations from landmark features, which is central to applications involving simultaneous localization and mapping. First, the optimization process is formulated with observation vectors extracted from point-cloud features. Then, error-covariance expressions are derived. The attitude and position estimates obtained via the derived optimization process are proven to reach the bounds defined by the Cramér–Rao lower bound under the small-angle approximation of attitude errors. A fully populated observation noise-covariance matrix is assumed as the weight in the cost function to cover the most general case of the sensor uncertainty. This includes more generic correlations in the errors than previous cases involving an isotropic noise assumption. The proposed solution is verified using Monte Carlo simulations, a Gazebo simulation in a robotics operating system, and an experiment with an actual light detection and ranging sensor to validate the error-covariance analysis.

FDGR-Net: Feature Decouple and Gated Recalibration Network for medical image landmark detection

Medical image landmark detection can automatically determine the pre-set position coordinates in medical images, so as to further assist disease diagnosis and treatment planning. Using Convolutional Neural Network (CNN) to estimate the heatmap of the landmark has become the mainstream method. However, some landmarks are not obvious and there are local similarities between different landmarks in the medical images. Existing methods usually directly map the image features to all the landmarks, without considering these two challenges. In this paper, we propose a Feature Decouple and Gated Recalibration Network (FDGR-Net) for the medical image landmark detection. The FDGR-Net first establishes independent pathways to decouple all landmark features into single landmark features. In this process, a dynamic multi-kernel convolution is designed as the decouple basic component to improve the feature robustness of the not obvious landmarks. Then, a novel Gated Feature Recalibration Module (GFRM) is proposed, which includes a retention gate and an elimination gate. The retention gate can further strengthen the information of the target landmark and the elimination gate can eliminate the interference of non-target landmarks. The proposed method is validated on one public dataset and two in-house datasets, and its performance is superior to the existing advanced method. The Mean Radial Error (MRE) of the proposed method can reach 3.04, 5.51 and 1.45 in the Knee dataset, Hand dataset and Head dataset, respectively.

GFFT: Global-local feature fusion transformers for facial expression recognition in the wild

Facial expression recognition in the wild has become more challenging owing to various unconstrained conditions, such as facial occlusion and pose variation. Previous methods usually recognize expressions by holistic or relatively coarse local methods, but only capture limited features and are susceptible to be influenced. In this paper, we propose the Global–local Feature Fusion Transformers (GFFT) that is centered on cross-patch communication between features by self-attentive fusion. This method solves the problems of facial occlusion and pose variation effectively. Firstly, the Global Contextual Information Perception (GCIP) is designed to fuse global and local features, learning the relationship between them. Subsequently, the Facial Salient Feature Perception (FSFP) module is proposed to guide the fusion features to understand the key regions of facial features using facial landmark features to further capture face-related salient features. In addition, the Multi-scale Feature Fusion (MFF) is constructed to combine different stages of fusion features to reduce the sensitivity of the deep network to facial occlusion. Extensive experiments show that our GFFT outperforms existing state-of-the-art methods with 92.05% on RAF-DB, 67.46% on AffectNet-7, 63.62% on AffectNet-8, and 91.04% on FERPlus, demonstrating its effectiveness and robustness.

A Novel Visual SLAM Based on Multiple Deep Neural Networks

The current visual simultaneous localization and mapping (SLAM) systems require the use of matched feature point pairs to estimate camera pose and construct environmental maps. Therefore, they suffer from poor performance of the visual feature matchers. To address this problem, a visual SLAM using deep feature matcher is proposed, which is mainly composed of three parallel threads: Visual Odometry, Backend Optimizer and LoopClosing. In the Visual Odometry, the deep feature extractor with convolutional neural networks is utilized for extracting feature points in each image frame. Then, the deep feature matcher is used for obtaining the corresponding feature landmark pairs. Afterwards, a fusion method based on the last and the reference frame is proposed for camera pose estimation. The Backend Optimizer is designed to execute local bundle adjustment for a part of camera poses and landmarks (map points). While LoopClosing, consisting of a lightweight deep loop closure detector and the same matcher as the one used in Visual Odometry is utilized for loop correction based on pose graph. The proposed system has been tested extensively on most of benchmark KITTI odometry dataset. The experimental results show that our system yields better performance than the existing visual SLAM systems. It can not only run in real-time at a speed of 0.08 s per frame, but also reduce estimation error by at least 0.1 m.

Energy Sources of Mobile Robot Power Systems: A Systematic Review and Comparison of Efficiency

Mobile robots can perform tasks on the move, including exploring terrain, discovering landmark features, or moving a load from one place to another. This group of robots is characterized by a certain level of intelligence, allowing the making of decisions and responding to stimuli received from the environment. As part of Industry 5.0, such mobile robots and humans are expected to co-exist and work together in a shared environment to make human work less tiring, quicker, and safer. This can only be realized when clean, dense, and economical energy sources are available. The aim of the study is to analyze the state of the art and to identify the most important directions for future developments in energy sources of robotic power systems based mainly on batteries. The efficiency and performance of the battery depends on the design using different materials. Work environments and performance requirements are considered in this systematic review to classify solutions that help developers choose the best-suited power system for specific application. Indirectly, the aim of the work is to generate discussion within the scientific and engineering community. A narrative review of publications from six major bibliographic databases according to preset inclusion criteria is combined with a critical analysis of current and future technologies. The main findings of the review allow answering the question of what is the role of modern power source technologies, artificial intelligence, and ground-breaking research work in global policies related to energy saving, green policies, and sustainable development. The main opportunities and threats are discussed, and a brief feasibility analysis is carried out. The novelty of the article relates not only to the analysis of technologies, but also to approaches and their use under conditions of limited resource availability, when resource usage must be minimized. The article provides an overview of batteries, their specifications, classifications, and their advantages and disadvantages. In addition, we propose (1) an algorithm for selecting the main energy source for robot application, and (2) an algorithm for selecting an electrical system power supply. Current mobile robot batteries are, in most cases, the robot’s biggest limitation. Progress in battery development is currently too slow to catch up with the demand for robot autonomy and range requirements, limiting the development of mobile robots. Further intensive research and implementation work is needed to avoid years of delay in this area.

A Quasiconformal-Based Geometric Model for Craniofacial Analysis and Its Application

We address the problem of craniofacial morphometric analysis using geometric models, which has important clinical applications for the diagnosis of syndromes associated with craniofacial dysmorphologies. In this work, a novel geometric model is proposed to analyze craniofacial structures based on local curvature information and Teichmüller mappings. A key feature of the proposed model is that its pipeline starts with few two-dimensional images of the human face captured at different angles, from which the three-dimensional craniofacial structure can be reconstructed. The 3D surface reconstruction from 2D images is based on a modified 3D morphable model (3DMM) framework. Geometric quantities around important feature landmarks according to different clinical applications can then be computed on each three-dimensional craniofacial structure. Together with the Teichmüller mapping, the landmark-based Teichmüller curvature distances (LTCDs) for every classes can be computed, which are further used for three-class classification. A composite score model is used and the parameter optimization is carried out to further improve the classification accuracy. Our proposed model is applied to study the craniofacial structures of children with and without the obstructive sleep apnoea (OSA). Sixty subjects, with accessible multi-angle photography and polysomnography (PSG) data, are divided into three classes based on the severity of OSA. Using our proposed model, our proposed model achieves a high 90% accuracy, which outperforms other existing models. This demonstrates the effectiveness of our proposed geometric model for craniofacial analysis.

Online facial expression recognition based on graph convolution and long short memory networks

Video‐based facial expression recognition (FER) models have achieved higher accuracy with more computation, which is not suitable for online deployment in mobile intelligent terminals. Facial landmarks can model facial expression changes with their spatial location information instead of texture features. But classical convolution operation cannot make full use of landmark information. To this end, in this paper, we propose a novel long short memory network (LSTM) by embedding graph convolution named GELSTM for online video‐based FER in mobile intelligent terminals. Specifically, we construct landmark‐based face graph data from the client. On the server side, we introduce graph convolution which can effectively mine spatial dependencies information in a landmark‐based facial graph. Moreover, the extracted landmark's features are fed to LSTM for temporal feature aggregation. We conduct experiments on the facial expression dataset and the results show our proposed method shows superior performance compared to other deep models.

View-Invariant Center-of-Pressure Metrics Estimation With Monocular RGB Camera

Center of pressure (CoP) metrics, including CoP path length and sway area, have been used as gold standard measurements of postural and balance control in biomechanical studies. A recent study of computer-vision-based CoP metrics estimation from 3D body landmark sequences offers a more portable and comprehensive solution than conventional force plate methods to obtain these important metrics for real-time evaluation of balance control. However, obtaining accurate 3D body landmarks requires a calibrated motion capture system or on-body markers, which involves lengthy data collection and processing time and limits their implementation in home and clinical environments. Existing methods that instead use 2D body landmarks fail to adapt to different camera positions. To overcome these challenges, we propose a view-invariant deep learning framework for video-level CoP metrics estimation, including CoP path length and sway area, using pose dimension lifting and graph convolutional network (GCN). This work is the first step toward obtaining gold-standard CoP metrics with an accessible, monocular RGB camera. We propose to use a dimension lifting convolutional neural network (CNN) to obtain view-invariant 3D body landmark features from 2D body landmarks. We also propose a two-stream regression model using GCN and discrete cosine transform (DCT) for a robust CoP metrics estimation. To facilitate the line of research, we release a novel multi-view body landmark dataset containing 2D body landmarks of a wide variety of action patterns from four different camera views with synchronized CoP labels and corresponding 3D body landmarks, which enables cross-view evaluation with different camera angles. We subsequently validate the proposed method through a cross-dataset training by training the dimension lifting model on an existing balance dataset and evaluating the CoP metrics estimation on the multi-view body landmark dataset. The experiments validate that our framework achieves state-of-the-art accuracy for both CoP path length and CoP sway area using a monocular RGB camera input for unseen views.

Hybrid dense matching features for cloud-based face recognition

Cloud computing is a computing service done not on a local device but an internet connection to a data centre infrastructure. The cloud computing system also provides a scalability solution where cloud computing can increase the resources needed when doing larger data processing. In recent years, face bio-metric plays an important role in biometric authentication, where security is the primary concern. The two major difficulties to be addressed in face recognition are illumination and pose variation. This work proposes a cloud computing-based face recognition technique by consolidating four-patch LBP and the local landmark features to the acquired feature vector. After that, the SVM is utilised to recognise the individual. The feature set used for training and testing is also made available in the cloud. At some stage in the training phase, the kernel parameters of SVM are optimised with GWO to improve the recognition performance. This method yields a maximum recognition accuracy of 99.00% with LFW and 98.0% accuracy with YTF.