Robust Landmark Research Articles

Self-supervised context-aware correlation filter for robust landmark tracking in liver ultrasound sequences.

Respiratory motion-induced displacement of internal organs poses a significant challenge in image-guided radiation therapy, particularly affecting liver landmark tracking accuracy. Addressing this concern, we propose a self-supervised method for robust landmark tracking in long liver ultrasound sequences. Our approach leverages a Siamese-based context-aware correlation filter network, trained by using the consistency loss between forward tracking and back verification. By effectively utilizing both labeled and unlabeled liver ultrasound images, our model, Siam-CCF , mitigates the impact of speckle noise and artifacts on ultrasonic image tracking by a context-aware correlation filter. Additionally, a fusion strategy for template patch feature helps the tracker to obtain rich appearance information around the point-landmark. Siam-CCF achieves a mean tracking error of 0.79±0.83 mm at a frame rate of 118.6 fps, exhibiting a superior speed-accuracy trade-off on the public MICCAI 2015 Challenge on Liver Ultrasound Tracking (CLUST2015) 2D dataset. This performance won the 5th place on the CLUST2015 2D point-landmark tracking task. Extensive experiments validate the effectiveness of our proposed approach, establishing it as one of the top-performing techniques on the CLUST2015 online leaderboard at the time of this submission.

Segmentation-guided coordinate regression for robust landmark detection on X-rays: application to automated assessment of lower limb alignment

Segmentation-guided coordinate regression for robust landmark detection on X-rays: application to automated assessment of lower limb alignment

Landmark Recognition Model for Smart Tourism using Lightweight Deep Learning and Linear Discriminant Analysis

Scene recognition algorithm is crucial for landmark recognition model development. Landmark recognition model is one of the main modules in the intelligent tour guide system architecture for the use of smart tourism industry. However, recognizing the tourist landmarks in the public places are challenging due to the common structure and the complexity of scene objects such as building, monuments and parks. Hence, this study proposes a super lightweight and robust landmark recognition model by using the combination of Convolutional Neural Network (CNN) and Linear Discriminant Analysis (LDA) approaches. The landmark recognition model was evaluated by using several pretrained CNN architectures for feature extraction. Then, several feature selections and machine learning algorithms were also evaluated to produce a super lightweight and robust landmark recognition model. The evaluations were performed on UMS landmark dataset and Scene-15 dataset. The results from the experiments have found that the Efficient Net (EFFNET) with CNN classifier are the best feature extraction and classifier. EFFNET-CNN achieved 100% and 94.26% classification accuracy on UMS-Scene and Scene-15 dataset respectively. Moreover, the feature dimensions created by EFFNet are more compact compared to the other features and even have significantly reduced for more than 90% by using Linear Discriminant Analysis (LDA) without jeopardizing classification performance but yet improved its performance.

Robust landmark graph-based clustering for high-dimensional data

High-dimensional data has attracted much attention because it contains more comprehensive information about samples. How to cluster these high-dimensional data has become a crucial topic in unsupervised learning. Existing clustering methods often show limited applicability due to their high computational complexity and low anti-noise ability. To address this issue, we propose a novel robust landmark graph-based clustering algorithm for high-dimensional data (RLGCH), which inherits the advantages of both k-means++ and graph-based clustering by using the results of k-means++ as pseudo labels for landmark graph-based clustering. In particular, RLGCH can achieve more reasonable clustering effectiveness than methods that just operate in the low-dimensional space or the original space since it performs k-means++ in the low-dimensional space and landmark graph-based spectral clustering in the original feature space. To avoid post-processing after optimization, the embedded factor matrix is constrained as an indicator matrix rather than a simple nonnegative matrix. To enhance the clustering robustness, the L2,1-norm is adopted to minimize the error of results between k-means++ and landmark graph-based clustering. To solve the model of RLGCH, we established a novel efficient optimization strategy to obtain all sample categories directly. Combining our clustering model and optimization strategy, the computational complexity is reduced to linear and insensitive to data dimensions. Extensive experiments on seven real-world datasets and sixteen noisy datasets show that compared with other state-of-the-art methods, RLGCH can improve the clustering efficiency and robustness greatly while guaranteeing comparable or even better clustering effectiveness.

Touchless Control of Heavy Equipment Using Low-Cost Hand Gesture Recognition

Human-machine interaction using remote hand gestures is becoming increasingly prevalent across various industries. However, their potential application to heavy construction equipment is often overlooked. This article presents a robust and inexpensive hand gesture recognition system that was implemented and tested on a robotic 1-tonne wheel loader. The system uses an RGB camera paired with a laptop to process, in real time, hand gestures to control the loader. We first design four unique gestures for controlling the loader and then collect 26,000 images to train and test a neural network for hand gesture recognition. Our system uses robust landmark detection using an off-the-shelf system prior to gesture recognition. We successfully controlled the loader to excavate in a rock pile by using the proposed hand gesture recognition system.

Styloid Process Is the Robust Landmark for Identification of the Facial Nerve during Parotid Surgery

Objective: To identify the facial nerve trunk (FNT) very easily. quickly and accurately considering the styloid process as the robust anatomically consistent landmark during parotid surgery. Patients and Methods: This prospective study was conducted between January 2010 to December 2022 at a Tertiary care hospital. It included 130 patients: 84 (65%) males and 46 (35%) females with a mean age of 43.52 ± 15.63 years. Patients presented with pleomorphic adenoma (90), chronic sialectasis (05), parotid cyst (03), parotid sinus (01), mucoepidermoid carcinoma (28), adenoid cystic carcinoma (02) and squamous cell carcinoma (01) were subjected to superficial parotidectomy and total parotidectomy. The outcome was evaluated regarding the clinical success of facial nerve function that considered the location of the styloid process in the triangle technique for identification of the FNT during parotid surgery. Results: All 130 patients underwent superficial and total parotidectomy. The facial nerve trunk was successfully identified as the location of the styloid process in the triangle technique in all the patients with no intra-operative complications. Operative time ranged from 50 to 180 minutes with a mean time 90.70 ± 15.68 minutes. Dysfunction of the angle of the mouth due to the deficit of the marginal mandibular nerve (MMN) was noted in 5 patients (3.8%), 02 had difficulty in eye closure (1.5%). No patient reported surgical site infection, skin flap necrosis and hematoma in this series. Conclusion: Styloid process is the prominent robust landmark in the triangle technique for the identification of facial nerve trunk with relative ease, safety and accuracy in the parotid surgical procedure. This can be a very useful method to locate the FNT and minimize facial nerve injury during parotid surgery.

3FO: The Three-Frame-Only Approach for Fast and Accurate Monocular SLAM Initialization

The monocular simultaneous localization and mapping (SLAM) system is one of the most important among all visual or visual-inertial SLAM (VSLAM or VISLAM) systems due to its low cost, easy calibration and identification. Initialization is always crucial to bootstrap the monocular SLAM system. With the rapid growth of some quick-start required SLAM applications, e.g., augmented reality (AR) and unmanned aerial vehicles (UAVs), devising faster and more accurate initialization has become a central problem. Traditional initialization uses the first two frames to create landmarks and then computes camera poses for the subsequential frames, using a 3D-to-2D perspective-n-points (PnP) mechanism. In this paper, we propose a novel three-frame-only (3FO) initialization approach for the monocular SLAM system, which consists of two steps. In the first step, we use the first two frames to preinitialize poses and landmarks, and in the second step, we use the second and third frames to improve the preinitialization by using the scale consistency of the landmarks generated in the first step to filter out outliers and using inliers to generate more robust landmarks. In both steps, we use the pretrained multilayer perceptron (MLP) combined with homotopy continuation to solve the essential matrices. Finally, we perform a global bundle adjustment (BA) to refine the three camera poses and all the created landmarks. The proposed 3FO initialization approach is evaluated experimentally on the EuRoC benchmark data set with the initialization time and trajectory metrics. The results show that, compared to the traditional ORB-SLAM2 initialization, the 3FO approach reduces the initialization time by 5 times and improves the accuracy by 36.7% on average.

Accuracy of joint line restoration based on three-dimensional registration of the contralateral tibial tuberosity and the fibular tip

PurposeTo assess a novel method of three-dimensional (3D) joint line (JL) restoration based on the contralateral tibia and fibula.Methods3D triangular surface models were generated from computed tomographic data of 96 paired lower legs (48 cadavers) without signs of pathology. Three segments of the tibia and fibula, excluding the tibia plateau, were defined (tibia, fibula, tibial tuberosity (TT) and fibular tip). A surface registration algorithm was used to superimpose the mirrored contralateral model onto the original model. JL approximation and absolute mean errors for each segment registration were measured and its relationship to gender, height, weight and tibia and fibula length side-to-side differences analyzed. Fibular tip to JL distance was measured and analyzed.ResultsMean JL approximation did not yield significant differences among the three segments. Mean absolute JL error was highest for the tibia 1.4 ± 1.4 mm (range: 0 to 6.0 mm) and decreased for the fibula 0.8 ± 1.0 mm (range: 0 to 3.7 mm) and for TT and fibular tip segment 0.7 ± 0.6 (range: 0 to 2.4 mm) (p = 0.03). Mean absolute JL error of the TT and fibular tip segment was independent of gender, height, weight and tibia and fibula length side-to-side differences. Mean fibular tip to JL distance was 11.9 ± 3.4 mm (range: 3.4 to 22.1 mm) with a mean absolute side-to-side difference of 1.6 ± 1.1 mm (range: 0 to 5.3 mm).Conclusion3D registration of the contralateral tibia and fibula reliably approximated the original JL. The registration of, TT and fibular tip, as robust anatomical landmarks, improved the accuracy of JL restoration independent of tibia and fibula length side-to-side differences.Level of evidenceIV

Tracking 3D ultrasound anatomical landmarks via three orthogonal plane-based scale discriminative correlation filter network.

In abdominal interventional therapy, accurate motion tracking of the target is regarded as crucial to minimize trauma and optimize dosage delivery. Meanwhile, three-dimensional (3D) ultrasound (US) is an attractive modality to show the real-time motion pattern of the target. In this work, we developed an accurate and robust landmark tracking algorithm for 3D US sequences. The proposed algorithm introduces a three orthogonal planes (TOPs) based scale discriminative correlation filter network for 3D US landmarks tracking. First, we couple the fully convolutional network (FCN) with scale discriminative correlation filter (SDCF) to generate an effective tracker. And SDCF is reformulated as a differentiable layer, which ensures the network can perform scale learning and end-to-end training. Next, we train the end-to-end network on millions of natural images. Finally, we convert 3D US image to 2D three-channel image by TOP transformation and feed them to the proposed network for performing online tracking. Online tracking performance was evaluated on the Challenge of Liver Ultrasound Tracking (CLUST) dataset with 22 sets of 3D US sequences, obtaining mean error of 1.63±1.04mm and 95th percentile (95%ile) error of 3.37mm, when compared with manual annotations annotated by surgeons. Ablation study indicates that the promising results benefit from SDCF and scale learning, which alleviates the influence from deformation. The findings of the clinical analysis support that the proposed algorithm can work well with different initial patch sizes, which means that our algorithm has potential to lighten the burden of surgeons. We propose a flexible, accurate and robust landmark tracking algorithm for the image-guided interventions, and our algorithm is comparable with the state-of-the-art approaches. The tracking accuracy and robustness show that our algorithm has potential in 3D US-guided abdominal interventional therapies. Furthermore, more researches are needed to improve the computing speed of the algorithm to achieve real-time tracking.

Category-Aware Aircraft Landmark Detection

Aircraft landmark detection (ALD) aims at detecting the keypoints of aircraft, which can serve as an important role for subsequent applications such as fine-grained aircraft recognition. In ALD, the physical size discrepancy between different kinds of aircraft may lead to inconsistent landmark structure, which significantly harms landmark detection results. In this letter, we take advantage of the category prior to alleviate the size discrepancy in ALD. The proposed category-aware landmark detection network (CALDN) possesses two streams: a classification stream for size categorization and a localization stream for landmark detection. Instance-level size category information captured by classification stream serves as the guidance in the localization stream for robust landmark detection. Moreover, a category attention module (CAM) is proposed for better-utilizing category information to guide ALD. Benefitting from the adaptive attention mechanism, CAM can automatically highlight category-specific features for ulteriorly reducing the influence of size discrepancy. Furthermore, to advance ALD research, we contribute the first perspective-variant aircraft landmark dataset. Solid experiments demonstrate the superiority of our method.

Deep Reinforcement Learning (DRL): Another Perspective for Unsupervised Wireless Localization

Location is key to spatialize internet-of-things (IoT) data. However, it is challenging to use low-cost IoT devices for robust unsupervised localization (i.e., localization without training data that have known location labels). Thus, this paper proposes a deep reinforcement learning (DRL) based unsupervised wireless-localization method. The main contributions are as follows. (1) This paper proposes an approach to model a continuous wireless-localization process as a Markov decision process (MDP) and process it within a DRL framework. (2) To alleviate the challenge of obtaining rewards when using unlabeled data (e.g., daily-life crowdsourced data), this paper presents a reward-setting mechanism, which extracts robust landmark data from unlabeled wireless received signal strengths (RSS). (3) To ease requirements for model re-training when using DRL for localization, this paper uses RSS measurements together with agent location to construct DRL inputs. The proposed method was tested by using field testing data from multiple Bluetooth 5 smart ear tags in a pasture. Meanwhile, the experimental verification process reflected the advantages and challenges for using DRL in wireless localization.

Region-Based Context Enhanced Network for Robust Multiple Face Alignment

The recent studies for face alignment have involved developing an isolated algorithm on well-cropped face images. It is difficult to obtain the expected input by using an off-the-shelf face detector in practical applications. In this paper, we attempt to bridge between face detection and face alignment by establishing a novel joint multi-task model, which allows us to simultaneously detect multiple faces and their landmarks on a given scene image. In contrast to the pipeline-based framework by cascading separate models, we aim to propose an end-to-end convolutional network by sharing and transform feature representations between the task-specific modules. To learn a robust landmark estimator for unconstrained face alignment, three types of context enhanced blocks are designed to encode feature maps with multi-level context, multi-scale context, and global context. In the post-processing step, we develop a shape reconstruction algorithm based on point distribution model to refine the landmark outliers. Extensive experiments demonstrate that our results are robust for the landmark location task and insensitive to the location of estimated face regions. Furthermore, our method significantly outperforms recent state-of-the-art methods on several challenging datasets including 300 W, AFLW, and COFW.

Robust facial landmark extraction scheme using multiple convolutional neural networks

Facial landmarks are a set of features that can be distinguished on the human face with the naked eye. Typical facial landmarks include eyes, eyebrows, nose, and mouth. Landmarks play an important role in human-related image analysis. For example, they can be used to determine whether there is a human being in the image, identify who the person is, or recognize the orientation of a face when taking a photograph. General techniques for detecting facial landmarks can be classified into two groups: One is based on traditional image processing techniques, such as Haar cascade classifiers and edge detection. The other is based on machine learning techniques in which landmarks can be detected by training neural network using facial features. However, such techniques have shown low accuracy, especially in some special conditions such as low luminance and overlapped faces. To overcome these problems, we proposed in our previous work a facial landmark extraction scheme using deep learning and semantic segmentation, and demonstrated that with even a small dataset, our scheme could achieve reasonable facial landmark extraction performance under such conditions. Nevertheless, for more extensive dataset, we found several exceptional cases where the scheme could not detect face landmarks precisely. Hence, in this paper, we revise our facial landmark extraction scheme using a deep learning model called Faster R-CNN and show how our scheme can improve the overall performance by handling such exceptional cases appropriately. Also, we show how to expand the training dataset by using image filters and image operations such as rotation for more robust landmark detection.

Robust Landmark Detection and Position Measurement Based on Monocular Vision for Autonomous Aerial Refueling of UAVs

In this paper, a position measurement system, including drogue's landmark detection and position computation for autonomous aerial refueling of unmanned aerial vehicles, is proposed. A multitask parallel deep convolution neural network (MPDCNN) is designed to detect the landmarks of the drogue target. In MPDCNN, two parallel convolution networks are used, and a fusion mechanism is proposed to accomplish the effective fusion of the drogue's two salient parts' landmark detection. Considering the drogue target's geometric constraints, a position measurement method based on monocular vision is proposed. An effective fusion strategy, which fuses the measurement results of drogue's different parts, is proposed to achieve robust position measurement. The error of landmark detection with the proposed method is 3.9%, and it is obviously lower than the errors of other methods. Experimental results on the two KUKA robots platform verify the effectiveness and robustness of the proposed position measurement system for aerial refueling.

Ventricular diastolic dimension over maximal myocardial thickness is robust landmark of systolic impairment in patients with hypertrophic cardiomyopathy.

BackgroundThe effects of focal hypertrophy on geometry of the left ventricle and systolic function have not been studied in patients with hypertrophic cardiomyopathy (HCM), despite the fact that the former is the most prominent disease characteristic. The aim of our study was to analyze systolic function over ventricle geometry, generating a functional index made from left ventricle end diastolic dimension (LVEDD) divided by end diastolic thickness of the region with maximal extent of hypertrophy and interventricular septum.Material/MethodsOur hospital database of cardiac magnetic resonance was screened for HCM. Geometric functional index (GFI) was calculated for LVEDD over maximal end diastolic thickness (MaxEDT) giving GFI-M, while LVEDD over interventricular septum was expressed as GFI-I. There were 55 consecutive patients with HCM.ResultsThere were 43 males (78.2%) and 12 females (21.8%). The mean age was 52.3±16.7 years (range: 15.5–76.4 years). A significant difference of GFI was found for preserved versus impaired systolic function of the left ventricle (preserved systolic function); GFI-M 2.28±0.60 versus 3.66±0.50 (p<0.001), and GFI-I 2.75±0.88 versus 3.81±0.87 (p<0.001), respectively. Diagnostic value was tested using receiver operating curve (ROC) analyzes, with GFI-M area under curve (AUC)=0.959 (95% CI: 0.868–0.994); (p<0.001) and GFI-I-AUC=0.847 (0.724–0.930); (p<0.001). GFI-M was superior to GFI-I for appraisal of left ventricle systolic dysfunction in HCM; ΔAUC=0.112 (0.018–0.207); (p=0.020).ConclusionsGFI is a simple tool, with high sensitivity and specificity for detecting impairment of systolic function in patients with HCM. Further studies would be necessary to investigate its clinical and prognostic impacts, as well as reproducibility with prospective validation.

Automatic selection of landmarks in T1-weighted head MRI with regression forests for image registration initialization.

Medical image registration establishes a correspondence between images of biological structures, and it is at the core of many applications. Commonly used deformable image registration methods depend on a good preregistration initialization. We develop a learning-based method to automatically find a set of robust landmarks in three-dimensional MR image volumes of the head. These landmarks are then used to compute a thin plate spline-based initialization transformation. The process involves two steps: (1)identifying a set of landmarks that can be reliably localized in the images and (2)selecting among them the subset that leads to a good initial transformation. To validate our method, we use it to initialize five well-established deformable registration algorithms that are subsequently used to register an atlas to MR images of the head. We compare our proposed initialization method with a standard approach that involves estimating an affine transformation with an intensity-based approach. We show that for all five registration algorithms the final registration results are statistically better when they are initialized with the method that we propose than when a standard approach is used. The technique that we propose is generic and could be used to initialize nonrigid registration algorithms for other applications.

Heading estimation fusing inertial sensors and landmarks for indoor navigation using a smartphone in the pocket

Principal component analysis (PCA)-based approach for user heading estimation using a smartphone in the pocket suffers from an inaccurate estimation of device attitude, which plays a central role in both obtaining acceleration signals in the horizontal plane and the ultimate global walking direction extraction. To solve this problem, we propose a novel heading estimation approach based on two unscented Kalman filters (UKFs) fusing inertial sensors and landmarks. The first UKF is developed for the recalibration of device attitude estimation. We mathematically derive the measurement equation connecting observed user heading from landmarks with the quaternion vector representing device attitude. To decrease the nonlinearity of the measurement equation and make the filter more robust, we deploy the difference between user heading derived from the landmark and estimation result of PCA-based approach as the observation variable. The second UKF is developed for user heading estimation fusing estimation results of PCA-based approach and observed user headings from landmarks. Besides, we develop a robust landmark identification method by exploiting the acceleration and device pitch patterns, while noisy barometers are no longer required as previous methods. Experiments show that the proposed landmark-aided user heading estimation approach may improve accuracy performance significantly, which is very useful for continuous indoor navigation.

Automatic selection of landmarks in T1-weighted head MRI with regression forests for image registration initialization.

Medical image registration establishes a correspondence between images of biological structures and it is at the core of many applications. Commonly used deformable image registration methods are dependent on a good preregistration initialization. The initialization can be performed by localizing homologous landmarks and calculating a point-based transformation between the images. The selection of landmarks is however important. In this work, we present a learning-based method to automatically find a set of robust landmarks in 3D MR image volumes of the head to initialize non-rigid transformations. To validate our method, these selected landmarks are localized in unknown image volumes and they are used to compute a smoothing thin-plate splines transformation that registers the atlas to the volumes. The transformed atlas image is then used as the preregistration initialization of an intensity-based non-rigid registration algorithm. We show that the registration accuracy of this algorithm is statistically significantly improved when using the presented registration initialization over a standard intensity-based affine registration.

Effective Multi-Query Expansions: Collaborative Deep Networks for Robust Landmark Retrieval.

Given a query photo issued by a user (q-user), the landmark retrieval is to return a set of photos with their landmarks similar to those of the query, while the existing studies on the landmark retrieval focus on exploiting geometries of landmarks for similarity matches between candidate photos and a query photo. We observe that the same landmarks provided by different users over social media community may convey different geometry information depending on the viewpoints and/or angles, and may, subsequently, yield very different results. In fact, dealing with the landmarks with low quality shapes caused by the photography of q-users is often nontrivial and has seldom been studied. In this paper, we propose a novel framework, namely, multi-query expansions, to retrieve semantically robust landmarks by two steps. First, we identify the top- k photos regarding the latent topics of a query landmark to construct multi-query set so as to remedy its possible low quality shape. For this purpose, we significantly extend the techniques of Latent Dirichlet Allocation. Then, motivated by the typical collaborative filtering methods, we propose to learn a collaborative deep networks-based semantically, nonlinear, and high-level features over the latent factor for landmark photo as the training set, which is formed by matrix factorization over collaborative user-photo matrix regarding the multi-query set. The learned deep network is further applied to generate the features for all the other photos, meanwhile resulting into a compact multi-query set within such space. Then, the final ranking scores are calculated over the high-level feature space between the multi-query set and all other photos, which are ranked to serve as the final ranking list of landmark retrieval. Extensive experiments are conducted on real-world social media data with both landmark photos together with their user information to show the superior performance over the existing methods, especially our recently proposed multi-query based mid-level pattern representation method [1].

Robust facial landmark detection using mixture of discriminative visibility‐aware models

Facial landmark detection is a fundamental process included in many face analysis tasks. However, it faces great challenges for the facial images to be detected usually containing large variations, which will deteriorate the detection precision. Among such variations, partial occlusions and pose variations take great effects. In this study, the authors present a mixture of discriminative visibility-aware models (MDVMs) for facial landmark detection, to improve the generalisation ability of the model to occlusion and pose variation. By adopting different structure constrains for different poses as well as selecting different appearance model for the occluded parts, the MDVMs method can efficiently address the problem of partial occlusion and pose variation. Experiment results demonstrate that their proposed MDVMs method outperforms the well-known template-based methods, and can obtain much more accurate and robust facial landmarks detection results under both occlusions and pose variations.