Accurate Orientation Estimation Research Articles

Convolutional Cross-View Pose Estimation.

We propose a novel end-to-end method for cross-view pose estimation. Given a ground-level query image and an aerial image that covers the query's local neighborhood, the 3 Degrees-of-Freedom camera pose of the query is estimated by matching its image descriptor to descriptors of local regions within the aerial image. The orientation-aware descriptors are obtained by using a translationally equivariant convolutional ground image encoder and contrastive learning. The Localization Decoder produces a dense probability distribution in a coarse-to-fine manner with a novel Localization Matching Upsampling module. A smaller Orientation Decoder produces a vector field to condition the orientation estimate on the localization. Our method is validated on the VIGOR and KITTI datasets, where it surpasses the state-of-the-art baseline by 72% and 36% in median localization error for comparable orientation estimation accuracy. The predicted probability distribution can represent localization ambiguity, and enables rejecting possible erroneous predictions. Without re-training, the model can infer on ground images with different field of views and utilize orientation priors if available. On the Oxford RobotCar dataset, our method can reliably estimate the ego-vehicle's pose over time, achieving a median localization error under 1 m and a median orientation error of around 1 ° at 14 FPS.

Multi-sensor Attitude Estimation using Quaternion Constrained GNSS Ambiguity Resolution and Dynamics-Based Observation Synchronization

Recently, high accuracy and low-cost navigation hardware is becoming increasingly available that can be efficiently used for the control of autonomous vehicles. We present a sensor fusion method providing tightly coupled integration of pseudorange, carrier phase, and Doppler satellite measurements taken at multiple vehicle-mounted GNSS antennas with onboard inertial sensor observations. The key of accurate GNSS position and orientation estimation is the successful integer ambiguity resolution. We propose a method that uses the quaternion states as constraints to improve ambiguity resolution and to increase the accuracy of the GNSS based attitude determination. Generally, the low-cost hardware neither allows a hardware-level time synchronization between the GNSS receivers due to a lack of a common external oscillator nor provides the clock steering function available in geodetic GNSS receivers. The lack of observation synchronization causes several degrees of error in attitude estimation. To eliminate this effect, a dynamics-based solution is presented that synchronizes the observations by taking the dynamics of the moving platform into account. Compared to common external oscillator based sensor setups, our solution allows to increase both the number of rover receivers on the platform and the baselines between them easily, thus it opens up new possibilities in the attitude determination of large vehicles. We validate our approach against a tactical grade inertial navigation system. The results show that our approach using low-cost sensors provides the ambiguity success rate of 100% for the moving baselines, and the positioning and attitude error reached the centimeter and half a degree level, respectively.

Deep Learning-Based Localization and Orientation Estimation of Pedicle Screws in Spinal Fusion Surgery.

This study investigated the application of a deep learning-based object detection model for accurate localization and orientation estimation of spinal fixation surgical instruments during surgery. We employed the You Only Look Once (YOLO) object detection framework with oriented bounding boxes (OBBs) to address the challenge of non-axis-aligned instruments in surgical scenes. The initial dataset of 100 images was created using brochure and website images from 11 manufacturers of commercially available pedicle screws used in spinal fusion surgeries, and data augmentation was used to expand 300 images. The model was trained, validated, and tested using 70%, 20%, and 10% of the images of lumbar pedicle screws, with the training process running for 100 epochs. The model testing results showed that it could detect the locations of the pedicle screws in the surgical scene as well as their direction angles through the OBBs. The F1 score of the model was 0.86 (precision: 1.00, recall: 0.80) at each confidence level and mAP50. The high precision suggests that the model effectively identifies true positive instrument detections, although the recall indicates a slight limitation in capturing all instruments present. This approach offers advantages over traditional object detection in bounding boxes for tasks where object orientation is crucial, and our findings suggest the potential of YOLOv8 OBB models in real-world surgical applications such as instrument tracking and surgical navigation. Future work will explore incorporating additional data and the potential of hyperparameter optimization to improve overall model performance.

Motion Reconstruction of Fast-Rotating Rigid Bodies

Abstract Motion reconstruction and navigation require accurate orientation estimation. Modern orientation estimation methods utilize filtering algorithms, such as the Kalman filter or Madgwick's algorithm. However, these methods do not address potential sensor saturation, which may occur within short time periods in highly dynamic applications, such as, e.g., particle tracking in snow avalanches, leading to inaccurate orientation estimates. In this paper, we present two algorithms for orientation estimation combining magnetometer and partially saturated gyrometer readings. One algorithm incorporates magnetic field vector observations and the full nonlinearity of the exponential map. The other, computationally more efficient algorithm builds on a linearization of the exponential map and is solved analytically. Both algorithms are then applied to measurement data from four different experiments, with two of them being snow avalanche experiments. Moreover, Madgwick's filtering algorithm was used to validate the proposed algorithms. The two algorithms improved the orientation estimation significantly in all experiments. Hence, the proposed algorithms can improve the performance of existing sensor fusion algorithms significantly.

A hybrid framework for efficient and accurate orientation estimation based on single and multiple orientation vector fields

This article presents a hybrid framework for efficient and accurate orientation estimation. The proposed scheme combines the single orientation information given by a novel method and the multiple orientation information provided by a bank of linear orientated morphological openings. The single orientations are estimated by means of an energy-minimization Gaussian filtering which solves the drawback related to phase changes of other methods. After describing the formulation of these two approaches for estimating the existing orientations in the pixels of an image, several strategies have been analyzed to fuse and discriminate the information of both orientation vector fields in the resulting hybrid orientation vector field. The objective of the proposed hybrid method is to reduce the computational cost involved in calculating multiple orientations only in those pixels where they exist while maintaining the accuracy provided by the single orientation method in the remaining pixels. To this end, strategies ranging from a threshold in the multiple orientation vector field to a convolutional neural network trained with a set of patterns specifically designed to detect pixels with multiple orientations, passing through the Harris corner detector, have been tested to identify those pixels where multiple orientations exist. Results on natural and synthetic images show the accuracy and the computational efficiency achieved by the proposed hybrid framework to provide the vector field with single and multiple orientations.

2OC: A General Automated Orientation and Orthorectification Method for Corona KH-4B Panoramic Imagery

Due to a lack of geographical reference information, complex panoramic camera models, and intricate distortions, including radiation, geometric, and land cover changes, it can be challenging to effectively apply the large number (800,000+) of high-resolution Corona KH-4B panoramic images from the 1960s and 1970s for surveying-related tasks. This limitation hampers their significant potential in the remote sensing of the environment, urban planning, and other applications. This study proposes a method called 2OC for the automatic and accurate orientation and orthorectification of Corona KH-4B images, which is based on generalized control information from reference images such as Google Earth orthophoto. (1) For the Corona KH-4B panoramic camera, we propose an adaptive focal length variation model that ensures accuracy and consistency. (2) We introduce a robust multi-source remote sensing image matching algorithm, which includes an accurate primary orientation estimation method, a multi-threshold matching enhancement strategy based on scale, orientation, and texture (MTE), and a model-guided matching strategy. These techniques are employed to extract high-accuracy generalized control information for Corona images with significant geometric distortions and numerous weak texture areas. (3) A time-iterative Corona panoramic digital differential correction method is proposed. The orientation and orthorectification results of KH-4B images from multiple regions, including the United States, Russia, Austria, Burkina Faso, Beijing, Chongqing, Gansu, and the Qinghai–Tibet Plateau in China, demonstrate that 2OC not only achieves automation but also attains a state-of-the-art level of generality and accuracy. Specifically, the standard deviation of the orientation is less than 2 pixels, the mosaic error of orthorectified images is approximately 1 pixel, and the standard deviation of ground checkpoints is better than 4 m. In addition, 2OC can provide a longer time series analysis of data from 1962 to 1972, benefiting various fields such as environmental remote sensing and archaeology.

Coded Estimation: Design of Backscatter Array Codes for 3D Orientation Estimation

We consider the problem of estimating the orientation of a 3D object with the assistance of configurable backscatter tags. We explore the idea of designing tag response codes to improve the accuracy of orientation estimation. To minimize the difference between the true and estimated orientation, we propose two code design criteria. We also derive a lower bound on the worst-case error using Le Cam’s method and provide simulation results for multiple scenarios including line-of-sight only and multipath, comparing the theoretical bounds to those achieved by the designs.

Effects of Data Augmentation on the Nine-Axis IMU-Based Orientation Estimation Accuracy of a Recurrent Neural Network.

The nine-axis inertial and measurement unit (IMU)-based three-dimensional (3D) orientation estimation is a fundamental part of inertial motion capture. Recently, owing to the successful utilization of deep learning in various applications, orientation estimation neural networks (NNs) trained on large datasets, including nine-axis IMU signals and reference orientation data, have been developed. During the training process, the limited amount of training data is a critical issue in the development of powerful networks. Data augmentation, which increases the amount of training data, is a key approach for addressing the data shortage problem and thus for improving the estimation performance. However, to the best of our knowledge, no studies have been conducted to analyze the effects of data augmentation techniques on estimation performance in orientation estimation networks using IMU sensors. This paper selects three data augmentation techniques for IMU-based orientation estimation NNs, i.e., augmentation by virtual rotation, bias addition, and noise addition (which are hereafter referred to as rotation, bias, and noise, respectively). Then, this paper analyzes the effects of these augmentation techniques on estimation accuracy in recurrent neural networks, for a total of seven combinations (i.e., rotation only, bias only, noise only, rotation and bias, rotation and noise, and rotation and bias and noise). The evaluation results show that, among a total of seven augmentation cases, four cases including 'rotation' (i.e., rotation only, rotation and bias, rotation and noise, and rotation and bias and noise) occupy the top four. Therefore, it may be concluded that the augmentation effect of rotation is overwhelming compared to those of bias and noise. By applying rotation augmentation, the performance of the NN can be significantly improved. The analysis of the effect of the data augmentation techniques presented in this paper may provide insights for developing robust IMU-based orientation estimation networks.

CoVRage: Millimeter-Wave Beamforming for Mobile Interactive Virtual Reality

Contemporary Virtual Reality (VR) setups often include an external source delivering content to a Head-Mounted Display (HMD). “Cutting the wire” in such setups and going truly wireless will require a wireless network capable of delivering enormous amounts of video data at an extremely low latency. The massive bandwidth of higher frequencies, such as the millimeter-wave (mmWave) band, can meet these requirements. Due to high attenuation and path loss in the mmWave frequencies, beamforming is essential. In wireless VR, where the antenna is integrated into the HMD, any head rotation also changes the antenna’s orientation. As such, beamforming must adapt, in real-time, to the user’s head rotations. An HMD’s built-in sensors providing accurate orientation estimates may facilitate such rapid beamforming. In this work, we present coVRage, a receive-side beamforming solution tailored for VR HMDs. Using built-in orientation prediction present on modern HMDs, the algorithm estimates how the Angle of Arrival (AoA) at the HMD will change in the near future, and covers this AoA trajectory with a dynamically shaped oblong beam, synthesized using sub-arrays. We show that this solution can cover these trajectories with consistently high gain, even in light of temporally or spatially inaccurate orientational data.

Trust–Region Nonlinear Optimization Algorithm for Orientation Estimator and Visual Measurement of Inertial–Magnetic Sensor

This paper proposes a novel robust orientation estimator to enhance the accuracy and robustness of orientation estimation for inertial–magnetic sensors of the small consumer–grade drones. The proposed estimator utilizes a trust–region strategy within a nonlinear optimization framework, transforming the orientation fusion problem into a nonlinear optimization problem based on the maximum likelihood principle. The proposed estimator employs a trust–region Dogleg gradient descent strategy to optimize orientation precision and incorporates a Huber robust kernel to minimize interference caused by acceleration during the maneuvering process of the drone. In addition, a novel method for evaluating the performance of orientation estimators is also presented based on visuals. The proposed method consists of two parts: offline calibration of the basic cube using Augmented Reality University of Cordoba (ArUco) markers and online orientation measurement of the sensor carrier using a nonlinear optimization solver. The proposed measurement method’s accuracy and the proposed estimator’s performance are evaluated under low–dynamic (rotation) and high–dynamic (shake) conditions in the experiment. The experimental findings indicate that the proposed measurement method obtains an average re–projection error of less than 0.1 pixels. The proposed estimator has the lowest average orientation error compared to conventional orientation estimation algorithms. Despite the time–consuming nature of the proposed estimator, it exhibits greater robustness and precision, particularly in highly dynamic environments.

A Multiscale Unsupervised Orientation Estimation Method With Transformers for Remote Sensing Image Matching

Estimating the orientations of remote sensing images is a very important step in remote sensing image matching and is now gradually receiving widespread attention. However, due to the inability to explicitly define the standard orientations of feature points, the current methods still produce feature point orientation estimation errors, resulting in reduced matching accuracy. In this letter, we propose a multiscale unsupervised orientation estimation method with transformers, in which we use a multiscale feature extraction module to aggregate rich semantic features and a transformer-based attention mechanism module to address robust feature extraction in weakly textured regions while predicting the orientations of feature points through a carefully designed loss function. We set up image matching experiments on remote sensing images in different scenes for comparison purposes, and the experimental results show that our proposed method achieves substantially improved orientation estimation accuracy and improved image matching performance.

9 축 IMU 기반 자세 추정을 위한 순환 신경망: 교란조건에서의 3 차원 자세 추정 성능

The estimation of 3D orientation based on nine-axis inertial measurement unit (IMU) is an essential technology in various applications, from unmanned aerial vehicle to human motion tracking. Various sensor fusion filter algorithms such as Kalman filter (KF) or complementary filter (CF) have been proposed for accurate 3D orientation estimation. However, the degradation of estimation performance due to disturbance components such as magnetic distortion and external acceleration is still a critical issue. An alternative approach for the orientation estimation task is to train a neural network end-to-end with a variety of massive experimental datasets consisting of the raw IMU signals and the ground truth orientations. This paper proposes a recurrent neural network (RNN) for robust IMU-based 3D orientation estimation. Overall, this paper is an extension of the previous work by Weber et al., where the RNN model was used to estimate the quaternion, but only the attitude estimation performance was investigated without considering the heading estimation. The proposed RNN receives a nine-dimensional sensor signal as an input and outputs a unit quaternion representing 3D orientation, and it can robustly estimate the orientation across various motion characteristics and magnetic environments without needing an additional filter algorithm. Verification results showed that the proposed method outperformed the conventional CF and KF algorithms. In particular, in magnetically disturbed conditions, the averaged root mean square error of the proposed RNN approach was reduced by 41.4% and 60.3%, respectively, when compared to the CF and the KF algorithm.

VQF: Highly accurate IMU orientation estimation with bias estimation and magnetic disturbance rejection

The miniaturization of MEMS-based inertial measurement units (IMUs) facilitates their widespread use in a growing number of application domains. The fundamental sensor fusion task of orientation estimation is a prerequisite for most further data processing steps in inertial motion tracking, such as position and velocity estimation, joint angle estimation, and 3D visualization. Errors in the estimated orientations severely affect all further processing steps. Recent systematic comparisons of existing algorithms show that out-of-the-box accuracy is often low and that application-specific tuning is required to obtain high accuracy. In the present work, we propose and extensively evaluate a quaternion-based orientation estimation algorithm that is based on a novel approach of filtering the acceleration measurements in an almost-inertial frame and that includes extensions for gyroscope bias estimation and magnetic disturbance rejection, as well as a variant for offline data processing. In contrast to all existing work, we perform an extensive evaluation, using a large collection of publicly available datasets and eight literature methods for comparison. The proposed method consistently outperforms all eight literature methods and achieves an average RMSE of 2.9°, while the errors obtained with literature methods range from 5.3° to 16.7°. This improved accuracy with respect to the state of the art is observed not only in average but also for each of several different motion characteristics, as well as for gyroscope bias estimation. Since the evaluation was performed with one single fixed parametrization across a very diverse dataset collection, we conclude that the proposed method provides unprecedented out-of-the-box performance for a broad range of motions, sensor hardware, and environmental conditions. This gain in orientation estimation accuracy is expected to advance the field of IMU-based motion analysis and provide performance benefits in numerous applications. The provided open-source implementation makes it easy to employ the proposed method.

A quaternion-based sensor fusion approach using orthogonal observations from 9D inertial and magnetic information

The micro-electro-mechanical technology facilitates the development of low-cost orientation estimation solutions using inertial and magnetic sensors. However, the magnetic disturbances challenge the robust pitch and roll measurements on some occasions, such as the quadrotor, ship, and railway. This work analyzes the coupling mechanism of the magnetic information on the roll and pitch estimates. Subsequently, it proposes a quaternion-based decoupled sensor fusion approach under the frame of an extended Kalman filter by taking the orthogonal gravitational vector and horizontal magnetic vector as absolute observation references. Three experimental tests validated its performances in robust roll and pitch measurements under magnetic disturbances, static and dynamic robotic motion estimation, and human upper body pose tracking. Results show that the proposed sensor fusion approach could keep the roll and pitch immune to the magnetic disturbances, and provide reliable static and dynamic orientation estimation accuracy. Moreover, it is promising to provide feedback for rehabilitation training.

Simultaneous Localization and Calibration for Cooperative Radio Navigation

Cooperative radio localization and navigation systems can be used in scenarios where the reception of global navigation satellite system (GNSS) signals is not possible or impaired. While the benefit of cooperation has been highlighted by many papers, calibration is not widely considered, but equally important in practice. Utilizing the signal propagation time requires group delay or ranging bias calibration and estimating the directions-of-arrival (DoAs) requires antenna response calibration. Often, calibration parameters are determined only once before operation. However, the calibration parameters are influenced by e.g. changing temperatures of radio frequency (RF) components or changing surroundings of antennas. To cope with that, we derive a cooperative simultaneous localization and calibration (SLAC) algorithm based on Bayesian filtering, which estimates antenna responses and ranging biases simultaneously with positions and orientations. By simulations, we show that the calibration parameters can be estimated during operation without additional sensors. We further proof practical applicability of SLAC by evaluating measurement data from robotic rovers. With SLAC, both ranging and DoAs estimation performance is improved, resulting in better position and orientation estimation accuracy. SLAC is thus able to provide reliable calibration and to mitigate model mismatch. Finally, we discuss open research questions and possible extensions of SLAC.

Steer'n'Detect: fast 2D template detection with accurate orientation estimation.

MotivationRotated template matching is an efficient and versatile algorithm to analyze microscopy images, as it automates the detection of stereotypical structures, such as organelles that can appear at any orientation. Its performance however quickly degrades in noisy image data.ResultsWe introduce Steer’n’Detect, an ImageJ plugin implementing a recently published algorithm to detect patterns of interest at any orientation with high accuracy from a single template in 2D images. Steer’n’Detect provides a faster and more robust substitute to template matching. By adapting to the statistics of the image background, it guarantees accurate results even in the presence of noise. The plugin comes with an intuitive user interface facilitating results analysis and further post-processing.Availability and implementation https://github.com/Biomedical-Imaging-Group/Steer-n-Detect.Supplementary information Supplementary data are available at Bioinformatics online.

PolarMesh: A Star-Convex 3D Shape Approximation for Object Pose Estimation

In this letter, we introduce PolarMesh as a star-convex approximation of a 3D object based on spherical projection and can be applied to monocular object pose and shape estimation. The proposed PolarMesh can be stored in a discrete 2D map that allows a trivial conversion between it and the object surface, as the adjacent map elements form faces of the 3D model. The fixed-grid 2D nature of this object surface approximation ensures that it can be estimated from an image patch of the object with an auto-encoder neural network. Moreover, the PolarMesh not only encodes the object shape but also its orientation. Consequently, it is used for accurate estimation of object orientation and can be extended to the full 6D pose estimation using a projection-based method for translation. An exhaustive quantitative evaluation on LineMod, LineMod-Occlusion and HomebrewedDB shows the competitive performance of the proposed method compared to other monocular RGB pose estimation pipelines. Furthermore, we tested the generalization capabilities of the representation on the recent NOCS dataset for category-level pose estimation and achieved very compelling results in terms of predicted rotations.

VRfRNet: Volumetric ROI fODF reconstruction network for estimation of multi-tissue constrained spherical deconvolution with only single shell dMRI

Diffusion MRI (dMRI) is one of the most popular techniques for studying the brain structure, mainly the white matter region. Among several sampling methods in dMRI, the high angular resolution diffusion imaging (HARDI) technique has attracted researchers due to its more accurate fiber orientation estimation. However, the current single-shell HARDI makes the intravoxel structure challenging to estimate accurately. While multi-shell acquisition can address this problem, it takes a longer scanning time, restricting its use in clinical applications. In addition, most existing dMRI scanners with low gradient-strengths often acquire single-shell up to b=1000s/mm2 because of signal-to-noise ratio issues and severe image artefacts. Hence, we propose a novel generative adversarial network, VRfRNet, for the reconstruction of multi-shell multi-tissue fiber orientation distribution function from single-shell HARDI volumes. Such a transformation learning is performed in the spherical harmonics (SH) space, as raw input HARDI volume is transformed to SH coefficients to soften gradient directions. The proposed VRfRNet consists of several modules, such as multi-context feature enrichment module, feature level attention, and softmax level attention. In addition, three loss functions have been used to optimize network learning, including L1, adversarial, and total variation. The network is trained and tested using standard qualitative and quantitative performance metrics on the publicly available HCP data-set.

A Visual Compass Based on Point and Line Features for UAV High-Altitude Orientation Estimation

The accurate and reliable high-altitude orientation estimation is of great significance for unmanned aerial vehicles (UAVs) localization, and further assists them to conduct some fundamental functions, such as aerial mapping, environmental monitoring, and risk management. However, the traditional orientation estimation is susceptible to electromagnetic interference, high maneuverability, and substantial scale variations. Hence, this paper aims to present a new visual compass algorithm to estimate the orientation of a UAV employing the appearance and geometry structure of the point and line features in the remote sensing images. In this study, a coarse-to-fine feature tracking method is used to locate the matched keypoints precisely. An LK-ZNCC algorithm is proposed to match line segments in real-time. A hierarchical fusion method for point and line features is designed to expand the scope of the usage of this system. Many comparative experiments between this algorithm and others are conducted on a UAV. Experimental results show that the proposed visual compass algorithm is a reliable, precise, and versatile system applicable to other UAV navigation systems, especially when they do not work in particular situations.

The Orientation Estimation of Elongated Underground Objects via Multipolarization Aggregation and Selection Neural Network

The horizontal orientation angle and the vertical inclination angle of an elongated subsurface object are key parameters for object identification and imaging in ground-penetrating radar (GPR) applications. Conventional methods can only extract the horizontal orientation angle or estimate both angles in narrow ranges due to limited polarimetric information and detection capability. To address these issues, this letter, for the first time, explores the possibility of leveraging neural networks with multipolarimetric GPR data to estimate both angles of an elongated subsurface object in the entire spatial range. Based on the polarization-sensitive characteristic of an elongated object, we propose a multipolarization aggregation and selection network (MASNet), which takes the multipolarimetric radargrams as inputs, integrates their characteristics in the feature space, and selects discriminative features of reflected signal patterns for accurate orientation estimation. Numerical results show that our proposed MASNet achieves high estimation accuracy with an angle estimation error of less than 5°. The promising results obtained by the proposed method encourage one to think of new solutions for GPR-related tasks by integrating multipolarization information with deep learning techniques. The data and code implemented in the letter can be found at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://haihan-sun.github.io/GPR.html</uri> .