Orientation Estimation Method Research Articles

Multilayer Perceptron-Based Error Compensation for Automatic On-the-Fly Camera Orientation Estimation Using a Single Vanishing Point from Road Lane

This study introduces a multilayer perceptron (MLP) error compensation method for real-time camera orientation estimation, leveraging a single vanishing point and road lane lines within a steady-state framework. The research emphasizes cameras with a roll angle of 0°, predominant in autonomous vehicle contexts. The methodology estimates pitch and yaw angles using a single image and integrates two Kalman filter models with inputs from image points (u, v) and derived angles (pitch, yaw). Performance metrics, including avgE, minE, maxE, ssE, and Stdev, were utilized, testing the system in both simulator and real-vehicle environments. The outcomes indicate that our method notably enhances the accuracy of camera orientation estimations, consistently outpacing competing techniques across varied scenarios. This potency of the method is evident in its adaptability and precision, holding promise for advanced vehicle systems and real-world applications.

Motion Reconstruction of Fast-Rotating Rigid Bodies

Abstract Motion reconstruction and navigation require accurate orientation estimation. Modern orientation estimation methods utilize filtering algorithms, such as the Kalman filter or Madgwick's algorithm. However, these methods do not address potential sensor saturation, which may occur within short time periods in highly dynamic applications, such as, e.g., particle tracking in snow avalanches, leading to inaccurate orientation estimates. In this paper, we present two algorithms for orientation estimation combining magnetometer and partially saturated gyrometer readings. One algorithm incorporates magnetic field vector observations and the full nonlinearity of the exponential map. The other, computationally more efficient algorithm builds on a linearization of the exponential map and is solved analytically. Both algorithms are then applied to measurement data from four different experiments, with two of them being snow avalanche experiments. Moreover, Madgwick's filtering algorithm was used to validate the proposed algorithms. The two algorithms improved the orientation estimation significantly in all experiments. Hence, the proposed algorithms can improve the performance of existing sensor fusion algorithms significantly.

Self-supervised rotation-equivariant spherical vector network for learning canonical 3D point cloud orientation

The perception of orientation in augmented reality, robot grasping, and 3D scene understanding is commonly addressed through the utilization of hand-crafted geometric features. However, machines can also learn the inherent orientation of 3D point clouds through experience, similar to humans. In this paper, we propose a self-supervised spherical vector network that is rotation-equivariant. Specifically, we use density-aware adaptive sampling to construct spherical signal samples to handle distorted point distributions in spherical space. Spherical convolutional vector layers and spherical routing layers are proposed to extract the rotation-equivariant vectors that represent the existence probability of the entity and orientations. Our method learns rotational representations from 3D point clouds through a self-supervised training process. We also provide theoretical proof that our proposed spherical vector networks are rotation-equivariant. Experiments on a variety of public datasets directly and indirectly demonstrate the effectiveness of the proposed method for canonical orientation estimation, even on unknown classes.

2OC: A General Automated Orientation and Orthorectification Method for Corona KH-4B Panoramic Imagery

Due to a lack of geographical reference information, complex panoramic camera models, and intricate distortions, including radiation, geometric, and land cover changes, it can be challenging to effectively apply the large number (800,000+) of high-resolution Corona KH-4B panoramic images from the 1960s and 1970s for surveying-related tasks. This limitation hampers their significant potential in the remote sensing of the environment, urban planning, and other applications. This study proposes a method called 2OC for the automatic and accurate orientation and orthorectification of Corona KH-4B images, which is based on generalized control information from reference images such as Google Earth orthophoto. (1) For the Corona KH-4B panoramic camera, we propose an adaptive focal length variation model that ensures accuracy and consistency. (2) We introduce a robust multi-source remote sensing image matching algorithm, which includes an accurate primary orientation estimation method, a multi-threshold matching enhancement strategy based on scale, orientation, and texture (MTE), and a model-guided matching strategy. These techniques are employed to extract high-accuracy generalized control information for Corona images with significant geometric distortions and numerous weak texture areas. (3) A time-iterative Corona panoramic digital differential correction method is proposed. The orientation and orthorectification results of KH-4B images from multiple regions, including the United States, Russia, Austria, Burkina Faso, Beijing, Chongqing, Gansu, and the Qinghai–Tibet Plateau in China, demonstrate that 2OC not only achieves automation but also attains a state-of-the-art level of generality and accuracy. Specifically, the standard deviation of the orientation is less than 2 pixels, the mosaic error of orthorectified images is approximately 1 pixel, and the standard deviation of ground checkpoints is better than 4 m. In addition, 2OC can provide a longer time series analysis of data from 1962 to 1972, benefiting various fields such as environmental remote sensing and archaeology.

Orientation estimation for instrumented helmet using neural networks

This work presents an integrated solution for head orientation estimation, which is a critical component for applications of virtual and augmented reality systems. The proposed solution builds upon the measurements from the inertial sensors and magnetometer added to an instrumented helmet, and an orientation estimation algorithm is developed to mitigate the effect of bias introduced by noise in the gyroscope signal. Convolutional Neural Network (CNN) techniques are introduced to develop a dynamic orientation estimation algorithm with a structure motivated by complementary filters and trained on data collected to represent a wide range of head motion profiles. The proposed orientation estimation method is evaluated experimentally and compared to both learning and non-learning-based orientation estimation algorithms found in the literature for comparable applications. Test results support the advantage of the proposed CNN-based solution, particularly for motion profiles with high acceleration disturbance that are characteristic of head motion.

Asynchronous Method of Simultaneous Object Position and Orientation Estimation with Two Transmitters

Asynchronous Method of Simultaneous Object Position and Orientation Estimation with Two Transmitters

A Multiscale Unsupervised Orientation Estimation Method With Transformers for Remote Sensing Image Matching

Estimating the orientations of remote sensing images is a very important step in remote sensing image matching and is now gradually receiving widespread attention. However, due to the inability to explicitly define the standard orientations of feature points, the current methods still produce feature point orientation estimation errors, resulting in reduced matching accuracy. In this letter, we propose a multiscale unsupervised orientation estimation method with transformers, in which we use a multiscale feature extraction module to aggregate rich semantic features and a transformer-based attention mechanism module to address robust feature extraction in weakly textured regions while predicting the orientations of feature points through a carefully designed loss function. We set up image matching experiments on remote sensing images in different scenes for comparison purposes, and the experimental results show that our proposed method achieves substantially improved orientation estimation accuracy and improved image matching performance.

3-DOF Orientation Motion Control of a Weight-Compensated Spherical Motor with Embedded Magnetic Field Sensing

This paper presents a real-time 3-DOF orientation motion control for a weight-compensated spherical motor with a rotor dynamic model derived using 3D axis-angle representation to facilitate the controller design. By integrating the sensor fusion-based real-time 3-DOF orientation estimation method, a sliding mode control method is developed employing the equivalent axis and angle as the control parameters, which is followed by the investigation of system stability using the Direct Lyapunov stability method. The effectiveness and accuracy of the control system are experimentally demonstrated and evaluated on a permanent-magnet spherical motor prototype fabricated using additive manufacturing with regulation and tracking control experiments.

HMM Based Adaptive Kalman Filter for Orientation Estimation

An adaptive orientation estimation method based on hidden Markov model (HMM) using multiple Magnetic Angular Rate Gravity (MARG) sensor is proposed to address the problem of linear acceleration and ferromagnetic disturbance. Two parallel Kalman filters containing first-order Gauss-Markov disturbance models are implemented to estimate the disturbances before estimating the orientation. It is important that the noise covariance of the disturbance model should match the magnitude of the disturbance to ensure the estimation performance; otherwise, the disturbance estimation accuracy will reduce significantly. Therefore, an adaptive Kalman filter based on HMM is designed to adjust the disturbance noise covariance. After the disturbances are separated, the estimated gravity acceleration and geomagnetic field as well as the gyroscope measurement are taken as the inputs, and the orientation estimation procedure is established by using the multiplicative extended Kalman filter (MEKF). Experiments show that the proposed method achieves a higher level of accuracy in comparison with other orientation estimation methods under various conditions.

A Fast and Simple Method for Absolute Orientation Estimation Using a Single Vanishing Point

Absolute orientation estimation is one of the key steps in computer vision, and n 2D–3D point correspondences can be used to obtain the absolute orientation, which is known as the perspective-n-point problem (PnP). The lowest number of point correspondences is three if there is no other information, and the corresponding algorithm is called the P3P solver. In practice, the real scene may consist of some geometric information, e.g., the vanishing point. When scenes contain parallel lines, they intersect at vanishing points. Hence, to reduce the number of point correspondences and increase the computational speed, we proposed a fast and simple method for absolute orientation estimation using a single vanishing point. First, the inertial measurement unit (IMU) was used to obtain the rotation of the camera around the Y-axis (i.e., roll angle), which could simplify the orientation estimation. Then, one vanishing point was used to estimate the coarse orientation because it contained direction information in both the camera frame and world frame. Finally, our proposed method used a non-linear optimization algorithm for solution refining. The experimental results show that compared with several state-of-the-art orientation estimation solvers, our proposed method had a better performance regarding numerical stability, noise sensitivity, and computational speed in synthetic data and real images.

Estimation of Crystal Orientation of Grains on Polycrystalline Silicon Substrate by Recurrent Neural Network

To analyze crystal defects in polycrystalline silicon substrates, it is necessary to measure the crystal orientation at high speed. In this paper, we propose a method for simultaneous and fast estimation of the crystal orientation of an entire substrate by measuring the reflected light from a single rotation of a light source. The effectiveness of the proposed method is verified by the orientation estimation experiments. © 2022 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

ADAPTIVE MORPHOLOGICAL FRAMEWORK FOR 3D DIRECTIONAL FILTERING

Engineering materials often feature lower dimensional and directed structures such as cracks, fibers, or closedfacets in foams. The characterization of such structures in 3D is of particular interest in various applications inmaterials science. In image processing, knowledge of the local structure orientation can be used for structureenhancement, directional filtering, segmentation, or separation of interacting structures. The idea of usingbanks of directed structuring elements or filters parameterized by a discrete subset of the orientation space isproven to be effective for these tasks in 2D. However, this class of methods is prohibitive in 3D due to the highcomputational burden of filtering on a sufficiently fine discretization of the unit sphere.This paper introduces a method for 3D pixel-wise orientation estimation and directional filtering inspiredby the idea of adaptive refinement in discretized settings. Furthermore, an operator for distinction betweenisotropic and anisotropic structures is defined based on our method. This operator utilizes orientationinformation to successfully preserve structures with one or two dominant dimensions. Finally, feasibilityand effectiveness of the method are demonstrated on 3D micro-computed tomography images in three usecases: detection of a misaligned region in a fiber-reinforced material, crack detection in concrete, and facetdetection in partially closed foams.

Target magnetic moment orientation estimation method based on full magnetic gradient orthonormal basis function

Magnetic anomaly detection (MAD) is an effective method to detect the existence and localization of magnetic targets. Magnetic signal processing technology can extract target signals from complex background noise. However, traditional magnetic signal processing methods cannot greatly improve the signal-to-noise ratio (SNR) while also restoring information concerning the target. This is because the main existing method to calculate a target’s magnetic moment requires a pure target signal. Our research regarding the full magnetic gradient orthonormal basis function (FMG-OBF) addresses the problem of low SNRs for the target magnetic anomaly (TMA) signal. However, this method can only detect the presence of the target and cannot obtain the magnetic moment characteristics of the target. Benefiting from the FMG tensor, which contains large amounts of spatial magnetic field information, this paper is devoted to characterizing TMA in different orientations from the signal energy point of view. We analyze the influences of the target magnetic moment’s variation on the energy components of the TMA signal in various orientations, and further propose a target magnetic moment orientation estimation method. Compared with the traditional signal processing method, the proposed method can estimate the magnetic moment orientation of the target while greatly improving the SNR. Therefore, this method has significant application potential for the classification and identification of weak TMA signals in MAD.

Drift-Free 3D Orientation and Displacement Estimation for Quasi-Cyclical Movements Using One Inertial Measurement Unit: Application to Running.

A Drift-Free 3D Orientation and Displacement estimation method (DFOD) based on a single inertial measurement unit (IMU) is proposed and validated. Typically, body segment orientation and displacement methods rely on a constant- or zero-velocity point to correct for drift. Therefore, they are not easily applicable to more proximal segments than the foot. DFOD uses an alternative single sensor drift reduction strategy based on the quasi-cyclical nature of many human movements. DFOD assumes that the quasi-cyclical movement occurs in a quasi-2D plane and with an approximately constant cycle average velocity. DFOD is independent of a constant- or zero-velocity point, a biomechanical model, Kalman filtering or a magnetometer. DFOD reduces orientation drift by assuming a cyclical movement, and by defining a functional coordinate system with two functional axes. These axes are based on the mean acceleration and rotation axes over multiple complete gait cycles. Using this drift-free orientation estimate, the displacement of the sensor is computed by again assuming a cyclical movement. Drift in displacement is reduced by subtracting the mean value over five gait cycle from the free acceleration, velocity, and displacement. Estimated 3D sensor orientation and displacement for an IMU on the lower leg were validated with an optical motion capture system (OMCS) in four runners during constant velocity treadmill running. Root mean square errors for sensor orientation differences between DFOD and OMCS were 3.1 ± 0.4° (sagittal plane), 5.3 ± 1.1° (frontal plane), and 5.0 ± 2.1° (transversal plane). Sensor displacement differences had a root mean square error of 1.6 ± 0.2 cm (forward axis), 1.7 ± 0.6 cm (mediolateral axis), and 1.6 ± 0.2 cm (vertical axis). Hence, DFOD is a promising 3D drift-free orientation and displacement estimation method based on a single IMU in quasi-cyclical movements with many advantages over current methods.

Orientation Estimation of Rotated Sonar Image Targets via the Wavelet Subimage Energy Ratio

Precise orientation information facilitates target recognition of sonar images. Traditional orientation estimation methods, including the Hough transform method, have poor antinoise abilities, so they cannot achieve high estimation precision for sonar images, which usually have low signal-to-noise ratios. The convolutional neural network (CNN) regression method is sensitive to image affine transformations and requires extensive computation, so it cannot achieve strong robustness or high speed. To achieve high-precision, high-speed, and robust orientation estimation of sonar image targets, we present a novel orientation estimation method via the wavelet subimage energy ratio (WSER). The WSER varies with the rotation angles of images and has the highest value when the long axes of targets are vertical. It is translational and scale invariant and does not need supervised training. Therefore, we propose estimating orientations of sonar image targets by finding the max of the WSER to achieve high precision, high speed, and strong robustness to the translation and scale transformations of images. The results of experiments on a self-made sonar image dataset show that the mean absolute error (MAE) of the proposed method is 7.9°, while the MAEs of the CNN regression method and traditional methods are 10.1 and 22.9°, respectively. In addition, the proposed method is six times faster than the CNN regression method. The proposed orientation estimation method has also been applied to align the orientation of sonar images and CNN can achieve state-of-the-art performance for rotated target recognition using the aligned images. Data and codes are publicly available.

A Novel Pixel Orientation Estimation Based Line Segment Detection Framework, and Its Applications to SAR Images

In this paper we propose a new line segment detection framework based on a novel Pixel Orientation Estimation (POE) method, which detects line segments from binary edge maps and can be combined with any edge detectors. We show its efficiency by testing it in 1-look SAR images. The novel Pixel Orientation Estimation method estimates the orientation of each edge pixel by counting the number of edge pixels along a set of orientations. As the most edge pixels exist along the orientation of the line segment, the orientation of the edge pixel is given by the orientation that gives the maximum number of counts. Counting the number of edge pixels along different orientations is equivalent to convolving the local neighbourhood of an edge pixel with a set of carefully designed window functions with each window function corresponding to a fixed orientation. With the estimated orientations of pixels in the edge map, pixels can be grouped with a region growing step to form line support regions. Regions with their size larger than a size threshold will be accepted. Finally, rectangles are used to approximate the accepted regions and those rectangles are detected line segments. Experiments in both simulated SAR dataset and three 1-look Sentinel-1 images demonstrate the efficiency of the proposed method. In particular, we advance the state-of-the-art performances by 18 percent (F1-score) on the 1-look dataset simulated from YorkUrban-LineSegment Dataset.

METHOD AND ALGORITHMS FOR CALCULATING HIGH-PRECISION ORIENTATION AND MUTUAL BINDING OF COORDINATE SYSTEMS OF SPACECRAFT STAR TRACKERS CLUSTER BASED ON INACCURATE MEASUREMENTS

The problem of increasing the accuracy of determining the orientation of a spacecraft (SC) using a system of star trackers (ST) is considered. Methods are proposed that make it possible to use a joint field of view and refine the relative position of ST to improve the accuracy of orientation determination. The use of several star trackers leads to an increase in the angle between the directions to the stars into the joint field of view, which makes it possible to reduce the condition number of the matrices used in calculating the orientation parameters. The paper develops a combinatorial method for interval estimation of the SC orientation with an arbitrary number of star trackers. To calculate the ST orientation, a linear problem of interval estimation of the orthogonal orientation matrix for a sufficiently large number of stars is solved. The orientation quaternion is determined under the condition that the corresponding orientation matrix belongs to the obtained interval estimates. The case is considered when the a priori estimate of the mutual binding of star trackers can have an error comparable to or greater than the error in measuring the angular coordinates of stars. With inaccurately specified matrices of the mutual orientation of the star trackers, the errors in the mutual orientations of the STs are added to the errors of measuring the directions to the stars, which leads to an expansion of the uncertainty intervals of the right-hand sides of the system of linear algebraic equations used to determine the orientation parameters. A method is proposed for solving the problem of refining the mutual reference of the internal coordinate systems of a pair of ST as an independent task, after which the main problem of increasing the accuracy of spacecraft orientation is solved. The developed method and algorithms for solving such a complex problem are based on interval estimates of orthogonal orientation matrices. For additional narrowing of the intervals, the property of orthogonality of orientation matrices is used. The numerical simulation carried out made it possible to evaluate the advantages and disadvantages of each of the proposed methods.

Insect 3-D Orientation Estimation Based on Cooperative Observation From Two Views of Entomological Radars

The ability of insect orientation measurement of entomological radars supported the study of insect heading behaviors. However, the current entomological radars can only measure the two-dimension (2D) orientation that is the projection of the three-dimension (3D) orientation on the horizontal plane. The ability to measure the 3D orientation of insect will promote the study of the vertical and 3D heading behaviors of insect. In this study, an insect 3D orientation estimation method based on cooperative observation from two views of entomological radars is proposed. The expression of 3D orientation is deduced based on 2D orientation, azimuth and elevation measured with two radars at different stations through vector operation. The simulations are conducted to verify the effectiveness of the proposed method. The result shows that the estimation error of 3D orientation depends on that of the 2D orientation and the included angle between two lines of sight of two radars. A multi-aspect fully polarimetric rig is designed to measure insect echo signals from two aspects in a microwave anechoic chamber. The experiment data is used to further validate the effectiveness of the proposed method.

NICUface: Robust Neonatal Face Detection in Complex NICU Scenes

The development of non-contact patient monitoring applications for the neonatal intensive care unit (NICU) is an active research area, particularly in facial video analysis. Recent studies have used facial video data to estimate vital signs, assess pain from facial expression, differentiate sleep-wake status, detect jaundice, and in face recognition. These applications depend on an accurate definition of the patient’s face as a region of interest (ROI). Most studies have required manual ROI definition, while others have leveraged automated face detectors developed for adult patients, without systematic validation for the neonatal population. To overcome these issues, this paper first evaluates the state-of-the-art in face detection in the NICU setting. Finding that such methods often fail in complex NICU environments, we demonstrate how fine-tuning can increase neonatal face detector robustness, resulting in our NICUface models. A large and diverse neonatal dataset was gathered from actual patients admitted to the NICU across three studies and gold standard face annotations were completed. In comparison to state-of-the-art face detectors, our NICUface models address NICU-specific challenges such as ongoing clinical intervention, phototherapy lighting, occlusions from hospital equipment, etc. These analyses culminate in the creation of robust NICUface detectors with improvements on our most challenging neonatal dataset of +36.14, +35.86, and +32.19 in AP30, AP50, and mAP respectively, relative to state-of-the-art CE-CLM, MTCNN, img2pose, RetinaFace, and YOLO5Face models. Face orientation estimation is also addressed, leading to an accuracy of 99.45%. Fine-tuned NICUface models, gold-standard face annotation data, and the face orientation estimation method are also released here.

Human Activity Classification Based on Moving Orientation Determining Using Multistatic Micro-Doppler Radar Signals

Traditional micro-Doppler (m-D)-based human activity classification system using monostatic radar suffers from the drawback that classification performance is vulnerable to the variation of human motion aspect angle. This leads to a performance degradation if the human movements are not directly toward or away with respect to the radar line of sight. The multistatic radar system has been suggested as an effective solution to solve the problem, as it can observe the target from multiple views and achieve favorable aspect angles to the targets. In this article, a novel human activity classification method based on motion orientation determining using multistatic m-D signals is proposed. First, the aspect angles of target motion direction with respect to each radar nodes are inferred by using the proposed motion orientation estimation method. The multistatic m-D data are then divided into several intervals based on the measured angle, and the data in the same interval are fused at the data level. Finally, the classification results are obtained through the adaptive weighted decision-level fusion. Compared with the traditional multistatic classification method, due to the consideration of the time-varying human motion aspect angle, the proposed method is more reasonable in data fusion and has better classification performance.