3D Orientation Estimation Research Articles

Effects of Data Augmentation on the Nine-Axis IMU-Based Orientation Estimation Accuracy of a Recurrent Neural Network.

The nine-axis inertial and measurement unit (IMU)-based three-dimensional (3D) orientation estimation is a fundamental part of inertial motion capture. Recently, owing to the successful utilization of deep learning in various applications, orientation estimation neural networks (NNs) trained on large datasets, including nine-axis IMU signals and reference orientation data, have been developed. During the training process, the limited amount of training data is a critical issue in the development of powerful networks. Data augmentation, which increases the amount of training data, is a key approach for addressing the data shortage problem and thus for improving the estimation performance. However, to the best of our knowledge, no studies have been conducted to analyze the effects of data augmentation techniques on estimation performance in orientation estimation networks using IMU sensors. This paper selects three data augmentation techniques for IMU-based orientation estimation NNs, i.e., augmentation by virtual rotation, bias addition, and noise addition (which are hereafter referred to as rotation, bias, and noise, respectively). Then, this paper analyzes the effects of these augmentation techniques on estimation accuracy in recurrent neural networks, for a total of seven combinations (i.e., rotation only, bias only, noise only, rotation and bias, rotation and noise, and rotation and bias and noise). The evaluation results show that, among a total of seven augmentation cases, four cases including 'rotation' (i.e., rotation only, rotation and bias, rotation and noise, and rotation and bias and noise) occupy the top four. Therefore, it may be concluded that the augmentation effect of rotation is overwhelming compared to those of bias and noise. By applying rotation augmentation, the performance of the NN can be significantly improved. The analysis of the effect of the data augmentation techniques presented in this paper may provide insights for developing robust IMU-based orientation estimation networks.

MmWave 6D Radio Localization With a Snapshot Observation From a Single BS

Accurate and ubiquitous localization is crucial for a variety of applications such as logistics, navigation, intelligent transport, monitoring, control, and also for the benefit of communications. Exploiting millimeter-wave (mmWave) signals in 5G and Beyond 5G systems can provide accurate localization with limited infrastructure. We consider the single base station (BS) localization problem and extend it to 3D position and 3D orientation estimation of an unsynchronized multi-antenna user equipment (UE), using downlink multiple-input multiple-output orthogonal frequency-division multiplexing (MIMO-OFDM) signals. Through a Fisher information analysis, we show that the problem is often identifiable, provided that there is at least one multipath component in addition to the line-of-sight (LoS), even if the position of corresponding incidence point (IP) is a priori unknown. Subsequently, we pose a maximum likelihood (ML) estimation problem, to jointly estimate the 3D position and 3D orientation of the UE as well as several nuisance parameters (the UE clock offset and the positions of IPs corresponding to the multipath). The ML problem is a high-dimensional nonconvex optimization problem over a product of Euclidean and non-Euclidean manifolds. To avoid complex exhaustive search procedures, we propose a geometric initial estimate of all parameters, which reduces the problem to a 1-dimensional search over a finite interval. Numerical results show the efficiency of the proposed ad-hoc estimation, whose gap to the Cramér-Rao bound (CRB) is tightened using the ML estimation.

Cramér–Rao Lower Bounds on 3D Position and Orientation Estimation in Distributed Ranging Systems

As radio frequency (RF) hardware continues to improve, many technologies that were traditionally impractical have suddenly become viable alternatives to legacy systems. Two-way ranging (TWR) is often considered a poor positioning solution for airborne and other vehicular navigation systems due to its low precision, poor angular resolution, and precise timing requirements. With the advent of modern RF hardware and advanced processing techniques, however, modern studies have experimentally demonstrated TWR systems with an unprecedented, sub-centimeter ranging precision with low size, weight, power, and cost (SWaP-C) consumer-grade hardware. This technique enables a new class of positioning, navigation, and timing (PNT) capabilities for urban and commercial aircraft but also instigates new system design challenges such as antenna placement, installation of new electronics, and design of supporting infrastructure. To inform these aircraft design decisions, we derive 2D and 3D Cramér–Rao lower bounds (CRLBs) on position and orientation estimation in a multi-antenna TWR system. We specifically formulate these bounds as a function of the number of antennas, platform geometry, and geometric dilution of precision (GDoP) to inform aircraft design decisions under different mission requirements. We simulate the performance of several classic position and orientation estimators in this context to validate these bounds and to graphically depict the expected performance with respect to these design considerations. To improve the accessibility of these highly theoretical results, we also present a simplified discussion of how these bounds may be applied to common airborne applications and suggest best practices for using them to inform aircraft design decisions.

A Parallel Recurrent Neural Network for Robust Inertial and Magnetic Sensor-Based 3D Orientation Estimation

A Parallel Recurrent Neural Network for Robust Inertial and Magnetic Sensor-Based 3D Orientation Estimation

9 축 IMU 기반 자세 추정을 위한 순환 신경망: 교란조건에서의 3 차원 자세 추정 성능

The estimation of 3D orientation based on nine-axis inertial measurement unit (IMU) is an essential technology in various applications, from unmanned aerial vehicle to human motion tracking. Various sensor fusion filter algorithms such as Kalman filter (KF) or complementary filter (CF) have been proposed for accurate 3D orientation estimation. However, the degradation of estimation performance due to disturbance components such as magnetic distortion and external acceleration is still a critical issue. An alternative approach for the orientation estimation task is to train a neural network end-to-end with a variety of massive experimental datasets consisting of the raw IMU signals and the ground truth orientations. This paper proposes a recurrent neural network (RNN) for robust IMU-based 3D orientation estimation. Overall, this paper is an extension of the previous work by Weber et al., where the RNN model was used to estimate the quaternion, but only the attitude estimation performance was investigated without considering the heading estimation. The proposed RNN receives a nine-dimensional sensor signal as an input and outputs a unit quaternion representing 3D orientation, and it can robustly estimate the orientation across various motion characteristics and magnetic environments without needing an additional filter algorithm. Verification results showed that the proposed method outperformed the conventional CF and KF algorithms. In particular, in magnetically disturbed conditions, the averaged root mean square error of the proposed RNN approach was reduced by 41.4% and 60.3%, respectively, when compared to the CF and the KF algorithm.

Depth-aware gaze-following via auxiliary networks for robotics

Gaze-Following aims to predict the gaze target of a subject within an image, and information on orientation and depth greatly improves this task. However, previous methods require additional datasets to obtain depth or orientation information, leading to cumbersome training or inference processes. To this end, we propose an end-to-end depth-aware gaze-following approach that incorporates depth and orientation information without additional datasets. Our approach identifies a primary task, gaze-following, supervised by true labels from the gaze-following dataset and two auxiliary tasks, scene depth estimation and 3D orientation estimation, supervised by generated pseudo labels. Intermediate auxiliary features are integrated into the primary task network as implicit information. We propose a residual filter module for screening useful information that can enhance gaze-following prediction performance. Extensive experiments on GazeFollow and VideoAttentionTarget show that our approach achieves state-of-the-art results (0.120 Ave. Dist. achieved on GazeFollow and 0.104 L2 Dist. achieved on VideoAttentionTarget). Finally, we apply our approach to a real robot for understanding human attention and intention. Compared to the previous depth considered gaze-following method, our method saves half of the computation time.

ADAPTIVE MORPHOLOGICAL FRAMEWORK FOR 3D DIRECTIONAL FILTERING

Engineering materials often feature lower dimensional and directed structures such as cracks, fibers, or closedfacets in foams. The characterization of such structures in 3D is of particular interest in various applications inmaterials science. In image processing, knowledge of the local structure orientation can be used for structureenhancement, directional filtering, segmentation, or separation of interacting structures. The idea of usingbanks of directed structuring elements or filters parameterized by a discrete subset of the orientation space isproven to be effective for these tasks in 2D. However, this class of methods is prohibitive in 3D due to the highcomputational burden of filtering on a sufficiently fine discretization of the unit sphere.This paper introduces a method for 3D pixel-wise orientation estimation and directional filtering inspiredby the idea of adaptive refinement in discretized settings. Furthermore, an operator for distinction betweenisotropic and anisotropic structures is defined based on our method. This operator utilizes orientationinformation to successfully preserve structures with one or two dominant dimensions. Finally, feasibilityand effectiveness of the method are demonstrated on 3D micro-computed tomography images in three usecases: detection of a misaligned region in a fiber-reinforced material, crack detection in concrete, and facetdetection in partially closed foams.

A Two-LED Based Indoor Three-Dimensional Visible Light Positioning and Orienteering Scheme for a Tilted Receiver

Conventional visible light positioning (VLP) systems usually require at least three light-emitting diodes (LEDs) to enable trilateration or triangulation, which is infeasible when the LED condition is constrained. In this paper, we propose a novel indoor three-dimensional (3D) VLP and orienteering (VLPO) scheme. By using only two LEDs and two photo-detectors (PDs), our scheme can achieve simultaneous 3D localization and receiver orientation estimation efficiently. Further, to eliminate the location uncertainty caused by receiver tilt, we propose a location selection strategy which can effectively determine the true location of the receiver. Through extensive simulations, it is found that when the receiver faces upwards, the proposed scheme can achieve a mean 3D positioning error of 7.4 cm and a mean azimuthal error of 7.0°. Moreover, when the receiver tilts with a polar angle of 10°, accurate VLPO can still be achieved with 90.3% of 3D positioning errors less than 20 cm and 92.6% of azimuthal errors less than 5°. These results indicate that our scheme is a promising solution to achieve accurate VLPO when there is only two LEDs. Results also verify the effectiveness of the VLPO scheme when locating a tilted receiver.

Drift-Free 3D Orientation and Displacement Estimation for Quasi-Cyclical Movements Using One Inertial Measurement Unit: Application to Running.

A Drift-Free 3D Orientation and Displacement estimation method (DFOD) based on a single inertial measurement unit (IMU) is proposed and validated. Typically, body segment orientation and displacement methods rely on a constant- or zero-velocity point to correct for drift. Therefore, they are not easily applicable to more proximal segments than the foot. DFOD uses an alternative single sensor drift reduction strategy based on the quasi-cyclical nature of many human movements. DFOD assumes that the quasi-cyclical movement occurs in a quasi-2D plane and with an approximately constant cycle average velocity. DFOD is independent of a constant- or zero-velocity point, a biomechanical model, Kalman filtering or a magnetometer. DFOD reduces orientation drift by assuming a cyclical movement, and by defining a functional coordinate system with two functional axes. These axes are based on the mean acceleration and rotation axes over multiple complete gait cycles. Using this drift-free orientation estimate, the displacement of the sensor is computed by again assuming a cyclical movement. Drift in displacement is reduced by subtracting the mean value over five gait cycle from the free acceleration, velocity, and displacement. Estimated 3D sensor orientation and displacement for an IMU on the lower leg were validated with an optical motion capture system (OMCS) in four runners during constant velocity treadmill running. Root mean square errors for sensor orientation differences between DFOD and OMCS were 3.1 ± 0.4° (sagittal plane), 5.3 ± 1.1° (frontal plane), and 5.0 ± 2.1° (transversal plane). Sensor displacement differences had a root mean square error of 1.6 ± 0.2 cm (forward axis), 1.7 ± 0.6 cm (mediolateral axis), and 1.6 ± 0.2 cm (vertical axis). Hence, DFOD is a promising 3D drift-free orientation and displacement estimation method based on a single IMU in quasi-cyclical movements with many advantages over current methods.

Insect 3-D Orientation Estimation Based on Cooperative Observation From Two Views of Entomological Radars

The ability of insect orientation measurement of entomological radars supported the study of insect heading behaviors. However, the current entomological radars can only measure the two-dimension (2D) orientation that is the projection of the three-dimension (3D) orientation on the horizontal plane. The ability to measure the 3D orientation of insect will promote the study of the vertical and 3D heading behaviors of insect. In this study, an insect 3D orientation estimation method based on cooperative observation from two views of entomological radars is proposed. The expression of 3D orientation is deduced based on 2D orientation, azimuth and elevation measured with two radars at different stations through vector operation. The simulations are conducted to verify the effectiveness of the proposed method. The result shows that the estimation error of 3D orientation depends on that of the 2D orientation and the included angle between two lines of sight of two radars. A multi-aspect fully polarimetric rig is designed to measure insect echo signals from two aspects in a microwave anechoic chamber. The experiment data is used to further validate the effectiveness of the proposed method.

Six-Dimensional Localization of a Robotic Capsule Endoscope Using Magnetoquasistatic Field

This study presents a six-dimensional (6D) localization method and theoretical analysis for a robotic capsule endoscope (RCE). Herein, the challenge of implementing 6D localization for an RCE, with the simultaneous localization of 3D position and 3D orientation, is resolved using optimization with an induced magnetostatic field and RCE kinematics. The optimization is achieved using three transmitting coils (Txs) that are placed in a foundation bed to generate the magnetoquasistatic field and three orthogonal receiving coils (Rxs) that are embedded in the RCE, which is aligned with a capsule coordinate system (CCS), to detect the magnetoquasistatic field. For the 3D position estimation, a nonlinear net magnetic field model is linearized, and a polynomial equation optimization is formulated. The local existence and uniqueness of the solution in the region of interest (ROI) are proved based on the model and simulation. For the 3D orientation estimation, a rotation matrix describing the RCE orientation is computed using the measured magnetic field at the three orthogonal Rxs, and the orthogonality of the rotation matrix is enhanced by using the polar coordinate decomposition. Both simulations and experiments verify the suitability of the proposed method. The maximum error for position and orientation are 1.68+/−0.76 mm and 1.74+/−1.06°, respectively, under a 5 Hz sampling rate in an applicable spherical ROI with a diameter of 10 cm.

Gyroscope-Based Video Stabilization for Electro-Optical Long-Range Surveillance Systems.

Video stabilization is essential for long-range electro-optical systems, especially in situations when the field of view is narrow, since the system shake may produce highly deteriorating effects. It is important that the stabilization works for different camera types, i.e., different parts of the electromagnetic spectrum independently of the weather conditions and any form of image distortion. In this paper, we propose a method for real-time video stabilization that uses only gyroscope measurements, analyze its performance, and implement and validate it on a real-world professional electro-optical system developed at Vlatacom Institute. Camera movements are modeled with 3D rotations obtained by integration of MEMS gyroscope measurements. The 3D orientation estimation quality depends on the gyroscope characteristics; we provide a detailed discussion on the criteria for gyroscope selection in terms of the sensitivity, measurement noise, and drift stability. Furthermore, we propose a method for improving the unwanted motion estimation quality using interpolation in the quaternion domain. We also propose practical solutions for eliminating disturbances originating from gyro bias instability and noise. In order to evaluate the quality of our solution, we compared the performance of our implementation with two feature-based digital stabilization methods. The general advantage of the proposed methods is its drastically lower computational complexity; hence, it can be implemented for a low price independent of the used electro-optical sensor system.

A Complementary Filter Design on SE(3) to IdentifyMicro-Motions during 3D Motion Tracking.

In 3D motion capture, multiple methods have been developed in order to optimize the quality of the captured data. While certain technologies, such as inertial measurement units (IMU), are mostly suitable for 3D orientation estimation at relatively high frequencies, other technologies, such as marker-based motion capture, are more suitable for 3D position estimations at a lower frequency range. In this work, we introduce a complementary filter that complements 3D motion capture data with high-frequency acceleration signals from an IMU. While the local optimization reduces the error of the motion tracking, the additional accelerations can help to detect micro-motions that are useful when dealing with high-frequency human motions or robotic applications. The combination of high-frequency accelerometers improves the accuracy of the data and helps to overcome limitations in motion capture when micro-motions are not traceable with 3D motion tracking system. In our experimental evaluation, we demonstrate the improvements of the motion capture results during translational, rotational, and combined movements.

GPU-accelerated ray-casting for 3D fiber orientation analysis.

Orientation analysis of fibers is widely applied in the fields of medical, material and life sciences. The orientation information allows predicting properties and behavior of materials to validate and guide a fabrication process of materials with controlled fiber orientation. Meanwhile, development of detector systems for high-resolution non-invasive 3D imaging techniques led to a significant increase in the amount of generated data per a sample up to dozens of gigabytes. Though plenty of 3D orientation estimation algorithms were developed in recent years, neither of them can process large datasets in a reasonable amount of time. This fact complicates the further analysis and makes impossible fast feedback to adjust fabrication parameters. In this work, we present a new method for quantifying the 3D orientation of fibers. The GPU implementation of the proposed method surpasses another popular method for 3D orientation analysis regarding accuracy and speed. The validation of both methods was performed on a synthetic dataset with varying parameters of fibers. Moreover, the proposed method was applied to perform orientation analysis of scaffolds with different fibrous micro-architecture studied with the synchrotron μCT imaging setup. Each acquired dataset of size 600x600x450 voxels was analyzed in less 2 minutes using standard PC equipped with a single GPU.

A multi-task Faster R-CNN method for 3D vehicle detection based on a single image

Vehicle detection is an important part of robot environmental perception. In this paper, a 3D vehicle detection method using a single image is proposed to generate the 3D space coordinate information of the object using monocular vision for autonomous driving. The proposed method works under the multi-task framework and integrates 2D object detection, 3D object detection, orientation estimation and key point detection into one unified deep convolution neural network (DCNN) which could be trained by end-to-end learning. Besides, our proposed method is built by modifying Fast R-CNN using multi-task learning, and thus our proposed method is named multi-task Faster R-CNN (MT-Faster R-CNN). The experiments on KITTI dataset are conducted to evaluate our proposed method and the other 3D vehicle detection methods. The experimental results demonstrate that our proposed method is competitive and could significantly assist autonomous driving.

Augmented Autoencoders: Implicit 3D Orientation Learning for 6D Object Detection

We propose a real-time RGB-based pipeline for object detection and 6D pose estimation. Our novel 3D orientation estimation is based on a variant of the Denoising Autoencoder that is trained on simulated views of a 3D model using Domain Randomization. This so-called Augmented Autoencoder has several advantages over existing methods: It does not require real, pose-annotated training data, generalizes to various test sensors and inherently handles object and view symmetries. Instead of learning an explicit mapping from input images to object poses, it provides an implicit representation of object orientations defined by samples in a latent space. Our pipeline achieves state-of-the-art performance on the T-LESS dataset both in the RGB and RGB-D domain. We also evaluate on the LineMOD dataset where we can compete with other synthetically trained approaches. We further increase performance by correcting 3D orientation estimates to account for perspective errors when the object deviates from the image center and show extended results. Our code is available here https://github.com/DLR-RM/AugmentedAutoencoder.

자기교란에 의한 방위각 추정 오차 최소화를 위해 기구학적 구속조건을 투영한 칼만 필터

With regard to 3D orientation estimation based on IMMU (Inertial Magnetic Measurement Unit) signals, the yaw estimation accuracy may be significantly degraded as a result of magnetic distortions. Consequently, several yaw estimation Kalman filters (KFs) possessing distortion compensation mechanisms have been proposed. However, majority of the conventional methods fail to effectively curb inaccuracies due to distortion when magnetic fields are extremely distorted. In this paper, we propose a new KF projecting a kinematic constraint to minimize yaw estimation errors induced by magnetic distortions. After the measurement update using magnetometer signals, the proposed method additionally corrects the yaw estimation through projection of a kinematic constraint on a conventional unconstrained KF. Experimental results show that the proposed KF outperformed the conventional KF by approximately 52-67%.

Fusion of inertial and magnetic sensors for 3D position and orientation estimation.

This paper presents a new method for 3D position and orientation estimation based on the magnetic field generated by a single coil and the fusion of low power magnetometers and inertial sensors. The fixed spatial relationship between the sensors is exploited to solve the ambiguity problem that occurs when only a single coil is employed. In this context a new positioning algorithm is developed and verified by simulations. Using a single coil instead of three coils as common in conventional magnetic field based systems brings a benefit both regarding power consumption and construction size. Moreover, as the proposed system requires no line of sight between source and sensors, it is ideally suited for portable, location-independent applications such as the mobile assessment of a persons' head to torso posture and movement e.g. for the purpose of examining causes for chronic pain in the cervical spine area under conditions of daily life.

English

English

Comparative assessment of the effects of different error sources on the estimate of 3D orientation using magnetic-inertial measurement units

Main topics: Technical developments in movement science; Mathematical simulation in human mov. science Introduction and aim: The determination of the accurate 3D orientation of a body segment is a crucial aspect in clinical movement analysis [1]. Among motion capture technologies, wearable sensors, suchasmagnetic-inertialmeasurementunits (MIMUs), are gaining popularity. Several methods were developed to estimate 3D orientation from MIMU data, in order to effectively exploit the complementary properties of inertial (gyroscopes and accelerometers) and magnetic sensors embedded in the MIMUs [2]. It is well known that these sensors are affected by several error sources, including measurement noise, bias instability, calibration errors, ferromagneticdisturbances, external accelerations [3]. Thepurpose of the present study is to assess the contribution of the sources of error associated to each sensor to the uncertainty affecting the 3D orientation estimation, during a consolidated clinical test. Patients/materials andmethods:ATimedUpandGo test (TUG) was performed by ten healthy subjects (5M, 5 F, 28 5 y). OneMIMU (Opal, APDM Inc.) was secured to the lower back of each subject and angular velocity, acceleration and local magnetic field vector data were collected. The trajectories of four markers attached to theMIMUwere trackedby a stereophotogrammetric system (Vicon MX3), and the orientation of a marker-based frame was obtained andconsideredas reference. Thesemarker trajectorieswereused to calculate the same variablesmeasured by theMIMU.Measured and simulated MIMU data were then combined, as detailed in Table 1, obtaining five different scenarios. Each signal combination was fed as an input of an Extended Kalman Filter (EKF) [2] to estimate the 3D MIMU orientation. The rationale was to assess, both individually and jointly, the effect of the errors associated to the EKFmodel, as well as to each MIMU sensor. The accuracy of the obtained orientations was then evaluated in terms of orientation error [2]. A Kruskal–Wallis testwithpost-hocDunncomparisons ( =0.05)was performed to investigate whether significance differences existed among the obtained orientation errors. Results: Results (mean one standard deviation) about the error contribution associated to the EKF model and to the MIMU sensors