Target In 3D Space Research Articles

Localization and circumnavigation of an unknown target using distance measurements in 3D space

This paper investigates the problem of localization and circumnavigation of an unknown stationary target in 3D space using distance measurements. An estimator is first designed in 2D space to locate the target, with which a control protocol is proposed to force the agent to travel along a circular trajectory around the target. Then the algorithm is extended to 3D space to circumnavigate the target under the condition of obtaining gravity direction additionally. In particular, the agent moves alternately in the X-Y horizontal planes and the X-Z vertical planes in different patterns under different switching conditions, which is beneficial to monitor the target from a broader perspective. Meanwhile, the rigorous stability analyses of two cases are given in terms of the Lyapunov method. Finally, the effectiveness of the proposed algorithms is verified through numerical simulations and a real experiment.

Robust control of a dual-arm space robot to capture a non-cooperative target in 3D space

Non-cooperative objects in space, like defunct spinning satellites, are potentially dangerous to subsequent space missions. Thus, this paper proposes a new capture strategy by a dual-arm space robot to improve the efficiency of the capture operation. Compared with single-arm space robots, dual-arm space robots can carry out more complex missions, like capturing a spinning target in three-dimensional (3D) space, with a higher probability of success. Considering the possible model uncertainties of a space robot and a non-cooperative target, a sliding mode controller (SMC) with good robustness against the model uncertainties was developed. However, due to frequent switches in the controller, the SMC usually has chattering effects causing uncontrolled vibrations of the spacecraft structure. For comparison, a smoothed quasi-continuous second-order sliding mode controller (SQC2S) can provide the same high accuracy level as the SMC and remove the chattering effects caused by the SMC. In addition, the SQC2S can deliver better robust performance against the model uncertainties than the SMC. Finally, the proposed capture strategy is validated and compared in the numerical simulations by the SMC and the SQC2S.

Sensor Fusion with Asynchronous Decentralized Processing for 3D Target Tracking with a Wireless Camera Network.

We present a method to acquire 3D position measurements for decentralized target tracking with an asynchronous camera network. Cameras with known poses have fields of view with overlapping projections on the ground and 3D volumes above a reference ground plane. The purpose is to track targets in 3D space without constraining motion to a reference ground plane. Cameras exchange line-of-sight vectors and respective time tags asynchronously. From stereoscopy, we obtain the fused 3D measurement at the local frame capture instant. We use local decentralized Kalman information filtering and particle filtering for target state estimation to test our approach with only local estimation. Monte Carlo simulation includes communication losses due to frame processing delays. We measure performance with the average root mean square error of 3D position estimates projected on the image planes of the cameras. We then compare only local estimation to exchanging additional asynchronous communications using the Batch Asynchronous Filter and the Sequential Asynchronous Particle Filter for further fusion of information pairs' estimates and fused 3D position measurements, respectively. Similar performance occurs in spite of the additional communication load relative to our local estimation approach, which exchanges just line-of-sight vectors.

Joint Localization and Environment Sensing of Rigid Body With 5G Millimeter Wave MIMO

Accurately localizing a target in three-dimensional (3D) space assisted with the fifth generation (5G) wireless systems in an indoor environment could enable a wide variety of new applications, including precise control for factory automation, self-maneuver of the vehicle and so on. However, controlling a target in 3D space relies on a rigid body modelling with six degrees of freedom, which dramatically increases the localization difficulty and complexity. Furthermore, for radio-based localization methods, the lack of line-of-sight (LOS) and the existence of reflection points in the environment will also influence the rigid body localization process. To improve the rigid body localization accuracy as well as unravel useful environmental information from the received signal, a novel rigid body joint active localization and environment sensing scheme is proposed in this paper. Specifically, the multi-path effect of millimeter wave (mm-wave) signal with a single reflection can be exploited to enhance the rigid body localization accuracy, and it can also be utilized to locate the reflection points, which further enables a new way for environmental sensing. Hence, we first propose a two-step hierarchical compressive sensing algorithm to extract the angular and distance information of the LOS (if available) and single-bounce specular reflections. Then a particle swarm optimization (PSO) based method is derived to recover the posture of the rigid body and the location of reflection points. The Cramér-Rao lower bound (CRLB) on angle, rigid body posture and reflection points position uncertainty is also analyzed. The simulation results demonstrate that the proposed scheme can achieve high accuracy rigid body localization and locate the reflection points around the rigid body even under obstructed-line-of-sight (OLOS) conditions in an indoor scene.

Visual orienting to beyond field of view targets in 3D space: Effects of cue modality, eccentricity, and distractor presence

Visual orienting to beyond field of view targets in 3D space: Effects of cue modality, eccentricity, and distractor presence

A Novel Method for Space Circular Target Detection in Machine Vision.

Computer-vision-based space circular target detection has a wide range of applications in visual measurement, object detection, and other fields. The space circular target is projected into an ellipse in the camera for localization. Traditional methods based on monocular vision use a precise calculation model to calculate the center coordinate and normal vector of the space circular target according to the image’s elliptic parameters. However, this accurate calculation method has the disadvantage of poor anti-interference ability in practical application. Aiming at the shortcomings of the above traditional calculation method, this paper proposes an optimization method for fitting the circular target in 3D space, where the image ellipse is projected back into 3D space and then detects the center coordinate and normal vector of the space circular target. Unlike the traditional method, this approach is not sensitive to the image’s elliptic parameters; it has stronger noise resistance performance and notable application value. The feasibility and effectiveness of the proposed method were verified by both simulation and practical experimental results.

Integration of the 3D Environment for UAV Onboard Visual Object Tracking

Single visual object tracking from an unmanned aerial vehicle (UAV) poses fundamental challenges such as object occlusion, small-scale objects, background clutter, and abrupt camera motion. To tackle these difficulties, we propose to integrate the 3D structure of the observed scene into a detection-by-tracking algorithm. We introduce a pipeline that combines a model-free visual object tracker, a sparse 3D reconstruction, and a state estimator. The 3D reconstruction of the scene is computed with an image-based Structure-from-Motion (SfM) component that enables us to leverage a state estimator in the corresponding 3D scene during tracking. By representing the position of the target in 3D space rather than in image space, we stabilize the tracking during ego-motion and improve the handling of occlusions, background clutter, and small-scale objects. We evaluated our approach on prototypical image sequences, captured from a UAV with low-altitude oblique views. For this purpose, we adapted an existing dataset for visual object tracking and reconstructed the observed scene in 3D. The experimental results demonstrate that the proposed approach outperforms methods using plain visual cues as well as approaches leveraging image-space-based state estimations. We believe that our approach can be beneficial for trafficmonitoring, video surveillance, and navigation.

Detection of peripheral visual targets in 3D space is not affected by plane of fixation in a simulated driving context

Detection of peripheral visual targets in 3D space is not affected by plane of fixation in a simulated driving context

Experimental Analysis of a Spatialised Audio Interface for People with Visual Impairments

Sound perception is a fundamental skill for many people with severe sight impairments. The research presented in this article is part of an ongoing project with the aim to create a mobile guidance aid to help people with vision impairments find objects within an unknown indoor environment. This system requires an effective non-visual interface and uses bone-conduction headphones to transmit audio instructions to the user. It has been implemented and tested with spatialised audio cues, which convey the direction of a predefined target in 3D space. We present an in-depth evaluation of the audio interface with several experiments that involve a large number of participants, both blindfolded and with actual visual impairments, and analyse the pros and cons of our design choices. In addition to producing results comparable to the state-of-the-art, we found that Fitts’s Law (a predictive model for human movement) provides a suitable metric that can be used to improve and refine the quality of the audio interface in future mobile navigation aids.

Decoding of standard and non-standard visuomotor associations from parietal cortex

Objective. Neural signals can be decoded and used to move neural prostheses with the purpose of restoring motor function in patients with mobility impairments. Such patients typically have intact eye movement control and visual function, suggesting that cortical visuospatial signals could be used to guide external devices. Neurons in parietal cortex mediate sensory-motor transformations, encode the spatial coordinates for reaching goals, hand position and movements, and other spatial variables. We studied how spatial information is represented at the population level, and the possibility to decode not only the position of visual targets and the plans to reach them, but also conditional, non-spatial motor responses. Approach. The animals first fixated one of nine targets in 3D space and then, after the target changed color, either reached toward it, or performed a non-spatial motor response (lift hand from a button). Spiking activity of parietal neurons was recorded in monkeys during two tasks. We then decoded different task related parameters. Main results. We first show that a maximum-likelihood estimation (MLE) algorithm trained separately in each task transformed neural activity into accurate metric predictions of target location. Furthermore, by combining MLE with a Naïve Bayes classifier, we decoded the monkey’s motor intention (reach or hand lift) and the different phases of the tasks. These results show that, although V6A encodes the spatial location of a target during a delay period, the signals they carry are updated around the movement execution in an intention/motor specific way. Significance. These findings show the presence of multiple levels of information in parietal cortex that could be decoded and used in brain machine interfaces to control both goal-directed movements and more cognitive visuomotor associations.

Robotics-assisted visual-motor training influences arm position sense in three-dimensional space

BackgroundPerforming activities of daily living depends, among other factors, on awareness of the position and movements of limbs. Neural injuries, such as stroke, might negatively affect such an awareness and, consequently, lead to degrading the quality of life and lengthening the motor recovery process. With the goal of improving the sense of hand position in three-dimensional (3D) space, we investigate the effects of integrating a pertinent training component within a robotic reaching task.MethodsIn the proof-of-concept study presented in this paper, 12 healthy participants, during a single session, used their dominant hand to attempt reaching without vision to two targets in 3D space, which were placed at locations that resembled the functional task of self-feeding. After each attempt, participants received visual and haptic feedback about their hand’s position to accurately locate the target. Performance was evaluated at the beginning and end of each session during an assessment in which participants reached without visual nor haptic feedback to three targets: the same two targets employed during the training phase and an additional one to evaluate the generalization of training.ResultsCollected data showed a statistically significant [39.81% (p=0.001)] reduction of end-position reaching error when results of reaching to all targets were combined. End-position error to the generalization target, although not statistically significant, was reduced by 15.47%.ConclusionsThese results provide support for the effectiveness of combining an arm position sense training component with functional motor tasks, which could be implemented in the design of future robot-assisted rehabilitation paradigms to potentially expedite the recovery process of individuals with neurological injuries.

Coordinated control of a dual-arm space robot to approach and synchronise with the motion of a spinning target in 3D space

The non-cooperative objects in space such as space debris and spinning defunct satellites have attracted significant attention because of the rapid development of space exploration. To approach and capture the spinning targets in 3D space, the dual-arm space robot could be more efficient and safer than single-arm space robot to carry out the complicated tasks for spinning targets. The paper extends the model of a dual-arm space robot with Reaction Wheels into 3D space, which is more realistic in practice. The desired trajectories for end-effectors of the space robot are designed to securely approach and prepare to capture the spinning object through synchronising with the grasp points, while at the same time the desired attitude of the base could be maintained. Then coordinated control of the base and end-effectors could be carried out by Nonlinear Model Predictive Controller (NMPC) with the consideration of the uncertainties of the space robot system. NMPC has performed with a high accuracy without the uncertainties of the space robot system. Considering the imperfect knowledge of the mass and inertia as the uncertainties, NMPC could hold the accuracy of tracking error at the same level of accuracy in the presence of the uncertainties.

Scene Target 3D Point Cloud Reconstruction Technology Combining Monocular Focus Stack and Deep Learning

In order to obtain the depth information of the target in the scene and realize three-dimensional (3D) reconstruction, in this paper, a target reconstruction method combining monocular focus stack image and deep neural network is proposed. This method makes full use of the advantages of light field imaging technology and can generate the all focus image. The method first collects multiple frames of continuous images at different focal lengths of the scene, using a divide and conquer algorithm strategy, uplink uses YOLO neural network to identify the target in 3D space and track the position information; the downlink reconstructs the four-dimensional (4D) light field data based on the focus stack image frequency domain back projection, and then uses light field imaging technology to invert the scene parallax; subsequently, achieve scene depth estimation and reconstruction of all focus image; finally, the uplink and downlink are merged to realize the reconstruction of the 3D point cloud of the space target. Experimental results on real scenes show the effectiveness of the proposed algorithm.

Robot-Aided Upper-limb Proprioceptive Training in Three-Dimensional Space.

Proprioception, the ability to sense body position and limb movements in space without visual feedback, is one of the key factors in controlling body movements and performing activities of daily living. However, this capability might be affected after neural injuries such as stroke. Robotic platforms can be used to monitor and promote arm movements and, therefore, can assist in developing rehabilitation protocols that aim to improve proprioception through repetitive reaching motions without vision. The objective of this paper is to investigate if a robotic training protocol improves the end-position reaching proprioceptive sense in three-dimensional (3D) space. As an initial step towards clinical application, a robotic platform was employed to train the end-position proprioceptive sense in six healthy participants. During the training phase, volunteers used their dominant hand to reach without vision to two different targets in 3D space. Positions of these targets were carefully chosen to create a hand movement pattern similar to that used when self-feeding, which is an important activity of daily living. At the end of each training trial, participants were provided with visual feedback to help them move their hands to the exact locations confirmed through haptic feedback. Their performance was evaluated before and after the training in an assessment phase during which participants were asked to move from the start position to the same two targets as well as an additional third one without any visual or haptic feedback. The results from this study show significant improvements in overall reaching accuracy and trajectory smoothness, demonstrated by 41% decrease in the average end-position error and 13% reduction in the average index of curvature after the training. This research suggests the potential of designing robotic rehabilitation protocols for improving 3D proprioception.

A Method to Track Targets in Three-Dimensional Space Using an Imaging Sonar.

This paper introduces a methodology applying an imaging sonar for three-dimensional (3D) target tracking underwater. The key process in this work involves obtaining the target’s position in space using two images of the same scene, acquired by an adaptive resolution imaging sonar (ARIS) at different positions. A data association algorithm was designed to connect the same target in image sequences. The goal of this work was to track multiple targets in 3D space. The ARIS provides sequences of bi-dimensional images from the backscattered energy according to the range and azimuth. The challenge involved determining the missing elevation information for the observed object within the sonar detection range. By computing the geometrical transformation between the acquisition planar images and the cubical space, using only the sonar information that included the posture and moving speed of the ARIS, the target’s elevation information was obtained. To evaluate the performance of the proposed method, an indoor experiment was conducted using the ARIS. On the basis of the experimental results, we confirmed that the proposed method effectively obtained the target’s position in 3D space. A moving target simulation was also conducted, and the results showed that this method was effective for moving targets. Finally, a field experiment was performed to obtain the vertical distribution and track the 3D trajectories of fish.

Comparison of 2D and 3D modeled tumor motion estimation/prediction for dynamic tumor tracking during arc radiotherapy

Many real-time imaging techniques have been developed to localize a target in 3D space or in a 2D beam’s eye view (BEV) plane for intrafraction motion tracking in radiation therapy. With tracking system latency, the 3D-modeled method is expected to be more accurate even in terms of 2D BEV tracking error. No quantitative analysis, however, has been reported. In this study, we simulated co-planar arc deliveries using respiratory motion data acquired from 42 patients to quantitatively compare the accuracy between 2D BEV and 3D-modeled tracking in arc therapy and to determine whether 3D information is needed for motion tracking.We used our previously developed low kV dose adaptive MV–kV imaging and motion compensation framework as a representative of 3D-modeled methods. It optimizes the balance between additional kV imaging dose and 3D tracking accuracy and solves the MLC blockage issue.With simulated Gaussian marker detection errors (zero mean and 0.39 mm standard deviation) and ~155/310/460 ms tracking system latencies, the mean percentage of time that the target moved >2 mm from the predicted 2D BEV position are 1.1%/4.0%/7.8% and 1.3%/5.8%/11.6% for the 3D-modeled and 2D-only tracking, respectively. The corresponding average BEV RMS errors are 0.67/0.90/1.13 mm and 0.79/1.10/1.37 mm. Compared to the 2D method, the 3D method reduced the average RMS unresolved motion along the beam direction from ~3 mm to ~1 mm, resulting in on average only <1% dosimetric advantage in the depth direction. Only for a small fraction of the patients, when tracking latency is long, the 3D-modeled method showed significant improvement of BEV tracking accuracy, indicating potential dosimetric advantage. However, if the tracking latency is short (~150 ms or less), those improvements are limited. Therefore, 2D BEV tracking has sufficient targeting accuracy for most clinical cases. The 3D technique is, however, still important in solving the MLC blockage problem during 2D BEV tracking.

A numerical approach for tracking unknown number of individual targets in videos

Suppose that we wish to get a comprehensive match of a target in the next frame. Where would we search the target in the next frame? Brute-force search has an asymptotic runtime of O(n!) with problem size n. Yet we can search the target only from a number of automatically generated specific regions, named candidate regions, in the next frame. But how can we get those regions? Deeming the silhouettes of movers, this paper denotes a detailed deliberation of how to generate those candidate regions automatically and then how to track unknown number of individual targets with them. Phase-correlation technique aids to find the key befitting matches of the targets using them. Hungarian method in combination with a state estimation process called Kalman filter finds the best correspondence of the targets among those matches, allowing us to construct full trajectories of unknown number of individual targets in 3D space irresistibly swift as compared to brute-force search since the relative runtime reduced from O(n!) to O(n3). Favorable outcomes, upon conducting experiments on videos from three different datasets, show the robustness and effectiveness of our approach.

Block design enhances classification of 3D reach targets from electroencephalographic signals

To date, decoding accuracy of actual or imagined pointing movements to targets in 3D space from electroencephalographic (EEG) signals has remained modest. The reason may pertain to the fact that these movements activate essentially the same neural networks. In this study, we aimed at testing whether repetitive pointing movements to each of the targets promotes the development of segregated neural patterns, resulting in enhanced decoding accuracy. Six human subjects generated slow or fast repetitive pointing movements with their right dominant arm to one of five targets distributed in 3D space, followed by repetitive imagery of movements to the same target or to a different target. Nine naive subjects generated both repetitive and non-repetitive slow actual movements to each of the five targets to test the effect of block design on decoding accuracy. In order to assure that base line drift and low frequency motion artifacts do not contaminate the data, the data were high-pass filtered in 4–30Hz, leaving out the delta and gamma band. For the repetitive trials, the model decoded target location with 81% accuracy, which is significantly higher than chance level. The average decoding rate of target location was only 30% for the non-repetitive trials, which is not significantly different than chance level. A subset of electrodes, mainly over the contralateral sensorimotor areas, was found to provide most of the discriminative features for all tested conditions. Time proximity between trained and tested blocks was found to enhance decoding accuracy of target location both by target non-specific and specific mechanisms. Our findings suggest that movement repetition promotes the development of distinct neural patterns, presumably by the formation of target-specific kinesthetic memory.

SIMPLE METHOD OF UCOOPERATIVE HUMAN BEINGS LOCALISATION IN 3D SPACE BY UWB RADAR

In this paper, we will present a method for through-the-wall localisation of targets in 3D space by ultra-wideband (UWB) radar. The targets are moving or static uncooperative human beings and the localisation is based on the time-of-arrival (TOA) measurement. The presented method is based on the use of a specific antenna array layout, which simplifies the problem of the 3D multilateration method to the solution of two 2D localisation problems. Through-the-wall as well as the line-of-sight measurements were done for both static and moving person scenarios and the results of the selected scenarios are described in this paper.

Efficiency of visual feedback integration differs between dominant and non-dominant arms during a reaching task.

Recent studies have shown that patterns of endpoint variability following double-step reach sequences reflect the influence of both planning and execution-related processes, but are strongly dominated by noise associated with the online updating of movement plans based on visual feedback. However, it is currently unclear whether these results reflect the dominant arm/hemisphere's postulated specialization for visual feedback processing, or whether these effects reflect a more general "arm/hemisphere independent" preference for visual feedback in the control of reaching. To explore this, twelve subjects performed double-step reach sequences with their dominant and non-dominant arms to targets in 3D space with and without visual feedback of the arm. Variability was quantified using the volumes, aspect ratios, and orientations of 95% confidence ellipsoids fit to the distributions of reach endpoints. In consonance with previous findings, the availability of visual feedback resulted in ellipsoids that were significantly smaller, had greater aspect ratios, and were more aligned with the depth axis than those performed without visual feedback. Moreover, the effects of vision on aspect ratio and orientation were similar in magnitude for the dominant and non-dominant arms, suggesting that noise associated with planning and execution-related processes is managed in a similar way by the sensorimotor systems of each arm. However, the degree to which vision decreased ellipsoid volume was found to be significantly greater for the dominant arm. This suggests that the feedback control system of the dominant arm uses visual information more efficiently to control reaches to visual targets.