3D Localization Method Research Articles

Smart dimensional quality assessment of embedded steel plates based on images and laser data fusion

Abstract Accurate and efficient positioning is critical to ensuring the dimensional quality assessment of embedded steel plates. However, traditional manual measurement methods struggle to efficiently measure and evaluate these plates. Vision-based measurement methods offer advantages such as high resolution, fast data acquisition, and processing speed, allowing accurate measurement of 2D coordinates. LiDAR can capture highly accurate point clouds, due to the unordered nature of point clouds, processing and analysis require significant computational resources. This paper proposes a method for smart 3D localization of embedded steel plates using image and laser data. (1) We introduce an improved Rectangular Diagonal Constraint Harris (RDC-Harris) corner detection method and achieve subpixel 2D corner detection of embedded plates based on deep learning；(2) Given a calibrated camera-LiDAR, we develop a smart detection algorithm guided by 2D image bounding boxes, achieving 3D corner localization. In indoor testing and engineering applications, this method effectively ensures the dimensional quality of embedded steel plates. Compared to traditional manual inspection, the measurement efficiency reaches 10 minutes per station, with an accuracy of 2.12 mm.

3D Localization and Tracking Methods for Multiplatform Radar Networks

3D Localization and Tracking Methods for Multiplatform Radar Networks

3D magnetic seed localization for augmented reality in surgery

PurposeFor tumor resection, surgeons need to localize the tumor. For this purpose, a magnetic seed can be inserted into the tumor by a radiologist and, during surgery, a magnetic detection probe informs the distance to the seed for localization. In this case, the surgeon still needs to mentally reconstruct the position of the tumor from the probe’s information. The purpose of this study is to develop and assess a method for 3D localization and visualization of the seed, facilitating the localization of the tumor.MethodsWe propose a method for 3D localization of the magnetic seed by extending the magnetic detection probe with a tracking-based localization. We attach a position sensor (QR-code or optical marker) to the probe in order to track its 3D pose (respectively, using a head-mounted display with a camera or optical tracker). Following an acquisition protocol, the 3D probe tip and seed position are subsequently obtained by solving a system of equations based on the distances and the 3D probe poses.ResultsThe method was evaluated with an optical tracking system. An experimental setup using QR-code tracking (resp. using an optical marker) achieves an average of 1.6 mm (resp. 0.8 mm) 3D distance between the localized seed and the ground truth. Using a breast phantom setup, the average 3D distance is 4.7 mm with a QR-code and 2.1 mm with an optical marker.ConclusionTracking the magnetic detection probe allows 3D localization of a magnetic seed, which opens doors for augmented reality target visualization during surgery. Such an approach should enhance the perception of the localized region of interest during the intervention, especially for breast tumor resection where magnetic seeds can already be used in the protocol.

Vision-Based Object Localization and Classification for Electric Vehicle Driving Assistance

The continuous advances in intelligent systems and cutting-edge technology have greatly influenced the development of intelligent vehicles. Recently, integrating multiple sensors in cars has improved and spread the advanced drive-assistance systems (ADAS) solutions for achieving the goal of total autonomy. Despite current self-driving approaches and systems, autonomous driving is still an open research issue that must guarantee the safety and reliability of drivers. This work employs images from two cameras and Global Positioning System (GPS) data to propose a 3D vision-based object localization and classification method for assisting a car during driving. The experimental platform is a prototype of a two-sitter electric vehicle designed and assembled for navigating the campus under controlled mobility conditions. Simultaneously, color and depth images from the primary camera are combined to extract 2D features, which are reprojected into 3D space. Road detection and depth features isolate point clouds representing the objects to construct the occupancy map of the environment. A convolutional neural network was trained to classify typical urban objects in the color images. Experimental tests validate car and object pose in the occupancy map for different scenarios, reinforcing the car position visually estimated with GPS measurements.

Surgical navigation for guidewire placement from intraoperative fluoroscopy in orthopaedic surgery.

Objective. Surgical guidewires are commonly used in placing fixation implants to stabilize fractures. Accurate positioning of these instruments is challenged by difficulties in 3D reckoning from 2D fluoroscopy. This work aims to enhance the accuracy and reduce exposure times by providing 3D navigation for guidewire placement from as little as two fluoroscopic images.Approach. Our approach combines machine learning-based segmentation with the geometric model of the imager to determine the 3D poses of guidewires. Instrument tips are encoded as individual keypoints, and the segmentation masks are processed to estimate the trajectory. Correspondence between detections in multiple views is established using the pre-calibrated system geometry, and the corresponding features are backprojected to obtain the 3D pose. Guidewire 3D directions were computed using both an analytical and an optimization-based method. The complete approach was evaluated in cadaveric specimens with respect to potential confounding effects from the imaging geometry and radiographic scene clutter due to other instruments.Main results. The detection network identified the guidewire tips within 2.2 mm and guidewire directions within 1.1°, in 2D detector coordinates. Feature correspondence rejected false detections, particularly in images with other instruments, to achieve 83% precision and 90% recall. Estimating the 3D direction via numerical optimization showed added robustness to guidewires aligned with the gantry rotation plane. Guidewire tips and directions were localized in 3D world coordinates with a median accuracy of 1.8 mm and 2.7°, respectively.Significance. The paper reports a new method for automatic 2D detection and 3D localization of guidewires from pairs of fluoroscopic images. Localized guidewires can be virtually overlaid on the patient's pre-operative 3D scan during the intervention. Accurate pose determination for multiple guidewires from two images offers to reduce radiation dose by minimizing the need for repeated imaging and provides quantitative feedback prior to implant placement.

3D Localization With a Single Partially-Connected Receiving RIS: Positioning Error Analysis and Algorithmic Design

In this paper, we introduce the concept of partially-connected receiving reconfigurable intelligent surfaces (R-RISs), which refers to metasurfaces designed to efficiently sense electromagnetic waveforms impinging on them, and perform localization of the users emitting them. The presented R-RIS hardware architecture comprises subarrays of meta-atoms, with each of them incorporating a waveguide assigned to direct the waveforms reaching its meta-atoms to a reception radio-frequency (RF) chain, enabling signal/channel parameter estimation. We particularly focus on the scenarios where the user is located in the far-field of all the R-RIS subarrays, and present a three-dimensional (3D) localization method which is based on narrowband signaling and angle of arrival (AoA) estimates of the impinging signals at each single-receive-RF R-RIS subarray. For the AoA estimation, which relies on spatially sampled versions of the received signals via each subarray's phase configuration of meta-atoms, we devise an off-grid atomic norm minimization approach, which is followed by subspace-based root multiple signal classification (MUSIC). The AoA estimates are finally combined via a least-squared line intersection method to obtain the position coordinates of a user emitting synchronized localization pilots. Our derived theoretical Cramér Rao lower bounds (CRLBs) on the estimation parameters, which are compared with extensive computer simulation results of our localization approach, verify the effectiveness of the proposed R-RIS-empowered 3D localization system, which is showcased to offer cm-level positioning accuracy. Our comprehensive performance evaluations also demonstrate the impact of various system parameters on the localization performance, namely the training overhead and the distance between the R-RIS and the user, as well as the spacing among the R-RIS's subarrays and its partitioning patterns.

Buried gas pipe leak localization using dual-wave spectrum estimation

To accurately locate leaks in buried gas pipes, a three-dimensional (3D) localization method using dual-wave velocity pairing and multi-array fusion localization was developed in this study to locate leak-induced vibroacoustic sources. Two velocities at which the P1 and S waves propagated along soil were estimated when the peaks of the dual-wave spectrum coincided with each other. Root-MUSIC algorithm was introduced to quickly determine the superimposed peak, thereby reducing the computational complexity. To address the issue of 3D localization using the P1 wave that propagated in soil according to a far-field model, least-squares (LS) estimation for the 3D coordinates was performed using the fusion localization method. A numerical model of geometrical dilution of precision (GDOP) was constructed to analyze the performances of multi-array layouts. A uniform circular array of eight accelerometers was utilized to visualize the leak-induced vibroacoustic source, and the imaging showed that the vibroacoustic source coincided with the leak. Accurate direction of arrival (DOA) estimation under multiple soil conditions was achieved, which proved that the velocity near the array was accurately estimated. An equilateral-triangle multi-array layout was selected using GDOP analyzing, and the results of localization experiments revealed that the equilateral-triangle layouts with 1.5-m and 2-m spacings achieved relatively good accuracy. Compared to the 3D localization method using a single array, the fusion localization method showed better accuracy for the remote leak, and the maximum reduction in error is 84.75 %. GDOP of fusion localization was found less than 0.1 m when the internal pipe pressure ≥ 0.4 MPa and the signal-to-noise ratio (SNR) ≥ 0 dB under the conditions of experiments in this study.

An Analytical Method for 3-D Target Localization Based on a Four-Element Ultrasonic Sensor Array With TOA Measurement

A new analytical method for 3D target localization based on a four-element ultrasonic sensor array is proposed. In the sensor array, with the consideration of good real-time performance, one ultrasonic sensor (the reference ultrasonic sensor) works both as a transmitter and as a receiver, while the other three sensors work as receivers. A basic target localization model is developed. The necessary and sufficient condition of the proposed method to obtain the unique and analytical solution is analyzed. It is proved that if and only if the four ultrasonic sensors are not in the same plane, the four-element ultrasonic sensor array can obtain a unique and analytical solution. In addition, with the consideration of the stability and the complexity of the analytical solution, a practical four-element ultrasonic sensor array configuration is proposed, which is characterized in that the reference ultrasonic sensor is at the origin, the other three ultrasonic sensors are on the three coordinate axes respectively and the distances from the three ultrasonic sensors to the reference ultrasonic sensor are equal. 3D target localization experiments are carried out. The experimental results verify the effectiveness of the proposed method and the localization performance is satisfactory. The proposed sensor array can realize the 3D localization with the advantages of simple construction and less computation.

Experimental study on 3D source localization in indoor environments with weak airflow based on two bionic swarm intelligence algorithms

Existing studies on using robots to locate indoor pollutant sources mainly focused on ventilated indoor environments, and rarely addressed unventilated or poorly ventilated indoor environments. This study aims to locate pollutant sources with unknown heights in real-world indoor environments with weak airflow. For this purpose, a 3D source localization system was first developed, which consists of three robots equipped with gas sensors that can be controlled to move up and down in the height direction. Next, two 3D source localization methods based on bionic swarm intelligence algorithms, namely the 3D whale optimization algorithm (WOA_3D) method and the 3D particle swarm optimization (PSO_3D) method, were validated in an unventilated indoor space with sources at different heights. The experimental results show that both the WOA_3D method and PSO_3D methods were highly adaptable to the change in source height and performed well in terms of success rate and localization efficiency. The comparison between the WOA_3D method and the PSO_3D method shows that the former method has better comprehensive performance. The WOA_3D method was also compared with its 2D simplified version, namely, the WOA_2D method, which further demonstrated the necessity of using the 3D source localization method when the source height is unknown.

Tag Localization by Handheld UHF RFID Reader With Optical and RFID Landmarks

In this paper a prototype method for manual inventorying and real-time 3D localization of RFID tagged products by a low-cost handheld UHF RFID reader is introduced. The device is equipped with a UHF RFID reader and an optical camera. "ArUco" markers with RFID tags are placed at known positions. Those optical and RF landmarks are used for the estimation of the pose of the handheld reader over time, by fusing the corresponding measurements in a Kalman filter. The reader also measures RFID tagged items at unknown locations. The positions of the tags are estimated by unwrapping the collected phase measurements, combined with the estimated antenna trajectory. Experimental results show that increased accuracy can be achieved by increasing the spatial density of the optical and RFID markers. The proposed method is expected to enhance low-cost manual inventorying, delivering 3D localization of the tagged products.

Feasibility of simultaneous multichannel magnetocardiographic mapping, multiple monophasic action potential recording and laser photocoagulation of isolated and perfused rabbit ventricular myocardium

Abstract Background Multichannel magnetocardiographic mapping (MCG) has been validated as an accurate non-invasive method for 3D localization of arrhythmogenic substrates suitable for ablation treatment and of intracardiac electrophysiology catheters (1,2). High resolution (HR) MCG-guided multiple simultaneous monophasic action potential (multi-MAP) recording and pacing with a single patented amagnetic catheter (AC) detects local arrhythmogenic mechanisms (3). Laser photocoagulation (LP) providing more controlled and homogeneous myocardial lesions (4), may be an alternative to radiofrequency (RF) ablation, without unwanted electromagnetic noise effect during real-time MCG. Purpose This study aimed to experimentally test the feasibility of simultaneous multi-MAP recording and LP guided by real-time MCG. Methods An unshielded 36 channels SQUID system (sensitivity of 30 fT/Hz1/2, at 1 Hz) (CMI, USA) was used to study Langendorff-perfused isolated rabbit heart with MCG (Figure 1A). Four epicardial MAPs were simultaneously recorded (inter MAP distance: 1 mm) with a single multi-MAP AC (bandpass: DC-500 Hz; sampling frequency: 2 kHz) (GE-Prucka Cardiolab). LP was performed with a NdYAG laser (Dornier Medilas 5100) and a catheter system for safe myocardium LP (5). The study was carried out according to NIH guidelines for animal care and use. Results MCG imaging of rabbit ventricular magnetic field distribution and inverse source localization with the magnetic dipole model are shown in Figure 1B. The difference in morphology of local HR ventricular multi-MAP signals was compatible with myocardial anisotropy (Figure 1C). With appropriate cooling of the optic fiber-tissue interface, no epicardial damage was induced by LP, whereas deeper well defined intramural myocardial lesions were appreciable, of different size as a function of the energy delivered (Figure 1D). Multi-MAP recording was not significantly disturbed by LP. Keeping the Medilas unit at least 5 meters away from the MCG sensors, no electromagnetic artifacts were observed during MCG. Conclusion MCG, single catheter multi-MAP recording and well-defined LP lesions of ventricular myocardium were simultaneously feasible on Langendorff-perfused isolated rabbit heart, without major electromagnetic interferences, in an unshielded catheterization laboratory. With appropriate technological developments, safe MCG-guided single-catheter laser ablation of focal arrhythmogenic substrates is foreseen as a possible minimally invasive alternative to current interventional approaches. Funding Acknowledgement Type of funding sources: None.

Wi-Fi indoor 3D localization algorithm based on multi-classifier fusion

Indoor localization technology based on Received Signal Strength (RSS) fingerprint is widely used in life and industry. Compared with the traditional localization methods, the localization technology integrating multiple machine learning methods has better localization accuracy. However, in multi-floor localization, the existing fusion localization technology ignores the interaction between different floors in the localization process, resulting in low indoor localization accuracy. This paper proposes a Wi-Fi indoor 3D localization method based on multi-classifier fusion named FLMCF. Firstly, floor classification training is carried out to reduce the location deviation in the vertical direction. Secondly, for each floor, multiple classifiers are used for model training and training the optimal weight set by minimizing the average localization error. In this case, the advantages of each classifier can be fully integrated to improve the localization accuracy. Finally, the Reliability Fusion Weight Selection (RFWS) algorithm determines the weight and calculates the final location estimation. The experimental results indicate that FLMCF is nearly 14.1% better than DIFMIC in the 90th percentile of CDF.

3D Localization Method of Partial Discharge in Air-Insulated Substation Based on Improved Particle Swarm Optimization Algorithm

Partial discharge (PD) localization in an air-insulated substation (AIS) can be used to assess insulation conditions efficiently. Many localization methods have been reported during the past few years. However, the error of the localization results has been large or the localization algorithm too inefficient. The reason is that the localization equation set is nonlinear and non-symmetrical. In this paper, an improved particle swarm optimization (PSO) algorithm is proposed to improve the localization accuracy in 3D. Firstly, the proposed localization method is based on the symmetrical antenna array and the location error distribution is analyzed. Secondly, the objective function of PSO is constructed using the error distribution. Specifically, the 3D location target is divided into two steps—plane coordinates and height. The two targets are optimized respectively. To verify the method, a test is carried out by a prefabricated fault bushing in the laboratory to compare with the existing methods. According to the results, the localization error is 0.21 m, which can locate the PD source accurately. A complete calculation takes 42.29 s, and the efficiency is increased by 16.13 times under the same accuracy. The comparison results show that the proposed method can greatly improve the efficiency while ensuring accuracy.

A review on structural development and recognition–localization methods for end-effector of fruit–vegetable picking robots

The excellent performance of fruit and vegetable picking robots is usually contributed by the reasonable structure of end-effector and recognition–localization methods with high accuracy. As a result, efforts are focused on two aspects, and diverse structures of end-effector, target recognition methods as well as their combinations are yielded continuously. A good understanding for the working principle, advantages, limitations, and the adaptability in respective fields is helpful to design picking robots. Therefore, depending on different grasping ways, separating methods, structures, materials, and driving modes, main characteristics existing in traditional schemes will be depicted firstly. According to technical routes, advantages, potential applications, and challenges, underactuated manipulators and soft manipulators representing future development are then summarized systematically. Secondly, partial recognition and localization methods are also demonstrated. Specifically, current recognition manners adopting the single-feature, multi-feature fusion and deep learning are explained in view of their advantages, limitations, and successful instances. In the field of 3D localization, active vision based on the structured light, laser scanning, time of flight, and radar is reflected through the respective applications. Also, another 3D localization method called passive vision is also evaluated by advantages, limitations, the degree of automation, reconstruction effects, and the application scenario, such as monocular vision, binocular vision, and multiocular vision. Finally portrayed from structural development, recognition, and localization methods, it is possible to develop future end-effectors for fruit and vegetable picking robots with superior characteristics containing the less driving element, rigid–flexible–bionic coupling soft manipulators, simple control program, high efficiency, low damage, low cost, high versatility, and high recognition accuracy in all-season picking tasks.

3D Global Localization in the Underground Mine Environment Using Mobile LiDAR Mapping and Point Cloud Registration.

This study proposes a 3D global localization method that implements mobile LiDAR mapping and point cloud registration to recognize the locations of objects in an underground mine. An initial global point cloud map was built for an entire underground mine area using mobile LiDAR; a local LiDAR scan (local point cloud) was generated at the point where underground positioning was required. We calculated fast point feature histogram (FPFH) descriptors for the global and local point clouds to extract point features. The match areas between the global and the local point clouds were searched and aligned using random sample consensus (RANSAC) and iterative closest point (ICP) registration. The object’s location on the global coordinate system was measured using the LiDAR sensor trajectory. Field experiments were performed at the Gwan-in underground mine using three mobile LiDAR systems. The local point cloud dataset formed for the six areas of the underground mine precisely matched the global point cloud, with a low average error of approximately 0.13 m, regardless of the type of mobile LiDAR system used. In addition, the LiDAR senor trajectory was aligned on the global coordinate system to confirm the change in the dynamic object’s position over time.

Object localization and tracking in three dimensions by space-to-time encoding.

In this paper, we present a novel method for measuring the location and estimating the dynamics of fast-moving small objects in free space. The proposed 3D localization method is realized by a space-to-time optical transform and measurement of time-of-flight. We present the underlying physical and mathematical model of the method and provide an example based on a simple configuration. In the simplest mode, the method is implemented by two plane mirrors, a spherical light pulse illuminator, and a single fast response photodetector. The 3D spatial information is retrieved from the temporal measurements by solving an inverse problem that uses a sparse approximation of the scene. System simulation shows the ability to track fast small objects that are moving in space using only a single time-resolved detector.

Four-dimensional multi-particle tracking in living cells based on lifetime imaging

Abstract Research on dynamic events in living cells, such as intracellular transportation, is important for understanding cell functions. As movements occur within cells, the microenvironment of the moving vesicles or biomacromolecules may affect the behavior of them. Herein, we propose a method of simultaneously monitoring changes in spatial positions and the local environment related to the fluorescence lifetime, i.e., four-dimensional (4D) multi-particle parallel-tracking in living cells. Based on double-helix point spread function (DH-PSF) microscopy and streak camera, the method combines three-dimensional (3D) localization methods and fluorescence lifetime imaging. By modifying the PSF of the system, the 3D positions and fluorescence lifetime information for several molecules within a depth of a few microns can be acquired simultaneously from a single snapshot. The feasibility of this method is verified by simulating the real-time tracking of a single particle with a given trajectory. In addition, a proof-of-concept 4D tracking system based on the DH-PSF and streak camera was built. The experimental results show that the 3D localization and lifetime precision are σ(x, y, z) = (26 nm, 35 nm, 53 nm) and σ(τ) = 103 ps, respectively, and the effective depth of field is approximately 4 μm. Finally, intracellular endocytosis in a living cell was observed using the system, which demonstrated the successful 4D tracking of two microspheres moving within an axial depth of 4 μm. This work opens a new perspective for research of dynamic processes, by providing information about the chemical (microenvironments) and physical (positions) changes of moving targets in living cells.

Real-Time, Robot-Based, 3D Localization of RFID Tags, by Transforming Phase Measurements to a Linear Optimization Problem

In this paper, we propose a prototype method for fast and accurate 3D localization of RFID-tagged items by a mobile robot. The robot performs Simultaneous Localization of its own pose and Mapping of the surrounding environment (SLAM). It is equipped with RFID readers and antennas placed at different heights, collecting phase-measurements by all tags. Thanks to the self-localization property of the robot, a synthetic aperture is created for each tag. In this paper, we have manipulated the set of phase-measurements, combined with the known poses of the robot to craft an overdetermined system of linear equations which pinpoints the location of the tag. The linearity of the system preserves that localization is achieved rapidly. The problem is solved for any arbitrary movement of the robot and extended in three dimensions. The proposed method is experimentally compared against the state-of-the-art in SAR-based RFID localization. It is the fastest and the most accurate, when the robot moves along straight-paths. Its accuracy slightly worsens, when the robot moves along non-straight paths, while preserving its exceptionally small running-time. It can be applied in real-time 3D localization problems, demonstrating mean 3D accuracy below 20 cm.

A 3D source localization method based on whale optimization algorithm: Experiments for locating a time-varying source in a dynamic indoor environment

To solve the problem of pollutant source localization in an indoor environment with dynamic mechanical ventilation better, this paper designed and built a 3D source localization system composed of three 3D source localization robots whose sensors can move under control in the height direction (0.5 m–1.5 m). Through adopting the 3D source localization system, we realized the application of the previously developed the improved whale optimization algorithm (IWOA) method in 3D source localization and proposed the 3D_IWOA method. To validate the feasibility of the 3D_IWOA method and compare the performance of the 3D_IWOA method and other three 3D source localization methods (the 3D_SPSO, 3D_WUⅡ, and 3D_IPSO methods), each method repeated 15 independent experiments in a Training-Center under the same conditions, respectively. For locating the periodic source at a height of 1.05 m in the downwind zone of this environment, the numbers of successful experiments of the 3D_SPSO, 3D_WUⅡ, 3D_IPSO, and 3D_IWOA methods were 5, 6, 11, and 13 (the success rates were 33.3%, 40.0%, 73.3%, and 86.7%), respectively. In addition, the means of total steps of the 3D_SPSO, 3D_WUⅡ, 3D_IPSO, and 3D_IWOA methods were 36.29 steps, 35.96 steps, 31.75 steps, and 38.15 steps, respectively.

Universal gated recurrent unit-based 3D localization method for ultra-wideband systems

In this study, a universal gated recurrent unit (u-GRU)-based location estimation (LE) method is proposed to obtain the location of an ultra-wideband (UWB) transmitter. The proposed u-GRU-LE system consists of a GRU-based classifier and nine localizers for nine channel models (CMs). The classifier first predicts the CM, and then, the proper localizer is selected according to the predicted CM to estimate the location of the transmitter. Rigorous simulations are executed with various CMs. From the results, it is verified that the proposed universal GRU-based 3D localization method generally performs well irrespective of the channel environments.