3D Localization Research Articles

Creating speech zones with self-distributing acoustic swarms

Imagine being in a crowded room with a cacophony of speakers and having the ability to focus on or remove speech from a specific 2D region. This would require understanding and manipulating an acoustic scene, isolating each speaker, and associating a 2D spatial context with each constituent speech. However, separating speech from a large number of concurrent speakers in a room into individual streams and identifying their precise 2D locations is challenging, even for the human brain. Here, we present the first acoustic swarm that demonstrates cooperative navigation with centimeter-resolution using sound, eliminating the need for cameras or external infrastructure. Our acoustic swarm forms a self-distributing wireless microphone array, which, along with our attention-based neural network framework, lets us separate and localize concurrent human speakers in the 2D space, enabling speech zones. Our evaluations showed that the acoustic swarm could localize and separate 3-5 concurrent speech sources in real-world unseen reverberant environments with median and 90-percentile 2D errors of 15 cm and 50 cm, respectively. Our system enables applications like mute zones (parts of the room where sounds are muted), active zones (regions where sounds are captured), multi-conversation separation and location-aware interaction.

Deep Learning Algorithms in Mobile Edge with Real-Time Abnormal Event Detection for 5G-IoT Devices

IoT is becoming increasingly popular due to its quick expansion and variety of applications. In addition, 5G technology helps with communication and network connectivity. This work integrates C-RAN with IoT networks to provide an experimental 5G testbed. In a 5G IoT environment, this experience is utilized to enhance both perpendicular and flat localization (3D localization). DRCaG, an acronym for the proposed model, stands for a deep, complicated network with a gated layer on top. The performance of the proposed model has been demonstrated through extensive simulations in terms of learning reduction, accuracy, and matrix disorientation, with a variable signal-to-noise ratio (SNR) spanning from 20 dB to + 20 dB, which illustrates the superiority of DRCaG compared to others. An online, end-to-end solution based on deep learning techniques is presented in this study for the fast, precise, reliable, and automatic detection of diverse petty crime types. By detecting tiny crimes like hostility, bag snatching, and vandalism, the suggested system may not only identify unusual passenger behavior like vandalism and accidents but also improve passenger security. The solution performs admirably in a variety of use cases and environmental settings.

Improving pedicle screw path planning by vertebral posture estimation

Objective. Robot-assisted pedicle screw placement in spinal surgery can reduce the complications associated with the screw placement and reduce the hospital return counts due to malfunctions. However, it requires accurate planning for a high-quality procedure. The state-of-the-art technologies reported in the literature either ignore the anatomical variations across vertebrae or require substantial human interactions. We present an improved approach that achieves pedicle screw path planning through multiple projections of a numerically re-oriented vertebra with the estimated posture. Approach. We proposed an improved YOLO-type neural network model (YOLOPOSE3D) to estimate the posture of a vertebra before pedicle path planning. In YOLOPOSE3D, the vertebral posture is given as a rotation quaternion and 3D location coordinates by optimizing the intersection over union of the vertebra with the predicted posture and the actual posture. Then, a new local coordinate system is established for the vertebra based on the estimated posture. Finally, the optimal pedicle screw path trajectory is determined from the multiple projections of the vertebra in the local coordinates. Main results. The experimental results in difficult cases of scoliosis showed that the new YOLOPOSE3D network could accurately detect the location and posture of the vertebra with average translation and orientation errors as small as 1.55 mm and 2.55°. The screw path planning achieved 83.1% success rate without breaking the pedicle cortex for the lumbar vertebral L1–L5, which is better than that of a doctor’s manual planning, 82.4%. With the clinical class A requirement to allow less than 2 mm out of the pedicle cortex, the success rate achieved nearly 100%. Significance. The proposed YOLOPOSED3D method can accurately determine the vertebral postures. With the improved posture prior, better clinical outcomes can be achieved for pedicle screw placement in spine internal fixation procedures.

A Clustering-Based 3D Localization in Wireless Sensor Networks Using RSSI and AoA

A Clustering-Based 3D Localization in Wireless Sensor Networks Using RSSI and AoA

Topological Flat Bands in 2D Breathing-Kagome Lattice Nb3 TeCl7.

Flat bands(FBs) can appear in two-dimensional (2D) geometrically frustrated systems caused by quantum destructive interference (QDI). However, the scarcity of pure 2D frustrated crystal structures in natural materials makes FBs hard to be identified, let alone modulate FBs relating to electronic properties. Here, the experimental evidence of the complete electronic QDI induced FB contributed by the 2D breathing-kagome layers of Nb atoms in Nb3 TeCl7 (NTC) is reported. An identical chemical state and 2D localization characteristics of the Nb breathing-kagome layers are experimentally confirmed, based on which NTC is demonstrated to be a superior concrete candidate for the breathing-kagome tight-binding model. Furthermore, it theoretically establishes the tunable roles of the on-site energy over Nb sites on bandwidth, energy position, and topology of FBs in NTC. This work opens an aveanue to manipulate FB characteristics in these 4d transition-metal-based breathing-kagome materials.

On Performance Analysis for Random 3D Mobile AUV Networks With Limited Data Buffers

In this paper, we consider a finite network of multiple mobile autonomous underwater vehicles (AUVs) collectingdata in a given region and transferring them to a surface sink. All AUVs patrol a cylindrical region and dynamically control their 3-Dimensional (3D) location according to the random way-point (RWP) mobility model. Our specific interest focuses on the system throughput under different locations of the serving AUV. The following three serving AUV selection policies are investigated: 1)The serving AUV is the closest AUV among all active AUVs; 2) The serving AUV is randomly selected among the active AUVs; 3)The serving AUV is the farthest AUV among all active AUVs. Under the time slotted network scenario, the system throughput is defined as the probability that the sink can successfully recover the data packet from the serving AUV in one time slot. Further, we consider a practical constraint that all AUVs are equipped with limited data buffers. Based on the probability distributions of the distances between the AUVs and the surface sink, this paper presents a mathematical framework to characterize the performance of the random 3D mobile network. With the help of this framework, we obtain the theoretical performance in terms of the system throughput of the surface sink and the average queue delay of any AUV. Finally, extensive simulations are presented to verify the accuracy of our analysis and study the effects of various parameters.

ANALYSIS OF 3D COASTAL TERRAIN BY UAV IMAGE AND BUNDLE ADJUSTMENT

Recently, studies on the construction of 3D terrain modeling and location accuracy evaluation using UAV have been conducted, but the application to coastal areas lacking image matching points has not been made in detail yet. In coastal topography survey using UAV, ortho-images and DSM (Digital Surface Model) data were constructed through imaging and ground control point (GCP) surveying on the ground. In the case of ortho-images, VRS survey was used to analyze horizontal and vertical errors between GCP points, to evaluate accuracy, and to construct terrain data from the ortho-image and DSM data.

Cyber Mobility Mirror: A Deep Learning-Based Real-World Object Perception Platform Using Roadside LiDAR

Object perception plays a fundamental role in Cooperative Driving Automation (CDA) which is regarded as a revolutionary promoter for next-generation transportation systems. However, the vehicle-based perception may suffer from the limited sensing range and occlusion as well as low penetration rates in connectivity. In this paper, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Cyber Mobility Mirror</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CMM</i> ), a next-generation real-world object perception system for 3D object detection, tracking, localization, and reconstruction, to explore the potential of roadside sensors for enabling CDA in the real world. The CMM system consists of six main components: i) the data pre-processor to retrieve and preprocess the raw data; ii) the roadside 3D object detector to generate 3D detection results; iii) the multi-object tracker to identify detected objects; iv) the global locator to generate geo-localization information; v) the mobile-edge-cloud-based communicator to transmit perception information to equipped vehicles, and vi) the onboard advisor to reconstruct and display the real-time traffic conditions. An automatic perception evaluation approach is proposed to support the assessment of data-driven models without human-labeling requirements and a CMM field-operational system is deployed at a real-world intersection to assess the performance of the CMM. Results from field tests demonstrate that our CMM prototype system can achieve 96.99% precision and 83.62% recall for detection and 73.55% ID-recall for tracking. High-fidelity real-time traffic conditions (at the object level) can be geo-localized with a root-mean-square error (RMSE) of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.69m$</tex-math> </inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.33m$</tex-math> </inline-formula> for lateral and longitudinal direction, respectively, and displayed on the GUI of the equipped vehicle with a frequency of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3-4 Hz$</tex-math> </inline-formula> .

Human ear-inspired ultrasonic transducers able to locate objects in three dimensions

Adding spirals to acoustic lenses encodes directional information, enabling 3D localization by a single detector.

A human ear-inspired ultrasonic transducer (HEUT) for 3D localization of sub-wavelength scatterers

The proposed technology aims to enable 3D localization of scatterers using single element ultrasonic transducers, which are traditionally limited to 1D measurements. This is achieved by designing a bespoke acoustic lens with a spiral-shaped pattern similar to the human outer ear, a shape that has evolved for sound source localization. This lens breaks the surface symmetry of the transducer, allowing ultrasonic waves arriving from different directions to be encoded in a certain way that can later be decoded to extract directional information. By employing the mechanism of spatial-encoding of the received signals and decoding via signal processing, the location of sub-wavelength scatterers can be detected in 3D with a single measurement for sparsely distributed scatterers. The proposed technology is first verified through a simulation study, and then 3D printed acoustic lenses are used to demonstrate the 3D encoding functionality of the Human Ear-inspired Ultrasonic Transducer (HEUT) experimentally. A framework is created to localize scatterers in 3D by processing received signals acquired by a HEUT prototype. With this technology, a single transducer can obtain multi-dimensional information with a single pulse-echo measurement, reducing the number of elements required for performing 3D ultrasound localization. The proposed spatial-encoding and -decoding technology can be applied to other wave-based imaging methods to develop affordable, practical, and compact sensing devices.

CHSNet: Automatic lesion segmentation network guided by CT image features for acute cerebral hemorrhage

Stroke is a cerebrovascular disease that can lead to severe sequelae such as hemiplegia and mental retardation with a mortality rate of up to 40%. In this paper, we proposed an automatic segmentation network (CHSNet) to segment the lesions in cranial CT images based on the characteristics of acute cerebral hemorrhage images, such as high density, multi-scale, and variable location, and realized the three-dimensional (3D) visualization and localization of the cranial lesions after the segmentation was completed. To enhance the feature representation of high-density regions, and capture multi-scale and up-down information on the target location, we constructed a convolutional neural network with encoding-decoding backbone, Res-RCL module, Atrous Spatial Pyramid Pooling, and Attention Gate. We collected images of 203 patients with acute cerebral hemorrhage, constructed a dataset containing 5998 cranial CT slices, and conducted comparative and ablation experiments on the dataset to verify the effectiveness of our model. Our model achieved the best results on both test sets with different segmentation difficulties, test1: Dice = 0.918, IoU = 0.853, ASD = 0.476, RVE = 0.113; test2: Dice = 0.716, IoU = 0.604, ASD = 5.402, RVE = 1.079. Based on the segmentation results, we achieved 3D visualization and localization of hemorrhage in CT images of stroke patients. The study has important implications for clinical adjuvant diagnosis.

Instance Segmentation of Lotus Pods and Stalks in Unstructured Planting Environment Based on Improved YOLOv5

Accurate segmentation of lotus pods and stalks with pose variability is a prerequisite for realizing the robotic harvesting of lotus pods. However, the complex growth environment of lotus pods causes great difficulties in conducting the above task. In this study, an instance segmentation model, LPSS-YOLOv5, for lotus pods and stalks based on the latest YOLOv5 v7.0 instance segmentation model was proposed. The CBAM attention mechanism was integrated into the network to improve the model’s feature extraction ability. The scale distribution of the multi-scale feature layer was adjusted, a 160 × 160 small-scale detection layer was added, and the original 20 × 20 large-scale detection layer was removed, which improved the model’s segmentation accuracy for small-scale lotus stalks and reduced the model size. On the medium-large scale test set, LPSS-YOLOv5 achieved a mask mAP0.5 of 99.3% for all classes. On the small-scale test set, the mAP0.5 for all classes and AP0.5 for stalks were 88.8% and 83.3%, which were 2.6% and 5.0% higher than the baseline, respectively. Compared with the mainstream Mask R-CNN and YOLACT models, LPSS-YOLOv5 showed a much higher segmentation accuracy, speed, and smaller size. The 2D and 3D localization tests verified that LPSS-YOLOv5 could effectively support the picking point localization and the pod–stalk affiliation confirmation.

Optimizing Voronoi-based quantifications for reaching interactive analysis of 3D localizations in the million range.

Over the last decade, single-molecule localization microscopy (SMLM) has revolutionized cell biology, making it possible to monitor molecular organization and dynamics with spatial resolution of a few nanometers. Despite being a relatively recent field, SMLM has witnessed the development of dozens of analysis methods for problems as diverse as segmentation, clustering, tracking or colocalization. Among those, Voronoi-based methods have achieved a prominent position for 2D analysis as robust and efficient implementations were available for generating 2D Voronoi diagrams. Unfortunately, this was not the case for 3D Voronoi diagrams, and existing methods were therefore extremely time-consuming. In this work, we present a new hybrid CPU-GPU algorithm for the rapid generation of 3D Voronoi diagrams. Voro3D allows creating Voronoi diagrams of datasets composed of millions of localizations in minutes, making any Voronoi-based analysis method such as SR-Tesseler accessible to life scientists wanting to quantify 3D datasets. In addition, we also improve ClusterVisu, a Voronoi-based clustering method using Monte-Carlo simulations, by demonstrating that those costly simulations can be correctly approximated by a customized gamma probability distribution function.

Deep Learning Approach to Source Localization of Electromagnetic Waves in the Presence of Various Sources and Noise

In this paper, the 3D localization and signal enhancement problem of a source in a noisy environment is addressed using an antenna array to ensure symmetry in communication engineering. The use of machine-learning-dependent convolutional recurrent neural networks (CRNN) and a minimum variance distortionless response (MVDR) beamformer for the localization of the source is developed. Furthermore, to ensure the adaptability of the signal enhancement module during deployment in a new environment or in new conditions, the training of a meta-learning model is conducted. At first, during the localization, the direction of arrival (DoA) estimation in both azimuth and elevation angles is generated. This is generated in a noisy three-dimensional plane and multi-source signal. Employing the DoA estimates, the MVDR is used for the enhancement of the signal source. Verifying the proposed method in the presence of mutual coupling, the two scenarios in communication engineering were simulated using a ray-tracing tool in the form of a real-world problem towards enhancing a signal source in a noisy environment and in the presence of various sources. The results obtained demonstrate how the proposed method outperforms the machine learning and parametric methods. In addition, the trained meta-learning model is employed to demonstrate how the proposed method is adaptable to any environment and still maintains an appreciable quality performance index after retraining with few data. Finally, the results obtained are motivating enough for the practical application of the proposed method.

Captivates: A Smart Eyeglass Platform for Across-Context Physiological Measurements

We present Captivates, an open-source smartglasses system designed for long-term, in-the-wild psychophysiological monitoring at scale. Captivates integrate many underutilized physiological sensors in a streamlined package, including temple and nose temperature measurement, blink detection, head motion tracking, activity classification, 3D localization, and head pose estimation. Captivates was designed with an emphasis on (1) manufacturing and scalability, so we can easily support large-scale user studies for ourselves and offer the platform as a generalized tool for ambulatory psychophysiology research; (2) robustness and battery life, so long-term studies result in trustworthy data across an individual's entire day in natural environments without supervision or recharge; and (3) aesthetics and comfort, so people can wear them in their normal daily contexts without self-consciousness or changes in behavior.

Research on 3D Particle Field Reconstruction Method Based on Deep Learning

Reconstructing 3D particle fields from single holograms is an important topic in the computer vision field. To increase network accuracy, we incorporate a channel attention mechanism based on the U-Net architecture in this research; meanwhile, we use the LeakyReLU activation function to accelerate the network convergence. Deep learning is used to extract information from a single hologram that can recreate the 3D particle field. The network receives the digital hologram as input, and the radius and 3D locations of the particles are converted into 2D grayscale images as real labels. Simulation and experimental results show that 2D grayscale images with clear edge textures can be rapidly encoded using the neural network, and the average SSIM and PSNR of the network output results with the real target on the test data set can reach 0.989 and 32.56.

A Fully Wireless and Batteryless Localization System with 50 Micrometre Motion Detection Capability and Adaptive Transmitter Power Control for Point-of-Care Biomedical Applications.

Localization has varied applications in biomedicine, such as wireless capsule endoscopy (WCE), detection of cancerous tissue, drug delivery, robotic surgeries, and brain mapping. Currently, most localization systems are battery-powered and suffer from issues regarding battery leakage and limited battery life, resulting in potential health hazards and inconveniences when using them for continuous health monitoring applications. This paper proposes an entirely wireless and battery-less 2D localization system consisting of an integrated circuit (IC) that is wirelessly powered at a distance of 4 cm by a 40.68 MHz radio frequency (RF) power of only 2 W. The proposed localization system wirelessly transmits a locked sub-harmonic 13.56 MHz signal generated from the wirelessly received 40.68 MHz RF power signal, eliminating the need for a power-hungry oscillator. Additionally, the system, having a measurement latency of 11.3 ms, has also been verified to sense motion as small as 50 μm as well as measure the rate of motion up to 10 beats per minute, therefore extending its application to the detection of physiological motions such as diaphragm motion during breathing. The localizer has a small form factor of 17 mm × 12 mm × 0.2 mm and consumes an average power of 6 μW. Ex vivo measurements using the localizer inside the porcine intestine demonstrate a localization accuracy of less than 5 mm.

Automated localization and segmentation of cervical lymph nodes on contrast-enhanced CT using a 3D foveal fully convolutional neural network

BackgroundIn the management of cancer patients, determination of TNM status is essential for treatment decision-making and therefore closely linked to clinical outcome and survival. Here, we developed a tool for automatic three-dimensional (3D) localization and segmentation of cervical lymph nodes (LNs) on contrast-enhanced computed tomography (CECT) examinations.MethodsIn this IRB-approved retrospective single-center study, 187 CECT examinations of the head and neck region from patients with various primary diseases were collected from our local database, and 3656 LNs (19.5 ± 14.9 LNs/CECT, mean ± standard deviation) with a short-axis diameter (SAD) ≥ 5 mm were segmented manually by expert physicians. With these data, we trained an independent fully convolutional neural network based on 3D foveal patches. Testing was performed on 30 independent CECTs with 925 segmented LNs with an SAD ≥ 5 mm.ResultsIn total, 4,581 LNs were segmented in 217 CECTs. The model achieved an average localization rate (LR), i.e., percentage of localized LNs/CECT, of 78.0% in the validation dataset. In the test dataset, average LR was 81.1% with a mean Dice coefficient of 0.71. For enlarged LNs with a SAD ≥ 10 mm, LR was 96.2%. In the test dataset, the false-positive rate was 2.4 LNs/CECT.ConclusionsOur trained AI model demonstrated a good overall performance in the consistent automatic localization and 3D segmentation of physiological and metastatic cervical LNs with a SAD ≥ 5 mm on CECTs. This could aid clinical localization and automatic 3D segmentation, which can benefit clinical care and radiomics research.Relevance statementOur AI model is a time-saving tool for 3D segmentation of cervical lymph nodes on contrast-enhanced CT scans and serves as a solid base for N staging in clinical practice and further radiomics research.Key points• Determination of N status in TNM staging is essential for therapy planning in oncology.• Segmenting cervical lymph nodes manually is highly time-consuming in clinical practice.• Our model provides a robust, automated 3D segmentation of cervical lymph nodes.• It achieves a high accuracy for localization especially of enlarged lymph nodes.• These segmentations should assist clinical care and radiomics research.Graphical

Vertical nanoscale strain-induced electronic localization in epitaxial La2/3Sr1/3MnO3 films with ZrO2 nanopillar inclusions

Unusual electrical transport properties associated with weak or strong localization are sometimes found in disordered electronic materials. Here, we report experimental observation of a crossover of electronic behavior from weak localization to enhanced weak localization due to the spatial influence of disorder induced by ZrO2 nanopillars in (La2/3Sr1/3MnO3)1−x:(ZrO2)x (x = 0, 0.2, and 0.3) nanocomposite films. The spatial strain regions, identified by scanning transmission electron microscopy and high-resolution x-ray diffraction, induce a coexistence of two-dimentional (2D) and three-dimentional (3D) localization and switches to typical 2D localization with increasing density of ZrO2 pillars due to length scale confinement, which interestingly accords with enhancing vertically interfacial strain. Based on the excellent agreement of our experimental results with one-parameter scaling theory of localization, the enhanced weak localization exists in metal range close to the fixed point. These films provide a tunable experimental model for studying localization in particular the transition regime by appropriate choice of the second epitaxial phase.Graphical

Improving Monocular Camera Localization for Video-Based Three-Dimensional Outer Ear Reconstruction Tasks

This work addresses challenges related to camera 3D localization while reconstructing a 3D model of an ear. This work explores the potential solution of using a cap, specifically designed not to obstruct the ear, and its efficiency in enhancing the camera localization for structure-from-motion (SfM)-based object reconstruction. The proposed solution is described, and an elaboration of the experimental scenarios used to investigate the background textures is provided; data collection and software tools used in the research are reported. The results show that the proposed method is effective, and using the cap with texture leads to a reduction in the camera localization error. Errors in the 3D location reconstruction of the camera were calculated by comparing cameras localized within typical ear reconstruction situations to those of higher-accuracy reconstructions. The findings also show that caps with sparse dot patterns and a regular knitted patterned winter hat are the preferred patterns. The study provides a contribution to the field of 3D modeling, particularly in the context of creating 3D models of the human ear, and offers a step towards more accurate, reliable, and feasible 3D ear modeling and reconstruction.