Coarse Localization Research Articles

Fingerprint-Based Localization and Channel Estimation Integration for Cell-Free Massive MIMO IoT Systems

In this article, we propose a novel localization and channel estimation integration framework for cell-free massive multiple-input–multiple-output (MIMO) Internet of Things (IoT) systems, in which position information supports accurate channel estimation and accurate channel information can, in turn, improve positioning accuracy. Under this integration framework, we propose a two-phase fingerprint-based localization method consisting of both initial and accurate localization phases and a coarse-location-based (CLB) pilot reassignment scheme. The coarse location information for pilot reassignment is obtained in the initial localization phase of the two-phase localization method, and the fingerprint information used in the accurate localization phase is extracted through channel estimation based on the CLB scheme. Furthermore, for localization, two different fingerprint similarity criteria are proposed to meet the requirements of the different localization phases. Simulation results demonstrate that our proposed two-phase fingerprint-based localization method achieves better positioning performance than existing methods, although there is a slight increase in computational complexity compared to the initial localization. Moreover, our proposed CLB pilot reassignment scheme outperforms the conventional pilot assignment schemes in the comprehensive performance considering both channel estimation performance and complexity.

Accurate Long‐Term Evolution/Wi‐Fi hybrid positioning technology for emergency rescue

AbstractIt is critical to estimate the location using only Long‐Term Evolution (LTE) and Wi‐Fi information gathered by the user's smartphone and deployable for emergency rescue, regardless of whether the Global Positioning System is received. In this research, we used a vehicle to gather LTE and Wi‐Fi wireless signals over a large area for an extended period of time. After that, we used the learning technique to create a positioning database that included both collection and noncollection points. We presented a two‐step positioning algorithm that utilizes coarse localization to discover a rough location in a wide area rapidly and fine localization to estimate a particular location based on the coarse position. We confirmed our technology utilizing different sorts of devices in four regional types that are generally encountered: dense urban, urban, suburban, and rural. Results presented that our algorithm can satisfactorily achieve the target accuracy necessary in emergency rescue circumstances.

Learn then match: A fast coarse-to-fine depth image-based indoor localization framework for dark environments via deep learning and keypoint-based geometry alignment

Image-based indoor localization provides fundamental support for applications such as indoor navigation, virtual reality, and location-based services. Most research focuses on developing methods in good lighting conditions via RGB images; while for low lighting situations, especially at night, RGB-based methods cannot perform well. Depth images are promising alternative in such conditions as they record geometrical information instead of texture information, making it possible to work in low lighting scenarios. Current depth image-based methods, either retrieval-based methods or 3D registration-based methods, are inefficient due to its high computation overhead, preventing the wide applications. To address this issue, we propose a fast coarse-to-fine localization framework for dark environment via deep learning and keypoint-based geometry alignment. In the coarse localization phase, we jointly perform the depth completion and pose regression to relieve the occlusion caused appearance variance in depth images. In the refinement phase, keypoints are used instead of whole depth image points under the ICP alignment framework to increase the localization efficiency. The keypoints are detected on depth feature maps weakly supervised with pose regression. The experiments on the open available 7Scenes dataset show that the proposed method obtain positional accuracy of 0.143 m and orientational accuracy of 5.275°in average and only cost 0.8s for a single depth image. The code for the proposed work is available at https://github.com/lqing900205/KeyPointDepthLocalization

Weakly Supervised Instance Segmentation via Category-aware Centerness Learning with Localization Supervision

Deep convolutional neural networks (DCNN) trained with pixel-level segmentation masks achieve high performance in the task of instance segmentation. The difficulty of acquiring such annotation limits the application and popularization of the DCNN-based approaches. To address the issue, a weakly supervised approach is proposed in the paper which performs instance segmentation only with the supervision of bounding box or coarse localization annotation. A novel DCNN model is constructed which consists of two branches: the centerness branch and the segmentation branch. The former branch is to learn the semantically spatial importance over the areas of object instances under the localization supervision. Object proposals with exact boundaries are automatically generated and are then ranked under the guidance of the output of the centerness branch. The most matched instance proposal is assigned to each object, which is then used to supervise the segmentation branch. The losses are calculated by both the outputs of the two branches and the entire DCNN model is trained end-to-end. Experiments are extensively conducted to verify the effectiveness. With the supervision of precise bounding box annotation, our approach achieves state-of-the-art (SOTA) accuracy in the comparison with recent related works. And in the case of coarse localization annotation, our approach only deduces a slight reduction in accuracy, which significantly outperforms other approaches. The excellent performance demonstrates that our approach would be helpful to further alleviate the workload of image annotation while maintaining competitive accuracy.

MT-GCNN: Multi-Task Learning with Gated Convolution for Multiple Transmitters Localization in Urban Scenarios.

With the advance of the Internet of things (IoT), localization is essential in varied services. In urban scenarios, multiple transmitters localization is faced with challenges such as nonline-of-sight (NLOS) propagation and limited deployment of sensors. To this end, this paper proposes the MT-GCNN (Multi-Task Gated Convolutional Neural Network), a novel multiple transmitters localization scheme based on deep multi-task learning, to learn the NLOS propagation features and achieve the localization. The multi-task learning network decomposes the problem into a coarse localization task and a fine correction task. In particular, the MT-GCNN uses an improved gated convolution module to extract features from sparse sensing data more effectively. In the training stage, a joint loss function is proposed to optimize the two branches of tasks. In the testing stage, the well-trained MT-GCNN model predicts the classified grids and corresponding biases jointly to improve the overall performance of localization. In the urban scenarios challenged by NLOS propagation and sparse deployment of sensors, numerical simulations demonstrate that the proposed MT-GCNN framework has more accurate and robust performance than other algorithms.

Multilayer map construction and vision-only multi-scale localization for intelligent vehicles in underground parking

High-accuracy localization is a major problem for intelligent vehicles. With the continuation of traffic scenarios, underground parking lots have become an essential link to intelligent vehicle localization. Current localization methods suitable for ordinary traffic scenes, such as the global localization system and Beidou, cannot obtain stable localization results because the satellite signal is blocked. Thus, localization is a complex problem to solve in this scenario to achieve highly accurate localization. This study proposes a coarse-to-fine multi-scale localization method that can achieve highly precise localization with only one monocular camera based on the constructed multilayer map. First, the visual features of the images to be located are extracted, and the proposed features are counted and matched with the map using the bag of word model to obtain the nearest node for completing node-level coarse localization. Second, the perspective N-point problem model is formed using visual features and structural information in the nearest node. Finally, the positions and attitudes of the vehicles relative to the map node are calculated to complete degree-level fine localization. The error rate of the localization algorithm is verified to be below 7.3%, which is superior to that of other methods. This study provides a new solution for the highly accurate localization of underground parking and other indoor scenes.

DS-UNet: A dual streams UNet for refined image forgery localization

Nowadays, image manipulation has become very easy with the help of image editing applications. The abuse of tampered images would mislead people’s judgment, or even make the society suffer from instability if without any interference. To address this issue, a dual-stream UNet named DS-UNet is presented in this paper to detect image tampering and localize forgery areas. The DS-UNet employs an RGB stream to extract both high-level and low-level manipulated traces for a coarse localization, and adopts a Noise stream to expose the local noise inconsistency for a refined localization. The lightweight hierarchical fusion way provides the DS-UNet with the ability to perceive tampered objects at different scales for the reason that tampered objects always vary in shapes and sizes. After that, the DS-UNet applies a single decoder to receive the abundant low-level manipulated traces and spatial localization information in the skip connection path. Through the decoder, the object details and spatial dimensions are gradually recovered, and the high-resolution prediction maps are generated. In the comparative analysis, more evaluation indices are introduced than the existing works in order to obtain a more comprehensive assessment. Extensive experiments with satisfying results on five datasets demonstrate its superior performance over state-of-the-art methods.

Guided multi-scale refinement network for camouflaged object detection.

The purpose of camouflaged object detection (COD) is to identify the hidden camouflaged object in an input image. Compared with other binary segmentation tasks like salient object detection, COD needs to deal with more complex scenes, such as low contrast, similar foreground and background. In this work, we proposed a novel guided multi-scale refinement network for COD. Specifically, we first design a global perception module for coarse localization by stacking multi-scale residual block on the top of the backbone in a recurrent manner. Then, we propose the guided multi-scale refinement module to refine such initial prediction progressively, which is combined with multi-level side-output features in a prediction-to-feature fusion strategy. By plugging into side-output features for multi-scale guidance, the missing object parts and false detection can be well remedied. Experimental results show that our proposed network can more accurately locate the camouflaged object and salient object with sharpened details than existing state-of-the-art approaches. In addition, our model is also very efficient and compact, which enables potential real-world applications.

Improving Weighted Multiple Linear Regression Algorithm for Radiolocation Estimation in LoRaWAN

In location based services, Weighted Multiple Linear Regression (WMLR) algorithm is used for radio device position estimation. Nevertheless, WMLR provides coarse location estimate, because weights apportioned to the received signal strength (RSS) for each hearable base station during matrix weight formation are not properly distributed. In an attempt to address the problem articulated above, an improved WMLR that enhanced the accuracy of radio device position estimate is proposed in this work. Min-Max scaling was used to determine the weight for each RSS values logged at different BS, as such forming a refined matrix weight. Public on-site outdoor Long Range Wide Area Network (LoRaWAN) RSS data set was used to assess the improved WMLR estimation algorithm on the basis of accuracy. The location accuracy of the proposed method is validated with the existing WMLR algorithm and Federal Communication Commission (FCC) maximum location error benchmark. Results show that the location accuracy of the improved approach outperformed that of the existing WMLR localization method.

Real-Time Sound Source Localization for Low-Power IoT Devices Based on Multi-Stream CNN.

Voice-activated artificial intelligence (AI) technology has advanced rapidly and is being adopted in various devices such as smart speakers and display products, which enable users to multitask without touching the devices. However, most devices equipped with cameras and displays lack mobility; therefore, users cannot avoid touching them for face-to-face interactions, which contradicts the voice-activated AI philosophy. In this paper, we propose a deep neural network-based real-time sound source localization (SSL) model for low-power internet of things (IoT) devices based on microphone arrays and present a prototype implemented on actual IoT devices. The proposed SSL model delivers multi-channel acoustic data to parallel convolutional neural network layers in the form of multiple streams to capture the unique delay patterns for the low-, mid-, and high-frequency ranges, and estimates the fine and coarse location of voices. The model adapted in this study achieved an accuracy of 91.41% on fine location estimation and a direction of arrival error of 7.43° on noisy data. It achieved a processing time of 7.811 ms per 40 ms samples on the Raspberry Pi 4B. The proposed model can be applied to a camera-based humanoid robot that mimics the manner in which humans react to trigger voices in crowded environments.

An Improved Canny-Zernike Subpixel Detection Algorithm

This paper proposed a new subpixel detection method that detects subpixel edges directly, as opposed to the previous method, which requires crossing the entire image. It shows superior subpixel detection on the edge directly, which enhances subpixel edge detection speed significantly. In order to overcome the problem of noise reduction, this paper employs bilateral filtering. The method first performed coarse localization with the improved operator to determine the coordinates and gradient direction of the edge points. Then, the Zernike moment algorithm was used for subpixel repositioning of edge points. Finally, subpixel level edge positioning of the image is obtained. The detection algorithm is used to identify the edges of large-size workpieces, and the results reveal that the approach has superior positioning accuracy, noise immunity, and fast detection speed.

Urban Vehicle Localization in Public LoRaWan Network

Location-based services (LBS) such as LoRa geolocation are important aspects of IoT applications. In this article, we propose a hierarchical clustering-based technique for urban vehicle localization using received signal strength indicator (RSSI) measurements in a public LoRaWan network. The solution relies on a two-layer hierarchy: the first layer consists of a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> -Means clustering to partition a large urban area into several regions based on geographical coordinates of the datapoints. A coarse localizer utilizes kernel density estimation to model the received signal distribution of each gateway and determine in which the most probable regions of interests the vehicle is located, followed by a finer localization step at the second layer. For each region, reference points are grouped based on the similarity between the gateway coverage vectors. A spatial kernel-based fingerprint method that adopts spatial co-location patterns between neighbors is introduced to provide support for further fine granularity positioning within each region. The Kullback–Leibler divergence is used to measure similarities between observations and fingerprints, and the final position estimation is based on a weighted kernel regression model. The system is evaluated using a publicly available LoRaWan data set collected in large urban areas in the city of Antwerp, Belgium. We are able to achieve a median error of 158.41 m and a mean error of 346.03 m based on the raw LoRa RSSI data, and it is reported as the best accuracy based on the same data set in the literature.

Robust hand tracking for surgical telestration

PurposeAs human failure has been shown to be one primary cause for post-operative death, surgical training is of the utmost socioeconomic importance. In this context, the concept of surgical telestration has been introduced to enable experienced surgeons to efficiently and effectively mentor trainees in an intuitive way. While previous approaches to telestration have concentrated on overlaying drawings on surgical videos, we explore the augmented reality (AR) visualization of surgical hands to imitate the direct interaction with the situs.MethodsWe present a real-time hand tracking pipeline specifically designed for the application of surgical telestration. It comprises three modules, dedicated to (1) the coarse localization of the expert’s hand and the subsequent (2) segmentation of the hand for AR visualization in the field of view of the trainee and (3) regression of keypoints making up the hand’s skeleton. The semantic representation is obtained to offer the ability for structured reporting of the motions performed as part of the teaching.ResultsAccording to a comprehensive validation based on a large data set comprising more than 14,000 annotated images with varying application-relevant conditions, our algorithm enables real-time hand tracking and is sufficiently accurate for the task of surgical telestration. In a retrospective validation study, a mean detection accuracy of 98%, a mean keypoint regression accuracy of 10.0 px and a mean Dice Similarity Coefficient of 0.95 were achieved. In a prospective validation study, it showed uncompromised performance when the sensor, operator or gesture varied.ConclusionDue to its high accuracy and fast inference time, our neural network-based approach to hand tracking is well suited for an AR approach to surgical telestration. Future work should be directed to evaluating the clinical value of the approach.

Map-based localization for intelligent vehicles from bi-sensor data fusion

Accurate localization is a challenging task for intelligent vehicles (IVs). Map-based localization methods are promising as they have no cumulative errors and high accuracy. Most methods are based on a single sensor, such as a camera or laser rangefinder (LRF). However, these sensors have disadvantages. Thus, this study proposes a map-based localization method from bi-sensor data fusion. We proposed a method using a camera, LRF, differential Global Positioning System, Inertial Navigation System to generate a multi-scale map. The multi-map consists of scenario nodes, and each node encodes scenario features, road 3D structure, and trajectories. Based on the multi-scale map, we propose an accurate localization method that includes coarse, node-level, and metric localizations. A bitopological localization model is set up for coarse localization. In this model, we use previous feature matching results to predict the current position. The features include image and light detection and ranging (LIDAR) data, and each of them set up one topological model. The two models make up a bitopological localization model. In node-level localization, we extract the LIDAR and image features and match them with the multi-scale map. The closest node is found by K-nearest neighbors from multiple feature spaces. In this step, we keep the image feature extraction in mind and propose a novel LIDAR feature extraction method called the LIDAR-image feature. In metric localization, we solve the perspective n-points problem between the current position and the closest node to compute the pose of IVs. In the experiments, our method has been tested on open road and campus. Each route covers different times and weather conditions. The proposed method can achieve less than 0.20 m mean localization errors. Compared with existing map-based methods, our method utilizes bi-sensors to improve localization performance.

Point cloud registration and localization based on voxel plane features

The 3D point cloud can directly provide accurate distance information, which facilitates many applications such as autonomous driving and environmental modeling. In order to construct more complete environmental information for these applications, it is necessary to register point clouds that are obtained from different poses. Registration after coarse localization is an effective method for understanding the pose of the device among the context of its environment. Here, a method for extracting plane features based on voxels is proposed and used for coarse registration and localization. The point cloud is divided into voxels by an octree, and voxels with the same plane characteristics are merged to obtain plane features. The candidate transformation matrix is calculated by using the corresponding plane set, and the RANSAC process with two-level transformation matrix verification, including quick verification and fine verification, is used to find the optimum transformation matrix from the candidates after clustering. Then, the coarse registration can be achieved. With the extracted plane features, a combined plane feature description of the point cloud frame is constructed to fulfill fast frame-level global localization. With the integration of the registration method presented in the current paper, pose localization can also be achieved. Experimental results show that the proposed method can achieve more than 85% successful registration rate with short time consumptions. This implies that the proposed method is more efficient than the benchmark method. Even when the map is large, frame-level localization is still fast, and has a successful localization rate of over 90%. The corresponding code are available at https://github.com/zhanjiawang/VPFBR-L.

Star spot centroid extraction method in high dynamic condition based on difference hash algorithm

In high dynamic conditions, star spot exhibits a tailing phenomenon, which makes it impossible to accurately extract the star centroid. Adjusting the exposure time can shorten the tail length but it will reduce the energy received by the detector, which also makes it difficult to extract the star target. Aiming at the above two problems, this paper proposes a star spot centroid extraction method under high dynamic conditions based on different hash algorithm. The method is divided into three steps. Firstly, establish a mathematical model of dynamic star spots. Then using difference hash algorithm and Hamming distance to realize coarse positioning of star targets in star tracking window. Finally, adopt threshold segmentation and connected domain method to extract star spot centroid in the coarse location area. The experimental results show that the proposed method can adapt to various exposure time, making the star sensor realize the stable tracking at the angular velocity of 3°/s. When the exposure time is 50ms and the angular velocity is 3°/s, the angular distance error is 13 arcseconds, the average extraction rate is 96%, which is 29.6% and 29.7% better than the traditional method.

MinkLoc3D-SI: 3D LiDAR Place Recognition With Sparse Convolutions, Spherical Coordinates, and Intensity

The 3D LiDAR place recognition aims to estimate a coarse localization in a previously seen environment based on a single scan from a rotating 3D LiDAR sensor. The existing solutions to this problem include hand-crafted point cloud descriptors (e.g., ScanContext, M2DP, LiDAR IRIS) and deep learning-based solutions (e.g., PointNetVLAD, PCAN, LPD-Net, DAGC, MinkLoc3D), which are often only evaluated on accumulated 2D scans from the Oxford RobotCar dataset. We introduce MinkLoc3D-SI, a sparse convolution-based solution that utilizes spherical coordinates of 3D points and processes the intensity of 3D LiDAR measurements, improving the performance when a single 3D LiDAR scan is used. Our method integrates the improvements typical for hand-crafted descriptors (like ScanContext) with the most efficient 3D sparse convolutions (MinkLoc3D). Our experiments show improved results on single scans from 3D LiDARs (USyd Campus dataset) and great generalization ability (KITTI dataset). Using intensity information on accumulated 2D scans (RobotCar Intensity dataset) improves the performance, even though spherical representation doesn’t produce a noticeable improvement. As a result, MinkLoc3D-SI is suited for single scans obtained from a 3D LiDAR, making it applicable in autonomous vehicles.

DualMMP-GAN: Dual-scale multi-modality perceptual generative adversarial network for medical image segmentation

Multi-modality magnetic resonance imaging (MRI) can reveal distinct patterns of tissue in the human body and is crucial to clinical diagnosis. But it still remains a challenge to obtain diverse and plausible multi-modality MR images due to expense, noise, and artifacts. For the same lesion, different modalities of MRI have big differences in context information, coarse location, and fine structure. In order to achieve better generation and segmentation performance, a dual-scale multi-modality perceptual generative adversarial network (DualMMP-GAN) is proposed based on cycle-consistent generative adversarial networks (CycleGAN). Dilated residual blocks are introduced to increase the receptive field, preserving structure and context information of images. A dual-scale discriminator is constructed. The generator is optimized by discriminating patches to represent lesions with different sizes. The perceptual consistency loss is introduced to learn the mapping between the generated and target modality at different semantic levels. Moreover, generative multi-modality segmentation (GMMS) combining given modalities with generated modalities is proposed for brain tumor segmentation. Experimental results show that the DualMMP-GAN outperforms the CycleGAN and some state-of-the-art methods in terms of PSNR, SSMI, and RMSE in most tasks. In addition, dice, sensitivity, specificity, and Hausdorff95 obtained from segmentation by GMMS are all higher than those from a single modality. The objective index obtained by the proposed methods are close to upper bounds obtained from real multiple modalities, indicating that GMMS can achieve similar effects as multi-modality. Overall, the proposed methods can serve as an effective method in clinical brain tumor diagnosis with promising application potential.

Multitask deep active contour-based iris segmentation for off-angle iris images

Iris recognition has been considered as a secure and reliable biometric technology. However, iris images are prone to off-angle or are partially occluded when captured with fewer user cooperations. As a consequence, iris recognition especially iris segmentation suffers a serious performance drop. To solve this problem, we propose a multitask deep active contour model for off-angle iris image segmentation. Specifically, the proposed approach combines the coarse and fine localization results. The coarse localization detects the approximate position of the iris area and further initializes the iris contours through a series of robust preprocessing operations. Then, iris contours are represented by 40 ordered isometric sampling polar points and thus their corresponding offset vectors are regressed via a convolutional neural network for multiple times to obtain the precise inner and outer boundaries of the iris. Next, the predicted iris boundary results are regarded as a constraint to limit the segmentation range of noise-free iris mask. Besides, an efficient channel attention module is introduced in the mask prediction to make the network focus on the valid iris region. A differentiable, fast, and efficient SoftPool operation is also used in place of traditional pooling to keep more details for more accurate pixel classification. Finally, the proposed iris segmentation approach is combined with off-the-shelf iris feature extraction models including traditional OM and deep learning-based FeatNet for iris recognition. The experimental results on two NIR datasets CASIA-Iris-off-angle, CASIA-Iris-Africa, and a VIS dataset SBVPI show that the proposed approach achieves a significant performance improvement in the segmentation and recognition for both regular and off-angle iris images.

A fully convolutional network (FCN) based automated ischemic stroke segment method using chemical exchange saturation transfer imaging.

Chemical exchange saturation transfer (CEST) MRI is a promising imaging modality in ischemic stroke detection due to its sensitivity in sensing postischemic pH alteration. However, the accurate segmentation of pH-altered regions remains difficult due to the complicated sources in water signal changes of CEST MRI. Meanwhile, manual localization and quantification of stroke lesions are laborious and time-consuming, which cannot meet the urgent need for timely therapeutic interventions. The goal of this study was to develop an automatic lesion segmentation approach of the ischemic region based on CEST MR images. A novel segmentation framework based on the fully convolutional neural network was investigated in our task. Z-spectra from 10 rats were manually labeled as ground truth and split into two datasets, where the training dataset including 3 rats was used to generate a segmentation model, and the remaining rats were used as test datasets to evaluate the model's performance. Then a 1D fully convolutional neural network equipped with bottleneck structures was set up, and a Grad-CAM approach was used to produce a coarse localization map, which can reflect the relevancy to the "ischemia" class of each pixel. As compared with the ground truth, the proposed network model achieved satisfying segmentation results with high values of evaluation metrics including specificity (SPE), sensitivity (SEN), accuracy (ACC), and Dice similarity coefficient (DSC), especially in some intractable situations where conventional MRI modalities and CEST quantitative method failed to distinguish between ischemic and normal tissues; also the model with augmentation was robust to input perturbations. The Grad-CAM maps performed clear tissue change distributions and interpreted the segmentations, showed a strong correlation with the quantitative method, and gave extended thinking to the function of networks. The proposed method can segment ischemia region from CEST images, with the Grad-CAM maps giving access to interpretative information about the segmentations, which demonstrates great potential in clinical routines.