Real-time Position Research Articles

Accurate Vision-Based PCB Positioning Using Cosine-Convolutional Neural Networks

Object positioning is a fundamental task for assembly and inspection operations in the electronic industry. Traditional methods such as template matching and feature-point detection have been applied for object positioning. They are however computationally expensive and are generally affected by environmental changes. Deep learning models based on Convolutional Neural Network (CNN) have also been used for object positioning. They are computationally very efficient but can be sensible under environmental variations such as illumination and noise. In this article, a Cosine-Convolution operation is proposed with the objective of minimizing the effects of illumination variations. The proposed Cosine-Convolution can substitute any convolutional operation in a CNN-based regressor for positioning tasks. The proposed convolution is based on cosine-measure that normalizes the convolution between an image window and a filter at local scope. The proposed Cosine-Convolutional Neural Network (Cosine-CNN) performs more accurate than the traditional CNN under environment variations, including illumination changes, noise, defocusing and template occlusion. The experimental results reveal that the proposed model can reach mean prediction errors smaller than 1 pixel and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1^\circ$</tex-math> </inline-formula> for object shift and rotation, regardless of the environmental variations. The method is computationally very fast, with an average evaluation time of 1.2-ms for real-time PCB positioning. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The training of deep learning models based on Convolutional Neural Network (CNN) can be tedious. Image samples with different illumination settings must be provided to have an efficient and robust model, which may not be easy to acquire or synthesize. This study proposes a solution to the challenge of illumination variations in images for the object positioning task in Printed Circuit Boards (PCB). A new Convolutional operation called Cosine-Convolutional that normalizes the traditional Convolution is introduced to overcome the illumination variations in the input images. Experimental results demonstrate that the proposed Cosine-Convolutional operation is less sensitive against lightning variations with respect to the training images.

Real-Time Low Energy Indoor Positioning System to Efficient Use of Operating Theaters with Medical Asset and Staff Tracking

Real-Time Low Energy Indoor Positioning System to Efficient Use of Operating Theaters with Medical Asset and Staff Tracking

Temporal Characteristics Based Outlier Detection and Prediction Methods for PPP-B2b Orbit and Clock Corrections

The BeiDou Global Navigation Satellite System (BDS-3) provides real-time precise point positioning (PPP) service via B2b signals, offering real-time decimeter-level positioning for users in China and surrounding areas. However, common interruptions and outliers in PPP-B2b services arise due to factors such as the Geostationary Orbit (GEO) satellite “south wall effect”, Issue of Data (IOD) matching errors, and PPP-B2b signal broadcast priorities, posing challenges to continuous high-precision positioning. This study meticulously examines the completeness, continuity, and jumps in PPP-B2b orbit and clock correction using extensive observational data. Based on this analysis, a two-step method for detecting outliers in PPP-B2b orbit and clock corrections is devised, leveraging epoch differences and median absolute deviation. Subsequently, distinct prediction methods are developed for BDS-3 and GPS orbit and clock corrections. Results from simulated and real-time dynamic positioning experiments indicate that predicted corrections can maintain the same accuracy as normal correction values for up to 10 min and sustain decimeter-level positioning accuracy within 30 min. The adoption of predicted correction values significantly enhances the duration of sustaining real-time PPP during signal interruptions.

Real-time underwater wireless optical communication and positioning an integrated system based on angular diversity SiPMs.

Underwater communication and positioning are essential for autonomous underwater vehicle (AUV) docking and formation. The traditional methods for communication and positioning are mainly independent from each other, increasing the redundancy and integration difficulty for AUVs. In this Letter, we demonstrate a real-time underwater wireless optical communication and positioning (UWOCP) integrated system. The LED array is adopted as a light source, and the pulse-position modulation (PPM) is used for a maximum transmission and sensing distance. By employing the silicon photomultiplier (SiPM) array, which consists of five SiPMs with different angles, the high sensitivity and ability to distinguish angles are obtained. Through calculating the relationship between the received pulse signal intensity of the five SiPMs, the pitch angle and yaw angle can be obtained. The experimental results in the pool show that the Ethernet bandwidth of 2.2 Mbps with an average angular error of 3.08° for one-dimensional positioning can be realized at a 50 m distance. To the best of our knowledge, this is the longest distance at which a real-time UWOCP system has been demonstrated. The proposed UWOCP system has the advantages of high sensitivity, computing efficiency, and compact structure, presenting great potential for underwater applications.

Improving the stochastic model for code pseudorange observations from Android smartphones

In recent years, there has been increasing attention to positioning, navigation, and timing applications with smartphones. Because of frequently disrupted carrier phase observations, code observations remain critical for smartphone-based positioning. Considering a realistic stochastic model is mandatory to obtain the utmost positioning performance, this study proposes a sound stochastic approach for code observations from Android smartphones. The proposed approach includes a modified version of the SIGMA-ɛ variance model with different coefficients for each GNSS constellation and a robust Kalman filter method. First the noise characteristics of observations from the Xiaomi Mi 8 smartphone are analyzed utilizing code-minus-phase combinations to estimate the coefficients for each GNSS constellation. This includes the determination of a variance model as well as a check of the probability distribution. Finally, the proposed approach is validated in the positioning domain using single-frequency code observation-based real-time standalone positioning. The results show that more than 95% of observations follow the normal distribution when the proposed approach is applied. Compared with the conventional stochastic approach, including a C/N0-dependent model and standard Kalman filter, it improves the positioning accuracy by 45.8% in a static experiment, while its improvement is equal to 26.6% in a kinematic experiment. For the static and kinematic experiments, in 50% of the epochs, the 3D positioning errors are smaller than 3.0 m and 3.4 m for the proposed stochastic approach. The results exhibit that the stochastic properties of code observations from smartphones can be successfully represented by the proposed approach.

Use of Mixed Reality in Attachment of Surgical Site Measurement Robot to Surgical Bed

Recently, we have observed that the digital potential function defined by the difference between the real and virtual organ depth images is globally stable where the real and virtual livers coincide. This globality is then used to overlay the real and virtual livers. In this study, we consider the installation of a robotic mechanical system for measuring the depth images of real organs in the surgical bed. In general, virtual organs measured by CT or MRI show the position and posture of blood vessel groups and malignant tumors, and if these can be presented to the physician during surgery, he or she can operate while confirming their positions in real time. Although this robotic mechanical system is designed such that the camera can be raised or lowered as necessary to avoid interfering with the movement of the doctor, assistant, or nurse during surgery, it may still shift owing to contact with the hands or head of the doctor or nurse. In this study, an experiment was conducted in which a surgical measurement robotic mechanical system was constructed in a VR environment, and an actual robot was installed using this as a model. In the experiment, a video image of a virtual object was superimposed on that of a real object to confirm whether the surgical robotic mechanical system was able to accurately measure the surgical site.

A Comprehensive Comparison of Photogrammetric and RTK-GPS Methods for General Order Land Surveying

One of the main objectives of modern-day surveying is to maximize the efficiency and accuracy of mapping a landscape for natural features and elevations prior to the start of a construction project. This paper focuses on a comparison between terrestrial and aerial photogrammetry and real-time kinematic global positioning systems (RTK-GPSs) in terms of elevation accuracy, data expenditure, and time for each survey to be completed. Two sites in San Diego County were chosen to be studied with a combined area of about 1.14 acres, and a total station system was used to establish 572 control points between both areas. Two of the three methods investigated produced similar results in elevation and were well within the established standard, as the terrestrial photogrammetry averaged 0.0583 feet of error, the aerial photogrammetry averaged 0.345 feet of error, and the RTK-GPS averaged 0.0432 feet of error when compared to the total station ground truth. If data consumption is not a concern, the terrestrial photogrammetric method should be preferred to the aerial photogrammetric and RTK-GPS methods in topographic mapping and land monitoring due to the increase in time efficiency and in surface model detail while keeping within the Caltrans specified tolerance of error of 0.2 feet. For general order land surveys, the photogrammetric approach utilized with a Looq scanner would provide the most efficient and cost-effective survey while staying within the 0.2 foot tolerance of error. This method also allows for the utmost clarity of the resulting point cloud when analyzing terrain, break lines, or other features in the survey area.

Use of Yolo Detection for 3D Pose Tracking of Cardiac Catheters Using Bi-Plane Fluoroscopy

The increasing rate of minimally invasive procedures and the growing prevalence of cardiovascular disease have led to a demand for higher-quality guidance systems for catheter tracking. Traditional methods for catheter tracking, such as detection based on single points and applying masking techniques, have been limited in their ability to provide accurate pose information. In this paper, we propose a novel deep learning-based method for catheter tracking and pose detection. Our method uses a Yolov5 bounding box neural network with postprocessing to perform landmark detection in four regions of the catheter: the tip, radio-opaque marker, bend, and entry point. This allows us to track the catheter’s position and orientation in real time, without the need for additional masking or segmentation techniques. We evaluated our method on a dataset of fluoroscopic images from two distinct datasets and achieved state-of-the-art results in terms of accuracy and robustness. Our model was able to detect all four landmark features (tip, marker, bend, and entry) used to generate a pose for a catheter with 0.285 ± 0.143 mm, 0.261 ± 0.138 mm, 0.424 ± 0.361 mm, and 0.235 ± 0.085 mm accuracy. We believe that our method has the potential to significantly improve the accuracy and efficiency of catheter tracking in medical procedures that utilize bi-plane fluoroscopy guidance.

Compaction quality assessment of highway using roller-integrated positive maximum kinetic energy increment detection technique

During conventional highway construction, quality control and acceptance (QC/QA) are based on limited spot tests of material density at random locations which may not comprehensively represent the entire compacted area and are vulnerable to potential biases. Based on roller-integrated positive maximum kinetic energy increment detection technique with real-time kinematic global positioning systems, this research adopted kinetic energy compaction value (KECV) as a real-time monitoring index for the highway compaction quality, and subsequently proposed an KECV-based assessment method to estimate the compaction quality of subgrade and pavement materials. Additionally, a global interpolation method (Green spline) was adopted to obtain estimates for both KECVs and compaction quality at any location on the work area, enabling the analysis of the passing rate of compaction quality for the entire work area. A case study on the Hengyong highway project in China indicates a highly linear correlation between the KECV and the compaction parameters, the compactness of LLL silt, and the relative density of cement-stabilized macadam; therefore, it can serve as a reliable index for monitoring compaction quality. Assessment of compaction quality on the entire work area can be fast and continuously achieved using the proposed method, which can effectively overcome the above limitation of conventional testing, timely feedback compaction information to avoid quality defects, and availably improve the construction quality of highways.

Real-time position and pose prediction for a self-propelled undulatory swimmer in 3D space with artificial lateral line system

This study aims to investigate the feasibility of using an artificial lateral line (ALL) system for predicting the real-time position and pose of an undulating swimmer with Carangiform swimming patterns. We established a 3D computational fluid dynamics simulation to replicate the swimming dynamics of a freely swimming mackerel under various motion parameters, calculating the corresponding pressure fields. Using the simulated lateral line data, we trained an artificial neural network to predict the centroid coordinates and orientation of the swimmer. A comprehensive analysis was further conducted to explore the impact of sensor quantity, distribution, noise amplitude and sampling intervals of the ALL array on predicting performance. Additionally, to quantitatively assess the reliability of the localization network, we trained another neural network to evaluate error magnitudes for different input signals. These findings provide valuable insights for guiding future research on mutual sensing and schooling in underwater robotic fish.

Positioning of a lunar surface rover on the south pole using LCNS and DEMs

With the renewed interest in the lunar surface exploration, the European Space Agency is supporting, as part of the Moonlight program, the development of a dedicated Lunar Communications and Navigation Service (LCNS) infrastructure. This should enable to achieve real time positioning and autonomous navigation in the cislunar space, including, among others, lunar landing and surface operations. One of the services proposed, as part of the LunaNet framework (NASA, 2022), is called the Lunar Augmented Navigation Service (LANS) and it is based on a GNSS-like concept. However, compared to Earth-based GNSS, the expected number of available satellites in lunar orbit will initially be limited, which triggers the interest of coupling these technologies at the receiver level with other navigation data sources. In the case of surface rovers, use of digital elevation model (DEM) data has shown to be a potential other source of information to support positioning. The purpose of this contribution is to look at the actual navigation performances achievable when using a simple (four satellites constellation) lunar radio navigation constellation combined with a DEM, through a state estimation filter, for different rover dynamic and different DEM accuracy levels. It is shown that a 5 meter real-time horizontal position accuracy can be achieved, for 99.7% of the time, using a precise enough DEM (5 m/pixel) and a 4 satellite-constellation under different rover’s dynamic conditions (i.e. velocities).

Survey of Techniques and Application in AR Integrated Home Furniture App

Abstract: This paper focuses on combining Augmented Reality technology with furniture stores to give customers the same shopping experience from the comfort of their homes. The mobile application uses the power of AR to allow users to place and view furniture in their positions in real time. Through a smartphone camera or tablet, users can see that there are many products available to add to their existing decor, such as a sofa, table, or chair, allowing them to make a purchasing decision. This AR furniture application not only simplifies the purchasing process, but also increases customer satisfaction by reducing the guesswork in the furniture purchasing process, and finally transforms the traditional rules of interior design into a conversational, interactive, and personal one. This paper studies the methodology and implementation of the AR app.

A low-cost, open-source-based optical surgical navigation system using stereoscopic vision

AbstractComputer-assisted surgical navigation systems have gained popularity in surgical procedures that demand high amounts of precision. These systems aim to track the real-time positioning of surgical instruments in relation to anatomical structures. Typically, state-of-the-art methods involve tracking reflective 3D marker spheres affixed to both surgical instruments and patient anatomies with infrared cameras. However, these setups are expensive and financially impractical for small healthcare facilities. This study suggests that a fully optical navigation approach utilizing low-cost, off-the-shelf parts may become a viable alternative. We develop a stereoscopic camera setup, costing around $120, to track and monitor the translational movement of open-source based fiducial markers on a positioning platform. We evaluate the camera setup based on its reliability and accuracy. Using the optimal set of parameters, we were able to produce a root mean square error of 2 mm. These results demonstrate the feasibility of real-time, cost-effective surgical navigation using off-the-shelf optical cameras.

Correlation between rate of TEC index and positioning error during solar flares and geomagnetic storms using navigation with Indian constellation receiver measurements

Abstract The real-time position accuracy of the Global Navigation Satellite System (GNSS)/Navigation with Indian Constellation (NavIC) receiver is limited by the dynamic behavior of the ionosphere, particularly in adverse conditions like solar flares and geomagnetic storms. The NavIC satellites broadcast dual coherent radio beacon signals on L5 (1,164.5 MHz) and S (2,472.5 MHz) bands for providing position, velocity, and timing services in all weather conditions. The Total Electron Content (TEC) and Rate of TEC Index (ROTI) are the potential indicators for characterizing the ionosphere and its irregularities. In this research work, the TEC and ROTI are computed from the code and carrier phase observations of the NavIC receiver located at Kurnool low latitude station (15.79° N, 78.07° E) with geomagnetic coordinates (7.30° N, 151.65° E). This paper presents a statistical study of TEC, ROTI, and the correlation between ROTI and NavIC positioning error during highly intense solar flares (X9.3 and X2.2) and geomagnetic storm conditions. Compared to quiet days mean TEC, the enhancement is 3 TECU due to X9.3 flares, and the maximum peak of TEC on storm day (September 8, 2017) is 80.92 TECU. Moreover, the correlation coefficient between ROTI and position error is 0.76 on a quiet day (September 4, 2017), 0.54 on an intense solar flares day (September 6, 2017), and 0.24 on a storm day (September 8, 2017), this indicates positional accuracy degradation on a geomagnetic storm day. The outcome of this research work would be helpful for investigating characteristics of the northern low latitude ionospheric irregularities and, in turn, useful for developing suitable ionospheric nowcasting/prediction models for GNSS applications.

Online Bilateral Predictive Control for Time-Delayed Teleoperation of Snake-like Robots

Controlling the teleoperation of snake-like robots is challenging due to complex nonlinear dynamics and communication delays. This research proposes an online bilateral predictive control architecture to address these issues. This control structure is established by predicting environment force and the user’s future motion. The former uses a model-mediated approach by creating a virtual environment on the master side and the latter adopts an artificial neural network (ANN) for online operator’s motion prediction. The slave controller utilizes transmitted data from ANN to generate required backbone lengths, which are then transformed into the slave's local bending and torsional degrees of freedom through the inverse kinematics of the robot. Motion prediction is examined in two scenarios: when the ANN predicts the trained motions, and when it predicts a different motion. Simulation studies demonstrate that the proposed online bilateral predictive teleoperation structure successfully achieves real-time position synchronization and force feedback, by effectively bypassing communication delays.

Simulation of performance skills teaching system based on machine vision and sensor audio signal processing

In the traditional network media education system, there is little content about skill performance, and the malleability is poor. With the combination of Internet technology, online teaching videos of performance skills began to be promoted on a large scale, and with the help of the Internet, the learning of online performance skills has been updated. The aim of this study is to develop a performance skills teaching system based on machine vision and sensor audio signal processing technology to provide a simulated performance environment and personalized instruction. In this study, multiple sensors were used to capture various movement data during the performance of students. Through the use of sensor technology, the system is able to capture information about students' posture, finger position and force in real time, and analyze posture and movement in combination with machine vision technology. The system can also process and analyze the audio signals in the students' performance to evaluate the accuracy and fluency of the playing technique. The application of sensor technology enables the system to accurately capture students' movements and provide timely feedback. The use of machine vision technology allows the system to accurately analyze the student's posture and movements to provide more targeted guidance. The application of audio signal processing algorithm enables the system to objectively evaluate the students' playing skills and help them improve the accuracy and fluency in the playing process.

Research on Locating and Searching for Missing Submersibles Based on Differential Equations and SAR-A* Algorithm

In recent years, the application of submersibles has become increasingly widespread. The current focus of research is on real-time and accurate positioning of submersibles to cope with potential emergencies. This paper aims to establish a trajectory model for faulty submersibles. Firstly, a differential equation model for submersible dynamics is established, numerical solutions are obtained and fitted to determine the exponential relationship between velocity and time, resulting in a theoretical formula for submersible displacement. Subsequently, based on the trajectory, a probability model for the presence of targets is established using a two-dimensional Gaussian distribution. The SAR-A* algorithm is employed for regional search, combined with a greedy algorithm to obtain initial deployment points for search and rescue, thus deriving the search and rescue trajectory. This study provides theoretical support for locating and rescuing missing submersibles and is expected to play an important role in future submersible applications.

Crowdsourcing RTK: a new GNSS positioning framework for building spatial high-resolution atmospheric maps based on massive vehicle GNSS data

High-quality spatial atmospheric delay correction information is essential for achieving fast integer ambiguity resolution (AR) in precise positioning. However, traditional real-time precise positioning frameworks (i.e., NRTK and PPP-RTK) depend on spatial low-resolution atmospheric delay correction through the expensive and sparsely distributed CORS network. This results in limited public appeal. With the mass production of autonomous driving vehicles, more cost-effective and widespread data sources can be explored to create spatial high-resolution atmospheric maps. In this study, we propose a new GNSS positioning framework that relies on dual base stations, massive vehicle GNSS data, and crowdsourced atmospheric delay correction maps (CAM). The map is easily produced and updated by vehicles equipped with GNSS receivers in a crowd-sourced way. Specifically, the map consists of between-station single-differenced ionospheric and tropospheric delays. We introduce the whole framework of CAM initialization for individual vehicles, on-cloud CAM maintenance, and CAM-augmented user-end positioning. The map data are collected and preprocessed in vehicles. Then, the crowdsourced data are uploaded to a cloud server. The massive data from multiple vehicles are merged in the cloud to update the CAM in time. Finally, the CAM will augment the user positioning performance. This framework forms a beneficial cycle where the CAM’s spatial resolution and the user positioning performance mutually improve each other. We validate the performance of the proposed framework in real-world experiments and the applied potency at different spatial scales. We highlight that this framework is a reliable and practical positioning solution that meets the requirements of ubiquitous high-precision positioning.

Research on the drone endoscopic system for fusion reactors

This paper proposes an air-ground separation drone endoscopic system (DES) design scheme for the fusion reactor internal inspection task. A foldable rotor drone with vertical takeoff and landing capability is used as the main body of DES. The nest platform is designed separately from the drone to reduce the weight. Meanwhile, the nest platform can be installed with a safety cable and 2D code to realize emergency rescue and precise take-off and landing. DES adds 3D lidar sensors for 360° stereo sensing, realizing real-time positioning in dark or even lightless environments in vacuum chambers. The first-generation DES prototype is developed, and the flight test is performed inside the PF6 superconducting coil shell by fusing 3D lidar point cloud data with visual camera image data. The results show that the position error of DES is stabilized within ±0.05 m during the hovering phase. The error is mainly due to the narrow space inside the PF6 coil shell, which produces a perturbed flow field on the DES. In general, the DES accomplished circumferential autonomous flight control and 3D point cloud data acquisition in a confined space, proving the feasibility of drone inspection in vacuum chambers.

Development of a combined harvester navigation control system based on visual simultaneous localization and mapping-inertial guidance fusion

Recently, the existing unmanned systems of combine harvesters mostly adopts satellite navigation scheme, lacking real-time observation of harvesting adjustment. To improve the operational efficiency of combine harvester assisted navigation operation, this paper designs a combine harvester navigation control system based on vision simultaneous localization and mapping (SLAM)-inertial guidance fusion. The system acquires field image information and extracts the crop boundary line as the navigation datum by binocular camera. First, the system acquires field image information through binocular camera and extracts the crop boundary line as the navigation datum. Second, fusing camera and inertial guidance information to obtain the real-time relative position of a combine harvester. Third, constrained optimization of image and inertial guidance information is achieved through a sliding window optimization method based on tightly coupled nonlinear optimization. Finally, obtain the position of the combine harvester relative to the navigation datum line, and output a signal to the steering mechanism to realize the combine harvester in the field intelligent positioning navigation control. The system consists of binocular camera, inertial measurement unit, motorized steering wheel, monitor display, angle sensor and microcontroller. During field testing, the system underwent repetitive harvesting trials over a distance of 25 m.. The testing machine performs field operations at a speed of 0.9-1.5 m/s, with an average lateral deviation range of 2.21-8.62 cm, a standard deviation range of 0.13-4.21 cm and an average cutting rate range of 92.2%-96.0%, achieving the expected harvesting effect.