Real-time Trajectory Research Articles

Distributed short-term predictive control for AUV clusters in underwater cooperative hunting tasks

A distributed short-term predictive control method is proposed to be applied to autonomous underwater vehicle (AUV) clusters for underwater cooperative hunting tasks. To support AUV clusters in unknown underwater dynamic environments, the constraints of information acquisition, AUV motion constraints, collision avoidance, and smoothness requirements of motion paths in the tasks are considered. First, a distributed multi-constraint optimization model including target attraction, interactions between AUVs, continuous obstacle avoidance, and control effort is proposed. Effective information between AUVs is extended under limited information conditions. Second, each AUV solves the distributed optimization problem and determines the desired trajectory in real time using information from the prediction horizon. In the simulation experiments, the proposed method is demonstrated to provide at least 12.19%, 11.08%, and 79.89% improvement in terms of task completion time, energy consumption, and path smoothing, respectively, when compared to other methods under the same information conditions. The robustness of the proposed method under different noise conditions is verified by positioning noise experiments, and the task can still be accomplished even if the noise level reaches 75% of the safe distance. The adaptability of the proposed method is demonstrated by simulations in different environments.

Feasibility of stereotactic optical navigation for needle positioning in percutaneous nephrolithotomy

BackgroundThis study assessed the feasibility of acquiring single-attempt access to the pelvicalyceal system during percutaneous nephrolithotomy (PCNL) using stereotactic optical navigation combined with cone-beam CT (CBCT) imaging.MethodsPatients with a PCNL indication were prospectively included in this IRB approved study. After sterile preparation, fiducial markers were attached to patients’ skin. An initial intraprocedural CBCT scan was acquired, on which the urologist planned the needle trajectory using the navigation software. After verifying that no critical structures were crossed, the needle guide was aligned with the plan. A needle was manually inserted through the needle guide to the indicated depth and a second CBCT scan was performed for needle position confirmation. Both, scanning and needle insertion, were performed under apnea. The study evaluated technical success, accuracy, procedure time, complication rate, and radiation dose.ResultsBetween June 2022 and April 2023, seven patients were included. In all patients, the navigation system allowed safe puncture. However, the technical success rate was only 29%. In 42% of the cases, pelvicalyceal access was achieved by a small manual adjustment. In the remaining 29%, the needle was retracted and positioned per clinical standard. The average deviation between the needle and target was 5.9 ± 2.3 mm. The average total procedure time was 211 ± 44 min. The average radiation exposure was 6.4 mSv, with CBCT scanning contributing to 82% of this exposure.ConclusionsThe optical navigation system facilitated safe needle insertion but did not consistently ensure accurate one-attempt needle positioning for PCNL. Real-time visualization and trajectory correction may improve the technical success rate.

Bio-Inspired Neural Network for Real-Time Evasion of Multi-Robot Systems in Dynamic Environments.

In complex and dynamic environments, traditional pursuit-evasion studies may face challenges in offering effective solutions to sudden environmental changes. In this paper, a bio-inspired neural network (BINN) is proposed that approximates a pursuit-evasion game from a neurodynamic perspective instead of formulating the problem as a differential game. The BINN is topologically organized to represent the environment with only local connections. The dynamics of neural activity, characterized by the neurodynamic shunting model, enable the generation of real-time evasive trajectories with moving or sudden-change obstacles. Several simulation and experimental results indicate that the proposed approach is effective and efficient in complex and dynamic environments.

Real-Time UAV Trajectory Prediction for UTM Surveillance Using Machine Learning

The Unmanned Aerial System (UAS) Traffic Management (UTM) surveillance plays an important role in monitoring the safety compliance of UAV flights within specific operational zones. However, the quality of data reception from UAVs depends on transmission signal quality, leading to variable data latency during flights. As a means of addressing this issue and improving UAV operational safety, trajectory prediction emerges as a promising solution. This study aims to implement real-time trajectory prediction through the integration of machine learning techniques within the UTM surveillance system using Broadcast Remote ID. To facilitate this, a homemade receiver for broadcast Remote ID is developed using the ESP32 microcontroller. Subsequently, two distinctive flight tests are executed using a DJI Phantom 4 UAV. In the first flight, UAV trajectory data are used to serve as training input for the machine learning algorithms. The second flight focuses on the implementation of real-time trajectory prediction. The trajectory prediction model uses inputs such as latitude, longitude, height, speed, direction, latency time, and the Received Signal Strength Indicator (RSSI). Through an analysis of the first flight’s offline data, the Gated Recurrent Unit (GRU) algorithm is preferred to the Long Short-Term Memory (LSTM) algorithm. The outcomes of the second flight test prove the feasibility of the GRU-based trajectory prediction. The GRU model successfully produces real-time predictions during UAV flight, showcasing accuracy levels similar to those derived from offline prediction analyses with short processing time. This real-time trajectory prediction could improve the safety of UAV operation by providing the estimated position when the latency time is higher than the threshold.

Warhead fragments motion trajectories tracking and spatio-temporal distribution reconstruction method based on high-speed stereo photography

High speed photography technique is potentially the most effective way to measure the motion parameter of warhead fragment benefiting from its advantages of high accuracy, high resolution and high efficiency. However, it faces challenge in dense objects tracking and 3D trajectories reconstruction due to the characteristics of small size and dense distribution of fragment swarm. To address these challenges, this work presents a warhead fragments motion trajectories tracking and spatio-temporal distribution reconstruction method based on high-speed stereo photography. Firstly, background difference algorithm is utilized to extract the center and area of each fragment in the image sequence. Subsequently, a multi-object tracking (MOT) algorithm using Kalman filtering and Hungarian optimal assignment is developed to realize real-time and robust trajectories tracking of fragment swarm. To reconstruct 3D motion trajectories, a global stereo trajectories matching strategy is presented, which takes advantages of epipolar constraint and continuity constraint to correctly retrieve stereo correspondence followed by 3D trajectories refinement using polynomial fitting. Finally, the simulation and experimental results demonstrate that the proposed method can accurately track the motion trajectories and reconstruct the spatio-temporal distribution of 1.0×103 fragments in a field of view (FOV) of 3.2 m×2.5 m, and the accuracy of the velocity estimation can achieve 98.6%.

Event-Triggered Deep Learning Control of Quadrotors for Trajectory Tracking

This paper proposes an event-triggered deep learning control strategy to achieve real-time trajectory tracking control for quadrotors. In the training data collection phase, the event-triggered model predictive control (MPC) method is applied to the quadrotor in the simulation environment to generate training data. Then, a deep neural network (DNN) controller is trained to approximate the optimal control policy of the event-triggered MPC. To further save computing resources of on-board processor, the event-triggered mechanism is incorporated with the DNN controller, and the dual-mode approach is employed in it. Finally, simulation and experimental results show that the proposed controller can ensure almost similar trajectory tracking performance to the event-triggered MPC controller while requiring a lower control computation cost.

A time-varying driving style oriented model predictive control for smoothing mixed traffic flow

Connected and automated vehicles (CAVs) have great potential to smooth mixed traffic flow. To focus on the fact that the driving styles of HDVs are time-varying, a new control framework based on time-varying model predictive control (MPC) is proposed for mixed traffic flow in longitudinal control. Firstly, a genetic algorithm can be employed to calibrate the parameters of the car-following model based on offline trajectory data, representing the long-term driving style of HDVs. Secondly, the short-term driving style is recognized through modifying the long-term driving style based on real-time trajectory data within a sliding window. Then, this paper seeks to design a time-varying MPC control framework to realize a smoother mixed traffic flow. Finally, compared to the baseline model, the method we proposed achieves the high-performance both in fuel consumption and driving comfort under various scenarios, and maintains fast online calculations. Besides, it also displays the significant potential of saving fuel consumption in a large-scale traffic system, reducing by over 11.28% with 17∼33% penetration rates of CAVs.

Optimized Real-Time Tracking Radar Trajectory Simulation with Multi-Threaded Data Transmission and Visualization

Optimized Real-Time Tracking Radar Trajectory Simulation with Multi-Threaded Data Transmission and Visualization

Laser Rangefinder Methods: Autonomous-Vehicle Trajectory Control in Horticultural Plantings

This article presents a developed motion control system for a robotic platform based on laser-ranging methods, a graph traversal algorithm and the search for the optimal path. The algorithm was implemented in an agricultural building and in the field. As a result, the most efficient algorithm for finding the optimal path (A*) for the robotic platform was chosen when performing various technological operations. In the Rviz visualization environment, a program code was developed for planning the movement path and setting the points of the movement trajectory in real time. To find the optimal navigation graph in an artificial garden, an application was developed using the C# programming language and Visual Studio 2019. The results of the experiments showed that field conditions can differ significantly from laboratory conditions, while the positioning accuracy is significantly lower. The statistical processing of the experimental data showed that, for the movement of a robotic platform along a given trajectory in the field, the most effective conditions are as follows: speed: 2.5 km/h; illumination: 109,600 lux; distance to the tree: 0.5 m. An analysis of the operating parameters of the LiDAR sensor showed that it provides a high degree of positioning accuracy under various lighting conditions at various speeds in the aisles of a garden 3 m wide with an inter-stem distance of 1.5 m and a tree crown width of 0.5 m. The use of sensors—rangefinders of the optical range—allows for the performance of positional movements of the robotic platform and ensures the autonomous performance of the basic technological operations of the units in intensive gardens with a deviation from the specified trajectory of no more than 8.4 cm, which meets the agrotechnical requirements.

A comparative study of EM-CCD and CMOS cameras for particle ion trajectory imaging

High-resolution and real-time imaging of particle ion trajectories is essential in nuclear medicine and nuclear engineering. One potential method to achieve high-resolution real-time trajectory imaging of particle ions involves utilizing an imaging system that integrates a scintillator plate with a magnifying unit and a cooled electron multiplying charge-coupled device (EM-CCD) camera. However, acquiring an EM-CCD camera might prove challenging due to the discontinuation of CCD sensor manufacturing by vendors. As an alternative imaging approach, a low-noise, high-sensitivity camera utilizing a cooled complementary metal-oxide-semiconductor (CMOS) sensor offers a promising solution for imaging particle ion trajectories. Yet, it remains uncertain whether CMOS-based cameras can perform as effectively as CCD-based cameras in capturing particle ion trajectories. To address these concerns, we conducted a comparative analysis of the imaging performance between a CMOS-based system and an EM-CCD-based system for capturing alpha particle trajectories. The results revealed that both systems could image the trajectories of alpha particle, but the spatial resolution with the CMOS-based camera exceeded that of the EM-CCD-based camera, primarily due to the smaller pixel size of the sensor. While the signal-to-noise ratio (SNR) of the trajectory image from the CMOS-based camera initially lagged behind that from the EM-CCD-based camera, this disparity was mitigated by implementing binning techniques on the CMOS-based camera images. In conclusion, our findings suggest that a cooled CMOS camera could serve as a viable alternative for imaging particle ion trajectories.

Computer Vision-Based Real-Time Identification of Vehicle Loads for Structural Health Monitoring of Bridges

Coupling effects of various loading conditions can cause deflections, settlements and even failure of in-service bridges. Although it is one of the most critical loads, unfortunately, loading conditions of moving vehicles are difficult to capture in real time by bridge monitoring systems currently in place for sustainable operation. To fully understand the status of a bridge, it is essential to obtain instantaneous vehicle load distributions in a dynamic traffic environment. Although there are some methods that can identify overweight vehicles, the captured vehicle-related information is scattered and incomplete and thus cannot support effective bridge structural health monitoring (BSHM). This study proposes a noncontact, vision-based approach to identification of vehicle loads for real-time monitoring of bridge structural health. The proposed method consists of four major steps: (1) establish a dual-object detection model for vehicles using YOLOv7, (2) develop a hybrid coordinate transformation model on a bridge desk, (3) develop a multiobject tracking model for real-time trajectory monitoring of moving vehicles, and (4) establish a decision-level fusion model for fusing data on vehicle loads and positions. The proposed method effectively visualizes the 3D spatiotemporal vehicular-load distribution with low delay at a speed of over 30FPS. The results show that the hybrid coordinate transformation ensures that the vehicle position error is within 1 m, a 5-fold reduction compared with the traditional method. Wheelbase is calculated through dual-object detection and transformation and is as the primary reference for vehicle position correction. The trajectory and real-time speed of vehicles are preserved, and the smoothed speed error is under 5.7%, compared with the speed measured by sensors. The authors envision that the proposed method could constitute a new approach for conducting real-time SHM of in-service bridges.

A walking energy harvesting device based on miniature water turbine

The rapid development of wearable electronics highlights the urgence to develop the portable energy harvester with excellent output performance, comfortability, and sustainability. This work designs an electromagnetic walking energy harvester based on water turbine that can be embedded in shoes with good comfortability. Its working principle is that the walking generated pressure energy drives a miniature hydraulic turbine to output electricity. Experimental results show that an average power of 300 and 180 mW can be produced at heel and toe, respectively, when a man of 80 kg walks at a speed of 1.8 m s−1. This power output exceeds the piezoelectric, triboelectric, and electromagnetic walking energy harvesters reported in the past. Additionally, the simpler structure endows it better comfortability as compared with the electrostatic capacitances. Computational fluid dynamics simulations provide a further insight that the efficiency of turbine can reach 13.5% by optimizing parameters of blade number and outlet flow ratio. Finally, user real-time positioning and trajectory recording are successfully demonstrated via a wearable GPS means Global Positioning System module powered by the harvester. Due to the combination of high output performance, simple structure and low discomfort, the water turbine based walking energy harvester will provide a wide application potential in wearable devices.

Video-Based Identification and Prediction Techniques for Stable Vessel Trajectories in Bridge Areas.

In recent years, the global upswing in vessel-bridge collisions underscores the vital need for robust vessel track identification in accident prevention. Contemporary vessel trajectory identification strategies often integrate target detection with trajectory tracking algorithms, employing models like YOLO integrated with DeepSORT or Bytetrack algorithms. However, the accuracy of these methods relies on target detection outcomes and the imprecise boundary acquisition method results in erroneous vessel trajectory identification and tracking, leading to both false positives and missed detections. This paper introduces a novel vessel trajectory identification framework. The Co-tracker, a long-term sequence multi-feature-point tracking method, accurately tracks vessel trajectories by statistically calculating the translation and heading angle transformation of feature point clusters, mitigating the impact of inaccurate vessel target detection. Subsequently, vessel trajectories are predicted using a combination of Long Short-Term Memory (LSTM) and a Graph Attention Neural Network (GAT) to facilitate anomaly vessel trajectory warnings, ensuring precise predictions for vessel groups. Compared to prevalent algorithms like YOLO integrated with DeepSORT, our proposed method exhibits superior accuracy and captures crucial heading angle features. Importantly, it effectively mitigates the common issues of false positives and false negatives in detection and tracking tasks. Applied in the Three Rivers area of Ningbo, this research provides real-time vessel group trajectories and trajectory predictions. When the predicted trajectory suggests potential entry into a restricted zone, the system issues timely audiovisual warnings, enhancing real-time alert functionality. This framework markedly improves vessel traffic management efficiency, diminishes collision risks, and ensures secure navigation in multi-target and wide-area vessel scenarios.

Short-term voltage instability prediction using pre-identified voltage templates and machine learning classifiers

System operators use different procedures to detect disturbances which can lead to short-term voltage instability. These schemes are designed using the domain knowledge and generally specific for the corresponding system. The design procedure of such schemes is heuristic and cannot be directly applied to any other power system. This paper proposes a data-driven approach to predict short-term voltage stability using set of post-disturbance voltage templates and a machine learning-based classifier. In this scheme, a set of voltage templates corresponding to stable/unstable events is identified for selected buses using a suitably generated training data set. The proximity of real-time voltage trajectory to each of the template is computed and input to a trained machine learning classifier to predict the short-term voltage stability status. The proposed approach is validated using IEEE Nordic test system. The investigations show that this method provides accurate short-term voltage stability predictions. Furthermore, impact of adverse phenomena such as measurement noise and missing data on the proposed method is analyzed to confirm the robustness of this scheme in particle environments. © 2017 Elsevier Inc. All rights reserved.

Kinetic analysis and numerical tests of an adaptive car-following model for real-time traffic in ITS

Abundant real-time vehicle trajectory information provides an important guarantee for the driving safety and drivers’ decision-making in the intelligent transportation system (ITS), which improves the transport efficiency of traffic flow at the micro level. In this paper, we propose an adaptive car-following model to study the influence of the preceding vehicles’ information on the motion state of the target and the kinetic characteristics of traffic flow in the ITS, where the space headway, the velocity difference and the acceleration of preceding vehicles are considered simultaneously. According to the generalized diagram structure of road traffic, we mainly consider the two vehicles ahead and introduce adaptive parameters to control the proportion of different influencing factors. The rationality of the extended model is verified via real-world vehicle trajectory fitting experiments with the collected empirical data. Theoretical derivations are then carried out to describe the evolution of traffic flow under different scenarios, where the linear stability conditions and nonlinear analysis on small disturbance are included. Numerical tests are made to simulate the driving conditions and verify the accuracy of theoretical results. Computing the engine power of vehicles, we summarize the impact of traffic congestions and stop-and-go behaviors on the energy consumption. Diverse micro motion states and traffic flow dynamics can be revealed based on our car-following model, which will serve as both theoretical basis and experimental examples for the real-world traffic.

Neural Optimal Control for Constrained Visual Servoing via Learning From Demonstration

This paper proposes a novel optimal control scheme for constrained image based visual servoing of a robot manipulator. For a robot manipulator with an eye-on-hand configuration, visibility constraint is an essential requirement to avoid servo failure, while robot’s actuator limits must also be satisfied. To ensure this, the constraints are modelled implicitly via learning the task and defining safe regions using expert human demonstrations via mixture of Dynamic Movement Primitives (DMPs). The visual servoing problem is then formulated as a closed-loop optimal control problem using these constraint model where a desired target (possibly time-varying) is obtained by acting upon the feedback from the real-time visual sensors. The visual servo control loop consists of a single network adaptive critic optimal tracking control scheme whose weights are tuned using Lyapunov stability criteria. The stability and the performance of the proposed control scheme is shown theoretically via Lyapunov approach and also verified experimentally using a seven degree of freedom (DOF) Franka Emika and six DOF Universal Robot (UR) 10 manipulator. The approach is also demonstrated on a use case scenarios in mock-up convenience store and warehouse setup. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The applications of robots in busy warehouses, healthcare sectors and convenience store, has a big societal impact. The scenarios in these real-world problems often consist of a dynamic environment. Therefore, sensor-based localization and planning, along with satisfying robot and task constraints are needed. These naturally adds up to the programming costs in addition to the robot platform and actuation. However, modern robots need intuitive and easy programming for more pervasive in a practical societal application. In our proposed method, this sensor-based planning with environmental constraints is modelled implicitly using the Programming by Demonstration framework. The user needs to catch the robot arm by his hand and teach the task at hand, and the framework captures the task and robot constraints while generalizing to new goals. This modelling, along with cost optimal controller, generates real-time constraint aware robot manipulation trajectories for the demonstrated task in dynamic environments.

Differential Game-Based Deep Reinforcement Learning in Underwater Target Hunting Task.

To meet requirements for real-time trajectory scheduling and distributed coordination, underwater target hunting task is challenging in terms of turbulent ocean environments and dynamic adversarial environment. Despite the existing research in game-based target hunting area, few approaches have considered dynamic environmental factors, such as sea currents, winds, and communication delay. In this article, we focus on a target hunting system consisted of multiple unmanned underwater vehicles (UUVs) and a target with high maneuverability. Besides, differential game theory is leveraged to analyze adversarial behaviors between hunters and the escapee. However, it is intractable that UUVs have to deploy an adaptive scheme to guarantee the consistency and avoid the escape of the target without collision. Therefore, we conceive the Hamiltonian function with Leibniz's formula to obtain feedback control policies. In addition, it proves that the target hunting system is asymptotically stable in the mean, and the system can satisfy Nash equilibrium relying on the proposed control policies. Furthermore, we design a modified multiagent reinforcement learning (MARL) to facilitate the underwater target hunting task under the constraints of energetic flows and acoustic propagation delay. Simulation results show that the proposed scheme is superior to the typical MARL algorithm in terms of reward and success rate.

Discrete-Target Prosthesis Control Using Uncertainty-Aware Classification for Smooth and Efficient Gross Arm Movement.

Current control approaches for gross prosthetic arm movement mainly regulate movement over a continuous range of target poses. However, these methods suffer from output fluctuation caused by input signal variations during gross arm movements. Prosthesis control approaches with a finite number of discrete target poses can address this issue and reduce the complexity of the pose control process. However, it remains under-explored in the literature and suffers from the consequences of misclassifying the target poses. Here, we propose a novel Uncertainty-Aware Discrete-Target Prosthesis Control (UA-DPC) approach. This approach consists of (1) an uncertainty-aware classification scheme to reduce unintended pose switches caused by misclassifications, and (2) real-time trajectory planning that adjusts motion to be rapid or conservative based on low or high quantified uncertainty, respectively. By addressing the impact of misclassification, this approach facilitates more efficient and smooth movements. Human-in-the-loop experiments were conducted in a virtual reality environment with 12 non-disabled participants. The participants controlled a transhumeral prosthesis using three approaches: the proposed UA-DPC, a discrete-target approach based on a traditional off-the-shelf classifier, and a continuous-target approach. The results demonstrate the superior performance of UA-DPC, which provides more efficient task completion with fewer misclassification instances as well as smoother residual limb and prosthesis movement.

Real-time Trajectory Planning and Tracking Control of Bionic Underwater Robot in Dynamic Environment.

In this article, we study the trajectory planning and tracking control of a bionic underwater robot under multiple dynamic obstacles. We first introduce the design of the bionic leopard cabinet underwater robot developed in our lab. Then, we model the trajectory planning problem of the bionic underwater robot by combining its dynamics and physical constraints. Furthermore, we conduct global trajectory planning for bionic underwater robots based on the temporal-spatial Bezier curves. In addition, based on the improved proximal policy optimization, local dynamic obstacle avoidance trajectory replanning is carried out. In addition, we design the fuzzy proportional-integral-derivative controller for tracking control of the planned trajectory. Finally, the effectiveness of the real-time trajectory planning and tracking control method is verified by comparative simulation in dynamic environment and semiphysical simulation of UWSim. Among them, the real-time trajectory planning method has advantages in trajectory length, trajectory smoothness, and planning time. The error of trajectory tracking control method is controlled around 0.2 m.

Real-Time Trajectory Planning and Obstacle Avoidance for Human–Robot Co-Transporting

Real-Time Trajectory Planning and Obstacle Avoidance for Human–Robot Co-Transporting