3D Trajectory Research Articles

A computer vision approach to monitor activity in commercial broiler chickens using trajectory-based clustering analysis

To monitor changes in broiler behaviour related to animal health and welfare, farmers typically observe their flocks using manual observation. However, due to the labour intensive and continuous aspect of this task, the analysis of broiler behaviour could be automated using camera technology. This paper proposes a proof-of-concept camera surveillance system based on the automated detection and tracking of broilers to monitor activity bouts using unsupervised 2D trajectory clustering. Firstly, a convolutional neural network-based detector was trained and tested on our labelled dataset which resulted in a precision, recall and f score of 0.98, 0.90 and 0.94, respectively. Using a tracking-by-detection approach, the proposed system was able to track chickens across video frames with a multi-object tracking accuracy of 74.7%. A component-based feature saliency Gaussian mixture model (CFSGMM) was subsequently generated and applied to objectively cluster the trajectories based on their spatiotemporal information. Nineteen features were extracted from the trajectories, representing both static and dynamic characteristics of broiler movement, and three activity classes were identified: ‘least active/resting,’ ‘active’ and ‘highly-active.’ The proposed method was validated on one-minute monocular video sequences. CFSGMM was applied to cluster 2D trajectories relating to broiler activity bouts within the commercial rearing environment with an agreement ranging from 6.0 to 99.7% when compared to human observation. We demonstrate the potential of the computer vision system to monitor overt, short-term changes in broiler activity associated with on-farm events and discuss the opportunities of leveraging the technology to monitor longer term changes in welfare state. It is anticipated that further development of the detection and tracking systems will improve the performance of the trajectory clustering method.

Optimization for UAV-assisted simultaneous transmission and reception communications in the existence of malicious jammers

In this paper, we study an unmanned aerial vehicle (UAV)-assisted communication system, where the UAV is dispatched to implement simultaneous transmission and reception (STR) in the existence of multiple malicious jammers. Two schemes are investigated, namely frequency band-division-duplex (FDD) and time-fraction (TF). Based on the FDD scheme, the UAV can transmit information by using the portion of the bandwidth and receive information within the remaining portion of the bandwidth simultaneously. To perform the STR within the whole bandwidth, the TF-based scheme is considered by using a fraction of a time slot for the downlink, while the remaining fraction of the time slot is allocated for the uplink. We aim to maximize the worst-case throughput by optimizing the UAV three-dimensional (3D) trajectory and resource allocation for each scheme. The optimization problem is non-convex and thus computationally intractable. To handle the nonlinear problem, we use the block coordinate decomposition method to disaggregate the optimization problem into four subproblems and adopt the successive convex approximation technique to tackle non-convex problems. The simulation results demonstrate the performance of the TF-based scheme over the benchmark schemes.

Automated System for Continuous Green Shoot Thinning in Vineyards

Highlights An automated green shoot thinning system was developed by integrating machine vision and mechanical systems. Mask R-CNN used for segmenting cordons at different shoot growth stages. Point cloud registration used to accurately stitch cordon trajectories for continuous thinning operation. Field and lab tests demonstrated the integrated system can efficiently automate green shoot thinning operation. Abstract. Mechanized green shoot thinning in vineyards has helped reduce labor and associated costs. However, high variability in the thinning efficiency has been observed with the current machines in non-selective thinning due to challenges faced by the operators to precisely position thinning end-effectors along the cordon trajectories. In this study, an automated green shoot thinning system is proposed for cordon-trained grapevines to enhance the precision and efficiency of mechanical thinning operation. The automated system consisted of (1) a machine vision system for estimating cordon trajectories and (2) a mechanical system to position thinning rollers along the estimated trajectories during the real-time thinning operation. The vision pipeline used a Mask R-CNN model to determine cordon profiles at different shoot growth stages and a depth image to estimate the 3D cordon trajectory. In addition, point-cloud registration estimated transformation matrices for accurately stitching adjacent cordon trajectories and generating a long map of trajectory for the thinning rollers to follow. An image processing-based calibration method was used to obtain transformation parameters that transformed the estimated trajectories from the camera coordinate space to the thinning-roller coordinate space required for trajectory following. Lab tests, conducted using simulated cordons, showed the integrated system can precisely position the thinning rollers with root mean square errors (RMSE) of 2.0 cm and 1.7 cm for vertical and lateral positions, respectively. To quantify the thinning performance of the integrated system in the field, shoot thinning efficiency (the percentage of target shoots impacted) was determined. From the field evaluation, shoot thinning efficiency of 76% and 85% were achieved for two rows of grapevines, with an overall shoot thinning efficiency of 80%. The favorable test results demonstrated that the proposed integrated system is capable of effectively automating green shoot thinning operations, which has the potential to improve shoot thinning performance in vineyards. Keywords: Automated green shoot thinning, Cordon trajectory estimation, Mask R-CNN, Vineyard canopy management.

Human Pose Estimation for Clinical Analysis of Gait Pathologies.

Gait analysis serves as a critical diagnostic tool for identifying neurologic and musculoskeletal damage. Traditional manual analysis of motion data, however, is labor-intensive and heavily reliant on the expertise and judgment of the therapist. This study introduces a binary classification method for the quantitative assessment of gait impairments, specifically focusing on Duchenne muscular dystrophy (DMD), a prevalent and fatal neuromuscular genetic disorder. The research compares spatiotemporal and sagittal kinematic gait features derived from 2D and 3D human pose estimation trajectories against concurrently recorded 3D motion capture (MoCap) data from healthy children. The proposed model leverages a novel benchmark dataset, collected from YouTube and publicly available datasets of their typically developed peers, to extract time-distance variables (e.g. speed, step length, stride time, and cadence) and sagittal joint angles of the lower extremity (e.g. hip, knee, and knee flexion angles). Machine learning and deep learning techniques are employed to discern patterns that can identify children exhibiting DMD gait disturbances. While the current model is capable of distinguishing between healthy subjects and those with DMD, it does not specifically differentiate between DMD patients and patients with other gait impairments. Experimental results validate the efficacy of our cost-effective method, which relies on recorded RGB video, in detecting gait abnormalities, achieving a prediction accuracy of 96.2% for Support Vector Machine (SVM) and 97% for the deep network.

An investigation of methods to improve the accuracy of classical dance steps based on machine vision recognition technology

Abstract Methods that use motion capture data to analyze human behavior are highly interpretable and offer significant advantages in vision-based dance step analysis. In this study, we propose a fusion feature extraction method that describes the rotational information embedded in the skeleton and combines the advantages of single-skeleton feature extraction and Li group feature extraction methods for recognition. The feasibility and validity of the proposed model are verified by conducting recognition experiments on BVH data of classical dance steps performed by subjects. In the ablation experiments, the average accuracy of the classical dance step recognition model based on fused features for step movement recognition is improved by 3.01% and 1.86% compared with the neural network model and the Lie group network model, respectively. It has been proven that adding rotation information to skeletal features can effectively differentiate 3D motion trajectories in similar dance steps. Furthermore, the dance movement trajectories derived from this model are very clear and can be utilized to direct the correct joint point positions throughout the movement. The recognition accuracy of the fusion feature extraction-based recognition method for all seven classical dance basic foot positions is greater than 90%, which reflects the accuracy of the proposed machine vision model in recognizing classical dance steps.

Service Time Maximization for Data Collection in Multi-UAV-Aided Networks

Unmanned aerial vehicles (UAVs) have been enormously gaining attention to offload traffic or collect data in wireless networks due to their key attributes, such as mobility, flexibility, and cost-effective deployment. However, the limited onboard energy inhibits the UAV from serving for a longer duration. Therefore, this paper studies a UAV-aided network where multiple UAVs are launched to collect data from the mobile nodes. In particular, we aim to maximize the service time of the UAVs by jointly optimizing the three-dimensional (3D) trajectory of the UAVs and resources allocated to each node by the UAVs such that each mobile node receives a minimum specified data rate. To facilitate a solution, we construct an equivalent problem that considers the UAV's energy consumption. In particular, we minimize the maximum energy consumed by the UAVs in each time slot. To solve the problem, an iterative approach is presented that decouples the problem into two sub-problems. The optimal location of the UAVs is computed in the first sub-problem, while resource allocation is carried out in the second sub-problem. These two sub-problems are solved in an iterative manner using the alternating optimization approach. We show that the proposed approach improves the service time of the network by 20% on average compared to the existing approaches.

3D Trajectory Optimization for Multimission UAVs in Smart City Scenarios

There is a definite possibility that, in a recent future, Unmanned Aerial Vehicles (UAVs) will form the backbone of any smart city in terms of automation and networking. One approach to extend the UAVs' resources spectrum is to provide a mean for them to opportunistically recharge and connect to otherwise unreachable networks: provide Training and Recharge Areas (TRAs). In these dedicated areas, the UAVs could dock to Energy and Data Dispensers (EDD) devices to resupply their batteries and exploit a high-speed connection. To autonomously move through the smart city while accomplishing a set of given tasks but, at the same time, consider visiting the EDDs, is part of a tridimensional trajectory planning problem that needs to be addressed. In this paper, we formally define the combinatorial optimization problem representing the trajectory planning. We consider the case in which more than one UAV can be connected with the same EDD at the same time, by properly addressing the assignment of the bandwidth. Through simulative investigation, realistic values for the solution of the optimization problem are found. The behavior of the proposed model is compared with an ”online” approach that does not require the same resources and knowledge and whose evaluation and comparison with the ”offline” approach are performed through network simulation.

3D Trajectory Design for Energy-Constrained Aerial CRNs Under Probabilistic LoS Channel

3D Trajectory Design for Energy-Constrained Aerial CRNs Under Probabilistic LoS Channel

Direct measurements & simplified models of colloidal interactions & diffusion with adsorbed macromolecules.

We report total internal reflection microscopy measurements of 3D trajectories of ensembles of micron sized colloidal particles near interfaces with and without adsorbed macromolecules. Evanescent wave scattering reveals nanometer scale motion normal to planar surfaces and sub-diffraction limit lateral motion is resolved via image analysis. Equilibrium and non-equilibrium analyses of particle trajectories reveal self-consistent position dependent energies (energy landscapes) and position dependent diffusivities (diffusivity landscapes) both perpendicular and parallel to interfaces. For bare colloids and surfaces, electrostatic and hydrodynamic interactions are accurately quantified with established analytical theories. For colloids and surfaces with adsorbed macromolecules, conservative forces are accurately quantified with models for interactions between brush layers, whereas directly measured position dependent diffusivities require novel models of spatially varying permeability within adsorbed layers. Agreement between spatially resolved interactions and diffusivities and rigorous simplified models provide a basis to consistently interpret, predict, and design colloidal transport in the presence of adsorbed macromolecules for diverse applications.

3D Trajectory Planning for Real-Time Image Acquisition in UAV-Assisted VR

Nowadays, unmanned aerial vehicles (UAVs), empowered with the capability of high-definition image transmission, are used to capture the rapidly changing physical environment by leveraging its high flexibility to reconstruct an immersive realistic virtual environment for metaverse users. In this paper, we consider a novel UAV-assisted image acquisition system where a UAV is dispatched to take off from an initial location to capture real-time images of multiple ground targets and then transfer the captured images back to the ground user for virtual environment reconstruction. We aim to minimize the time for the UAV to complete the image acquisition task by optimizing the three-dimensional UAV trajectory under the constraints of image quality, information causality and energy consumption. To this end, we first formulate the investigated scenario into a mixed integer optimization problem, which, however, is difficult to solve due to the infinite time-varying variables closely coupled with each other. Then, a three-stage progressive algorithm is proposed to obtain an efficient solution to the formulated mixed integer optimization problem, where the constraints of image quality, information causality and energy consumption can be sequentially satisfied. Finally, comprehensive performance evaluation is conducted to verify the effectiveness of the proposed three-stage progressive trajectory design algorithm, and the results show that the proposed algorithm significantly outperforms the benchmark schemes.

A method for the reproduction of cello bow kinematics using a robotic arm and motion capture

Numerous experimental and theoretical methods have focused on the bow–string interaction in bowed string instruments, including several artificial bowing setups. The current research aims to present an experimental approach to reproduce bowing techniques using a robotic arm. First, optical motion capture is used to track the 3D kinematics of the bow. The cello bow and corpus are equipped with reflective markers. The cello is mounted on a playing platform. The recorded 3D trajectories of the bow markers are used to control the motion of the robotic arm. This process requires converting the 3D data between the coordinate frames of the two systems. This conversion is described in detail in this paper. To demonstrate the performance of the proposed method, an experienced cellist was asked to play an adapted piece on the cello, which was then repeated using the robotic arm. The robotic arm is capable of accurately reproducing the bow velocity, but even minimal variations in position can compromise proper bow–string contact. To illustrate this, the study compares two similar robotic situations and discusses the challenges of adapting the robot’s coordinates as a function of a given playing parameter or the sound produced.

3D trajectory optimization for UAV-Enabled wireless-assisted communication system

3D trajectory optimization for UAV-Enabled wireless-assisted communication system

Multi-Agent Deep Reinforcement Learning Based Optimizing Joint 3D Trajectories and Phase Shifts in RIS-Assisted UAV-Enabled Wireless Communications

Multi-Agent Deep Reinforcement Learning Based Optimizing Joint 3D Trajectories and Phase Shifts in RIS-Assisted UAV-Enabled Wireless Communications

3D Trajectory Planning Scheme for Shoe Upper Processing

3D Trajectory Planning Scheme for Shoe Upper Processing

Robust Fixed-Time 3D Trajectory Tracking Control for Underactuated AUVs Based on a Fixed-Time SSNN Disturbance Observer

Robust Fixed-Time 3D Trajectory Tracking Control for Underactuated AUVs Based on a Fixed-Time SSNN Disturbance Observer

Learning Short-Term Spatial-Temporal Dependency for UAV 2D Trajectory Forecasting

Learning Short-Term Spatial-Temporal Dependency for UAV 2D Trajectory Forecasting

Simple tracking of occluded self-propelled organisms

Deepening our understanding of animals’ collective motions represents a multidisciplinary goal. Yet, quantifying the motions of hundreds of animals in the laboratory and nature posits a fundamental challenge for digital image processing: How do we track each object out of the crowd while allowing them to move freely in a three-dimensional (3D) domain? Here, we present a simple tracking strategy to reconstruct 3D trajectories with the aid of a mirror, even if moving objects experience occlusion. We explain the method using synthetically generated datasets and apply it to measure collective motions of phototactic zooplankton, Daphnia magna, swimming in a lab-scale aquarium at intermediate Reynolds numbers, 1<Re<13 . The method enables measuring statistics of characteristic features of D. magna swarm, including sinking velocities and flapping frequencies. Beyond the lab-scale animal tracking, we foresee further implementations of the method to study wild animals freely behaving in 3D environments irrespective of their species.

Resource management for sum-rate maximization in SCMA-assisted UAV system

This work presents a resource management framework for optimizing the sum-rate in a sparse code multiple access (SCMA)-assisted UAV downlink system. We formulate two optimization problems for maximizing the overall sum-rate: the first problem addresses UAV 3D deployment and trajectory optimization with energy constraints, while the second focuses on optimizing SCMA subcarrier and power allocation optimization, subject to factor graph matrix (FGM) constraints and a minimum user data rate. Since the optimization problems are non-convex, the complexity of finding the global optimal solutions is prohibitive. We propose a gradient ascent-based iterative algorithm to compute the optimal UAV 3D deployment and trajectory. Further, an effective channel state information-based algorithm is proposed for FGM assignment, followed by a Lagrange dual decomposition method to solve the power allocation problem efficiently. Our research findings demonstrate that the optimization of the UAV trajectory gives improved sum-rate within the specified energy budget. Further, employing CSI-based multiple subcarrier allocation and strategic power allocation can significantly improve system performance compared to the benchmark schemes.

Predicting Trajectories of Plate-Type Wind-Borne Debris in Turbulent Wind Flow with Uncertainties

Debris poses multifaceted risks and jeopardizes various aspects of the environment, human health, safety, and infrastructure. The debris trajectory in turbulent wind flow is more dispersed due to the inherent randomness of the turbulent winds. This paper investigates the three-dimensional trajectories of plate-type wind-borne debris in turbulent wind fields via the method of numerical simulation. A 3D probabilistic trajectory model of plate-type wind-borne debris is developed. The debris trajectories are numerically calculated by solving the governing equation of debris motion and by introducing turbulent wind flows based on the near-ground wind field measured in the wind tunnel to account for the probability characteristics of the debris trajectory. The dimensionless velocities and displacements of the debris trajectory show good agreement with the experimental data in wind tunnel tests, confirming the rationality of the probabilistic trajectory model. Based on the validated trajectory model, the probability characteristics of the debris impact position, impact velocity, and kinetic energy, debris angular displacement, and angular velocity are analyzed in detail under five different wind attack angles. The proposed probabilistic model of plate-type debris in turbulent wind flow provides an accurate and effective method for predicting debris trajectory in three-dimensional space.

Hydrodynamics and shape reconstruction of single rising air bubbles in water using high-speed tomographic particle tracking velocimetry and 3D geometric reconstruction

Time-resolved tomographic particle tracking velocimetry (TR-3D-PTV), also called 4D-PTV, is used here to obtain the instantaneous 3D liquid flow field information in the wake of a single rising bubble in water. Simultaneously, the bubble shape, size and velocity are determined by tomographic reconstruction of the 3D bubble shape. Both, tracer particles for PTV and bubbles, are imaged in a shadow mode with background illumination. The Lagrangian method used in this paper, especially combined with the shake the box algorithm, has big advantages compared to particle image velocimetry, in situations, where only low particle per pixel values can be obtained. In this research, single air bubbles of different sizes, with diameters of around 2.4 mm, 4.0 mm, 6.0 mm and 9.6 mm, were injected into stagnant de-ionized water. Their shape was reconstructed in 3D, and an equivalent bubble diameter was determined from this reconstruction. Compared to conventionally used 2D shadow imaging, this diameter is about 13% smaller. The 3D bubble trajectory can be analysed and decomposed into a sinusoidal function curve lateral projection and an ellipsoidal shape vertical projection. As the bubble diameter increases, the radius of the spiral trajectory is decreasing as well as the amplitude of vertical sinusoidal oscillation. The wake structure in the liquid behind the bubbles is also changing with bubble size: from simple vortex pairs for smaller bubbles to an intertwined structure of several twisted vortices for the bigger ones.Graphical abstractThree-dimensional bubble reconstruction (grey surface) and liquid stream lines coloured with velocity magnitude around an ascending air bubble in de-ionized water.