Direct Linear Transformation Research Articles

Research on 3D humanoid robot pose estimation based on HRNet-Epipolar and CRF robot model by multiple view

Purpose This paper aims to present an unmarked method including entire two-dimensional (2D) and three-dimensional (3D) methods to recover absolute 3D humanoid robot poses from multiview images. Design/methodology/approach The method consists of two separate steps: estimating the 2D poses in multiview images and recovering the 3D poses from the multiview 2D heatmaps. The 2D one is conducted by High-Resolution Net with Epipolar (HRNet-Epipolar), and the Conditional Random Fields Humanoid Robot Pictorial Structure Model (CRF Robot Model) is proposed to recover 3D poses. Findings The performance of the algorithm is validated by experiments developed on data sets captured by four RGB cameras in Qualisys system. It illustrates that the algorithm has higher Mean Per Joint Position Error than Direct Linear Transformation and Recursive Pictorial Structure Model algorithms when estimating 14 joints of the humanoid robot. Originality/value A new unmarked method is proposed for 3D humanoid robot pose estimation. Experimental results show enhanced absolute accuracy, which holds important theoretical significance and application value for humanoid robot pose estimation and motion performance testing.

A New 3D reconstruction method for pressure-sensitive paint measurements under limited optical access

Reconstruction of 3D pressure field is crucial for pressure-sensitive paint measurements in turbomachinery experiments. Existing methods impose high requirements on feature points in terms of quantity, spatial distribution, and positional accuracy, and is difficult to apply under limited optical access, such as in planar cascade wind tunnels. We propose a novel direct linear transformation method based on contour matching (CM-DLT). This method leverages model contours captured from a tilted camera perspective, which provides the basis for identifying the projection matrix linking image pixels to 3D spatial points. The subsequent optimization of the projection matrix significantly enhances the accuracy and robustness of 3D pressure field reconstruction. In a benchmark experiment, the efficiency and reprojection accuracy of 3D reconstruction are investigated. Subsequently, the CM-DLT method is successfully applied in the cascade wind tunnel experiment. The obtained 3D pressure fields of two surfaces of adjacent blades are analyzed.

A calibration method for telecentric line-scan cameras with an enhanced dynamic imaging model and initial estimates derived from direct linear transformation

This study presents a calibration method for telecentric line-scan cameras, incorporating a dynamic imaging model and initial estimates derived from direct linear transformation. The enhanced dynamic imaging model includes an inclination parameter instead of velocities in three orthogonal spatial directions. It is applicable to any direction of motion, considers lens distortion, and accommodates the line sensor at an arbitrary position relative to the optical axis. Employing the enhanced camera model, the initial values of telecentric line-scan camera parameters are determined derived from direct linear transformation. Sensitivity of our method with respect to the noise level of image points and the number of pattern planes are assessed through numerical simulations, while effectiveness and robustness are validated through experiments using real-world data, achieving a reprojection error of 0.6386 pixels and a standard deviation of 0.0160 pixels. The versatility of the proposed method can be extended to measurement applications utilizing flatbed scanners.

A new method with C-means segmentation for non-uniform image coordinate system definition in panoramic imaging employing Ladybug2 camera

A panorama ensures a stunning wide-angle field of view up to 360° representation of a scene, exceeding the limits of a normal photograph. Panoramic cameras satisfy the single-viewpoint characteristic. There are several types of panoramic cameras for 360-degree imaging. Multi-camera panoramic imaging systems pose a difficulty in obtaining a single projection center for the cameras. In a variety of practical implementations of panoramic cameras, it is possible to calculate three-dimensional coordinates from a panoramic image, especially using the Direct Linear Transformation (DLT) method. In this study, not only a defining method of the non-uniform image coordinate system is presented by utilizing the C-Means algorithm for a single panoramic image, captured with a Ladybug2 panoramic camera in a panoramic calibration room but also the use of an elliptical panoramic projection coordinate system is defined by the Singular Value Decomposition method in a panoramic view. The results of the suggested method have been compared with the DLT algorithm for a single panoramic image which defined a conventional photogrammetric image coordinate system. It has been observed that the proposed method provides more accurate results for the 3D coordinate definition.

Skating techniques of ladies' world-class long-distance speed skaters to shorten curved-section time during the official 3,000 m race

This study aimed to identify the factors contributing to expedited passage through curved sections in skating by analyzing centripetal acceleration and skating motions during curving in a 3,000 m race for ladies' world-class speed skating. It included 14 elite skaters participating in the ladies' 3,000 m race held during the World Cup. The recorded area consisted of the first inner curve lane. Skaters were recorded as they passed through the measurement range at the initial, middle, and final stages of the race. Three synchronized high-speed cameras were used to record skaters from the front, back, and side. From the images obtained by the high-speed camera, 21 body endpoints and 4 blade edges were digitized at 50 Hz using specialized digitizing software. Three-dimensional coordinates of the 25 points were obtained using a panning direct linear transformation technique. The stroke-averaged centripetal acceleration and kinematic parameters were calculated based on the three-dimensional coordinates of the body during the curve-skating motion. Centripetal acceleration had a significant effect on the curved-section time in all three race stages (initial: F = 17.19, middle: F = 23.30, final: F = 16.64) and significantly decreased as the race progressed (left: F = 9.42, right: F = 8.05). Throughout the race, the right and left shanks and the body's center of mass (CM) during the stroke were raised (shank angle: left: F = 13.62, right: F = 11.02, CM height: left: F = 21.15, right: F = 21.69). The body-tilt angle for both strokes and shank-tilt angle for the right stroke were significantly correlated with centripetal acceleration in all race stages (body-tilt: left: initial: r = 0.80, middle: r = 0.75, final: r = 0.89, right: initial: r = 0.78, middle: r = 0.84, final: r = 0.67, right shank-tilt initial: r = 0.80, middle: r = 0.77, final: r = 0.63). These results suggested that to reduce the skating time through curved sections, maintaining an inward body tilt and minimizing the decrease in centripetal acceleration even in the final race stage are crucial considerations. They also suggested that when leaning the body inward and maintaining centripetal acceleration, the right shank should be leaned inward for the right stroke and the left shank should be leaned inward for the left stroke, or the left blade should be positioned farther to the right of the CM.

Innovative K-Means based machine learning method for determination of non-uniform image coordinate system in panoramic imaging: a case study with Ladybug2 camera.

Currently, the practical implementations of panoramic cameras range from vehicle navigation to space studies due to their 360-degree imaging capability in particular. In this variety of uses, it is possible to calculate three-dimensional coordinates from a panoramic image, especially using the Direct Linear Transformation (DLT) method. There are several types of omnidirectional cameras which can be classified mainly as central and non-central cameras for 360-degree imaging. The central omnidirectional cameras are those which satisfy the single-viewpoint characteristic. Multi-camera systems are usually developed for applications for which two-image stereo vision is not flexible enough to capture the environment surrounding a moving platform. Although the technology based on multi-view geometry is inexpensive, accessible, and highly customizable, multi-camera panoramic imaging systems pose a difficulty in obtaining a single projection center for the cameras. In this study, not only a defining method of the non-uniform image coordinate system is suggested by means of the K-Means algorithm for a single panoramic image, captured with a Ladybug2 panoramic camera in the panoramic calibration room but also the use of an elliptical panoramic projection coordinate system definition by Singular Value Decomposition (SVD) method in panoramic view. The results of the suggested method have been compared with the DLT algorithm for a single panoramic image which defined a conventional photogrammetric image coordinate system.

Enhancing Camera Calibration for Traffic Surveillance with an Integrated Approach of Genetic Algorithm and Particle Swarm Optimization.

Recent advancements in sensor technologies, coupled with signal processing and machine learning, have enabled real-time traffic control systems to effectively adapt to changing traffic conditions. Cameras, as sensors, offer a cost-effective means to determine the number, location, type, and speed of vehicles, aiding decision-making at traffic intersections. However, the effective use of cameras for traffic surveillance requires proper calibration. This paper proposes a new optimization-based method for camera calibration. In this approach, initial calibration parameters are established using the Direct Linear Transformation (DLT) method. Then, optimization algorithms are applied to further refine the calibration parameters for the correction of nonlinear lens distortions. A significant enhancement in the optimization process is achieved through the integration of the Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) into a combined Integrated GA and PSO (IGAPSO) technique. The effectiveness of this method is demonstrated through the calibration of eleven roadside cameras at three different intersections. The experimental results show that when compared to the baseline DLT method, the vehicle localization error is reduced by 22.30% with GA, 22.31% with PSO, and 25.51% with IGAPSO.

A revisit of the normalized eight-point algorithm and a self-supervised deep solution

The normalized eight-point algorithm has been widely viewed as the cornerstone in two-view geometry computation, where the seminal Hartley’s normalization has greatly improved the performance of the direct linear transformation algorithm. A natural question is, whether there are and how to find other normalization methods that can further improve the performance for each input sample. In this paper, we provide a novel perspective and propose two contributions to this fundamental problem: (1) we revisit the normalized eight-point algorithm and make a theoretical contribution by presenting the existence of different and better normalization algorithms; (2) we introduce a deep convolutional neural network with a self-supervised learning strategy for normalization. Given eight pairs of correspondences, our network directly predicts the normalization matrices, thus learning to normalize each input sample. Our learning-based normalization module can be integrated with both traditional (e.g., RANSAC) and deep learning frameworks (affording good interpretability) with minimal effort. Extensive experiments on both synthetic and real images demonstrate the effectiveness of our proposed approach.

Propagation of uncertainty for an epipole-dependent model for convergent stereovision structure computation

An analytic model incorporating stereo epipoles is proposed for structure computation using a convergent stereovision setup. The developed model is predicated on the image parameters of both CCD camera sensors, together with two extrinsic parameters, namely the stereo baseline distance and the stereo projection angle of the scene point of interest. In the model, the points on the image planes are measured relative to the principal points, stereo epipoles are featured, and only focal length-normalized camera sensor coordinates are required for structure computation. The reconstruction model could be employed in active vision-based metrology in which the stereo imaging cameras are systematically rotated about their vertical axes relative to each other. The performance of the model is studied, and its accuracy tested by comparing the 3-space coordinates it predicted to the those obtained by a gold standard triangulation and to the ground truth results. In terms of execution speed the proposed reconstruction model exhibited a computation time of 0.6 ms compared to 6.2 ms and 9.9 ms recorded for the direct linear transformation and gold standard triangulation algorithms respectively. The coordinate measurement uncertainties determined by experimental methods are subsequently compared with those obtained by a theoretical approach based on the analytic reconstruction model. Strong correlations were found to exist between the two sets of uncertainty values obtained.

Reconstruction of the Motion of Traffic Accident Vehicle in the Vehicle‐Mounted Video Based on Direct Linear Transform

Based on the principle of direct linear transformation (DLT) in close‐range photogrammetry, a method was proposed for reconstructing the motion states of the host vehicle and other vehicles based on vehicle‐mounted videos. To verify the effectiveness and accuracy of the method, validation experiments were designed. Under two typical operating states, steering and straight driving, the motion states of the host vehicle and other vehicles (including trajectory, distance, speed, and acceleration) were reconstructed from the vehicle‐mounted video. In the experiments, high‐precision inertial navigation was installed on the other vehicle to record real‐time motion data of the vehicle. Finally, in order to compare and analyze the reconstructed video results with the vehicle’s actual motion data, the recorded motion data were matched and synchronized to the same time axis as the vehicle‐mounted videos through a GPS timing device. The experimental result shows that the reconstructed trajectory results based on this method can generally reflect the vehicle’s actual trajectory, with an average deviation of less than 7.4%; the reconstructed distance results have an average deviation of less than 9.3%; the reconstructed speed results have an average deviation of less than 7.3%; the reconstructed acceleration results can reflect the vehicle’s acceleration or deceleration states. The results of this study provide an effective solution for obtaining important parameters of vehicles in accident reconstruction research, such as the trajectory, speed, distance, and acceleration or deceleration, and it has significant practical value for applications.

Two-Dimensional Analysis of Lower Extremities to Predict Best Initial Condition on Marching Movement

Himpunan Mahasiswa Mesin ITB has a specific type of marching that follows a march song’s rhythm called “derap”. In practice, derap was a hard move to maintain and unify in a group. This difficulty was observed by noting the differences that occur between the subjects' initial and middle march motion conditions. The most noticeable difference was the height of the group. This study intends to observe and find how the differences vary in this motion. The differences that occur corresponded to the relative knee and ankle angle. This study utilized five active markers to reconstruct the lower extremities' activity during motion, one action camera at 60 fps, and a workstation. This study was also constrained by using three cycles of march song at 115 bpm. Direct Linear Transformation was used to obtain the intrinsic factor of the camera to reconstruct the motion. Evaluating the angular kinematic parameters of relative knee and ankle angle, the authors found the relative knee angle has increased from 110° to around 130°, which happened at the second song cycle and kept stable at the rest of the song. This result brought the conclusion that more extended derap motion would tend to a steady condition of relative knee angle and suggested that the subject should begin with 130° of relative knee angle to give less effort on marching.

Pedestrian trajectory prediction method based on the Social-LSTM model for vehicle collision

Abstract Techniques for predicting the trajectory of vulnerable road users are important to the development of perception systems for autonomous vehicles to avoid accidents. The most effective trajectory prediction methods, such as Social-LSTM, are often used to predict pedestrian trajectories in normal passage scenarios. However, they can produce unsatisfactory prediction results and data redundancy, as well as difficulties in predicting trajectories using pixel-based coordinate systems in collision avoidance systems. There is also a lack of validations using real vehicle-to-pedestrian collisions. To address these issues, some insightful approaches to improve the trajectory prediction scheme of Social-LSTM were proposed, such methods included transforming pedestrian trajectory coordinates and converting image coordinates to world coordinates. The YOLOv5 detection model was introduced to reduce target loss and improve prediction accuracy. The DeepSORT algorithm was employed to reduce the number of target transformations in the tracking model. Image Perspective Transformation (IPT) and Direct Linear Transformation (DLT) theories were combined to transform the coordinates to world coordinates, identifying the collision location where the accident could occur. The performance of the proposed method was validated by training tests using MS COCO (Microsoft Common Objects in Context) and ETH/UCY datasets. The results showed that the target detection accuracy was more than 90% and the prediction loss tends to decrease with increasing training steps, with the final loss value less than 1%. The reliability and effectiveness of the improved method were demonstrated by benchmarking system performance to two video recordings of real pedestrian accidents with different lighting conditions.

A robust direct linear transformation for camera pose estimation using points

Camera pose estimation from 3D points and their 2D projections, known as the perspective-n-point (PnP) problem, is a fundamental task in computer vision. In this paper, we propose a robust direct linear transformation method for the PnP problem. The novelty lies in that a plane constraint based on the pinhole camera model is introduced, and an over-constrained linear equation system is obtained by merging the linear form of both the measured point correspondences and the plane constraint. As a result, our method successfully reduces the minimal number of point correspondences required by the linear solver to 4, and yields a unique solution for 4-point, 5-point, and n-point pose estimation. Experiment results, using both synthetic and real data, show that our method is substantially faster, and offers the accuracy and precision comparable with that of state-of-the-art methods.

Method for three-dimensional reconstruction of dynamic stereo vision based on line structured light using global optimization

To overcome the limitation of the measurement range of static stereo vision, a three-dimensional (3-D) reconstruction method of dynamic stereo vision based on line structured light for metal surface is presented in this paper. The dynamic stereo vision is mainly composed of binocular cameras and a laser line generator, and moves along a horizontal guide rail. The intrinsic parameters of the cameras and the rotation and translation of the right camera relative to the left camera are previously obtained by Zhang’s method. To obtain the 3-D coordinates of the line structured light fringe in each position, the sub-pixel coordinates of the fringe center are extracted by Steger algorithm, and a joint calibration method between single camera and a laser line generator is proposed. The joint calibration method calculates the plane equation of the line structured light by using multiple checkerboard images with projected structured light line. To unify the 3-D coordinates of structured light fringe in each position to the reference coordinate system, the rotation and translation of the camera in each position relative to the reference position (called absolute pose) should be solved. The direct linear transformation method is firstly used to calculate the rough relative pose between adjacent frames, and principle of re-projection error minimization in object space is used to obtain the refined relative pose. Instead of using step-by-step transfer method, the absolute pose of the camera in each position is eventually calculated by using global optimization algorithm after the relative pose between all adjacent frames are obtained. Computer simulation and abundant experimental data validate the effectiveness of proposed method. The proposed 3-D reconstruction method using global optimization is respectively compared with step-by-step transfer method based on pixel re-projection error minimization and re-projection error minimization in object space. The experimental results reflect that our method is superior to step-by-step transfer method, and the 3-D reconstruction accuracy is improved by 28.05 % after global optimization.

Two-Phase Positioning System Based on the Fusion of Wi-Fi Signal Strength and Pose Estimation

Due to the growing importance of positioning in recent years, global positioning system has become widely used. However, indoor positioning presents significant challenges due to multipath interference, which can cause large signal fluctuations and high positioning errors for signal-based approaches. Recent advances in deep learning have enabled researchers to estimate people's positions and locate them indoors using image recognition technology, but identification of individuals remains challenging. Therefore, we propose a highly accurate indoor positioning approach using Wi-Fi signals and images. This research employs smartphones to measure the signal strength of three Wi-Fi base stations' 2.4 and 5 GHz frequency bands. We then use machine learning techniques to estimate individuals' positions with coarse accuracy. Subsequently, we analyze the positions of individuals in surveillance camera images. We use OpenPose, an open-source multiperson pose estimation system, and propose a foot position extraction algorithm to extract the foot positions of individuals in the image. We obtain the coordinates of the people in the image through direct linear transformation and machine learning. Finally, we match the coordinates obtained from the two techniques to identify the user's device. Experimental results show that the proposed method achieves higher accuracy, with an average error of 0.43 meters.

Technical Skills Influences on Front Crawl Tumble Turn Performance in Elite Female Swimmers.

The objective of this research was to compare technical skills composed of kinematic and kinetic variables in the complex motor task of a tumble turn between 9 elites and 9 sub-elite female swimmers. The best tumble turn among three attempts was analyzed using a three-dimensional underwater protocol. A total of 37 kinematic variables were derived from a Direct Linear Transformation algorithm for 3D reconstruction, and 16 kinetic variables measured by a piezoelectric 3D force platform. Data were analyzed by Student's t-test and effect size statistics. Pearson correlations were applied to the data of the eighteen swimmers to relate the association of 53 kinematic, kinetic variables to the performance of the tumble-turn (3 meters Round Trip Time, 3m RTT). The approach and the whole turn times were faster for elite swimmers compared to sub elites (1.09±0.06 vs. 1.23±0.08 sec, and 2.89±0.07 vs. 3.15±0.11 sec.), as well as the horizontal speeds of the swimmers' head 1 m before the rotation (1.73±0.13 vs. 1.57±0.13 m/sec.), at the end of the push-off on force platform (2.55±0.15 vs. 2.31±0.22 m/sec.) and 3 m after the wall (2.01±0.19 vs. 1.68±0.12 m/sec.). Large differences (|d| > 0.8) in favor of the elite swimmers were identified for the index of upper body extension at the beginning of the push-off, the lower limb extension index at the end of push-off, and among the kinetic variables, the horizontal impulse and lateralization of the push-off. Correlations for the whole group revealed a moderate to strong relationship between 6 body extension indices and 3mRTT performance. For the kinetic variables, the correlations indicated the fastest swimmers in 3mRTT showed large lateral impulse during placement (r=0.46), maximum horizontal force during the push-off (r=0.45) and lateralization of the push-off (r=0.44) (all p<0.05). Elite female swimmers had higher approach and push-off speeds, were more streamlined through the contact, and showed a higher horizontal impulse and lateralization of the push-off, than their sub-elite counterparts.

A Four-Point Camera Calibration Method for Sport Videos

The main objective of camera calibration is to estimate the homography between a sports field template and the corresponding field in a video frame. State-of-the-art approaches exploit deep-learning technologies to achieve this aim. However, the lack of large training datasets leads to unreliable results. Search-based approaches deliver better accuracy but at the expense of a very high computational cost. In this paper, a four-point camera calibration method is proposed to address these challenges. It consists of four stages: first, a conditional generative adversarial network (cGAN) is used to generate meaningfully segmented video frames. This cGAN is a general image-to-image translation technique which has robust performance even for small training datasets. The aim is to eliminate foreground objects that greatly impede accurate estimation of four suitable points for the subsequent stage. Then, a regression network which can estimate four points from a single segmented video frame is introduced. In this stage we aim at keeping the low computational cost. In the next processing stage, the estimated four points and the original selected four points are used to calculate the homography. In this third stage a Direct Linear Transformation (DLT) algorithm is applied. In the last processing stage, the accuracy of the estimated homography is refined by running an Enhanced Correlation Coefficient (ECC) module. The proposed method is extensively evaluated using a standard 2014 World Cup dataset. The obtained results are compared to the state-of-the-art approaches and its variations based on U-net, VGG-16, ResNet-50. The obtained results show that the proposed method is superior in terms of accuracy and computational efficiency. To demonstrate the effectiveness and robustness of the proposed method, it is also validated on National Basketball, 2018 World Cup and cross dataset. In these diverse environments, the proposed method still maintains the competitive performance.

Integration of multi-photos and laser scanner data to form a complete 3d model

Nowadays, laser scanners are widely used to get three-dimensional modeling, but sometimes some obstacles on the site or choosing the wrong scan width and the shadow areas due to different planes lead to the appearance of some gaps in the objective to be scanned. Mobile phone images (Conventional terrestrial photogrammetry technique) can be used to compensate for these missing data after transforming the image coordinates to ground coordinates. So, the main objective is to develop a technique to transform image coordinates to ground ones through the Direct Linear Transformation model (DLT) depending on a computer algorithm using the least-squares adjustment method. Four field experiments were made on AL-ASHRAFI QAITBAY . The obtained results showed that the putative technique can be applied to form a full 3D model with precision (11–22 mm) and it is compatible with the precision of a laser scanner.

Dynamic stability during level walking and obstacle crossing in children aged 2-5 years estimated by marker-less motion capture.

In the present study, dynamic stability during level walking and obstacle crossing in typically developing children aged 2-5 years (n = 13) and healthy young adults (n = 19) was investigated. The participants were asked to walk along unobstructed and obstructed walkways. The height of the obstacle was set at 10% of the leg length. Gait motion was captured by three RGB cameras. 2D body landmarks were estimated using OpenPose, a marker-less motion capture algorithm, and converted to 3D using direct linear transformation (DLT). Dynamic stability was evaluated using the margin of stability (MoS) in the forward and lateral directions. All the participants successfully crossed the obstacles. Younger children crossed the obstacle more carefully to avoid falls, as evidenced by obviously decreased gait speed just before the obstacle in 2-year-olds and the increased in maximum toe height with younger age. There was no significant difference in the MoS at the instant of heel contact between children and adults during level walking and obstacle crossing in the forward direction, although children increased the step length of the lead leg to a greater extent than the adults to ensure base of support (BoS)-center of mass (CoM) distance. In the lateral direction, children exhibited a greater MoS than adults during level walking [children: 9.5%, adults: 6.5%, median, W = 39.000, p < .001, rank-biserial correlation = -0.684]; however, some children exhibited a smaller MoS during obstacle crossing [lead leg: -5.9% to 3.6% (min-max) for 4 children, 4.7%-6.4% [95% confidence interval (CI)] for adults, p < 0.05; trail leg: 0.1%-4.4% (min-max) for 4 children, 4.7%-6.4% (95% CI) for adults, p < 0.05]]. These results indicate that in early childhood, locomotor adjustment needed to avoid contact with obstacles can be observed, whereas lateral dynamic stability is frangible.

Keep Attention to the Mapping:Application of AI for Geometric X-Ray CT Scan Calibration

This paper presents a method for mapping markers from CT calibration bodies in X-ray projections. In twin robotic CT systems, determining projection geometry from projection views is essential for reconstruction. Many algorithms, such as direct linear transformation (DLT), use a calibration body for this purpose. This consists of individual markers that must be individually assigned in a single X-ray projection. In the described approach, a transformation model is used to map the coordinates of the 2D marker centers to the correct index. This allows the use of freely designed calibration bodies, where the arrangement of the markers is not limited to geometric arrangements.