Initial Pose Research Articles

Improving repeatability of surface electromyography measurement of sit-to-stand motions by using muscle synergy

This study proposes a method to improve the measurement stability of sit-to-stand (STS) motion, one of the most important motions for human mobility. STS motion is complex and uses numerous muscles ranging from the feet to the torso. Electromyography (EMG) is often used to analyze STS motion by precisely measuring the activity of the contributing muscles. However, such measurements depend strongly on the measurement conditions, such as electrode attachment conditions, and variations in the contributing muscles and initial poses. We examined the variability caused by these conditions and explored a method to stabilize STS measurements by EMG. The proposed method adjusts measurements using data aggregation and muscle synergy computation, which leads to better comparisons or integration of motion measurements with two or more motion strategies or motions acquired over multiple EMG electrode reinstallations. The experimental results show that the proposed method improves the similarity of measurements under different conditions.

Model-based monocular 6-degree-of-freedom pose tracking for asteroid

In this paper, we present a novel vision-based framework to track the 6-DoF pose of an asteroid in real time with the 3D contour of the asteroid as a feature. During pose tracking, at the beginning time of tracking, the tracking system is initialized by a pose retrieval method. At each subsequent time instant, given the 3D mesh model of an asteroid, with the initial pose and its covariance given by the square root cubature Kalman Filter (SCKF), the 3D mesh segments constituting the 3D asteroid contour are efficiently extracted from the 3D mesh model. Then, in the input asteroid image, we search the image points corresponding to the extracted 3D segments within the searching range defined by the initial pose and its covariance. After that, the asteroid pose is determined in real time by minimizing the angles between the back-projection lines of the searched image points and the projection planes of the corresponding 3D segments, which is much more robust to the position change of the asteroid and asteroid size. The covariance matrix of the pose is inferred from the Cartesian noise model in the first order. Eventually, the SCKF is derived from the second-order auto regression to generate the final pose estimate and give the initial pose and its covariance for the next time instant. The synthetic trials quantitatively validate the real-time performance, robustness, and accuracy of our algorithm in dark space, different imaging distances, lighting conditions, image noise, model error, and initial pose error, and meanwhile, the real trial qualitatively shows the effectiveness of our method.

Addressing Suboptimal Poses in Nonequilibrium Alchemical Calculations.

Alchemical transformations can be used to quantitatively estimate absolute binding free energies at a reasonable computational cost. However, most of the approaches currently in use require knowledge of the correct (crystallographic) pose. In this paper, we present a combined Hamiltonian replica exchange nonequilibrium alchemical method that allows us to reliably calculate absolute binding free energies, even when starting from suboptimal initial binding poses. Performing a preliminary Hamiltonian replica exchange enhances the sampling of slow degrees of freedom of the ligand and the target, allowing the system to populate the correct binding pose when starting from an approximate docking pose. We apply the method on 6 ligands of the first bromodomain of the BRD4 bromodomain-containing protein. For each ligand, we start nonequilibrium alchemical transformations from both the crystallographic pose and the top-scoring docked pose that are often significantly different. We show that the method produces statistically equivalent binding free energies, making it a useful tool for computational drug discovery pipelines.

SLAM-RAMU: 3D LiDAR-IMU lifelong SLAM with relocalization and autonomous map updating for accurate and reliable navigation

PurposeAutonomous mobile robots (AMRs) play a crucial role in industrial and service fields. The paper aims to build a LiDAR-based simultaneous localization and mapping (SLAM) system used by AMRs to overcome challenges in dynamic and changing environments.Design/methodology/approachThis research introduces SLAM-RAMU, a lifelong SLAM system that addresses these challenges by providing precise and consistent relocalization and autonomous map updating (RAMU). During the mapping process, local odometry is obtained using iterative error state Kalman filtering, while back-end loop detection and global pose graph optimization are used for accurate trajectory correction. In addition, a fast point cloud segmentation module is incorporated to robustly distinguish between floor, walls and roof in the environment. The segmented point clouds are then used to generate a 2.5D grid map, with particular emphasis on floor detection to filter the prior map and eliminate dynamic artifacts. In the positioning process, an initial pose alignment method is designed, which combines 2D branch-and-bound search with 3D iterative closest point registration. This method ensures high accuracy even in scenes with similar characteristics. Subsequently, scan-to-map registration is performed using the segmented point cloud on the prior map. The system also includes a map updating module that takes into account historical point cloud segmentation results. It selectively incorporates or excludes new point cloud data to ensure consistent reflection of the real environment in the map.FindingsThe performance of the SLAM-RAMU system was evaluated in real-world environments and compared against state-of-the-art (SOTA) methods. The results demonstrate that SLAM-RAMU achieves higher mapping quality and relocalization accuracy and exhibits robustness against dynamic obstacles and environmental changes.Originality/valueCompared to other SOTA methods in simulation and real environments, SLAM-RAMU showed higher mapping quality, faster initial aligning speed and higher repeated localization accuracy.

Automated rerailing of a road-rail shunting vehicle on road-level tracks using 2D-Lidar

Road-rail vehicles are emerging increasingly in small flat yards for shunting wagons. While numerous research works suggest automation of the shunting operation to improve its efficiency, the rerailing procedure of such vehicle, i.e., switching between road and rail mode, remains manually executed. Hence, we present a simple but novel method for a road-rail vehicle with skid-steer drive to rerail automatically on road-level tracks (e.g., level-crossing). The vehicle uses two 2D-Lidar sensors to scan the floor for (rail) in order to estimate its lateral displacement and angular orientation towards the track centre. A simplified model and a PID-type controller for the yaw rate were deployed while keeping the linear velocity constant for the vehicle to align itself at the centre of and parallel to the track. Upon reaching the goal, the rail wheels are descended and the rerailing is completed. Test runs were conducted from a random initial pose near the track centre while limiting the travel range in x-direction to 2.5 m, yet allowing a change of direction each time the vehicle reaches this limit. In 97% of the test runs, the vehicle rerailed successfully in 47 s on average (12 s of which was needed for lowering the rail wheels) with three changes of direction or less. In the remaining 3% of cases it took more attempts. Automated rerailing could be used not only as assistance function for the shunting personnel, but also as important feature of an autonomous shunting robot that is able to access both road and rail domains.

Accurate Monocular SLAM Initialization via Structural Line Tracking.

In this paper, we present a novel monocular simultaneous localization and mapping (SLAM) initialization algorithm that relies on structural features by tracking structural lines. This approach addresses the limitations of the traditional method, which can fail to account for a lack of features or their uneven distribution. Our proposed method utilizes a sliding window approach to guarantee the quality and stability of the initial pose estimation. We incorporate multiple geometric constraints, orthogonal dominant directions, and coplanar structural lines to construct an efficient pose optimization strategy. Experimental evaluations conducted on both the collected chessboard datasets and real scene datasets show that our approach provides superior results in terms of accuracy and real-time performance compared to the well-tuned baseline methods. Notably, our algorithm achieves these improvements while being computationally lightweight, without the need for matrix decomposition.

Robust Point Cloud Registration Network for Complex Conditions.

Point cloud registration is widely used in autonomous driving, SLAM, and 3D reconstruction, and it aims to align point clouds from different viewpoints or poses under the same coordinate system. However, point cloud registration is challenging in complex situations, such as a large initial pose difference, high noise, or incomplete overlap, which will cause point cloud registration failure or mismatching. To address the shortcomings of the existing registration algorithms, this paper designed a new coarse-to-fine registration two-stage point cloud registration network, CCRNet, which utilizes an end-to-end form to perform the registration task for point clouds. The multi-scale feature extraction module, coarse registration prediction module, and fine registration prediction module designed in this paper can robustly and accurately register two point clouds without iterations. CCRNet can link the feature information between two point clouds and solve the problems of high noise and incomplete overlap by using a soft correspondence matrix. In the standard dataset ModelNet40, in cases of large initial pose difference, high noise, and incomplete overlap, the accuracy of our method, compared with the second-best popular registration algorithm, was improved by 7.0%, 7.8%, and 22.7% on the MAE, respectively. Experiments showed that our CCRNet method has advantages in registration results in a variety of complex conditions.

MNET++: Music-Driven Pluralistic Dancing Toward Multiple Dance Genre Synthesis.

Numerous task-specific variants of autoregressive networks have been developed for dance generation. Nonetheless, a severe limitation remains in that all existing algorithms can return repeated patterns for a given initial pose, which may be inferior. We examine and analyze several key challenges of previous works, and propose variations in both model architecture (namely MNET++) and training methods to address these. In particular, we devise the beat synchronizer and dance synthesizer. First, generated dance should be locally and globally consistent with given music beats, circumvent repetitive patterns, and look realistic. To achieve this, the beat synchronizer implicitly catches the rhythm enabling it to stay in sync with the music as it dances. Then, the dance synthesizer infers the dance motions in a seamless patch-by-patch manner conditioned by music. Second, to generate diverse dance lines, adversarial learning is performed by leveraging the transformer architecture. Furthermore, MNET++ learns a dance genre-aware latent representation that is scalable for multiple domains to provide fine-grained user control according to the dance genre. Compared with the state-of-the-art methods, our method synthesizes plausible and diverse outputs according to multiple dance genres as well as generates remarkable dance sequences qualitatively and quantitatively.

H4MER: Human 4D Modeling by Learning Neural Compositional Representation with Transformer.

Despite the impressive results achieved by deep learning based 3D reconstruction, the techniques of directly learning to model 4D human captures with detailed geometry have been less studied. This work presents a novel neural compositional representation for Human 4D Modeling with transformER (H4MER). Specifically, our H4MER is a compact and compositional representation for dynamic human by exploiting the human body prior from the widely used SMPL parametric model. Thus, H4MER can represent a dynamic 3D human over a temporal span with the codes of shape, initial pose, motion and auxiliaries. A simple yet effective linear motion model is proposed to provide a rough and regularized motion estimation, followed by per-frame compensation for pose and geometry details with the residual encoded in the auxiliary codes. We present a novel Transformer-based feature extractor and conditional GRU decoder to facilitate learning and improve the representation capability. Extensive experiments demonstrate our method is not only effective in recovering dynamic human with accurate motion and detailed geometry, but also amenable to various 4D human related tasks, including monocular video fitting, motion retargeting, 4D completion, and future prediction. The video demos are in the project homepage: https://boyanjiang.github.io/H4MER/.

NOPE-SAC: Neural One-Plane RANSAC for Sparse-View Planar 3D Reconstruction.

This paper studies the challenging two-view 3D reconstruction problem in a rigorous sparse-view configuration, which is suffering from insufficient correspondences in the input image pairs for camera pose estimation. We present a novel Neural One-PlanE RANSAC framework (termed NOPE-SAC in short) that exerts excellent capability of neural networks to learn one-plane pose hypotheses from 3D plane correspondences. Building on the top of a Siamese network for plane detection, our NOPE-SAC first generates putative plane correspondences with a coarse initial pose. It then feeds the learned 3D plane correspondences into shared MLPs to estimate the one-plane camera pose hypotheses, which are subsequently reweighed in a RANSAC manner to obtain the final camera pose. Because the neural one-plane pose minimizes the number of plane correspondences for adaptive pose hypotheses generation, it enables stable pose voting and reliable pose refinement with a few of plane correspondences for the sparse-view inputs. In the experiments, we demonstrate that our NOPE-SAC significantly improves the camera pose estimation for the two-view inputs with severe viewpoint changes, setting several new state-of-the-art performances on two challenging benchmarks, i.e., MatterPort3D and ScanNet, for sparse-view 3D reconstruction. The source code is released at https://github.com/IceTTTb/NopeSAC for reproducible research.

Coarse–Fine Registration of Point Cloud Based on New Improved Whale Optimization Algorithm and Iterative Closest Point Algorithm

In this paper, a point cloud coarse–fine registration method based on a new improved version of the whale optimization algorithm (NIWOA) and iterative closest point (ICP) algorithm is proposed; we use three strategies to improve the whale optimization algorithm (WOA). Firstly, circle chaotic mapping is used to initialize the whale population to increase the diversity of the population and make the search space more comprehensively explored. In addition, a Newton inertia weight is proposed to flexibly adjust the proportion of global exploration and local optimization in order to achieve the balance between the exploitation performance and exploration ability of the algorithm. At the same time, we introduce the nonlinear convergence factor that can adjust the size adaptively so that the algorithm can find the global optimal solution faster and more accurately, allowing it to avoid falling into the local optimal solution to a certain extent. The NIWOA algorithm is used to optimize the objective function of point cloud coarse registration to obtain the optimal coordinate transformation, and the rotation and translation operation is carried out on the registered point cloud. The obtained position is used as the initial pose of the ICP fine registration, and the final registration is achieved through ICP iteration. We conduct coarse registration experiments on multiple model point clouds and scene point clouds using the Stanford 3D Scanning Repository dataset and Princeton 3Dmatch dataset, respectively. The experimental results confirm that the NIWOA algorithm can not only find the initial position that is closer to the target point cloud, but also provide reliable initial values for the ICP algorithm. Meanwhile, the NIWOA algorithm combined with ICP experiment results show that the method has a higher registration accuracy and operation efficiency.

PanoVLM: Low-Cost and accurate panoramic vision and LiDAR fused mapping

Cameras and LiDARs are currently two types of sensors commonly used for 3D mapping. Vision-based methods are susceptible to textureless regions and lighting, and LiDAR-based methods easily degenerate in scenes with insignificant structural features. Most current fusion-based methods require strict synchronization between the camera and LiDAR and need auxiliary sensors, such as IMU. All of these lead to an increase in device cost and complexity. To address that, in this paper, we propose a low-cost mapping pipeline called PanoVLM that only requires a panoramic camera and a LiDAR without strict synchronization. First, camera poses are estimated by a LiDAR-assisted global Structure-from-Motion, and LiDAR poses are derived with the initial camera-LiDAR relative pose. Then, line-to-line and point-to-plane associations are established between LiDAR point clouds, which are used to further refine LiDAR poses and remove motion distortion. With the initial sensor poses, line-to-line correspondences are established between images and LiDARs to refine their poses jointly. The final step, joint panoramic Multi-View Stereo, estimates the depth map for each panoramic image and fuses them into a complete dense 3D map. Experimental results show that PanoVLM can work on various scenarios and outperforms state-of-the-art (SOTA) vision-based and LiDAR-based methods. Compared with the current SOTA LiDAR-based techniques, namely LOAM, LeGO-LOAM, and F-LOAM, PanoVLM manifests a reduction in the absolute rotation error and absolute translation error by 20% and 35%, respectively. Our code and dataset are available at https://github.com/3dv-casia/PanoVLM.

Automatic registration with continuous pose updates for marker-less surgical navigation in spine surgery

Established surgical navigation systems for pedicle screw placement have been proven to be accurate, but still reveal limitations in registration or surgical guidance. Registration of preoperative data to the intraoperative anatomy remains a time-consuming, error-prone task that includes exposure to harmful radiation. Surgical guidance through conventional displays has well-known drawbacks, as information cannot be presented in-situ and from the surgeon’s perspective. Consequently, radiation-free and more automatic registration methods with subsequent surgeon-centric navigation feedback are desirable. In this work, we present a marker-less approach that automatically solves the registration problem for lumbar spinal fusion surgery in a radiation-free manner. A deep neural network was trained to segment the lumbar spine and simultaneously predict its orientation, yielding an initial pose for preoperative models, which then is refined for each vertebra individually and updated in real-time with GPU acceleration while handling surgeon occlusions. An intuitive surgical guidance is provided thanks to the integration into an augmented reality based navigation system. The registration method was verified on a public dataset with a median of 100% successful registrations, a median target registration error of 2.7 mm, a median screw trajectory error of 1.6°and a median screw entry point error of 2.3 mm. Additionally, the whole pipeline was validated in an ex-vivo surgery, yielding a 100% screw accuracy and a median target registration error of 1.0 mm. Our results meet clinical demands and emphasize the potential of RGB-D data for fully automatic registration approaches in combination with augmented reality guidance.

Towards reliable multi-person pose estimation using Conditional Random Fields

Multi-person pose estimation is the task of estimating the coordinates of body joints and predicting the body poses for multiple people in an images. This problem has made breakthroughs in recent years, but the solutions still suffer from some shortcomings. A serious weakness of the state-of-the-art models is the number of poses detected by these models which is generally much larger than the actual number of human instances in the input image. This makes the existing models unreliable and thus unusable in real-world tasks. In this paper, we propose a more reliable multi-person pose estimation method consisting of three main blocks: a top-down multi-person pose estimation, a human detection, and a pose selection block. The proposed method incorporates the bounding box of the segmented objects to select the best subset of the initial pose set. We formulate the pose selection problem using Conditional Random Fields. First, we introduce a set of potential functions to form a general probability model. Then an inference algorithm is proposed to select the best poses which maximize the probability function. Finally, the proposed solution is implemented by a neural network. The proposed pose selection model is a model-agnostic method that can be easily used in conjunction with other pose estimation and object detection models. Experiments demonstrate that the reliability and precision of the proposed model are higher than those of the state-of-the-art models.

3D Object Tracking for Rough Models

AbstractVisual monocular 6D pose tracking methods for textureless or weakly‐textured objects heavily rely on contour constraints established by the precise 3D model. However, precise models are not always available in reality, and rough models can potentially degrade tracking performance and impede the widespread usage of 3D object tracking. To address this new problem, we propose a novel tracking method that handles rough models. We reshape the rough contour through the probability map, which can avoid explicitly processing the 3D rough model itself. We further emphasize the inner region information of the object, where the points are sampled to provide color constrains. To sufficiently satisfy the assumption of small displacement between frames, the 2D translation of the object is pre‐searched for a better initial pose. Finally, we combine constraints from both the contour and inner region to optimize the object pose. Experimental results demonstrate that the proposed method achieves state‐of‐the‐art performance on both roughly and precisely modeled objects. Particularly for the highly rough model, the accuracy is significantly improved (40.4% v.s. 16.9%).

A Fast Registration Method for MEMS LiDAR Point Cloud Based on Self-Adaptive Segmentation

The Micro-Electro-Mechanical System (MEMS) LiDAR point cloud in autonomous vehicles has a large deflection range, which results in slow registration speed and poor applicability. To maximize speed, an improved Normal Distribution Transform (NDT) method that integrates point cloud density features has been proposed. First, the point cloud is reduced using a modified voxel filter and a pass-through filter. Next, the Intrinsic Shape Signature (ISS) algorithm is utilized to analyze the point cloud features and extract key points; the Four-Point Congruent Set (4PCS) algorithm is then employed to calculate the initial pose under the constraints of the key point set to complete the coarse registration. Finally, the self-adaptive segmentation model is constructed by using a K-D tree to obtain the density features of key points, and the NDT algorithm is combined with this model to form an SSM-NDT algorithm, which is used for fine registration. Each algorithm was compared on the autonomous vehicle dataset PandaSet and actual collected datasets. The results show that the novel method increases the speed by at least 60% and takes into account good registration accuracy and strong anti-interference.

Digital twin model-based smart assembly strategy design and precision evaluation for PCB kit-box build

Research concerning microelectronics assembly has attracted increasing attention from the manufacturing industry. In this context, achieving precise placement of the printed circuit board (PCB) inside the enclosure is the most critical aspect of the kit-box build assembly task. However, automating the PCB kit-box build assembly (PKBA) process remains a challenging work. This study presents a Digital Twin (DT) model that enables accurate perception of object pose for intelligent PKBA assembly, facilitating real-time monitoring and evaluation of its service status. In our system, a symmetry-drive method is proposed to optimize the initial pose in 6-DoF matching during DT assembly. Based on the developed technology, a three-stage learning method is established to achieve grasping point localization and robotic assembly trajectory planning. To ensure accurate and robust robot assembly, we developed a PKBA quality prediction model based on the DT system, which predicts the uncertainty of actual PCB assembly positioning by small displacement torsor (SDT) theory and Monte Carlo methods. Particularly, the assembly quality of the actual product is effectively monitored when the state of the virtual simulation model corresponds to the physical assembly object. Finally, a prototype system and a case study involving dexterous assembly tasks are conducted to verify the effectiveness and feasibility of the proposed method. The results indicate that the proposed PKBA strategy achieves an 82% assembly success rate. By employing well-designed strategies, our method ensures that the majority of errors are below 0.8 mm and 0.6 degrees.

Camera and LiDAR Fusion for Urban Scene Reconstruction and Novel View Synthesis via Voxel-Based Neural Radiance Fields

3D reconstruction of urban scenes is an important research topic in remote sensing. Neural Radiance Fields (NeRFs) offer an efficient solution for both structure recovery and novel view synthesis. The realistic 3D urban models generated by NeRFs have potential future applications in simulation for autonomous driving, as well as in Augmented and Virtual Reality (AR/VR) experiences. Previous NeRF methods struggle with large-scale, urban environments. Due to the limited model capability of NeRF, directly applying them to urban environments may result in noticeable artifacts in synthesized images and inferior visual fidelity. To address this challenge, we propose a sparse voxel-based NeRF. First, our approach leverages LiDAR odometry to refine frame-by-frame LiDAR point cloud alignment and derive accurate initial camera pose through joint LiDAR-camera calibration. Second, we partition the space into sparse voxels and perform voxel interpolation based on 3D LiDAR point clouds, and then construct a voxel octree structure to disregard empty voxels during subsequent ray sampling in the NeRF, which can increase the rendering speed. Finally, the depth information provided by the 3D point cloud on each viewpoint image supervises our NeRF model, which is further optimized using a depth consistency loss function and a plane constraint loss function. In the real-world urban scenes, our method significantly reduces the training time to around an hour and enhances reconstruction quality with a PSNR improvement of 1–2 dB, outperforming other state-of-the-art NeRF models.

MVS‐SLAM: Enhanced multiview geometry for improved semantic RGBD SLAM in dynamic environment

AbstractSimultaneous Localization and Mapping (SLAM) is a crucial technology for intelligent mobile robots to operate successfully in unknown environments. While many excellent SLAM systems have been developed in recent years, most assume that the environment is static, resulting in poor performance in dynamic environments. To address this limitation, we propose multiview stereo (MVS)‐SLAM (MVS‐SLAM), a real‐time semantic RGBD SLAM system with improved‐multiview geometry, built on the RGBD mode of ORB‐SLAM3. MVS‐SLAM tightly integrates semantic and geometric information to tackle the challenges posed by dynamic scenes. To meet the real‐time requirements, the semantic module leverages the latest and fastest object detection network, YOLOv7, to provide semantic prior knowledge for the geometric module. We introduce a novel geometric constraint method that capitalizes on depth images and semantic information to recover three‐dimensional (3D) feature points and initial camera pose. We use a 3D coordinate error threshold to identify dynamic points and remove them using the K‐means clustering algorithm. This approach effectively reduces the impact of dynamic points. We validate MVS‐SLAM using challenging dynamic sequences from the TUM data set, demonstrating that it significantly improves localization accuracy and system robustness in all types of dynamic environments.

Design, synthesis, computational study and cytotoxic evaluation of some new quinazoline derivatives containing pyrimidine moiety

Quinazoline derivatives, as an important category of heterocyclic compounds, have received much attention for the design and development of new drugs due to their various pharmacological properties. Besides, there is a great deal of evidence showing pyrimidine analogs as anticancer agents. Thus, in the present study, for the design of new target compounds with cytotoxic activity, we focused on various quinazolinone and pyrimidine hybrids. A new series of quinazoline-pyrimidine hybrid derivatives (6a-6n) have been designed and synthesized as novel antiproliferative agents. All the synthesized compounds characterized based on their IR, NMR and Mass spectroscopic data. Antiproliferative activities of the new compounds were evaluated against three human cancer cell lines (MCF-7, A549, SW-480). The compounds were found to have appropriate potential with IC50 values ranging from 2.3 ± 5.91 to 176.5 ± 0.7 μM against the tested cell lines. Compound 6n exerted the highest antiproliferative activity with IC50 values of 5.9 ± 1.69 μM, 2.3 ± 5.91 μM and 5.65 ± 2.33 μM against A549, SW-480 and MCF-7 respectively. The results indicated that 6n could induce apoptosis in A549 cell line in a dose dependent manner and arrest in the S phase of cell cycle. Docking studies were also done to investigate the detailed binding pattern of the synthesized compounds against EGFR. Furthermore, molecular dynamic simulation and binding free energy calculation have been done to rescore initial docking pose of the synthesized compounds using ensemble-based MMGB/PBSA free energy method. According to the results, free energy calculation confirmed biological activity of compounds and also, Arg 817 and Lys 721 residues had the pivotal role in the high potency of 6n. Finally, the drug likeness and in silico ADME study were also predicted.