Multiple Poses Research Articles

Poliface: A Multi-pose Synchronous Imaging System

To enhance the accuracy of face recognition technology in real-world scenarios, it is necessary to train deep learning models on datasets that contain a large number of labeled human face images under multiple poses, lighting, and accessory variations. In this paper, we introduce a novel acquisition system named the Poliface. This system can capture multiple high-resolution images simultaneously around the human head. We designed this system with a well-built aluminum structure, control electronic circuits, and high-performing in-house software. The results demonstrate the precise operation and exceptional stability of this system. Using this Poliface system, we have collected over 6 million photos, which can be used to train and evaluate facial recognition models, and exploited for three-dimensional (3D) virtual face reconstruction.

Binding of Inhibitors to Nuclear Localization Signal Peptide from Venezuelan Equine Encephalitis Virus Capsid Protein Explored with All-Atom Replica Exchange Molecular Dynamics.

Several small molecule inhibitors have been designed to block binding of the Venezuelan equine encephalitis virus (VEEV) nuclear localization signal (NLS) sequence to the importin-α nuclear transport protein. To probe the inhibition mechanism on a molecular level, we used all-atom explicit water replica exchange molecular dynamics to study the binding of two inhibitors, I1 and I2, to the coreNLS peptide, representing the core fragment of the VEEV NLS sequence. Our objective was to evaluate the possibility of masking wherein binding of these inhibitors to the coreNLS occurs prior to its binding to importin-α. We found that the free energy of I1 and I2 binding to the coreNLS is less favorable than that to importin-α. This outcome argues against preemptive inhibitor binding to the coreNLS prior to importin-α. Instead, both inhibitors are expected to compete with the coreNLS peptide for binding to importin-α. The two factors responsible for the low affinities of the inhibitors to the coreNLS peptide are (i) the low cooperativity of binding to the peptide and (ii) the strong hydrophobic effect associated with binding to importin-α. Our results further show that upon binding to the coreNLS peptide, the inhibitors form multiple diverse binding poses. The coreNLS peptide coincubated with I1 and I2 adopts several conformational states, including open and collapsed, which underscores the fluidity of the coreNLS conformational ensemble as a target for inhibitors. Taken together with our prior investigations, this study sheds light on the molecular mechanism by which I1 and I2 ligands inhibit binding of the VEEV capsid protein to importin-α.

A multi-channel spatial information feature based human pose estimation algorithm

Human pose estimation is an important task in computer vision, which can provide key point detection of human body and obtain bone information. At present, human pose estimation is mainly utilized for detection of large targets, and there is no solution for detection of small targets. This paper proposes a multi-channel spatial information feature based human pose (MCSF-Pose) estimation algorithm to address the issue of medium and small targets inaccurate detection of human key points in scenarios involving occlusion and multiple poses. The MCSF-Pose network is a bottom-up regression network. Firstly, an UP-Focus module is designed to expand the feature information while reducing parameter computation during the up-sampling process. Then, the channel segmentation strategy is adopted to cut the features, and the feature information of multiple dimensions is retained through different convolutional groups, which reduces the parameter lightweight network model and makes up for the loss of the feature information associated with the depth of the network. Finally, the three-layer PANet structure is designed to reduce the complexity of the model. With the aid of the structure, it also to improve the detection accuracy and anti-interference ability of human key points. The experimental results indicate that the proposed algorithm outperforms YOLO-Pose and other human pose estimation algorithms on COCO2017 and MPII human pose datasets.

A Study of Multi-Pose Effects On a Face Recognition System

Interpersonal and intrapersonal face variation interference caused by multiple poses is challenging for distance-based face recognition systems. In this paper, we investigate the face-feature distance distribution for Chinese multi-pose faces. The simulation shows that the number of individuals in the gallery database will greatly affect the recognition performance for near-profile face images. It also provides a prediction of the Top-N occurrence rates in different gallery-size environments.

CharacterGen: Efficient 3D Character Generation from Single Images with Multi-View Pose Canonicalization

BNRist, Department of Computer Science and Technology, Tsinghua University, China In the field of digital content creation, generating high-quality 3D characters from single images is challenging, especially given the complexities of various body poses and the issues of self-occlusion and pose ambiguity. In this paper, we present CharacterGen, a framework developed to efficiently generate 3D characters. CharacterGen introduces a streamlined generation pipeline along with an image-conditioned multi-view diffusion model. This model effectively calibrates input poses to a canonical form while retaining key attributes of the input image, thereby addressing the challenges posed by diverse poses. A transformer-based, generalizable sparse-view reconstruction model is the other core component of our approach, facilitating the creation of detailed 3D models from multi-view images. We also adopt a texture-back-projection strategy to produce high-quality texture maps. Additionally, we have curated a dataset of anime characters, rendered in multiple poses and views, to train and evaluate our model. Our approach has been thoroughly evaluated through quantitative and qualitative experiments, showing its proficiency in generating 3D characters with high-quality shapes and textures, ready for downstream applications such as rigging and animation.

Image-based non-isotropic point light source calibration using digital fringe projection

Accurate light source calibration is critical in numerous applications including physics-based computer vision and graphics. We propose an image-based method for calibrating non-isotropic point light sources, addressing challenges posed by their non-radially symmetric radiant intensity distribution and the non-Lambertian properties of the calibration target. We deduce an image formation model, and capture the intensity of the calibration target at multiple poses, coupled with accurate 3D geometry acquisition using the digital fringe projection technique. Finally, we design an iterative computational framework to optimize all the light source parameters simultaneously. The experiment demonstrated the high accuracy of the calibrated light source model and showcased its application by estimating the relative reflectance of diverse surfaces.

ClassyPose: A Machine‐Learning Classification Model for Ligand Pose Selection Applied to Virtual Screening in Drug Discovery

Determining the target‐bound conformation of a drug‐like molecule is a crucial step in drug design, as it affects the outcome of virtual screening (VS), and paves the way for hit‐to‐lead and lead optimization. While most docking programs usually manage to produce at least a near‐native pose for a bioactive molecule inside its binding pocket, their integrated classical scoring functions (SFs) generally fail to prioritize this pose. Many studies have been carried out to tackle this SF problem, offering multiple pose refinement and/or classification methods, albeit with limitations. This study presents a new support vector machine model for pose classification, called “ClassyPose”, which predicts the probability that a receptor‐bound ligand conformation could be near‐native, without any additional pose optimization step. Trained on protein‐ligand extended connectivity features extracted from over 21 600 crystal and docking poses of diverse ligands, this model outperformed other machine‐learning algorithms and three existing SFs in terms of docking power, identifying the native ligand pose as top‐ranked solution for more than 90% of entries in two test sets. It also achieved high specificity (above 0.96), and improved VS performance when used for pose selection. This efficient, user‐friendly tool and all related data are available at https://github.com/vktrannguyen/Classy_Pose.

Three-Dimensional Human Pose Estimation from Micro-Doppler Signature Based on SISO UWB Radar

In this paper, we propose an innovative approach for transforming 2D human pose estimation into 3D models using Single Input–Single Output (SISO) Ultra-Wideband (UWB) radar technology. This method addresses the significant challenge of reconstructing 3D human poses from 1D radar signals, a task traditionally hindered by low spatial resolution and complex inverse problems. The difficulty is further exacerbated by the ambiguity in 3D pose reconstruction, as multiple 3D poses may correspond to similar 2D projections. Our solution, termed the Radar PoseLifter network, leverages the micro-Doppler signatures inherent in 1D radar echoes to effectively convert 2D pose information into 3D structures. The network is specifically designed to handle the long-range dependencies present in sequences of 2D poses. It employs a fully convolutional architecture, enhanced with a dilated temporal convolutions network, for efficient data processing. We rigorously evaluated the Radar PoseLifter network using the HPSUR dataset, which includes a diverse range of human movements. This dataset comprises data from five individuals with varying physical characteristics, performing a variety of actions. Our experimental results demonstrate the method’s robustness and accuracy in estimating complex human poses, highlighting its effectiveness. This research contributes significantly to the advancement of human motion capture using radar technology. It presents a viable solution for applications where precision and reliability in motion capture are paramount. The study not only enhances the understanding of 3D pose estimation from radar data but also opens new avenues for practical applications in various fields.

Lifting by Image – Leveraging Image Cues for Accurate 3D Human Pose Estimation

The "lifting from 2D pose" method has been the dominant approach to 3D Human Pose Estimation (3DHPE) due to the powerful visual analysis ability of 2D pose estimators. Widely known, there exists a depth ambiguity problem when estimating solely from 2D pose, where one 2D pose can be mapped to multiple 3D poses. Intuitively, the rich semantic and texture information in images can contribute to a more accurate "lifting" procedure. Yet, existing research encounters two primary challenges. Firstly, the distribution of image data in 3D motion capture datasets is too narrow because of the laboratorial environment, which leads to poor generalization ability of methods trained with image information. Secondly, effective strategies for leveraging image information are lacking. In this paper, we give new insight into the cause of poor generalization problems and the effectiveness of image features. Based on that, we propose an advanced framework. Specifically, the framework consists of two stages. First, we enable the keypoints to query and select the beneficial features from all image patches. To reduce the keypoints attention to inconsequential background features, we design a novel Pose-guided Transformer Layer, which adaptively limits the updates to unimportant image patches. Then, through a designed Adaptive Feature Selection Module, we prune less significant image patches from the feature map. In the second stage, we allow the keypoints to further emphasize the retained critical image features. This progressive learning approach prevents further training on insignificant image features. Experimental results show that our model achieves state-of-the-art performance on both the Human3.6M dataset and the MPI-INF-3DHP dataset.

Focus on melanotic neuroectodermal tumor of infancy

Melanotic neuroectodermal tumor of infancy (MNTI) is a very rare benign neoplasm of probable neurocristic origin. It primarily affects children in the first year of life, with the median age at diagnosis of 4.5 months (range 0–804 months). It usually presents as a fast-growing, painless tumor developing within maxilla, skull bones or mandible but other locations are also possible, especially in older children. The cornerstone of treatment of MNTI is surgery, however local relapses after incomplete tumor excision are common, particularly in patients younger than 2 months of age. Rare cases of multiple recurrent, inoperable or metastatic MNTI pose therapeutic challenges. In such clinical scenarios, various regimens of neoadjuvant chemotherapy based on schemes for neuroblastoma or Ewing sarcoma have been used with partial regressions in some patients, enabling less mutilating delayed surgery. The use of radiotherapy is limited due to very young age of patients with MNTI. No targeted therapies have been found useful so far. Long-term prognosis of localized MNTI is favorable. However, extensive or recurrent lesions can result in functional or esthetic sequelae after surgical removal. Rare cases of malignant/metastatic tumors and MNTI diagnosed in older children have unfavorable outcomes. Further collaborative studies to establish standards of management in patients with MNTI are necessary to improve outcomes and diminish sequelae of surgery. This article presents a literature review on this very rare tumor entity, re-evaluated in the light of the experience gained in the national working groups joined together within the European Cooperative Study Group in Pediatric Rare Tumors (EXPeRT).

Implicit space pose consistent transfer network for deep face verification

Unconstrained face recognition has achieved state-of-the-art performance with the help of the powerful representation capabilities of deep convolutional neural networks. However, face recognition always suffers from other intra-class imbalances due to the influence of pose variations present in the real world, which significantly degrades the performance, especially for tilted face recognition under surveillance videos, which is rarely studied. Tilted face recognition is more challenging than frontal and profile face recognition due to the effect of self-obscuration. To this end, we consider promoting the frontalization of tilted faces in the implicit space, while preserving identity information. In this paper, we propose an implicit spatial pose consistent transfer network (PCTN) that exploits the prior guidance of pose information to enhance the model’s ability to perceive pose. Specifically, we first develop the multi-path attention block (MPAB), which is used as the main block for feature embedding in deep networks. Such a design is employed to enhance the relevance of multi-layer convolutional features and provide a powerful foundation for the deep asymptotic transfer of feature frontalization. Then, the pose consistent transfer module (PCTM) combines pose labels and deep embedding features to learn robust discriminative information. PCTM consists of two parts: the pose guidance block (PGB) explores the implicit nonlinear feature mapping from pose faces to frontal faces in feature space, and the attention compensation module (ACM) aims to exploit the complementarity of dense and sparse features, enhancing pose-related features and suppressing pose-unrelated features in terms of both spatial-wise and channel-wise. Furthermore, we investigate the performance comparison on datasets with multiple pose variations without performance degradation. Extensive experimental results on multiple benchmark datasets demonstrate the effectiveness of our proposed method.

Exploring synergy between azole antifungal drugs and statins for Candida auris.

Global emergence of rapidly developing resistance to multiple antifungal drugs and high mortality pose challenges to the treatment of invasive Candida auris infections. New therapeutic approaches are needed, such as repurposing drugs including combination with antifungals. Statins have been reported to exert antifungal effects against various Candida species. Our study investigated potential synergy between the statins (rosuvastatin and fluvastatin) and azoles (voriconazole, posaconazole and isavuconazole) on clinical isolates of C. auris. Twenty-one clinical isolates of C. auris were obtained. Chequerboard assays based on the CLSI broth microdilution method were used to assess synergy based on FIC index (FICI) calculations of MICs of individual drugs and in combinations. Single drug geometric mean (GM) MICs of fluvastatin and rosuvastatin were ≥128 mg/L in all 21 isolates. GM (range) MICs of posaconazole, voriconazole and isavuconazole were 0.259 (0.016-1 mg/L), 0.469 (0.016-2 mg/L) and 0.085 (0.004-1 mg/L), respectively. Combination of azoles with fluvastatin showed synergy in 70%-90% of C. auris isolates. In particular, voriconazole/fluvastatin resulted in 16-fold reduction in voriconazole MIC and synergy in 14/21 (67%) isolates. Posaconazole/fluvastatin resulted in 8-fold reduction in posaconazole MIC and synergy in 19/21 (90%) isolates.Combining rosuvastatin with the azoles also showed synergy against C. auris in 40%-60% of the isolates and additive effect in 40%-50%. None of the combinations was antagonistic. Our results provide a rationale for pursuing in vivo synergy tests as well as clinical studies to explore tolerability, treatment outcomes, optimal dose and exposure targets.

MHCanonNet: Multi-Hypothesis Canonical lifting Network for self-supervised 3D human pose estimation in the wild video

Recent advancements in 3D Human Pose Estimation using fully-supervised learning approach have shown impressive results; however, these methods heavily rely on large amounts of annotated 3D data, which are challenging to obtain outside controlled laboratory environments. Therefore, in this study, we propose a new self-supervised training method designed to train a 3D human pose estimation network using unlabeled multi-view images. The model trains relative depths between joints without any 3D annotation by satisfying multi-view consistency constraints from unlabeled multi-view videos without camera calibration, while simultaneously learning representations of multiple plausible pose hypotheses. For this reason, we call our proposed network a Multi-Hypothesis Canonical Lifting Network (MHCanonNet). By enriching the diversity of extracted features and keeping various possibilities open, our network accurately estimates the final 3D pose. The key idea lies in the design of a novel and unbiased reconstruction objective function that combines multiple hypotheses from different viewpoints. The proposed approach demonstrates state-of-the-art results not only on two popular benchmark datasets, Human3.6M and MPI-INF-3DHP but also on an in-the-wild dataset, Ski-Pose, surpassing existing self-supervised training methods.

Binding of viral nuclear localization signal peptides to importin-α nuclear transport protein

Using all-atom replica-exchange molecular dynamics simulations, we mapped the mechanisms of binding of the nuclear localization signal (NLS) sequence from Venezuelan equine encephalitis virus (VEEV) capsid protein to importin-α (impα) transport protein. Our objective was to identify the VEEV NLS sequence fragment that confers native, experimentally resolved binding to impα as well as to study associated binding energetics and conformational ensembles. The two selected VEEV NLS peptide fragments, KKPK and KKPKKE, show strikingly different binding mechanisms. The minNLS peptide KKPK binds non-natively and nonspecifically by adopting five diverse conformational clusters with low similarity to the x-ray structure 3VE6 of NLS-impα complex. Despite the prevalence of non-native interactions, the minNLS peptide still largely binds to the impα major NLS binding site. In contrast, the coreNLS peptide KKPKKE binds specifically and natively, adopting a largely homogeneous binding ensemble with a dominant, highly native-like conformational cluster. The coreNLS peptide retains most of native binding interactions, including π-cation contacts and a tryptophan cage. While KKPK binding is governed by a complex multistate free energy landscape featuring transitions between multiple binding poses, the coreNLS peptide free energy map is simple, exhibiting a single dominant native-like bound basin. We argue that the origin of the coreNLS peptide binding specificity is several electrostatic interactions formed by the two C-terminal amino acids, Lys10 and Glu11, with impα. The coreNLS sequence is then sufficient for native binding, but none of the amino acids flanking minNLS, including Lys10 and Glu11, are strictly necessary for the native pose. Our analyses indicate that the VEEV coreNLS sequence is virtually unique among human and viral proteins interacting with impα making it a potential target for VEEV-specific inhibitors.

3D Human Pose Estimation Based on Transformer

Currently, 3D human pose estimation has gradually been a well-liked subject. Although various models based on the deep neural network have produced an excellent performance, they still suffer from the ignorance of multiple feasible pose solutions and the problem of the relatively-fixed input length. To solve these issues, a coordinate transformer encoder based on a 2D pose is constructed to generate multiple feasible pose solutions, and multi-to-one pose mapping is employed to generate a reliable pose. A temporal transformer encoder is used to exploit the temporal dependencies of consecutive pose sequences, which avoids the issue of relatively-fixed input length caused by temporal dilated convolution. Adequate experiments indicate that our model achieves a promising performance.

3D global optimization of calibration parameters of deflectometry system by using a spherical mirror

The calibration of a deflectometry system is an essential requirement for achieving accurate specular surface reconstruction. In this paper, we propose a 3D-based optimization method for the calibration of system parameters, which achieves the same level of accuracy as the 2D multi-pose optimization, but utilizes only a single pose. Our approach introduces a high-precision spherical mirror as the target mirror (TM), and the cost function is the residual of the reconstruction of the TM, as opposed to the 2D re-projection error in image space. Single-pose calibration optimization is achieved because the spherical surface can be regarded as a combination of multiple pose plane mirrors. The simulations and experiments demonstrate that TM with a smaller radius outperforms other methods, and the surface reconstruction error of a spherical mirror with a radius of 500 mm is 186.3 nm RMS, offering a fast and efficient system calibration scheme for in-situ measurement in practical application scenarios.

2D pose estimation of multiple tomato fruit-bearing systems for robotic harvesting

Fruit detection is a priority for successful robotic harvesting. It should be able to provide geometrical information regarding the target fruit to be harvested. Recently, branch detection has been frequently used to determine cutting points for harvesting fruits, and studies have mostly focused on pedicel (or peduncle) detection in tomatoes. Previous studies detected the target tomato first and then estimated the cutting points on pedicels under the assumption that the target could be harvested; however, the cutting point of the detected tomato could not be guaranteed to be estimated visually by occlusions and postures. It is efficient to grasp the poses of all tomato-pedicel pairs in a scene simultaneously for determining the target object to be easily harvested. Thus, we proposed the 2D pose estimation of multiple tomatoes with pedicel based on the bottom-up method; we first detected all the tomato pose-related components in an image and then estimated the poses of each object. The proposed approach used a human pose estimation method as the backbone, and the model was trained suite to the four keypoints defined in this study: tomato-center, calyx, abscission zone, and branch point. The results demonstrate that the percentage of detected keypoints (PDK@0.3) was 0.85 and 0.80 for keypoints and their linkages, respectively, and the distal pedicel, which is a important part as a cutting point for harvesting, indicated a performance higher than 0.94. In conclusion, the proposed method can provide a multiple tomato-pedicel pose that can perform with a relatively constant amount of computation, regardless of the number of objects in a scene and can contribute to determine harvesting priorities based on accessibility evaluation during robotic harvesting.

A novel hand-eye calibration method for industrial robot and line laser vision sensor

PurposeThe purpose of this study is solving the hand–eye calibration issue for line structured light vision sensor. Only after hand–eye calibration the sensor measurement data can be applied to robot system.Design/methodology/approachIn this paper, the hand–eye calibration methods are studied, respectively, for eye-in-hand and eye-to-hand. Firstly, the coordinates of the target point in robot system are obtained by tool centre point (TCP), then the robot is controlled to make the sensor measure the target point in multiple poses and the measurement data and pose data are obtained; finally, the sum of squared calibration errors is minimized by the least square method. Furthermore, the missing vector in the process of solving the transformation matrix is obtained by vector operation, and the complete matrix is obtained.FindingsOn this basis, the sensor measurement data can be easily and accurately converted to the robot coordinate system by matrix operation.Originality/valueThis method has no special requirement for robot pose control, and its calibration process is fast and efficient, with high precision and has practical popularized value.

Multi-hypothesis representation learning for transformer-based 3D human pose estimation

Despite significant progress, estimating 3D human poses from monocular videos remains a challenging task due to depth ambiguity and self-occlusion. Most existing works attempt to solve both issues by exploiting spatial and temporal relationships. However, those works ignore the fact that it is an inverse problem where multiple feasible solutions (i.e., hypotheses) exist. To relieve this limitation, we propose a Multi-Hypothesis Transformer that learns spatio-temporal representations of multiple plausible pose hypotheses. In order to effectively model multi-hypothesis dependencies and build strong relationships across hypothesis features, we introduce a one-to-many-to-one three-stage framework: (i) Generate multiple initial hypothesis representations; (ii) Model self-hypothesis communication, merge multiple hypotheses into a single converged representation and then partition it into several diverged hypotheses; (iii) Learn cross-hypothesis communication and aggregate the multi-hypothesis features to synthesize the final 3D pose. Through the above processes, the final representation is enhanced and the synthesized pose is much more accurate. Extensive experiments show that the proposed method achieves state-of-the-art results on two challenging datasets: Human3.6M and MPI-INF-3DHP. The code and models are available at https://github.com/Vegetebird/MHFormer.

Error Covariance Estimation of 3D Point Cloud Registration Considering Surrounding Environment

To realize autonomous vehicle safety, it is important to accurately estimate the vehicle’s pose. As one of the localization techniques, 3D point cloud registration is commonly used. However, pose errors are likely to occur when there are few features in the surrounding environment. Although many studies have been conducted on estimating error distribution of 3D point cloud registration, the real environment is not reflected. This paper presents real-time error covariance estimation in 3D point cloud registration according to the surrounding environment. The proposed method provides multiple initial poses for iterative optimization in the registration method. Using converged poses in multiple searches, the error covariance reflecting the real environment is obtained. However, the initial poses were limited to directions in which the pose error was likely to occur. Hence, the limited search efficiently determined local optima of the registration. In addition, the process was conducted within 10 Hz, which is laser imaging detection and ranging (LiDAR) period; however, the execution time exceeded 100 ms in some places. Therefore, further improvement is necessary.