Probabilistic rotation modeling based on directional mixture density networks

Abstract

Predicting 3D rotations from a single image presents a significant challenge, primarily due to the inherent uncertainty arising from factors such as high symmetry, self-obscuration, and noise in the 3D environment. In this work, we propose a novel multimodal-based probabilistic model that integrates the matrix Fisher distribution and von Mises Fisher distribution into a mixture density network. Our model not only captures the inherent uncertainty of the object but also learns this uncertainty directly from the training data, thereby enhancing the robustness, flexibility, and efficiency of the model. To further refine the model's ability to handle ambiguities and recognize multiple distinct modes, we introduce a relaxed version of the winner-take-all loss function. This adaptation significantly improves the model's capability in accurately representing complex multimodal distributions. The performance of our model is rigorously assessed using two challenging datasets: Pascal3D+ and ModelNet10-SO(3). Extensive experimental analysis highlights the model's exceptional capability to fit complex multimodal distributions. Notably, when tested on the ModelNet10-SO(3) dataset, which is characterized by its ambiguity, and the more unequivocal Pascal3D+ dataset, our model outperforms the prevailing top baseline models by achieving accuracy improvements of 2.7% and 3.4%, respectively, at the minimum angle threshold. These results not only demonstrate our model's advanced capabilities in fitting complex distributions but also validate its effectiveness in accurately predicting 3D rotations in both ambiguous and unambiguous scenarios.