Abstract
It has been shown that global scene understanding tasks like layout estimation can benefit from wider field of views, and specifically spherical panoramas. While much progress has been made recently, all previous approaches rely on intermediate representations and postprocessing to produce Manhattan-aligned estimates. In this work we show how to estimate full room layouts in a single-shot, eliminating the need for postprocessing. Our work is the first to directly infer Manhattan-aligned outputs. To achieve this, our data-driven model exploits direct coordinate regression and is supervised end-to-end. As a result, we can explicitly add quasi-Manhattan constraints, which set the necessary conditions for a homography-based Manhattan alignment module. Finally, we introduce the geodesic heatmaps and loss and a boundary-aware center of mass calculation that facilitate higher quality keypoint estimation in the spherical domain. Our models and code are publicly available at https://github.com/VCL3D/SingleShotCuboids.
Highlights
Modern hardware advances have commoditized spherical cameras1 which have evolved beyond elaborate optics and camera clusters
Our work has focused on keypoint estimation on the sphere and in particular on layout corner estimation
While we have shown that end-to-end single-shot layout estimation is possible, our approach is rigid as it is based on a frequent and logical assumption, that the underlying room is, or can be approximated by, a cuboid
Summary
Modern hardware advances have commoditized spherical cameras which have evolved beyond elaborate optics and camera clusters. Datasets containing spherical panoramas like Matterport3D [3] and Stanford2D3D [1], were created using the Matterport camera, originally developed for virtual tours This signifies the importance of spherical panoramas for indoor 3D capturing, as they are (re-)used in multiple 3D vision tasks [55, 50, 67]. While modern deep priors produce higher quality results [68, 60], increasing the accuracy of their predictions and ensuring Manhattan-aligned layouts, requires postprocessing and hurts runtime efficiency. Spherical panoramas necessitate higher resolution processing, and increased computational complexity, as evidenced by recent data-driven layout estimation models [68, 60, 47]. Capitalizing on this, we further integrate full Manhattan alignment directly into the model, allowing for end-to-end training, lifting the postprocessing requirement
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
