Abstract
This paper addresses the challenge of 3D object detection from a single panoramic image under severe deformation. The advent of the two-stage approach has impelled significant progress in 3D object detection. However, most available methods only can localize region proposals by a single-scale architecture network, which are sensitive to deformation and distortion. To address this issue, we propose a multi-scale convolutional neural network (MSCNN) to estimate the 3D pose of an object. To be specific, the proposed MSCNN consists of three steps for effectively detecting the distorted object on the panoramic images. The MSCNN contains the CycleGAN network that converts rectilinear images into panoramas, a fused framework that improves both accuracy and speed for object detection, and an adversarial spatial transformer network (ASTN) that extracts the deformation features of the object on panoramic images. Additionally, we recover the 3D pose of the object using a coordinate projection and a 3D bounding box. Extensive experiments demonstrate that the proposed method can achieve a 3D detection accuracy of 38.7% in high-resolution panoramic images, which is higher than the current state-of-the-art algorithm of 5.2%. Moreover, the speed of detection is only about 0.6 seconds per image, which is six times faster than Faster R-CNN (COCO). The code will be available at https://github.com/Yanhui-He.
Highlights
The panoramic image visualization platform has enjoyed popularity in many applications, such as virtual reality, visual surveillance, autonomous vehicles and virtual interaction [1]
We propose a new method to detect the objects of 360◦ panoramic imagery using a multi-scale convolutional neural network (MSCNN)
We propose a novel fusion network for 3D object detection with a Multi-scale Convolutional Neural Network (MSCNN), which learns a distortional representation for robust 3D detection localization in panoramic images
Summary
The panoramic image visualization platform has enjoyed popularity in many applications, such as virtual reality, visual surveillance, autonomous vehicles and virtual interaction [1]. Panoramic images are typically represented using an equirectangular projection, The associate editor coordinating the review of this manuscript and approving it for publication was Li He. which creates severe geometric distortions for objects that are further from the central horizontal line [3]. Which creates severe geometric distortions for objects that are further from the central horizontal line [3] ERA images create new challenges for computer vision and image processing as i) we lack high-quality annotated 360◦ datasets, ii) imagery is difficult to treat due to its high-resolution and iii) equirectangular projection creates severe geometric distortions for objects away from the central horizontal line [3]. Panoramic images create new challenges for object detection, which is a crucial procedure
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
