Abstract
An opto-electronic neural network is designed for video object detection from a long-exposure blurred image. This network combines an optical encoder, convolutional neural network decoder, and object detection module, which are jointly optimized end-to-end. The joint loss is adopted for updating the network according to the physical constraints of hardware via back-propagation. A high-speed refreshed spatial light modulator is used as the encoder part of the network to generate coded sub-images, and then, a single blurred image is obtained after a common camera. The rest of the network is used for video object detection. Both simulations and experiments demonstrate that our framework can successfully retrieve object labels and bounding boxes at different moments in the long exposure. To the best of our knowledge, this is the first work investigating video object detection from a single motion-degraded image.
Highlights
With the flourishing development of deep learning, a number of computer vision tasks have received much attention that teaches machines to perceive the physical world
The parameter λ in the joint loss scitation.org/journal/app is set to 1 for the Adam optimizer and set to 0 for the stochastic gradient descent (SGD)
Training is performed on a workstation with a 3.3 GHz Intel Core i9-9940X central processing unit (CPU) (32 GB RAM) and two Nvidia GeForce RTX2080Ti GPUs
Summary
With the flourishing development of deep learning, a number of computer vision tasks have received much attention that teaches machines to perceive the physical world. Object detection has been widely used in a wide range of applications, including autonomous driving, robot vision, and video surveillance. These applications require high-quality images as input to extract precise target features. Regarding motion blur as noise and performing deblurring is a classic software solution.. All existing methods are limited to the task of generating only “one” deblurred image, which loses the information about the motion of the objects in the blurry image. The motion blur combines information about the texture and motion of the object, which can be used for video object detection (VID) in the motion process rather than just as noise
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
