A vision sensor for simultaneous imaging and distance sensing in real-time

Abstract

A vision sensor for simultaneous imaging and distance sensing is proposed herein. This vision sensor uses two images with a depth of field by two different aperture sizes to extract distance data. The aperture was fabricated through the microelectromechanical systems process. The optical parameter related to making a blur was precisely selected to reduce the active voltage and response time. The aperture measurement result showed that the maximum displacement of 1170 µm was obtained when 12 V was applied which enlarge aperture size from 2.75 mm to 3.92 mm. The response time was 16.6 ms with a rising and falling time of 6.2 and 10.4 ms, respectively. The distance data was obtained using the depth from defocus method, which compares the blurriness of two images using the aperture size. Through deep learning, the image and distance information were simultaneously obtained in a single camera. The result of a three-dimensional depth map showed an average accuracy of 98.7% when sensing the maximum distance of 10 m. To examine the accuracy of the device, experiments were conducted for different colors, and the result showed that maximum and minimum error rates of 3.46% and 1.83% were achieved, respectively. In addition, the error rate according to brightness was investigated, and the average error rate was maintained at 2.64% between 10 000 and 200 lx. The proposed sensor can be installed in self-driving robots, drones, and various smart devices.