Abstract
This study develops a projector–camera-based visible light communication (VLC) system for real-time broadband video streaming, in which a high frame rate (HFR) projector can encode and project a color input video sequence into binary image patterns modulated at thousands of frames per second and an HFR vision system can capture and decode these binary patterns into the input color video sequence with real-time video processing. For maximum utilization of the high-throughput transmission ability of the HFR projector, we introduce a projector–camera VLC protocol, wherein a multi-level color video sequence is binary-modulated with a gray code for encoding and decoding instead of pure-code-based binary modulation. Gray code encoding is introduced to address the ambiguity with mismatched pixel alignments along the gradients between the projector and vision system. Our proposed VLC system consists of an HFR projector, which can project 590 × 1060 binary images at 1041 fps via HDMI streaming and a monochrome HFR camera system, which can capture and process 12-bit 512 × 512 images in real time at 3125 fps; it can simultaneously decode and reconstruct 24-bit RGB video sequences at 31 fps, including an error correction process. The effectiveness of the proposed VLC system was verified via several experiments by streaming offline and live video sequences.
Highlights
With the recent rapid advances in computer and image sensor technologies, many high frame rate (HFR) vision systems that can capture and process images simultaneously at thousands of frames per second have been developed [1,2,3,4,5]; many tracking algorithms, such as optical flow estimation [6,7], cam-shift tracking [8], and feature-point tracking [9], have been accelerated by the parallel implementation of these algorithms on field programmable gate array (FPGA) and graphics processing units
Sensors 2020, 20, 5368 vibrating at hundreds or thousands of hertz, HFR projector systems based on the digital micro-mirror device (DMD) technology [22,23] can project binary image patterns at thousands of frames per second or more; several types of HFR projector–camera systems [24,25,26] have been reported in various applications, such as structured light based 3D sensing [27,28,29,30] and simultaneous projection mapping [31,32,33,34]
If a real-time HFR vision system could function as a communication receiver to perfectly capture and encode the HFR-blinking high-space-resolution image patterns with real-time video processing at thousands of frames per second, which are too fast for human eyes to see, and high-thorough visible light communication (VLC) could be realized for broadband video streaming by utilizing the high-resolution in mega-pixel order and high-frequency band in the kHz order in the HFR projection of an HFR projector
Summary
With the recent rapid advances in computer and image sensor technologies, many high frame rate (HFR) vision systems that can capture and process images simultaneously at thousands of frames per second have been developed [1,2,3,4,5]; many tracking algorithms, such as optical flow estimation [6,7], cam-shift tracking [8], and feature-point tracking [9], have been accelerated by the parallel implementation of these algorithms on field programmable gate array (FPGA) and graphics processing units. This system can simultaneously transmit and receive a 24-bit RGB 590 × 1060 video sequence at maximum 31 fps
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
