Camera-based Visible Light Communication Research Articles

Visual-Feedback-Based Frame-by-Frame Synchronization for 3000 fps Projector–Camera Visual Light Communication

This paper proposes a novel method for synchronizing a high frame-rate (HFR) camera with an HFR projector, using a visual feedback-based synchronization algorithm for streaming video sequences in real time on a visible-light communication (VLC)-based system. The frame rates of the camera and projector are equal, and their phases are synchronized. A visual feedback-based synchronization algorithm is used to mitigate the complexities and stabilization issues of wire-based triggering for long-distance systems. The HFR projector projects a binary pattern modulated at 3000 fps. The HFR camera system operates at 3000 fps, which can capture and generate a delay signal to be given to the next camera clock cycle so that it matches the phase of the HFR projector. To test the synchronization performance, we used an HFR projector–camera-based VLC system in which the proposed synchronization algorithm provides maximum bandwidth utilization for the high-throughput transmission ability of the system and reduces data redundancy efficiently. The transmitter of the VLC system encodes the input video sequence into gray code, which is projected via high-definition multimedia interface streaming in the form of binary images 590 × 1060. At the receiver, a monochrome HFR camera can simultaneously capture and decode 12-bit 512 × 512 images in real time and reconstruct a color video sequence at 60 fps. The efficiency of the visual feedback-based synchronization algorithm is evaluated by streaming offline and live video sequences, using a VLC system with single and dual projectors, providing a multiple-projector-based system. The results show that the 3000 fps camera was successfully synchronized with a 3000 fps single-projector and a 1500 fps dual-projector system. It was confirmed that the synchronization algorithm can also be applied to VLC systems, autonomous vehicles, and surveillance applications.

Rolling-shutter-effect camera-based visible light communication using RGB channel separation and an artificial neural network.

We propose and demonstrate a light-panel and rolling-shutter-effect (RSE) camera-based visible light communication (VLC) system using Z-score normalization, red/green/blue (RGB) color channel separation, and 1-D artificial neural network (ANN). The proposed scheme can mitigate the high inter-symbol interference (ISI) generated by the RSE pattern due to the low pixel-per-bit and high noise-ratio (NR) of the display contents.

HFR Projector Camera Based Visible Light Communication System for Real-Time Video Streaming.

This study develops a projector–camera-based visible light communication (VLC) system for real-time broadband video streaming, in which a high frame rate (HFR) projector can encode and project a color input video sequence into binary image patterns modulated at thousands of frames per second and an HFR vision system can capture and decode these binary patterns into the input color video sequence with real-time video processing. For maximum utilization of the high-throughput transmission ability of the HFR projector, we introduce a projector–camera VLC protocol, wherein a multi-level color video sequence is binary-modulated with a gray code for encoding and decoding instead of pure-code-based binary modulation. Gray code encoding is introduced to address the ambiguity with mismatched pixel alignments along the gradients between the projector and vision system. Our proposed VLC system consists of an HFR projector, which can project 590 × 1060 binary images at 1041 fps via HDMI streaming and a monochrome HFR camera system, which can capture and process 12-bit 512 × 512 images in real time at 3125 fps; it can simultaneously decode and reconstruct 24-bit RGB video sequences at 31 fps, including an error correction process. The effectiveness of the proposed VLC system was verified via several experiments by streaming offline and live video sequences.

Spaced color shift keying modulation for camera-based visible light communication system using rolling shutter effect

By controlling the on–off state of light emitting diode (LED), data can be modulated onto the visible light. By exploiting the rolling shutter effect (RSE) of camera image sensor, the data can be demodulated from the light correspondingly. Based on the principle, a kind of camera-based visible light communication (CVLC) system can be realized by using LED as transmitter and using mobile-phone camera as receiver. In this paper, to enhance the transmission performance and reduce the implementation complexity, a spaced color shift keying (SCSK) modulation scheme is proposed for RSE-based CVLC system. Meanwhile, a demodulation scheme based on HSB color space is proposed correspondingly. By using a commercial RGB-LED and a common smartphone, a CVLC system based on the SCSK scheme is experimentally demonstrated. The experiment results show that under the bit-error rate (BER) of 3.8e−3, the SCSK based VLC system can achieve a data capacity of up to 1.28 kbits per photo, although there are no clock recovery and blooming mitigation processes in the system.

Enhancement of Signal Performance in bidirectional RSE-based CVLC system employing NPM modulation scheme

Abstract Camera-based visible light communication (CVLC) exploiting rolling shutter effect (RSE) is a cost-effective technique that ensures secure data transmission. In this paper, a bidirectional CVLC system is experimentally demonstrated using smartphone/laptop as downlink/uplink transceiver in an experimental prototype testbed. Meanwhile, a Nyquist pre-distorted Manchester (NPM) modulation scheme is proposed and validated in the experimental testbed system. The proposed scheme is based on low-complexity transmitter-side digital signal processing (DSP) techniques, Nyquist shaping and linear pre-distortion, which can be employed for mitigation of the impact of extinction ratio (ER) fluctuation and exposure overlapping in the RSE-based CVLC system. The experimental results show that by using the proposed NPM modulation, the uplink and the downlink can both achieve about 4.8 Kb/s transmission with bit error rate (BER) of below than 3.8e−3. It verifies the effectiveness of the proposed NPM scheme and the corresponding transmitter-side DSP techniques.

Wide receiver orientation using diffuse reflection in camera-based indoor visible light communication

Abstract Camera-based visible light communication (CVLC) is a versatile and potential optical wireless communication using rapidly developing smartphone cameras as the receiver. In CVLC, the provision of both flicker-free illumination and high data rate has been a challenging issue. To some extent, however, this issue can be resolved by utilizing a rolling shutter property that is inherited from complementary metal-oxide semiconductor sensors adopted in all smartphone cameras. We propose a practical and versatile wide receiver orientation in CVLC on the basis of this rolling shutter property. That is, unlike previous studies focused on either a line-of-sight (LOS) or non-line-of-sight (NLOS) link only, an orientation independent rolling shutter based CVLC is presented. Based on a unique method of exploiting diffuse reflection of illumination, a wide orientation of the receiver is adequately supported in an indoor environment, thus making the CVLC data reception more practical and convenient. A special transmitter for this wide orientation is designed to ensure that adequate diffuse reflection is present in all directions, regardless of whether a LOS or NLOS link is established. In addition, adaptive algorithms are proposed to ensure an efficient and reliable wide orientation CVLC. Experiments are conducted and analyzed for practical orientations of the receiver in an indoor CVLC environment. While providing sufficient illumination and various orientations, it is found that the proposed scheme achieves a data rate of 6.72 kbps at a distance of up to 50 cm in a small-scale indoor experiment.

Thresholding Scheme Based on Boundary Pixels of Stripes for Visible Light Communication With Mobile-Phone Camera

In mobile-phone camera-based visible light communication (VLC) system, column matrix selection and threshold setting are very important steps for signal demodulation. As the blooming effect of complementary metal-oxide-semiconductor camera fluctuates grayscale values of pixels, thus causing a decrease in the accuracy of distinguishing data logic 0 and 1. Facing grayscale value fluctuation, using traditional polynomial fitting as the threshold to distinguish data logic 0 and 1, the achievable system performance is not very good. This limits applications of the VLC system. Therefore, advanced thresholding schemes are needed to increase the accuracy of distinguishing data logic 0 and 1, thereby improving the system performance. In this paper, we propose and experimentally demonstrate for the first time a thresholding scheme based on boundary pixels of stripes for improving the VLC system performance. First, we describe the process of boundary pixel position acquisition for the thresholding scheme in detail. Then, we analyze and discuss the selection of the optimum column matrix for signal demodulation. Finally, we demonstrate experiments to verify the feasibility of our proposed thresholding scheme. Experimental results show that our proposed thresholding scheme can achieve a good performance even when grayscale values fluctuate greatly. Compared with the conventional third-order polynomial fitting scheme, our proposed thresholding scheme has advantages in both of transmission distance and data rate. With aid of the optimum selected column matrix and our proposed thresholding scheme, a data rate of 4.5 Kb/s (2.5 Kb/s) is successfully achieved with the transmission distance of 50 cm (175 cm) under the 7% forward error correction limit (bit-error-rate = $3.8\times 10^{-3}$ ) for the VLC system with mobile-phone camera.

Interpixel Interference Mitigation in Visible Light Communication Using Image Sensor

To improve the data rate in a camera-based visible light communication system, multiple LEDs in an LED array are used to transmit multiple bits in one frame. The problem is that LEDs are always captured in the image with some degree of blooming effect. This effect increases the intensity of pixels surrounding the origin LEDs in the image. When multiple LEDs are used, the blooming effect from all LEDs causes the interpixel interference (IPI) problem that greatly degrades the quality of the received signal. This paper analyzes the blooming effect and finds that the IPI problem increases when the number of “ON” LEDs in the array increases. Through the analysis, this paper proposes a differential coding to minimize the number of “ON” LEDs in every transmitted frame and thus minimize the IPI effect. The proposed scheme is then verified through simulations. The results show that a lower bit error rate, which indicates a better signal quality, can be achieved by applying the proposed scheme.

2.38 Kbits/frame WDM transmission over a CVLC system with sampling reconstruction for SFO mitigation.

The camera-based visible light communication (CVLC) exploiting rolling shutter effect (RSE) is a cost-efficient technique that ensures secured data transmission. By controlling the on-off state of the light emitting diode (LED), data can be modulated onto the emitting visible light. The corresponding data recovery can be realized by exploiting the RSE of the mobile-phone camera. In this paper, based on a commercial RGB-LED and a single mobile-phone camera, a wavelength division-multiplexing (WDM) CVLC system exploiting RSE is experimentally demonstrated. For the generation and reception of the WDM signals, a structure of three parallel-independent channels is applied to the system. To mitigate the sampling frequency offset (SFO) effect, a low-complexity sampling reconstruction (SR) scheme is proposed and used in each channel of the system. Experimental results show that with the help of the proposed SR scheme, the system can achieve a theoretical data rate up to 2.38 Kbits/frame, with the bit error rate (BER) lower than 3.8 × 10-3.

Mobile-phone based visible light communication using region-grow light source tracking for unstable light source.

In order to increase the data rate of the camera-based visible light communication (VLC) system, using rolling shutter effect has been demonstrated successfully, in which the pixel rows of the complementary-metal-oxide-semiconductor (CMOS) image sensor are activated sequentially. Previous camera-based VLCs focused on using a stable LED light source, and its illumination area is positioned at the center of an image frame. In this work, we investigate the performance of a camera-based VLC with light source at different parts of an image frame. We propose and demonstrate using region-grow algorithm to track the light source. We also evaluate and discuss different scenarios when the light source is moved. Besides, a recorded > 5 kbit/s net data rate can be achieved by using only a single phosphor-based white-light LED source. Here, we demonstrate that 4.502 pixel/bit can be achieved.

Mobile Phone Camera-Based Indoor Visible Light Communications With Rotation Compensation

Downlink visible light communication (VLC) using a mobile phone camera is presented. The proposed scheme consists of an 8 × 8 light-emitting diode (LED) array for data transmission and eight white LEDs for an adequate level of illumination. The white LEDs are also used to acquire camera focus and light metering. Moreover, an efficient rotation compensation feature is provided in the form of a special header frame called keyframe to allow multiple users to receive the data simultaneously at 90°, 180°, and 270° orientations. Unlike conventional LED allocation methods for the rotation compensation, this keyframe-based method improves the data rate significantly. The experimental results demonstrate that the data rate achieves up to 1280 bps, whereas the theoretical simulations exhibit a potential data rate of up to 5120 bps using the identical LED array. The proposed scheme can be considered to be an efficient short-range camera-based VLC in an indoor environment with advanced features such as illumination, relatively high data rate, and robust camera rotation compensation for multiuser access.