Optical Camera Communication System Research Articles

Neural Network-Based Detection of OCC Signals in Lighting-Constrained Environments: A Museum Use Case

This research presents a novel approach by applying convolutional neural networks (CNNs) to enhance optical camera communication (OCC) signal detection under challenging indoor lighting conditions. The study utilizes a smartphone app to capture images of an LED lamp that emits 25 unique optical codes at distances of up to four meters. The developed CNN model demonstrates superior accuracy and outperforms traditional methodologies, which often struggle under variable illumination. This advancement provides a robust solution for reliable OCC detection where previous methods underperform, particularly in the tourism industry, where it can be used to create a virtual museum on the Unity platform. This innovation showcases the potential of integrating the application with a virtual environment to enhance tourist experiences. It also establishes a comprehensive visible light positioning (VLP) system, marking a significant advance in using CNN for OCC technology in various lighting conditions. The findings underscore the effectiveness of CNNs in overcoming ambient lighting challenges, paving the way for new applications in museums and similar environments and laying the foundation for future OCC system improvements.

Synchronous time multiplexing for non-line-of-sight multiple-input multiple-output optical camera communications.

In typical multiple-input multiple-output (MIMO) optical camera communication (OCC) systems, the spatial correlation of MIMO channels is large. Optical signals between light sources can easily interfere with each other, negatively impacting the overall transmission performance. In this work, we propose a time-multiplexing integral modulation scheme for a non-line-of-sight MIMO OCC system, where each LED transmits different signals to improve both the data rate and security of the system. A genetic algorithm (GA)-based adaptive multi-threshold scheme is designed to demodulate the blurred fringes in multi-level pulse amplitude modulation. The experimental results show that at a distance of 2.5 m, a data rate of 16.4 kb/s can reach with the BER performance of 3.01 × 10-3, which validates the superiority and reliability of our proposed schemes.

Investigating the angular distortion impact on vehicular Optical Camera Communication (OCC) systems

Investigating the angular distortion impact on vehicular Optical Camera Communication (OCC) systems

Optical Camera Communications in Healthcare: A Wearable LED Transmitter Evaluation during Indoor Physical Exercise.

This paper presents an experimental evaluation of a wearable light-emitting diode (LED) transmitter in an optical camera communications (OCC) system. The evaluation is conducted under conditions of controlled user movement during indoor physical exercise, encompassing both mild and intense exercise scenarios. We introduce an image processing algorithm designed to identify a template signal transmitted by the LED and detected within the image. To enhance this process, we utilize the dynamics of controlled exercise-induced motion to limit the tracking process to a smaller region within the image. We demonstrate the feasibility of detecting the transmitting source within the frames, and thus limit the tracking process to a smaller region within the image, achieving an reduction of 87.3% for mild exercise and 79.0% for intense exercise.

Compensation schemes for uneven illumination and LED light-emitting instability in optical camera communication system

In order to increase the data rate of the optical camera communication (OCC) system, the 8-composite-amplitude-shift-keying modulation (8CASK) OCC system is used in this work. However, if the static decision thresholds are employed to demodulate multi-level ASK signal, uneven illumination of LED lamps and LED light-emitting instability lead to the fluctuation of the gray range in the picture and degrade the bit-error-rate (BER) performance. In this work, we propose and demonstrate a demodulation scheme, using the uneven illumination compensation algorithm, the pixel matrix threshold overall update algorithm and the secondary decision algorithm, to mitigate the impact of illumination unevenness and LED light-emitting instability. The BER performance is evaluated and compared with other demodulation schemes. The experimental results demonstrate that the communication rate of our proposed scheme can reach 9kbit/s at a distance of 250 cm where the illumination is 135lux, and the BER is 8.01 × 10−5.

Joint interframe separation and gamma correction for asynchronous optical camera communication systems based on high-order statistics

Optical camera communication (OCC), which is enabled by large-scale light-emitting diodes (LEDs) arrays and image-sensor (IS) based cameras, has garnered significant attention from both researchers and industries. Existing OCC synchronization techniques typically rely on either super-Nyquist sampling or on computationally expensive asynchronous recovery algorithms to relax the required camera frame rate. In this paper, we propose a kurtosis-based asynchronous interference cancellation (K-AIC) algorithm, enabling the estimation for both the asynchronous interframe overlapping ratios and nonlinear Gamma distortion levels for each grayscale frame captured by camera. Through comprehensive numerical simulations, we demonstrate that the K-AIC algorithm exhibits low computational complexity, unique global optimum, high reliability and robust performance in mitigating asynchronous-induced bit errors across diverse scenarios. Short-range OCC experiment shows that the K-AIC scheme can effectively compensate for both interframe overlapping and Gamma distortions in a plesiochronous reception scenario, resulting in a Q-factor enhancement of approximately 12 dB with fluctuations of less than 1 dB. Consequently, the system achieves a net data rate of around 200 kbps.

Nearest neighbor bit assisted decision scheme for ISI mitigation in optical camera communications.

To enable a higher transmission rate in optical camera communication (OCC) systems, severe inter-symbol interference (ISI) occurs owing to the reduction of the number of pixel-row-per-bit (PPB). Therefore, those pixels representing data bit 0 or 1 may have same gray scale values, significantly deteriorating the bit decision when using the conventional thresholding scheme. In this Letter, a simple yet efficient scheme, referred to as nearest neighbor bit assisted decision (NNBAD) scheme, is proposed and experimentally demonstrated for signal decision in OCC systems. NNBAD leverages the nearest neighbor bit to jointly assist bit decision for pixels with severe ISI. Experimental results show that, for OCC systems with on-off keying (OOK) modulation, those pixels with severe ISI cannot be distinguished by the conventional thresholding scheme. Yet, the NNBAD scheme exhibits strong robustness against ISI, remarkably improving the bit error rate (BER) performance. The proposed scheme can achieve a throughput of 8.2 kbps with OOK modulation under an illuminance of 600 lx.

Effect of a vehicle's mobility on SNR and SINR in vehicular optical camera communication systems.

The widespread use of light-emitting diodes (LEDs) and cameras in vehicular environments provides an excellent opportunity for optical camera communication (OCC) in intelligent transport systems. OCC is a promising candidate for the Internet of Vehicles (IoV), and it uses LEDs as the transmitter and cameras as the receiver. However, the mobility of vehicles has a significant detrimental impact on the OCC system's performance in vehicular environments. In this paper, a traffic light that uses multiple-input multiple-output (MIMO) technology serves as the transmitter, and the receiver is a camera mounted on a moving vehicle. The impact of vehicle mobility on the vehicular MIMO-OCC system in the transportation environment is then examined using precise point spread function (PSF) analysis. The experimental results are used to evaluate the proposed PSF. A good agreement between the laboratory's recorded videos and this PSF model's simulations is observed. Moreover, the signal-to-noise ratio (SNR) and signal-to-interference-plus-noise ratio (SINR) values are evaluated. It is shown that they are greatly influenced by the vehicle's speed, direction of motion, and position of the camera. However, since the angular velocity in a typical transportation environment is low, it does not have a significant impact on the performance of the vehicular OCC systems.

Wavy water-to-air optical camera communication system using rolling shutter image sensor and long short term memory neural network.

We propose and experimentally demonstrate for the first time up to the authors' knowledge a wide field-of-view (FOV) water-to-air optical transmission using rolling-shutter (RS) based optical camera communication (OCC). Here, we evaluate the proposed OCC system without water ripple and with different percentage increases of water ripple. Long short term memory neural network (LSTM-NN) is utilized to mitigate the wavy water turbulence induced link outage and to decode 4-level pulse-amplitude-modulation (PAM4) RS pattern by meeting the pre-forward error correction bit-error-rate (pre-FEC BER = 3.8 × 10-3). We also evaluate the FOVs of the proposed water-to-air RS-based OCC system. This can be implemented by using different angular rotations of the camera. Experimental results show that the proposed OCC system can support ±70°, ± 30°, and ±30° rotations around the z-, y- and x-directions, respectively when operated at 6 kbit/s and decoded using LSTM-NN.

Design and Implementation of a Hybrid Optical Camera Communication System for Indoor Applications.

Optical wireless communication is a promising emerging technology that addresses the limitations of radio-frequency-based wireless technologies. This study presents a new hybrid modulation method for optical camera communication (OCC), which integrates two waveforms transmitted from a single transmitter light-emitting diode (LED) and receives data through two rolling shutter camera devices on the receiver side. Then, a smart camera with a high-resolution image sensor captures the high-frequency signal, and a low-resolution image sensor from a smartphone camera captures the low-frequency signal. Based on this hybrid scheme, two data streams are transmitted from a single LED, which reduces the cost of the indoor OCC device compared with transmitting two signals from two different LEDs. In the proposed scheme, rolling-shutter orthogonal frequency-division multiplexing is used for the high-frequency signals, and M-ary frequency-shift keying is used for the low-frequency signals in the time domain. This proposed scheme is compatible with smartphone and USB cameras. By controlling the OCC parameters, the hybrid scheme can be implemented with high performance for a communication distance of 10 m.

Color space-based autoencoder for optical camera communications

This paper proposes an end-to-end optical camera communications (OCC) system using an autoencoder neural network trained to recover the transmitted symbols. Although OCC techniques have been extensively studied in the literature, using an autoencoder that learns the transmitter and receiver functions jointly is a novel concept with significant prospects. Furthermore, we investigate the performance impact caused by the overlooked optical-to-electrical (O2E) conversion process of real-world OCC receivers. The autoencoder learning model captures these typically undesired changes in image sensors for the design of constellation symbols and reception schemes. For the simulation, we constructed an end-to-end autoencoder for a color space-based OCC system and measured the O2E performance effect. The proposed autoencoder communication system is analyzed and compared using the symbol error rate (SER) across various OCC detection systems. Despite the subtle spectral responsivity variations in image sensors, our numerical results indicate that the autoencoder model can learn to recover the transmitted data while minimizing SER and meeting the lighting requirements. These findings may interest a broad range of applications, particularly in IoT sensor networks. Among all the image sensors we studied, the OCC system with Bayer CFA-based signal detection showed superior performance.

Multi-Agent Deep Reinforcement Learning for Spectral Efficiency Optimization in Vehicular Optical Camera Communications

In this paper, we propose a vehicular optical camera communication system that can meet low bit error rate (BER) and ultra-low latency constraints. First, we formulate a sum spectral efficiency optimization problem that aims at finding the speed of vehicles and the modulation order that maximizes the sum spectral efficiency subject to reliability and latency constraints. This problem is mixed-integer programming with nonlinear constraints, and even for a small set of modulation orders, is NP-hard. To overcome the entailed high computational and time complexity which prevents its solution with traditional methods, we first model the optimization problem as a partially observable Markov decision process. We then solve it using an independent Q-learning framework, where each vehicle acts as an independent agent. Since the state-action space is large we then adopt deep reinforcement learning (DRL) to solve it efficiently. As the problem is constrained, we employ the Lagrange relaxation approach prior to solving it using the DRL framework. Simulation results demonstrate that the proposed DRL-based optimization scheme can effectively learn how to maximize the sum spectral efficiency while satisfying the BER and ultra-low latency constraints. The evaluation further shows that our scheme can achieve superior performance compared to radio frequency-based vehicular communication systems and other vehicular OCC variants of our scheme.

Non-line-of-sight optical camera communications based on CPWM and a convolutional neural network.

Non-line-of-sight (NLOS) optical camera communications (OCC) exhibit greater link availability and mobility than line-of-sight links, which are more susceptible to blocking and shadowing. In this work, we propose an NLOS OCC system, where the data signal is mapped into color pulse width modulation (CPWM) symbols prior to transmission using a red-, green-, and blue light-emitting diode. A convolutional-neural-network-based receiver is used to demodulate the CPWM signal. Based on experimental results, the proposed scheme effectively mitigates the effects of diffuse reflection induced intersymbol interference, resulting in an increased data transmission rate to 7.2kbps over a link span of more than 2m, which is typical for indoor applications.

Developing a comprehensive model for underwater MIMO OCC system.

Due to its spatial modulation feature and potential applications, optical camera communication (OCC) has gained significant attention in recent years for a range of applications including underwater. Nonetheless, due to the low frame rates of the camera, the OCC data rate is rather low, which is why multiple-input-multiple-output (MIMO) has been adopted to compensate. In MIMO systems, however, the signal from one light emitting diode (LED) may result in interference on the image sensor (i.e., the camera) resulting in inter-pixel interference (IPI). This paper presents a comprehensive model of the underwater OCC (UOCC) and experimentally verifies its performance under IPI by comparing signal to interference and noise ratio (SINR). The effect of distance between LEDs according to LED diameter D on signal to interference ratio (SIR) is presented and results indicate that coastal water has the SIR gain ∼2.5 dB for the link span of 1 to 6 m, and for harbor water channel length from 0.4 to 1.4 m the gain increased from ∼2 to ∼5 dB for d of 2D compared with d of 0.5D.

Design and Implementation of a 2D MIMO OCC System Based on Deep Learning.

Optical camera communication (OCC) is one of the most promising optical wireless technology communication systems. This technology has a number of benefits compared to radio frequency, including unlimited spectrum, no congestion due to high usage, and low operating costs. OCC operates in order to transmit an optical signal from a light-emitting diode (LED) and receive the signal with a camera. However, identifying, detecting, and extracting data in a complex area with very high mobility is the main challenge in operating the OCC. In this paper, we design and implement a real-time OCC system that can communicate in high mobility conditions, based on You Only Look Once version 8 (YOLOv8). We utilized an LED array that can be identified accurately and has an enhanced data transmission rate due to a greater number of source lights. Our system is validated in a highly mobile environment with camera movement speeds of up to 10 m/s at 2 m, achieving a bit error rate of 10-2. In addition, this system achieves high accuracy of the LED detection algorithm with mAP0.5 and mAP0.5:0.95 values of 0.995 and 0.8604, respectively. The proposed method has been tested in real time and achieves processing speeds up to 1.25 ms.

Integrated data detection and video restoration for optical camera communications

Optical Camera Communication (OCC) technology enables the use of Image Sensors (IS) with Visible Light Communications (VLC), one of the key technologies for the Sixth Generation (6G) of mobile communications. For Screen-to-Camera (S2C) applications in which Light-Emitting Diode (LED) screens modulate information symbols onto video pixels, increasing either the (transmitted) image resolution or frame rate may have a negative effect on the bit error performance and visual quality of OCC-encoded videos. This work proposes a novel OCC/S2C coding approach that lets both symbol transmission and video quality can be improved simultaneously. For this dual task, a Parallel Factor Analysis (PARAFAC) decomposition is applied for the first time in the literature to model an OCC system. In addition to validating the correct formulation of the proposed models, hypotheses, and algorithms, computational simulations demonstrate that increasing the number of video frames can result in time diversity gains. In accordance with the parameters of the system, this work also addresses the practical aspects of data rate, data volume, and computational complexity of the receiver.

Long-distance indoor optical camera communication using side-emitting fibers as distributed transmitters.

We present a design approach for a long-distance optical camera communication (OCC) system using side-emitting fibers as distributed transmitters. We demonstrate our approach feasibility by increasing the transmission distance by two orders up to 40 m compared to previous works. Furthermore, we explore the effect of the light-emitting diode (LED) modulation frequency and rolling shutter camera exposure time on inter-symbol interference and its effective mitigation. Our proposed OCC-fiber link meets the forward-error-correction (FEC) limit of 3.8 · 10-3 of bit error rate (BER) for up to 35 m (with BER= 3.35 · 10-3) and 40 m (with BER=1.13 · 10-3) using 2-mm and 3-mm diameter side-emitting fibers, respectively. Our results at on-off keying modulation frequencies of 3.54 kHz and 5.28 kHz pave the way to moderate-distance outdoor and long-distance indoor highly-reliable applications in the Internet of Things and OCC using side-emitting fiber-based distributed transmitters.

Frame recovery using multiple images in rolling shutter based systems

AbstractAn algorithm for recovering transmitted static identifiers (IDs) in rolling shutter based Optical Camera Communication systems is proposed by the authors, considering a system comprised of a camera and a circular light source. The goal is to allow the correct decoding when the ID frame is only partially detected in the image. A baseline algorithm as reference for the frame recovery success rate (FRSR) and a reconstruction algorithm based on the idea of capturing multiple frame fragments and reassembling them is proposed in order to recover the transmitted ID not entirely seen on a single image. It was proven, by simulation, that the maximum distance at which the IDs recovery can be guaranteed is increased by 2.5 fold with the proposed algorithm, for 6‐bit, 8‐bit and 10‐bit codewords. An experimental validation algorithm was also proposed, using image processing techniques to extract the bitstreams and test the ID recovery process. The proposed algorithm improves the FRSR for a given distance, even in the presence of considerable bit errors in the bitstreams extracted from the images.

BER analysis on exposure effect for optical camera communication.

The exposure effect of cameras in optical camera communication (OCC) distorts the received light pulses and generates inter-symbol interference (ISI), which adversely affects the bit error rate (BER) performance. In this Letter, we derive an analytical expression of BER based on the pulse response model of the camera-based OCC channel and analyze the impact of exposure time on BER performance considering asynchronous transmission characteristics. Numerical simulations and experimental results demonstrate that a long exposure time is beneficial in a noise-dominant communication scenario, while a short exposure time is preferable when ISI is dominant. This Letter provides a comprehensive analysis of the influence of exposure time on BER performance, offering a theoretical foundation for the design and optimization of OCC systems.

Empirical study of an underwater optical camera communication system under turbulent conditions.

This paper presents an experimental study of the turbulence impact caused by temperature inhomogeneity and air bubbles on a global shutter-based underwater optical camera communication (UOCC). The effects of these two phenomena on UOCC links are illustrated in terms of the intensity variations and an associated reduction in the average received intensity of the illuminated pixels corresponding to the optical source projection and the dispersion of the projection on the captured images. Additionally, it is shown that the area of illuminated pixels in the temperature-induced turbulence scenario is higher than in the bubbly water case. To analyze the effects of those two phenomena on the optical link performance, the signal-to-noise ratio (SNR) of the system is evaluated by considering different points as the regions of interest (ROI) from the light source projection of the captured images. The results indicate that the system performance is improved by averaging over the value of several pixels produced by the point spread function, compared to simply using the central and the maximum pixel value as the ROIs.