Optical Camera Communication Research Articles

Implementation of an indoor optical camera communication and localization fusion system using infrared LED markers

Recently, with the development of communication and positioning technologies, progressively higher demands have been placed on the timeliness of indoor communication and the effectiveness of indoor positioning. This paper proposes a low-cost and full-time domain coverage indoor communication and localization fusion system that uses an infrared camera to identify mobile LEDs based on region of interest optical camera communication (RoI-OCC) and calculate three-dimensional (3D) world coordinates based on perspective-n-point (PnP) algorithm. An intermediate delimiter modulation-demodulation method is designed to achieve asynchronous OCC. In processing infrared images, a region-growth-based absolute-directional-mean-difference (RG-ADMD) algorithm is developed to detect infrared targets and recover the scale. Additionally, a local-contrast-method-based Lucas-Kanade (LCM-LK) optical flow estimation algorithm is designed to track infrared targets and determine the centroid coordinates of pixels. In a 5×5 m experimental area, the established system can achieve a target detection recall rate of over 85%, the asynchronous OCC with error-free transmission and an average 3D positioning accuracy of 2.01 cm under a LCM-LK tracking algorithm.

Water-to-air unmanned aerial vehicle (UAV) based rolling shutter optical camera communication (OCC) system with gated recurrent unit neural network (GRU-NN)

Underwater sensors and autonomous underwater vehicles (AUVs) are widely adopted in oceanic research activities. As the number of underwater sensors and AUVs is growing quickly, the bandwidth requirements are increasing accordingly. In this work, we put forward and demonstrate a large field-of-view (FOV) water-to-air unmanned aerial vehicle (UAV) based optical camera communication (OCC) system with gated recurrent unit neural network (GRU-NN) for the first time to the best of our knowledge. As the UAVs are embedded with complementary-metal-oxide-semiconductor (CMOS) cameras, there is no need to install OCC receivers (Rxs), reducing the deployment cost. Moreover, the large photo-sensitive area of the CMOS camera can support a large FOV OCC transmission without the need for precise optical alignment. Here, by utilizing the column matrix identification during the rolling shutter pattern decoding in the CMOS image sensor, the scintillation caused by water turbulence can be reduced. Besides, in the outdoor and windy environment, the UAV will experience significant movement caused by the wind making it very difficult to capture stable OCC frames in the CMOS image sensor. Here, we propose and demonstrate utilizing GRU-NN, which is a special realization of the recurrent neural network (RNN) with memory cells capable of learning the time-domain dependent signals. It is shown that the GRU-NN can learn effectively from successive image frames in time-domain and produce correct prediction even under the windy and unstable UAV flying environment. Experimental results reveal that the proposed GRU-NN can outperform the previous pixel-per-symbol labeling neural network (PPS-NN), and also can significantly reduce the computation time when compared with long-short-term-memory-neural-network (LSTM-NN). The proposed system can decode 4-level pulse-amplitude-modulation (PAM4) rolling shutter OCC patterns at data rates of 5.4 kbit/s and 3.0 kbit/s under clear and cloudy water, respectively, fulfilling the pre-forward error correction bit-error rate (pre-FEC BER = 3.8 × 10−3). We also demonstrate that the UAV based OCC system can support data rates of 5.4 kbit/s, 4.2 kbit/s, and 3.0 kbit/s at distances of 2.2 m, 3.2 m, and 4.2 m, respectively, at outdoor and windy environments.

Robust License Plate Recognition in OCC-Based Vehicle Networks Using Image Reconstruction

With the help of traffic lights and street cameras, optical camera communication (OCC) can be adopted in Internet of Vehicles (IoV) applications to realize communication between vehicles and roadside units. However, the encoded light emitted by these OCC transmitters (LED infrastructures on the roadside and/or LED-based headlamps embedded in cars) will generate stripe patterns in image frames captured by existing license-plate recognition systems, which seriously degrades the accuracy of the recognition. To this end, we propose and experimentally demonstrate a method that can reduce the interference of OCC stripes in the image frames captured by the license-plate recognition system. We introduce an innovative pipeline with an end-to-end image reconstruction module. This module learns the distribution of images without OCC stripes and provides high-quality license-plate images for recognition in OCC conditions. In order to solve the problem of insufficient data, we model the OCC strips as multiplicative noise and propose a method to synthesize a pairwise dataset under OCC using the existing license-plate dataset. Moreover, we also build a prototype to simulate real scenes of the OCC-based vehicle networks and collect data in such scenes. Overall, the proposed method can achieve a recognition performance of 81.58% and 79.35% on the synthesized dataset and that captured from real scenes, respectively, which is improved by about 31.18% and 24.26%, respectively, compared with the conventional method.

ZEROES: Robust Derivative-Based Demodulation Method for Optical Camera Communication

Most of Optical Camera Communication (OCC) systems benefit from the rolling shutter mechanism of Complementary Metal-Oxide Semiconductor (CMOS) cameras to record the brightness evolution of the Light-Emitting Diode (LED) through dark and bright strips within images. While this technique enhances the maximum achievable data rate, the main difficulty lies in the demodulation of the signal extracted from images, subject to blooming effect. Thus, two main approaches were proposed to deal with this issue, using adaptive thresholds whose value evolves according to amplitude changes or detecting signal variations with the first-order derivative. As the second method is more robust, a new demodulation method based on the detection of the zeros of the first-order derivative of the extracted signal was proposed in this paper. Obtained results clearly show an improvement in the extracted signal demodulation compared to other methods, achieving a raw Bit Error Rate (BER) of 10−3 around 50 cm in a Line-Of-Sight scenario, and increasing the maximum communication distance by 43.5%, reaching 330 cm in the case of a Non-Line-Of-Sight transmission.

Performance Analysis of a Color-Code-Based Optical Camera Communication System

In this study, we present a visible light communication (VLC) system that analyzes the performance of an optical camera communication (OCC) system, utilizing a mobile phone camera as the receiver and a computer monitor as the transmitter. By creating color channels in the form of a 4 × 4 matrix within a frame, we determine the parameters that affect the successful transmission of data packets. Factors such as the brightness or darkness of the test room, the light color of the lamp in the illuminated environment, the effects of daylight when the monitor is positioned in front of a window, and issues related to dead pixels and light bleed originating from the monitor’s production process have been considered to ensure accurate data transmission. In this context, we utilized the PyCharm, Pydroid, Python, Tkinter, and OpenCV platforms for programming the transmitter and receiver units. Through the application of image processing techniques, we mitigated the effects of daylight on communication performance, thereby proposing a superior system compared to standard VLC systems that incorporate photodiodes. Additionally, considering objectives such as the maximum number of channels and the maximum distance, we regulated the sizes of the channels, the distances between the channels, and the number of channels. The NumPy library, compatible with Python–Tkinter, was employed to determine the color levels and dimensions of the channels. We investigate the effects of RGB and HSV color spaces on the data transmission rate and communication distance. Furthermore, the impact of the distance between color channels on color detection performance is discussed in detail.

A Security Protocol for Vehicle Platoon Verification Using Optical Camera Communications

A Security Protocol for Vehicle Platoon Verification Using Optical Camera Communications

Farm Monitoring System with Drones and Optical Camera Communication.

Drones have been attracting significant attention in the field of agriculture. They can be used for various tasks such as spraying pesticides, monitoring pests, and assessing crop growth. Sensors are also widely used in agriculture to monitor environmental parameters such as soil moisture and temperature. Due to the high cost of communication infrastructure and radio-wave modules, the adoption of high-density sensing systems in agriculture is limited. To address this issue, we propose an agricultural sensor network system using drones and Optical Camera Communication (OCC). The idea is to transmit sensor data from LED panels mounted on sensor nodes and receive the data using a drone-mounted camera. This enables high-density sensing at low cost and can be deployed in areas with underdeveloped infrastructure and radio silence. We propose a trajectory control algorithm for the receiving drone to efficiently collect the sensor data. From computer simulations, we confirmed that the proposed algorithm reduces total flight time by 30% compared to a shortest-path algorithm. We also conducted a preliminary experiment at a leaf mustard farm in Kamitonda-cho, Wakayama, Japan, to demonstrate the effectiveness of the proposed system. We collected 5178 images of LED panels with a drone-mounted camera to train YOLOv5 for object detection. With simple On-Off Keying (OOK) modulation, we achieved sufficiently low bit error rates (BERs) under 10-3 in the real-world environment. The experimental results show that the proposed system is applicable for drone-based sensor data collection in agriculture.

Error performance analysis for OOK modulated optical camera communication systems

Integrating image sensors’ imaging capabilities into the receiver for visible light communication is a prominent characteristic of optical camera communication (OCC). However, the exposure effect during imaging distorts received waveforms and introduces inter-symbol interference (ISI), leading to decreased OCC reliability. This paper aims to provide an in-depth analysis of the impact of image sensor exposure effects on the error performance of OCC systems. Analytical expressions of the pixel signal-to-interference and noise ratio (PSINR) are derived using the pulse response function (PRF) for an on-off keying (OOK) modulated OCC system with varying exposure times. Furthermore, the bit error rate (BER) performance is evaluated analytically using PSINR, and a straightforward BER measurement scheme is proposed for experimental validation. Results from analyses and experiments conducted under different exposure times indicate that longer exposures lead to increased ISI and decreased PSINR, thereby increasing the error probability of data demodulation. Additionally, a combined impact of noise and exposure on OCC system reliability is observed, highlighting noise-limited and interference-limited characteristics under low and high signal-to-noise ratio (SNR) conditions, respectively. By utilizing PSINR as a bridge, this paper precisely analyzes OCC system reliability under exposure effects, laying a theoretical foundation for system design and optimization.

Frame-rate adaptive fractionally spaced equalization enabled high-throughput optical camera communication.

Optical camera communication (OCC) has garnered worldwide research attention, due to its immunity to electromagnetic interference (EMI) and efficient utilization of spectrum resources. However, the limited bandwidth of the OCC system and the timing offset of the camera result in low system throughput. To enhance the OCC throughput, we propose and experimentally demonstrate a frame-rate adaptive fractionally spaced equalization algorithm (FA-FSE) for the joint mitigation of severe inter-symbol interference (ISI) and timing offset arising in OCC. Experimental results validate its correct and power-efficient function, leading to a record aggregated throughput of 250.96 kbit/s, when the 8-level pulse amplitude modulation (PAM-8) signals are independently modulated to eight chip-on-board light emitting diode (COB-LED) light strips, while simultaneously received by a smartphone 10 cm away.

Demonstration of symbol timing synchronization for rolling shutter-based optical camera communication

Optical camera communication (OCC) provides license-free communication channels without radio waves. The rolling shutter mechanism is expected to enhance the capacity of wireless links in OCC. However, signal erasure occurs due to waiting times between frames in consecutive signal transmission. To address this issue, we propose a symbol timing synchronization for OCC using a rolling shutter camera. The receiver camera synchronizes to the transmitter with additional waiting time which is computed based on the defined preamble signals. We analyzed the performance to clarify the theoretical limit. The feasibility of the proposed idea was confirmed with experimental results.

Research on performance enhancement scheme of optical camera communication based on adaptive exponential weighted moving average algorithm

AbstractWhen the COMS camera of optical camera communication (OCC) system acquired the information image, the grayscale value of pixels fluctuated due to the influence of blooming effect and light noise, so that the data logic could not be correctly judged by the decision threshold, which affected the bit error rate (BER) performance transmission distance and data rate, and degraded the system performance. In this paper, the adaptive exponential weighted moving average algorithm is proposed, and the optimal grayscale value matrix of signal demodulation is verified by experiment, so as to reduce the fluctuation of grayscale value and improve the performance of OCC system. The experimental results show that for data processing and analysis based on ZYNQ7020 experimental platform, a 7% forward error correction（BER = 3.8 × 10−3）requirements can be satisfied at 150 cm when the data rate is 3 Kbps, and a 7% forward error correction（BER = 3.8 × 10−3）requirements can be satisfied at 100 cm when the data rate is 5 Kbps.

Experimental evaluation of wearable LED strip and side-emitting fiber for optical camera communications systems.

This paper presents an experimental evaluation of two types of light-emitting diode (LED)-based distributed transmitters, namely an LED strip and an LED-coupled side-emitting optical fiber, in both laboratory and wearable optical camera communication (OCC) systems. We study the system performance in terms of success of reception (SoR) with regard to the transmission distance. The best value of SoR is achieved when the camera is facing directly to the transmitter (Tx) from a close distance of 1 m. Additionally, we compare the power consumption, the signal-to-noise ratio performance (SNR) and all the obtained values under optimal conditions are better than the forward error correction (FEC) limit in OCC systems.

Effect of an image compression scheme on optical camera communication

Optical camera communication (OCC) represents a subset of visible light communication with considerable potential applications in domains such as intelligent transportation systems. However, many related studies on OCC ignore the effect of image compression. Cameras typically compress captured image data to reduce the resources required to store the data. Taking Joint Photographic Experts Group (JPEG) and AV1 image file format (AVIF) compression as examples, the effect is studied based on a simplified OCC model. The simulation results indicate that image compression can achieve the effect of a low-pass filter. Image compression, especially AVIF compression, can effectively reduce the bit error ratio (BER) of the system. Compared with demodulating RAW images, the BER is reduced from 1.2×10−1 to 7.0×10−4 when the AVIF image compression ratio is approximately 140. In addition, image compression is one of the reasons for the different performance of the red, green, and blue channels.

Cost-Effective Optical Wireless Sensor Networks: Enhancing Detection of Sub-Pixel Transmitters in Camera-Based Communications.

In the domain of the Internet of Things (IoT), Optical Camera Communication (OCC) has garnered significant attention. This wireless technology employs solid-state lamps as transmitters and image sensors as receivers, offering a promising avenue for reducing energy costs and simplifying electronics. Moreover, image sensors are prevalent in various applications today, enabling dual functionality: recording and communication. However, a challenge arises when optical transmitters are not in close proximity to the camera, leading to sub-pixel projections on the image sensor and introducing strong channel dependence. Previous approaches, such as modifying camera optics or adjusting image sensor parameters, not only limited the camera's utility for purposes beyond communication but also made it challenging to accommodate multiple transmitters. In this paper, a novel sub-pixel optical transmitter discovery algorithm that overcomes these limitations is presented. This algorithm enables the use of OCC in scenarios with static transmitters and receivers without the need for camera modifications. This allows increasing the number of transmitters in a given scenario and alleviates the proximity and size limitations of the transmitters. Implemented in Python with multiprocessing programming schemes for efficiency, the algorithm achieved a 100% detection rate in nighttime scenarios, while there was a 89% detection rate indoors and a 72% rate outdoors during daylight. Detection rates were strongly influenced by varying transmitter types and lighting conditions. False positives remained minimal, and processing times were consistently under 1 s. With these results, the algorithm is considered suitable for export as a web service or as an intermediary component for data conversion into other network technologies.

Investigating the angular distortion impact on vehicular optical camera communication (OCC) systems

Optical camera communication (OCC) shows promise for optical wireless communication (OWC) in vehicular networks. However, vehicle mobility-induced angular distortions hinder system throughput by degrading non-isotropic vehicular OCC channel gain. Few of the prior works have ever made a comprehensive analysis of their impact, especially based on the pixel value which reflects the camera imaging features. To address this knowledge gap, a pixel value-described vehicular OCC system model accounting for transmitter imaging location and intensity from the geometry and radiometry aspects is presented in this paper with common types of the offset and rotation angles included. We integrate a MATLAB-based simulated vehicular OCC system with an experimentally designed testbed for validation and performance analysis. For a single-time snapshot, we investigate the impacts of common angular distortion types in vehicular OCC systems on maximum pixel value, imaging location, and communication-related metrics. Furthermore, we statistically analyze their influences by considering two driving scenarios with respective angular distributions. The angular distortion characterization from this work is expected to lay a stepping stone to addressing mobility in vehicular OCC systems.

Confidential multiple-input multiple-output optical camera communication aided by a two-dimensional pilot.

Optical camera communication (OCC) has attracted increased attention for its inherent security advantage. However, there still exists the risk of eavesdropping on the broadcasting channel of OCC. To achieve confidential communication, we propose the confidentiality-interference dual light-emitting diode (LED) communication (CIDLC) scheme at the transmitter (TX) and elimination of interference (EI) scheme at the receiver (RX). Meanwhile, interference signals refer to the bit shift of confidential signals. Further, we propose the two-dimensional pilot-aided channel estimation (2D-PACE) scheme to enhance the reliability of multiple-input multiple-output (MIMO) OCC. Experiment results validate the effectiveness of our schemes, which guarantee confidentiality while performing well at a 2 m non-line-of-sight (NLOS) distance. Finally, the communication-illumination integration OCC is constructed via the energy equalization coding (EEC) scheme.

Neural Network-Based Detection of OCC Signals in Lighting-Constrained Environments: A Museum Use Case

This research presents a novel approach by applying convolutional neural networks (CNNs) to enhance optical camera communication (OCC) signal detection under challenging indoor lighting conditions. The study utilizes a smartphone app to capture images of an LED lamp that emits 25 unique optical codes at distances of up to four meters. The developed CNN model demonstrates superior accuracy and outperforms traditional methodologies, which often struggle under variable illumination. This advancement provides a robust solution for reliable OCC detection where previous methods underperform, particularly in the tourism industry, where it can be used to create a virtual museum on the Unity platform. This innovation showcases the potential of integrating the application with a virtual environment to enhance tourist experiences. It also establishes a comprehensive visible light positioning (VLP) system, marking a significant advance in using CNN for OCC technology in various lighting conditions. The findings underscore the effectiveness of CNNs in overcoming ambient lighting challenges, paving the way for new applications in museums and similar environments and laying the foundation for future OCC system improvements.

Synchronous time multiplexing for non-line-of-sight multiple-input multiple-output optical camera communications.

In typical multiple-input multiple-output (MIMO) optical camera communication (OCC) systems, the spatial correlation of MIMO channels is large. Optical signals between light sources can easily interfere with each other, negatively impacting the overall transmission performance. In this work, we propose a time-multiplexing integral modulation scheme for a non-line-of-sight MIMO OCC system, where each LED transmits different signals to improve both the data rate and security of the system. A genetic algorithm (GA)-based adaptive multi-threshold scheme is designed to demodulate the blurred fringes in multi-level pulse amplitude modulation. The experimental results show that at a distance of 2.5 m, a data rate of 16.4 kb/s can reach with the BER performance of 3.01 × 10-3, which validates the superiority and reliability of our proposed schemes.

Optical Camera Communications in Healthcare: A Wearable LED Transmitter Evaluation during Indoor Physical Exercise.

This paper presents an experimental evaluation of a wearable light-emitting diode (LED) transmitter in an optical camera communications (OCC) system. The evaluation is conducted under conditions of controlled user movement during indoor physical exercise, encompassing both mild and intense exercise scenarios. We introduce an image processing algorithm designed to identify a template signal transmitted by the LED and detected within the image. To enhance this process, we utilize the dynamics of controlled exercise-induced motion to limit the tracking process to a smaller region within the image. We demonstrate the feasibility of detecting the transmitting source within the frames, and thus limit the tracking process to a smaller region within the image, achieving an reduction of 87.3% for mild exercise and 79.0% for intense exercise.

Compensation schemes for uneven illumination and LED light-emitting instability in optical camera communication system

In order to increase the data rate of the optical camera communication (OCC) system, the 8-composite-amplitude-shift-keying modulation (8CASK) OCC system is used in this work. However, if the static decision thresholds are employed to demodulate multi-level ASK signal, uneven illumination of LED lamps and LED light-emitting instability lead to the fluctuation of the gray range in the picture and degrade the bit-error-rate (BER) performance. In this work, we propose and demonstrate a demodulation scheme, using the uneven illumination compensation algorithm, the pixel matrix threshold overall update algorithm and the secondary decision algorithm, to mitigate the impact of illumination unevenness and LED light-emitting instability. The BER performance is evaluated and compared with other demodulation schemes. The experimental results demonstrate that the communication rate of our proposed scheme can reach 9kbit/s at a distance of 250 cm where the illumination is 135lux, and the BER is 8.01 × 10−5.