Image Sensor Communication Research Articles

High-speed image sensor communication using high dynamic range combining

High-speed image sensor communication using high dynamic range combining

A Review on Image Sensor Communication and Its Applications to Vehicles

Image sensor communication (ISC), also known as optical camera communication, is a form of visible light communication that utilizes image sensors rather than a single photodiode, for data reception. ISC offers spatial separation properties and robustness to ambient noise, making it suitable for outdoor applications such as intelligent transportation systems (ITSs). This review analyzes the research trends in ISC, specifically concerning its application in ITSs. Our focus is on various ISC receivers, including rolling shutter cameras, global shutter high-speed cameras, optical communication image sensors, and event cameras. We analyze how each of these receivers is being utilized in ISC vehicular applications. In addition, we highlight the use of ISC in range estimation techniques and the ability to achieve simultaneous communication and range estimation. By examining these topics, we aim to provide a comprehensive overview of the role of ISC technology in ITSs and its potential for future development.

Image sensor communication and its transmitting devices

This paper focuses on image sensor communication (ISC) using cameras to receive wireless signals transmitted by visible lights. We summarize the technical principles of ISC from the perspective of transmitting devices. An introduction to the development and applications of ISC was provided, starting with the history of image sensors and visible light communication (VLC). Moreover, the communication mechanism of ISC using complementary metal–oxide–semiconductor (CMOS) sensors is explained in this paper. Specifically, we investigate studies using an optical communication image sensor and the rolling shutter technique for high-speed ISC. Further, we highlight three distinctive transmitting devices in ISC: a LED array, rotating LEDs, and displays, and explain them in terms of structure and signal transmission methods. In particular, we present the parallel transmission of LED arrays and their application in traffic systems and introduce an optical flow-based signal transmission method using rotating LEDs. In addition, two visual transmission techniques in displays-based-ISC using brightness-difference and high-frequency flashing are introduced, respectively. This paper intuitively explains these ISC techniques and reveals ISC’s high versatility.

A study on detection of rotary LED transmitter for image sensor communication

A study on detection of rotary LED transmitter for image sensor communication

Simplified Alamouti-Type Space-Time Coding for Image Sensor Communication Using Rotary LED Transmitter

Simplified Alamouti-Type Space-Time Coding for Image Sensor Communication Using Rotary LED Transmitter

Rotary LED Transmitter for Improving Data Transmission Rate of Image Sensor Communication

This study focuses on image sensor communication (ISC), which is a form of visible light communication, using a light-emitting diode (LED) and a camera as the transmitting and receiving device, respectively. In an ISC system, the receiver captures optical signals transmitted by blinking LEDs as an image and demodulates the data using the captured image. The ISC system can spatially separate signals and noises in the image, thus providing superior anti-interference ability. However, because the speed of ISC depends on the shooting speed of the camera, the receiver suffers from a low data transmission rate when a low-speed camera is used. To improve the data rate of ISC, we have developed a rotary LED transmitter. This transmitter cylindrically rotates the blinking LEDs during the exposure time of the camera. The camera captures multiple blinking states of LEDs as afterimages in a single image, thereby increasing the amount of data received per image. In this paper, we propose an ISC system using a rotary LED transmitter and present an experiment for the evaluation of the data rate. The result indicates that the data rate of the rotary LED transmitter is 60 times that of the conventional method.

Comparison of distance performances of modulation schemes in intelligent transport system image sensor communication

Comparison of distance performances of modulation schemes in intelligent transport system image sensor communication

Sequential maximum likelihood decoding incorporating reliability determination for image sensor communication

In this study, the sequential maximum likelihood decoding (S-MLD) for image sensor based visible light communication, referred to as image sensor communication, is proposed. The conventional system cannot correctly decode degraded captured images. To overcome this problem, S-MLD partitions the LED area on the image into two segments and sequentially creates pseudo-captured images, which imitate all possible LED blinking patterns. This approach is based on MLD for each segment. Moreover, reliability determination is incorporated to improve communication quality. We performed computer simulations and confirmed that S-MLD exhibits superior decoding performance and approximately 90% computational cost reduction compared to the previous MLD.

Photography Trilateration Indoor Localization with Image Sensor Communication

Localization has become an important aspect in a wide range of mobile services with the integration of the Internet of things and service on demand. Numerous mechanisms have been proposed for localization, most of which are based on the estimation of distances. Depending on the channel modeling, each mechanism has its advantages and limitations on deployment, exhibiting different performances in terms of error rates and implementation. With the development of technology, these limitations are rapidly overcome with hybrid systems and enhancement schemes. The successful approach depends on the achievement of a low error rate and its controllability by the integration of deployed products. In this study, we propose and analyze a new distance estimation technique employing photography and image sensor communications, also named optical camera communications (OCC). It represents one of the most important steps in the implemented trilateration localization scheme with real architectures and conditions of deployment which is the second our contribution for this article. With the advantages of the image sensor hardware integration in smart mobile devices, this technology has great potential in localization-based optical wireless communication

Deep Learning Framework for Wireless Systems: Applications to Optical Wireless Communications

Optical wireless communication (OWC) is a promising technology for future wireless communications due to its potential for cost-effective network deployment and high data rate. There are several implementation issues in OWC that have not been encountered in radio frequency wireless communications. First, practical OWC transmitters need illumination control on color, intensity, luminance, and so on, which poses complicated modulation design challenges. Furthermore, signal-dependent properties of optical channels raise nontrivial challenges in both modulation and demodulation of the optical signals. To tackle such difficulties, deep learning (DL) technologies can be applied for optical wireless transceiver design. This article addresses recent efforts on DL-based OWC system designs. A DL framework for emerging image sensor communication is proposed, and its feasibility is verified by simulation. Finally, technical challenges and implementation issues for the DL-based optical wireless technology are discussed.

Maximum likelihood decoding based on pseudo-captured image templates for image sensor communication

This paper focuses on an image sensor communication system that uses an LED as the transmitter and a high-speed image sensor (camera) as the receiver. Communication in this scheme depends on the quality of images transmitted from the LED to the sensor. If the image becomes unfocused on the way to the receiver, the LED luminance that make up the signal cannot be detected, so the receiver cannot demodulate the signal data. To overcome this problem, this study proposes a novel demodulation scheme to recover data from a degraded image, based on a maximum likelihood decoding (MLD) algorithm. The proposed method creates template images that imitate all possible blinking patterns produced by the LED transmitter, and then calculates the Euclidean distances between pixels in the captured image and the pseudo images for all possible blinking patterns. Finally, the algorithm chooses the image template with the smallest Euclidian distance from the received signal as the recovered data. Though an exhaustive set of image templates must be prepared for the proposed MLD, the number of templates depends on the number of LEDs on the transmitter. Thus, the computational complexity of this method increases as the number of transmitter LEDs increases. To reduce the computational complexity of the proposed MLD algorithm, the binary differential evolution (BDE) algorithm is used, which is a swarm intelligence technique. Computer simulations are used to evaluate the BDE algorithm's usefulness for reducing computational complexity and improving the BER of the communication system.

Technical Issues on IEEE 802.15.7m Image Sensor Communication Standardization

This tutorial article presents an outline of image sensor communication (ISC) technologies realized by light sources and cameras. It discusses the revision of the IEEE 802.15.7-2011 standard, namely, the IEEE 802.15.7m Optical Wireless Communications Task Group, which has significantly influenced the development of the ISC technology. It also reviews the ISC technical proposals of the task group and compares them with related works. The essential technical considerations of the ISC specifications are presented, and the future directions of research and development are discussed.

Enhancement of Data Rate and Packet Size in Image Sensor Communications by Employing Constant Power 4-PAM

Image sensor communications (ISC) is a scheme to make the LED lamp replace access points or beacons in indoor environments. Its application has been limited, however, due to its low data rate and small packet size. To resolve these problems, we propose a new ISC scheme using 4-PAM modulation and a parallel packet structure. The constant power 4-PAM signal was designed to have constant optical power in each symbol to eliminate the flickering effect. The experiments show that a data rate more than 2.6 times higher than the existing on - off keying schemes was achieved. In addition, it is demonstrated that the packet length can reach more than several frames while it is limited to less than a frame using the existing schemes. The limiting factors of N-PAM levels and packet size are analyzed in this paper to further improvement. All these results are demonstrated through simulations and experiments.

Simplified Vehicle Vibration Modeling for Image Sensor Communication

Image sensor communication (ISC), derived from visible light communication (VLC) is an attractive solution for outdoor mobile environments, particularly for intelligent transport systems (ITS). In ITS-ISC, tracking a transmitter in the image plane is critical issue since vehicle vibrations make it difficult to selsct the correct pixels for data reception. Our goal in this study is to develop a precise tracking method. To accomplish this, vehicle vibration modeling and its parameters estimation, i.e., represetative frequencies and their amplitudes for inherent vehicle vibration, and the variance of the Gaussian random process represnting road surface irregularity, are required. In this paper, we measured actual vehicle vibration in a driving situation and determined parameters based on the frequency characteristics. Then, we demonstrate that vehicle vibration that induces transmitter displacement in an image plane can be modeled by only Gaussian random processes that represent road surface irregularity when a high frame rate (e.g., 1000fps) image sensor is used as an ISC receiver. The simplified vehicle vibration model and its parameters are evaluated by numerical analysis and experimental measurement and obtained result shows that the proposed model can reproduce the characteristics of the transmitter displacement sufficiently.

V2X communications with an image sensor

This paper introduces infrastructure-to-vehicle and vehicle-to-vehicle communications using VLC. A VLC coupled with a high-speed image sensor is introduced (i.e., image sensor communication). The high-speed image sensors provide eyes for autonomous and connected vehicles. VLC imparts data reception capability to image sensors with necessary functions, which can then be provided to autonomous and connected vehicles. In this paper, some of our research on coupling VLC to high-speed image sensors is introduced, including our key findings: the basics of ISC, a vehicle motion model, and range estimation.

Characteristics and Performance of Image Sensor Communication

A complementary metal-oxide semiconductor (CMOS) image sensor can be used for communication, in addition to imaging. This paper investigates noise characteristics and system performance for CMOS image sensor communication. Different signal-dependent and independent noise sources are categorized and comprehensively analyzed in conjunction with sensor measurements. Accordingly, a unified channel model in the pixel-matched case is proposed, and the communication signal-to-noise ratio is defined and demonstrated in different parameter settings. Moreover, the channel capacity with mixed noise and bounded input is derived and is proven to be achievable by a discrete input distribution of finite number of probability mass points. Simulation shows that capacity of more than 7 b/s/Hz per pixel or 8–10 b/s/Hz per block of multiple pixels is achievable.

Modulation and Coding Scheme (MCS) for Indoor Image Sensor Communication System

In this paper, we propose an indoor image sensor communication (ISC) system comprising an attentive indoor environment in which rolling-shutter cameras (including smart phone cameras) are used to receive data from light sources. The challenges of intended system mainly come from the receiver side, including its rolling shutter mechanism, its low and time-variant frame rate (e.g., no greater than 30 fps), and the narrow bandwidth response (i.e. several kHz baseband). Also, the requirements of flicker-free modulation enabling the dimming capacity, and indoor distance (i.e., up to 10 m) are unmitigated challenges. To resolve all mentioned challenges, we propose new hybrid modulation schemes, including M-ary frequency-shift keying (M-FSK) modulation and a hybrid modulation of M-FSK and phase-shift keying (2-PSK) for ISC. Simulation, numerical analysis and experiment results are given to illustrate the feasibility of the intended system.

Image Sensor Communication — Current Status and Future Perspectives

Image sensor communication (ISC), a type of visible light communication, is an emerging wireless communication technology that uses LEDs to transmit a signal and uses an image sensor in a camera to receive the signal. This paper discusses the present status of and future trends in ISC by describing the essential characteristics and features of ISC. Moreover, we overview the products and expected future applications of ISC.

Region-of-Interest Signaling Vehicular System using Optical Camera Communications

This technical paper introduces an optical camera communication system for vehicles. The purpose of the communication system incorporated into car (using existing light-emitting diode (LED) lights and ordinary cameras) is to add region-of-interest signaling functionality for cars via either their headlights or taillights. A low-data-rate but reliable link via the optical channel is the objective. In particular, the communication system, contributes a spatial modulation approach called spatial-2-phase-shift keying (S2-PSK) that uses a pair of optical light sources on a car (either a pair of rear-LEDs or a pair of front-LEDs) as a transmitter. A typical camera (i.e., less than 30 frames/s), either a global shutter or a rolling shutter, can be used to receive light-modulated data. As a remarkable physical layer (PHY) operating mode among image sensor communication PHY modes recently contributed to IEEE 802.15.7r1 (TG7r1), the communication system, along with technical details on the S2-PSK communication protocol, is described in this paper. In addition, in order to deal with the noisy optical channel and the low-performance challenge of a camera receiver, we have suggested a rate-1/2 line coding and an intelligent decoding approach in our intended vehicular communication scenario. Finally, numerical analyses on the performance of our system and experiment results are addressed.

신경망 기반 차량 이미지센서 시스템을 위한 플리커 프리 공간-PSK 변조 기법

This paper introduces a novel modulation scheme for vehicular communication in taking advantage of existing LED lights available on a car. Our proposed 2-Phase Shift Keying (2-PSK) is a spatial modulation approach in which a pair of LED light sources in a car (either rear LEDs or front LEDs) is used as a transmitter. A typical camera (i.e. low frame rate at no greater than 30fps) that either a global shutter camera or a rolling shutter camera can be used as a receiver. The modulation scheme is a part of our Image Sensor Communication proposal submitted to IEEE 802.15.7r1 (TG7r1) recently. Also, a neural network approach is applied to improve the performance of LEDs detection and decoding under the noisy situation. Later, some analysis and experiment results are presented to indicate the performance of our system