Integrated CMOS Image Sensor Research Articles

Metasurface-based polarization color routers

Polarization, an attribute of light, carries a lot of vital information independent of intensity and color content. Polarization imaging is very useful in applications including diffuse/specular component separation, material classification and reflectance estimation. Color routers applied in color imaging provide an efficient way to sort and guide different color lights to the corresponding output ports. Here, we report the design and experimental realization of a compact metasurface-based polarization color router. This color router shows a micron Bayer pattern (RGGB distribution) under the illumination of white light with any polarization. In addition, the green light intensity at the corresponding pixel changes regularly depending on the polarization light, illustrating the polarization color router can act as a polarizer at the same time. The measured light utilization efficiency of the polarization color router reaches a value of 88%. A correlation coefficient of 0.98, which is defined as the fitting effect between the measured and theoretical illuminating light polarization angles, was achieved. This device possesses the advantages of a lightweight, high light utilization efficiency and polarization-integrated detection functions, which could provide a new way for its applications in integrated CMOS image sensors.

Research on a pulse-based high-line-rate TDI CMOS image sensor

In this paper, a 256-stage pulse-based time delay integration (TDI) CMOS image sensor is presented. In this sensor, a capacitive trans-impedance amplifier (CTIA) based pipeline charge transfer (PCT) pixel structure is proposed, which achieves low noise signal accumulation. A pulse generator circuit is used after the PCT-pixel to detect and generate pulse, which starts processing self-reset operations when the pulse is generated. Therefore, 1-bit output is realized for each stage of TDI signal processing circuit. The pulse signal is easier to accumulate in the digital accumulator. With a high line rate of 100 KHz, SNR improvement of the proposed sensor is 24.15 dB and the energy consumption of the sensor is 0.44 nJ/pixel.

Hybrid‐mode single‐slope ADC with improved linearity and reduced conversion time for CMOS image sensors

SummaryIn the paper, a single‐slope analog‐to‐digital converter (ADC) for integrated CMOS image sensor applications with an improved technique of conversion has been proposed. The proposed hybrid‐mode ADC automatically uses one of the following conversion techniques: time based (i.e. PWM) or voltage based (i.e. single‐slope). During the ADC operation, the clock frequency and reference voltage are modified in order to reduce the conversion time and achieve the optimal linearity. Owing to this, the pixel using a photodiode working in the integration mode achieves a linear photoconversion characteristics (irradiance to digital number), and the conversion period, which is determined by the darkest parts of a scene, is reduced by an order of magnitude comparing with known ADC solutions. The proposed conversion technique has been validated with the ASIC prototype of a CMOS imager containing photosensors integrated with the ADCs. The ASIC was fabricated in standard 0.18 μm CMOS technology. A specialized measurement system has been used to optimize linearity in the hybrid‐mode conversion (integral nonlinearity below 2 LSB). The conversion period has been reduced 15 times compared with the standard technique. Measurements confirm functionality of the proposed approach, implemented within a small pixel area.

(Invited) Pixel-Parallel 3-D Integrated CMOS Image Sensors for Next-Generation Video Systems

Performances of image sensors including pixel counts, capturing speed, and dynamic range continue improving to meet the demands for high-reality video systems such as 8K/4K television and three-dimensional (3-D) imaging. Recent 3-D integration technology of silicon device has boosted sensor performances by using through-silicon vias (TSVs) and micro-bumps [1-2]. However, fully parallel signal processing has not been possible yet because the size of TSVs or micro-bumps is larger than the imaging pixel. We have proposed a new 3-D integration technology without using TSV or micro-bumps but by using direct bonding of silicon-on-insulator (SOI) wafers, in which gold electrodes are embedded in the SiO2 intermediate layer. The gold electrodes formed by the damascene process with chemical mechanical polishing (CMP) can be reduced to 1 μm or less in diameter. This 3-D integration technology enables us to develop pixel-parallel 3-D integrated CMOS image sensors [3-5] as illustrated in Fig. 1. Two SOI wafers are stacked and directly connected by the embedded gold electrodes within every pixel. We have also developed novel in-pixel A/D converters (ADCs) with pulse frequency output, where signal charges generated in photodiodes (PDs) are converted into pulse signals at a frequency corresponding to the input illuminance. The ADC has an advantage of wide dynamic range of illuminance because the signal charge accumulated in the PD is never saturated in principle. Prototype sensors of 128 × 96 resolution were developed with 80-μm square pixel each and 10-μm diameter gold electrodes. The upper layer was assigned for the PDs and ADCs, and the lower was for the 16-bit pulse counters. The two wafers were bonded by an image recognition of infra-red camera, where we achieved alignment accuracy as high as 1 μm or better. We evaluated the sensor and confirmed an excellent linear output with a wide dynamic range of 96 dB, which corresponds to the 16-bit output of the pulse counter. Fig. 2 shows an example image captured by the developed sensor. We successfully demonstrated the operation of the pixel-parallel sensor without any pixel defects. In summary, a pixel-parallel 3-D integrated CMOS image sensor with pulse-frequency-output A/D converters was developed by using direct bonding of SOI wafers and embedded gold electrodes. The developed process is promising for image sensors with ultimate performances and various More-than-Moore type devices.

(Invited) Pixel-Parallel 3-D Integrated CMOS Image Sensors for Next-Generation Video Systems

We report 3-D integrated CMOS image sensors with pixel-parallel A/D converters (ADCs). Direct bonding of silicon-on-insulator (SOI) wafers with gold electrodes embedded in a SiO2 surface achieves high-density pixel-wise interconnection in the sensor. Prototype 128 × 96 resolution sensors are developed on two SOI wafers with 16-bit ADCs for pulse frequency output integrated in every pixel. We have confirmed successful operation of capturing video images of 16-bit digital output and a wide dynamic range of 96 dB, demonstrating this type of 3-D integration provides functional diversification including circuits, sensors, and More-than-Moore type devices.

A Global Shutter High Speed TDI CMOS Image Sensor With Pipelined Charge Transfer Pixel

In this paper, a $120\,\times \, 45$ global shutter high speed time delay integration (TDI) CMOS image sensor with pipelined charge transfer pixel (PCT-pixel) is presented. Offset free and low noise pipelined signal accumulation are achieved by the PCT-pixel, and a novel layout is also proposed to increase the equivalent photosensitive area’s fill factor of the proposed pixel. Due to the parallelism of the proposed PCT-pixel, global shutter exposure method is applied in this sensor, which can eliminate nonsynchronous signal capturing problem. The proposed TDI sensor is implemented in a 0.11- $\mu \text{m}$ one-poly three-metal CMOS image sensor technology with a line rate of 100 KHz, a PCT-pixel size of $30\,\times \, 15 \,\,\mu \text{m}^{2}$ , and a fill factor of 43.5%. Measurement results show that the sensor can achieve a maximum sensitivity of 95 V/lux $\,\times \, $ sec and an energy consumption of 0.12 nJ/pixel. Measured signal-to-noise ratio boost value follows theoretical value well. The proposed sensor has the potential to achieve high scanning speed and high TDI stage while costing less power and silicon area.

An Antivibration Time-Delay Integration CMOS Image Sensor With Online Deblurring Algorithm

This paper presents an antivibration time-delay integration (TDI) CMOS image sensor (CIS) for small remote imaging systems, introducing a hardware-implemented online deblurring (ODB) algorithm to address the image blur problems caused by vibrations. The proposed sensor has eight TDI stages, column-parallel TDI accumulating and ODB circuits. A $256\times 8$ -pixel prototype chip was fabricated using a 0.18- $\mu \text{m}$ CIS technology with a pixel footprint of $6.5~\mu {\mathrm{ m}}\times 6.5~\mu \text{m}$ and a fill factor of 28%. Measurement results show that the sensor can achieve dynamic ranges of 45.1 and 51.8 dB, respectively, with and without enabling the ODB algorithm. Compared with a single-stage line scanner imager, it offers an improvement in signal-to-noise ratios of 1.9 and 8.6 dB, respectively, with and without the ODB algorithm.

(Invited) Pixel-Parallel CMOS Image Sensors with 16-Bit A/D Converters Developed by 3-D Integration of SOI Layers with Au/SiO2 Hybrid Bonding

To meet the increasing demands for higher performance of video imaging, several 3D integrated CMOS image sensors have been recently studied by using through-silicon vias (TSVs) and micro-bumps [1-2]. Although these sensors improved some of their performances by 3D integration, in-pixel interconnection was not realized because the TSV or bump size was larger than the imaging pixel size. We have studied on the pixel-parallel 3D integrated CMOS image sensors by using embedded Au electrodes [3-5] suitable for high density integration instead of TSVs or bumps. Fig. 1(a) shows a schematic of the developed CMOS image sensor. Pixels including 128 × 96 photodiodes (PDs) and A/D convertors (ADCs) in the upper layer were connected to 16-bit counters in the lower layer for each pixel. The two functional layers were formed on silicon-on-insulator (SOI) wafers and directly bonded, mediated by Au electrodes embedded in the intermediate SiO2 layer by electroplating and chemical mechanical polishing (CMP). Each pixel size was 80 μm square with a Au electrode of 10 μm in diameter. No voids were observed at the bonded interface. By measuring the input–output response of the developed sensor, we obtained a linear output to the light illumination intensity. The sensor also had a wide dynamic range of 96 dB, which corresponds to the 16-bit output. Signals were detected even under low illumination of 0.1 lx. Fig. 1(b) demonstrates an example of video images captured by the developed sensor. The 16-bit digital output contrast was adjusted to the standard display by compressed to 8-bit data. We confirmed the pixel-parallel operation without any pixel defects. In summary, a 128 × 96 pixel-parallel 3D integrated CMOS image sensor with 16-bit A/D converters was successfully demonstrated by applying Au/SiO2 hybrid bonding of SOI layers. The developed process is promising for CMOS image sensors of higher performance and More-than-Moore type devices for the next generation electronics.

Pixel-Parallel 3-D Integrated CMOS Image Sensors With Pulse Frequency Modulation A/D Converters Developed by Direct Bonding of SOI Layers

We have developed for the first time a 3-D integrated CMOS image sensor with pixel-parallel analog-to-digital converters (ADCs). Photodiode (PD) and inverter layers are prepared on separate silicon-on-insulator layers and directly bonded with damascened Au electrodes. The handle layer is then removed by grinding and XeF2 vapor phase etching to expose the PD surface. The developed process is suitable for pixelwise interconnection because it allows the damascened Au electrodes to be 1 $\mu \text{m}$ in diameter or less. An ADC circuit is designed based on pulse frequency modulation where pulses are generated proportional to the illumination intensity, and contains a PD, inverters, a reset transistor, and counters. A prototype 3-D integrated CMOS image sensor is also developed with 64 pixels, which acquires video images without pixel defects. A wide dynamic range of >80 dB is confirmed for the incident light intensity. The experimental results demonstrate the feasibility of pixel-level 3-D integration for high-performance CMOS image sensors.

Dual light field and polarization imaging using CMOS diffractive image sensors

In this Letter we present, to the best of our knowledge, the first integrated CMOS image sensor that can simultaneously perform light field and polarization imaging without the use of external filters or additional optical elements. Previous work has shown how photodetectors with two stacks of integrated metal gratings above them (called angle sensitive pixels) diffract light in a Talbot pattern to capture four-dimensional light fields. We show, in addition to diffractive imaging, that these gratings polarize incoming light and characterize the response of these sensors to polarization and incidence angle. Finally, we show two applications of polarization imaging: imaging stress-induced birefringence and identifying specular reflections in scenes to improve light field algorithms for these scenes.

A 12-bit high-speed column-parallel two-step single-slope analog-to-digital converter (ADC) for CMOS image sensors.

A 12-bit high-speed column-parallel two-step single-slope (SS) analog-to-digital converter (ADC) for CMOS image sensors is proposed. The proposed ADC employs a single ramp voltage and multiple reference voltages, and the conversion is divided into coarse phase and fine phase to improve the conversion rate. An error calibration scheme is proposed to correct errors caused by offsets among the reference voltages. The digital-to-analog converter (DAC) used for the ramp generator is based on the split-capacitor array with an attenuation capacitor. Analysis of the DAC's linearity performance versus capacitor mismatch and parasitic capacitance is presented. A prototype 1024 × 32 Time Delay Integration (TDI) CMOS image sensor with the proposed ADC architecture has been fabricated in a standard 0.18 μm CMOS process. The proposed ADC has average power consumption of 128 μW and a conventional rate 6 times higher than the conventional SS ADC. A high-quality image, captured at the line rate of 15.5 k lines/s, shows that the proposed ADC is suitable for high-speed CMOS image sensors.

A dynamic range extension scheme applied to a TDI CMOS image sensor

A dynamic range extension scheme applied to a time delay integration (TDI) CMOS image sensor (CIS) is presented. Two types of pixels with higher and lower conversion gain are adopted in the pixel array, which are suitable for capturing images in low and high illumination respectively. By fusing the two kinds of pixels' output signals in the process of TDI accumulation, a high dynamic range image can be achieved. Compared with the traditional multiple integration technique, no photoelectrons generated during the exposure time are discarded by the reset operation, and thus a higher level of signal-to-noise ratio (SNR) can be retained. A prototype chip with an 8 × 8 pixel array is implemented in a 0.18 μm CIS process, and the pixel size is 15 × 15 μm2. Test results show that a 76 dB dynamic range can be achieved in 8-stage TDI mode, when the SNR boost can reach 7.26 dB at 90.8 lux.

In-pixel charge addition scheme applied in time-delay integration CMOS image sensors

An addition scheme applicable to time-delay integration (TDI) CMOS image sensor is proposed, which adds signals in the charge domain in the pixel array. A two-shared pixel structure adopting two-stage charge transfer is introduced, together with the rolling shutter with an undersampling readout timing. Compared with the conventional TDI addition methods, the proposed scheme can reduce the addition operations by half in the pixel array, which decreases the power consumption of addition circuits outside the pixel array. The timing arrangement and pixel structure are analyzed in detail. The simulation results show that the proposed pixel structure can achieve the charge addition with negligible nonlinearity, therefore the power consumption of the periphery addition circuits can be reduced by half theoretically.

Characteristics of random telegraph signal noise in time delay integration CMOS image sensor

A statistical model based on large sample simulation is established to study the relationship between random telegraph signal (RTS) noise and the number of time delay integration (TDI) stages in TDI CMOS image sensor (CIS). Matlab simulation results show that the mean value of RTS noise increases by a factor greater than M0.5 when the number of TDI stages is M, and the factor approximates to M0.5 with larger TDI stages. In noise histogram, RTS noise exhibits Gaussian distribution when the number of TDI stages is more than a special value. These results serve as a guideline for the design of TDI stages and the analysis of noise.

An Integrating Wide Dynamic-Range Image Sensor With a Logarithmic Response

An integrating CMOS image sensor with a wide dynamic range is described. The dynamic range of these pixels is controlled by a user-defined reference voltage that creates a photocurrent-dependent effective integration time. The operation of these pixels and a method of obtaining a well-controlled logarithmic response are both described. Furthermore, described are the results of two alternative methods of correcting the fixed pattern noise in these pixels and measurements of the temporal noise from individual pixels. These results show that with these pixels, it is possible to match the contrast sensitivity of the human visual system.

High-speed laser Doppler perfusion imaging using an integrating CMOS image sensor

This paper describes the design and the performance of a new high-speed laser Doppler imaging system for monitoring blood flow over an area of tissue. The new imager delivers high-resolution flow images (256x256 pixels) every 2 to 10 seconds, depending on the number of points in the acquired time-domain signal (32-512 points). This new imaging modality utilizes a digital integrating CMOS image sensor to detect Doppler signals in a plurality of points over the area illuminated by a divergent laser beam of a uniform intensity profile. The integrating property of the detector improves the signal-to-noise ratio of the measurements, which results in high-quality flow images. We made a series of measurements in vitro to test the performance of the system in terms of bandwidth, SNR, etc. Subsequently we give some examples of flow-related images measured on human skin, thus demonstrating the performance of the imager in vivo. The perspectives for future implementations of the imager for clinical and physiological applications are discussed.