Metal Oxide Semiconductor Image Sensor Research Articles

Smartphone compatible on-site fluorescence analyzer for spilled crude oil based on CMOS image sensor

Oil spills, both in the ocean and inland waters, have negative effects such as mass mortality and habitat destruction. In this study, we develop a spilled oil point-of-testing device (SOPD), a field measurement system for immediate on-site fluorescence monitoring of oil concentrations. Photodetectors, such as photodiodes and photo multiplier tubes, which are commonly used in existing instrumentation, have inherent limitations because they are capable of only single-pixel detection. However, SOPD can cope with environmental noise caused by dust, dirt, and impurities, by adopting a multi-mega pixel complementary metal-oxide semiconductor image sensor as a detector. Furthermore, the manufacturing cost is remarkably reduced by a simple optical configuration using only two light-emitting diode light sources, five optical lenses, and a CMOS sensor. SOPD can quantitatively analyze fluorescence intensities at both low (0–1 ppm) and high (1–800 ppm) concentrations in marine and inland environments, respectively. The performance of the equipment reported in this paper was demonstrated by measuring the linear measurement range and the limit of detection using five different kinds of crude oil. A correlation analysis of crude oil samples having different collection dates was also completed. In addition, SOPD has been developed as a remote system that can be operated and analyzed on a smartphone, thereby enhancing user convenience, reducing analysis time, and simplifying device.

A single LED lamp positioning system based on CMOS camera and visible light communication

In this paper, a positioning system framework based on a single light emitting diode (LED) lamp is introduced. The LED lamp is used to transmit the positioning information, and the commercial mobile phone is used as the receiver. A fringe image carrying information is acquired by the rolling shutter effect of the complementary metal oxide semiconductor (CMOS) image sensor. Based on this, a piecewise fitting method is proposed to instead of the overall fitting method, which improves the decoding success rate. We also propose a method of LED light source recognition, which detects the edge of the light source based on brightness weakening, the Sobel operator and the least-squares fitting. In addition, the imaging principle of the camera in horizontal and tilted scenarios is analyzed. Accurate indoor positioning is achieved by the imaging principle of the camera lens and the attitude information of the mobile phone. The experimental results show that the system established in this paper can achieve centimeter-level positioning accuracy whether the mobile phone is placed horizontally or obliquely.

Chronic brain blood-flow imaging device for a behavioral experiment using mice.

A chronic brain blood-flow imaging device was developed for cerebrovascular disease treatment. This device comprises a small complementary metal-oxide semiconductor image sensor and a chronic fiber-optic plate window on a mouse head. A long-term cerebral blood-flow imaging technique was established in a freely moving mouse. Brain surface images were visible for one month using the chronic FOP window. This device obtained brain surface images and blood-flow velocity. The blood-flow changes were measured in behavioral experiments using this device. The chronic brain blood-flow imaging device may contribute to determining the cause of cerebrovascular disease and the development of cerebrovascular disease treatment.

Low-cost and portable UV holographic microscope for high-contrast protein crystal imaging

Imaging protein crystals and distinguishing them from salt crystals is an important task for protein crystallographers. The conventional tool used for this purpose is a dual-mode microscope composed of bright-field and ultraviolet (UV) induced fluorescence modes. The distinction between a protein and a salt crystal is made based upon the fluorescence response to the UV excitation, where most protein crystals absorb the UV excitation and emit fluorescence, unlike salt crystals. These dual-mode optical microscopes are sensitive; however, they are relatively bulky and expensive as they require UV-grade optics. As an alternative, here we demonstrate that on-chip UV holographic imaging offers a low-cost, portable, and robust technique to image and distinguish protein crystals from salt crystals, without the need for any expensive and bulky optical components. Only composed of a UV light-emitting-diode at 280 nm and a consumer-grade complementary metal–oxide–semiconductor image sensor de-capped and interfaced to a Raspberry Pi single-board computer, the necessary information from the crystal samples (placed very close to the sensor active area) is captured in the form of in-line holograms and extracted through digital back-propagation. In these holographic amplitude reconstructions, protein crystals appear significantly darker compared to the background due to the strong UV absorption, unlike salt crystals which do not show any contrast, enabling us to clearly distinguish between them. We believe that the on-chip UV holographic microscope could serve as a low-cost, sensitive, and robust alternative to conventional lens-based UV-microscopes used in protein crystallography.

High-precision indoor positioning algorithm based on visible light communication using complementary metal–oxide–semiconductor image sensor

Visible light positioning (VLP) is widely believed to be a cost-effective answer to the growing demand for indoor positioning. There are two critical elements in VLP: positioning accuracy and real-time ability. Nevertheless, because of the nonlinear and highly complicated relationship between three-dimensional world coordinate and two-dimensional image coordinate, developing an effective VLP location algorithm to locate the positioning facility using complementary metal–oxide–semiconductor image sensor is of great need. Additionally, due to the high computational cost of image processing, most existing VLP systems fail to deliver satisfactory performance regarding real-time ability and positioning accuracy, both of which are crucial for the performance of an indoor positioning algorithm. Therefore, we propose a positioning algorithm: the triple-light positioning algorithm based on a situation with three light-emitting diodes (LEDs). The proposed positioning algorithm takes positioning accuracy into consideration. As there are merely binary linear equations to solve, real-time ability is also reflected in the proposed algorithm, which is very worth mentioning because most current positioning algorithms fail to deliver the efficiency. In addition, we treated the LED-ID detection and recognition problem, which is an essential part of the algorithm, as a classification problem in machine learning field and machine learning algorithm into consideration. Experiment results show that the proposed algorithm provides an accuracy of 4.38 cm and the computational time is 65.50 ms, suggesting that the proposed algorithm has a good performance in accuracy as well as real-time ability.

Fully Integrated Light-Sensing Stimulator Design for Subretinal Implants.

This paper presents a fully integrated photodiode-based low-power and low-mismatch stimulator for a subretinal prosthesis. It is known that a subretinal prosthesis achieves 1600-pixel stimulators on a limited single-chip area that is implanted beneath the bipolar cell layer. However, the high-density pixels cause high power dissipation during stimulation and high fabrication costs because of special process technologies such as the complementary metal-oxide semiconductor CMOS image sensor process. In addition, the many residual charges arising from the high-density pixel stimulation have deleterious effects, such as tissue damage and electrode corrosion, on the retina tissue. In this work, we adopted a switched-capacitor current mirror technique for the single-pixel stimulator (SPStim) that enables low power consumption and low mismatch in the subretinal device. The customized P+/N-well photodiode used to sense the incident light in the SPStim also reduces the fabrication cost. The 64-pixel stimulators are fabricated in a standard 0.35-μm CMOS process along with a global digital controller, which occupies a chip area of 4.3 × 3.2 mm2 and are ex-vivo demonstrated using a dissected pig eyeball. According to measured results, the SPStim accomplishes a maximum biphasic pulse amplitude of 143 μA, which dissipates an average power of 167 μW in a stimulation period of 5 ms, and an average mismatch of 1.12 % between the cathodic and anodic pulses.

Analysis of Disparity Information for Depth Extraction Using CMOS Image Sensor with Offset Pixel Aperture Technique.

A complementary metal oxide semiconductor (CMOS) image sensor (CIS), using offset pixel aperture (OPA) technique, was designed and fabricated using the 0.11-µm CIS process. In conventional cameras, an aperture is located on the camera lens. However, in a CIS camera using OPA technique, apertures are integrated as left-offset pixel apertures (LOPAs) and right-offset pixel apertures (ROPAs). A color pattern is built, comprising LOPA, blue, red, green, and ROPA pixels. The disparity information can be acquired from the LOPA and ROPA channels. Both disparity information and two-dimensional (2D) color information can be simultaneously acquired from the LOPA, blue, red, green, and ROPA channels. A geometric model of the OPA technique is constructed to estimate the disparity of the image, and the measurement results are compared with the estimated results. Depth extraction is thus achieved by a single CIS using the OPA technique, which can be easily adapted to commercial CIS cameras.

3D Visible Light Indoor Positioning by Bokeh Based Optical Intensity Measurement in Smartphone Camera

This paper proposes a three dimensional (3D) high-resolution indoor positioning system that uses a single light-emitting diode (LED) and a complementary metal-oxide semiconductor image sensor (CIS). The proposed visible light positioning system is based on the bokeh effect for improving the positioning accuracy. The 3D position is determined by received signal strength (RSS) and angle of arrival (AOA). Optical saturation in pixels causes severe performance degradation in the RSS-based positioning based on the CIS. The bokeh-based image reception technique resolves this problem by spreading the optical power to adjacent pixels. Additionally, via the bokeh effect, the size of the LED image on the CIS is maintained regardless of the distances. The accuracy of the LED extraction for the AOA can be improved by fixed LED size. These characteristics of the proposed positioning scheme are analyzed via simulations, and the feasibility of the method is verified based on the experimental results. The experimental results indicate an estimated positioning average error rate within 4%.

Single particle transient response and displacement damage in CMOS image sensors induced by high energy neutrons at Back-n in CSNS facility

Neutron radiation experiments of the complementary metal-oxide semiconductor (CMOS) image sensors (CISs) at back-streaming white neutrons (Back-n) in China Spallation Neutron Source (CSNS) facility are presented. The displacement damages induced by neutron radiation are characterized by the analysis of the increase of dark signal and dark signal non-uniformity, and the occurrences of dark signal spike and random telegraph signal (RTS). The two-level and multi-level RTS behaviors induced by neutron displacement radiation are presented. The mechanisms of the dark signal spikes and RTS induced by neutron displacement radiation are analyzed respectively. The single particle transient response characterizations are described by analyzing the online dark images during neutron radiation. The mechanisms of the dark signal spikes and the white lines induced by the single transient neutron events are also demonstrated by analyzing the interactions between the higher energy neutrons and bulk silicon lattices of CISs.

A Low-Noise Image Sensor Readout Circuit With Internal Timing Generator

We describe the design, development, and performance of a complementary metal-oxide semiconductor (CMOS) image sensor read-out circuit. Designed in a 0.35-m CMOS technology and occupying an area of 2.3 13.6mm2, the proposed multipixel image sensor readout integrated circuit with internal timing generator achieves high performance with 188.7-V root mean square (RMS) of ultralow output noise, wide linear dynamic range of 2.3V, low power consumption of 42mA, and a simple timing scheme. The approaches to boost linear dynamic range are presented. The image sensor also provides the trigger signal for data acquisition purpose.

A 250 m Direct Time-of-Flight Ranging System Based on a Synthesis of Sub-Ranging Images and a Vertical Avalanche Photo-Diodes (VAPD) CMOS Image Sensor

We have developed a direct time-of-flight (TOF) 250 m ranging Complementary Metal Oxide Semiconductor (CMOS) image sensor (CIS) based on a 688 × 384 pixels array of vertical avalanche photodiodes (VAPD). Each pixel of the CIS comprises VAPD with a standard four transistor pixel circuit equipped with an analogue capacitor to accumulate or count avalanche pulses. High power near infrared (NIR) short (<50 ns) and repetitive (6 kHz) laser pulses are illuminated through a diffusing optics. By globally gating the VAPD, each pulse is counted in the in-pixel counter enabling extraction of sub-photon level signal. Depth map imaging with a 10 cm lateral resolution is realized from 1 m to 250 m range by synthesizing subranges images of photon counts. Advantages and limitation of an in-pixel circuit are described. The developed CIS is expected to supersede insufficient resolution of the conventional light detection and ranging (LiDAR) systems and the short range of indirect CIS TOF.

Real-time rolling shutter compensation for a complementary metal-oxide semiconductor image sensor

Complementary metal-oxide semiconductor image sensors suffer from undesirable geometric image distortion due to the rolling shutter effect. However, conventional algorithms cannot be used for strict real-time use in mobile devices. An extremely low computational method to compensate rolling shutter effect for the mobile devices is proposed. The proposed method efficiently predicts the rotational camera motion in three-dimensional space from the global motion vectors predicted in two-dimensional image space. Hence, the computational complexity is extremely reduced. To further improve the video quality, the video stabilization to reduce the high-frequency jitter of the camera motion without increasing the complexity is introduced. The results of simulation demonstrate that while the computational complexity is extremely low, the proposed algorithm provides acceptable performance.

Trapezoid pixel array complementary metal oxide semiconductor image sensor with simplified mapping method for traffic monitoring applications

A trapezoid (TZOID) pixel array complementary metal oxide semiconductor (CMOS) image sensor, and a pixel-resolution mapping (P-RM) method to extract information from the observed objects without requiring any specific patterns or man-made markings, are proposed for traffic applications. The size of each pixel in the TZOID imager was determined by pixel design equations based on the relationship between the pixel’s physical size and related spatial resolution. The TZOID imager trades detection accuracy for a tolerable data loss to reduce communication bandwidth and computation power. The P-RM method determines the pixels’ actual resolutions in the real world of the captured imaged by both standard rectangular single- and multiresolution TZOID pixel arrays. Both indoor and outdoor experiments were performed using single- and multiresolution imagers. The TZOID imager integrated circuit (IC) was fabricated in a 0.18-μm CMOS process. It was tested, and its performance parameters were compared with an off-the-shelf CMOS image sensor IC. Tests showed that the TZOID generates 98% less data than that of a standard image sensor, trading off only 2% detection accuracy. Results also show that the proposed method provides a general procedure for designing a multiresolution sensor for traffic applications.

Electrical Characterization of Charge Polarity in AlF₃ Anti-Reflection Layers for Complementary Metal Oxide Semiconductor Image Sensors.

In this study, the charge polarity of aluminum fluoride (AlF3) as a function of varying thickness (tAlF3 = 20, 35, 50, 65, and 80 nm) was discussed. AlF3 films were deposited onto p-Si wafers via electron beam sputtering. Thickness dependent charge polarity and reliability issues under bias-temperature stress conditions were identified using a capacitance-voltage (C-V) characterization method. AlF3 was found to possess negative fixed charges, leading to a C-V curve shift toward the positive gate bias direction as tAlF3 was increased up to 50 nm. On the contrary, the C-V characteristics were dominantly affected by the positive charges of mobile ions and/or fluorine vacancies when tAlF3 was increased to more than 50 nm. Additionally, negative bias temperature stress (1 MV/cm, 473 K for 10 mins) increased insulator trapped charges and decreased interface traps in 20 nm thick AlF3 films. These results could be attributed to positively charged fluorine vacancies introduced by broken Al-F bonds within AlF3 films and the passivation of Si dangling bonds due to broken fluorine ions at the interface, respectively. It was believed that 20 nm thick AlF3 films sufficiently attracted holes from the Si substrate, forming a hole accumulation layer on the surface due to total charge polarity of the AlF3 dielectric being entirely governed by negative fixed charges as the thickness of AlF3 decreased. Based on these results, AlF3 films are proposed for use as an anti-reflection layer to replace HfO2 in CMOS image sensors.

Orchard Free Space and Center Line Estimation Using Naive Bayesian Classifier for Unmanned Ground Self-Driving Vehicle

In the case of autonomous orchard navigation, researchers have developed algorithms that utilize features, such as trunks, canopies, and sky in orchards, but there are still various difficulties in recognizing free space for autonomous navigation in a changing agricultural environment. In this study, we applied the Naive Bayesian classification to detect the boundary between the trunk and the ground and propose an algorithm to determine the center line of free space. The naïve Bayesian classification requires a small number of samples for training and a simple training process. In addition, it was able to effectively classify tree trunk’s points and noise points of the orchard, which are problematic in vision-based processing, and noise caused by small branches, soil, weeds, and tree shadows on the ground. The performance of the proposed algorithm was investigated using 229 sample images obtained from an image acquisition system with a Complementary Metal Oxide Semiconductor (CMOS) Image Sensor (CIS) camera. The center line detected by the unaided-eye manual decision and the results extracted by the proposed algorithm were compared and analyzed for several parameters. In all compared parameters, extracted center line was more stable than the manual center line results.

(Invited) Proximity Gettering Design of Hydrocarbon Molecular Ion Implanted Silicon Wafers Using Direct Bonding Technique for Advanced CMOS Image Sensors: A Review

Complementary metal oxide semiconductor (CMOS) image sensors have become widely used ubiquitous devices such as smart phone and personal computer type tablets. Consumer markets strongly demands more high sensitivity imaging and more high speed image data processing for achieving advanced CMOS image sensor fabrication. However, there are some serious improvement technical issues in CMOS image sensor fabrication process. Extremely important technical issue is metallic impurity contaminated device active region during device fabrication process such as high temperature rapid thermal annealing, dry etching and plasma heat treatment. Metallic impurities formed deep energy level defects in the silicon band-gap. Thus, these deep energy level defects strongly affects electrical device characteristic such as pn-junction leakage current, recombination lifetime and gate oxide breakdown voltage. Another technical issue is oxygen impurity out-diffused to the device active region from a Czochralski (CZ) grown silicon substrate during device heat process. Oxygen impurity formed oxygen related deep energy level defects in the device active region of photo-diode space charge region and transfer transistor gate channel region. These oxygen related defects strongly influenced on electrical device performance such as image lag. Therefore, in recent years, CMOS image sensor manufacturers have strongly demanded silicon wafers with the highest metal and oxygen impurities gettering capability during CMOS device processes. In response, we have developed a novel gettering wafer production concept using a hydrocarbon molecular ion implantation technique for advanced CMOS image sensors [1][2][3]. This technique can implant a silicon wafer surface simultaneously with carbon and hydrogen elements that form the projection range using a hydrocarbon compound molecular gas source. In our previous study, it was found that a hydrocarbon molecular ion implanted silicon wafer had three unique characteristics for high performance of advanced CMOS image sensors. First, a hydrocarbon molecular ion projection range has high gettering capability of metallic impurities [1][2][4]. Second, this projection range also has a diffusion barrier effect for oxygen impurities out-diffusing to the device active region from a CZ grown silicon wafer substrate [5]. Third, it is expected that diffusing hydrogen to the device active region from the hydrogen of hydrocarbon molecular ions gettered in the projection range during the device fabrication process will have a passivation effect on unreconstructed surface dangling bonds and Si/SiO2 interface state densities such as local oxidation of silicon or shallow trench isolation [6][7]. Moreover, we achieved more improvement for gettering capability of hydrocarbon molecular ion implanted silicon wafer which formed epitaxial layer without epitaxial growth process using room temperature direct bonding technique. In this presentation, we demonstrate that a hydrocarbon molecular ion implanted silicon wafers using room temperature direct bonding technique has high gettering capability characteristics for transition metal and light element impurities such as oxygen, and hydrogen during the CMOS image sensor fabrication process.

Adaptive window thresholding for noise-robust photo detection in OCC

In an image sensor-based visible-light communication, noise reduction is essential for improving the transmission performance because each frame contains received data with various types of noises. In this study, we demonstrate the structures of receive (Rx) and transmit (Tx) frames for optical camera communication (OCC) by using the rolling shutter effect of a complementary metal-oxide semiconductor (CMOS) image sensor. We propose and demonstrate a three-step noise-reduction process for each frame. The additive white Gaussian noise is reduced by using “signal averaging,” and the background noise is reduced via “adaptive window thresholding” with “normalization.” By using the proposed method, this system shows noise tolerance. We experimentally demonstrate an 8.55-kBd error-free optical camera transmission.

Development of novel complementary metal-oxide semiconductor-based colorimetric sensors for rapid detection of industrially important gases

A sensitive and low-cost colorimetric sensor array was developed to detect industrially important gases. We selected several target gases and indicators, and then used a printing method to fabricate an array to read the color value and to analyze the data. Using the printing method, various indicators can be produced in a uniform array on a desired substrate in a short time and at a low-cost. Four gases (ammonia, methylamine, trimethylamine, and hydrazine) and ten indicators were used to confirm the ability of the gas detection system. The color change of the dyes in the array was extracted in real time using an array reader with an integrated complementary metal-oxide semiconductor (CMOS) image sensor. The developed gas detection system successfully detected gases using an array reader, which provided data for sensitive color changes in real time using the CMOS image sensor. This mechanism could be applied industrially for detecting toxic gases because the developed sensor is highly sensitive and inexpensive.

Temporal Noise Analysis of Charge-Domain Sampling Readout Circuits for CMOS Image Sensors.

This paper presents a temporal noise analysis of charge-domain sampling readout circuits for Complementary Metal-Oxide Semiconductor (CMOS) image sensors. In order to address the trade-off between the low input-referred noise and high dynamic range, a Gm-cell-based pixel together with a charge-domain correlated-double sampling (CDS) technique has been proposed to provide a way to efficiently embed a tunable conversion gain along the read-out path. Such readout topology, however, operates in a non-stationery large-signal behavior, and the statistical properties of its temporal noise are a function of time. Conventional noise analysis methods for CMOS image sensors are based on steady-state signal models, and therefore cannot be readily applied for Gm-cell-based pixels. In this paper, we develop analysis models for both thermal noise and flicker noise in Gm-cell-based pixels by employing the time-domain linear analysis approach and the non-stationary noise analysis theory, which help to quantitatively evaluate the temporal noise characteristic of Gm-cell-based pixels. Both models were numerically computed in MATLAB using design parameters of a prototype chip, and compared with both simulation and experimental results. The good agreement between the theoretical and measurement results verifies the effectiveness of the proposed noise analysis models.

Novel approach to improve the attitude update rate of a star tracker

The star tracker is widely used in attitude control systems of spacecraft for attitude measurement. The attitude update rate of a star tracker is important to guarantee the attitude control performance. In this paper, we propose a novel approach to improve the attitude update rate of a star tracker. The electronic Rolling Shutter (RS) imaging mode of the complementary metal-oxide semiconductor (CMOS) image sensor in the star tracker is applied to acquire star images in which the star spots are exposed with row-to-row time offsets, thereby reflecting the rotation of star tracker at different times. The attitude estimation method with a single star spot is developed to realize the multiple attitude updates by a star image, so as to reach a high update rate. The simulation and experiment are performed to verify the proposed approaches. The test results demonstrate that the proposed approach is effective and the attitude update rate of a star tracker is increased significantly.