Metal Oxide Semiconductor Image Sensor Research Articles

Reliability modelling and assessment of CMOS image sensor under radiation environment

The Complementary Metal-Oxide Semiconductor (CMOS) image sensor is a critical component with the function of providing accurate positioning in many space application systems. Under long-time operation in space environments, there are radiation related degradation and various uncertainties affecting the positioning accuracy of CMOS image sensors, which further leads to a reliability reduction of CMOS image sensors. Obviously, the reliability of CMOS image sensors is related to their specified function, degradation, and uncertainties; however, current research has not fully described this relationship. In this paper, a comprehensive approach to reliability modelling of CMOS image sensors is proposed based on the reliability science principles. Firstly, the performance margin modelling of centroid positioning accuracy is conducted. Then, the degradation model of CMOS image sensors is derived considering the dark current increase induced by the total ionizing dose effects. Finally, various uncertainties are analyzed and quantified, and the measurement equation of reliability is proposed. A case study of a CMOS image sensor is conducted to apply the proposed method, and the sensitivity analysis can provide suggestions for design and use of CMOS image sensors to ensure reliability. A simulation study is conducted to present the advantages of the proposed comprehensive approach.

Counting alpha particles produced by radon daughters using commercial off-the-shelf complementary metal oxide semiconductor (COTS CMOS) image sensors

Radon (Rn-222) is the main source of radiation exposure to human beings from natural radiation; it is of great significance to study the measuring methods and measuring instruments of radon for natural radiation protection. In recent years, the application of pixel detectors in radiation detection has attracted attention. In this paper, a Commercial Off-the-Shelf Complementary Metal Oxide Semiconductor (COTS CMOS) image sensor which replaces the glass for protection covering in front of the sensors was used to perform a series of measurements to identify alpha particles. During the experiment, the CMOS image sensor was used to record a video during the sampling period, and then use the FFmpeg software to take screenshots of the video by frame. MATLAB was used to count bright spots from the image frames. A measurement chamber was designed to measure radon concentrations, and when the relative humidity was constant, the count of alpha particles by the CMOS image sensor increased along with the increase of the concentration. The experiments verified the feasibility of the low-cost COTS CMOS image sensors to monitor radon.

Analysis of the Interference Effects in CMOS Image Sensors Caused by Strong Electromagnetic Pulses

With the electromagnetic environment becoming increasingly complex, it is crucial to address the risk posed by electromagnetic pulse, which critically impairs the performance and reliability of electronic systems based on complementary metal oxide semiconductor (CMOS) image sensors. In this context, research on the failure types of CMOS image sensors in a high-power electromagnetic environment, caused by strong electromagnetic pulses and the rapid evaluation method of interference immunity, has garnered significant interest. This paper conducts electromagnetic pulse simulation experiments on CMOS image sensors to first study their failure types, such as image abnormalities and functional interruption, and then identify the corresponding failure criteria. Furthermore, this study builds on the small sample test evaluation method to investigate the interference threshold of functional interruptions in CMOS image sensors by calculating the failure probability at different field strengths. The obtained data were combined with the Weibull distribution function for fitting, the results of which found the interference threshold to be at 40.4 kV/m. The findings of this study provide a basis for evaluating the survivability of CMOS image sensors and their associated reinforcement technology in high-power electromagnetic environments.

Motion-resistant three-wavelength spatial frequency domain imaging system with ambient light suppression using an 8-tap CMOS image sensor.

We present a motion-resistant three-wavelength spatial frequency domain imaging (SFDI) system with ambient light suppression using an 8-tap complementary metal-oxide semiconductor (CMOS) image sensor (CIS) developed at Shizuoka University. The system addresses limitations in conventional SFDI systems, enabling reliable measurements in challenging imaging scenarios that are closer to real-world conditions. Our study demonstrates a three-wavelength SFDI system based on an 8-tap CIS. We demonstrate and evaluate the system's capability of mitigating motion artifacts and ambient light bias through tissue phantom reflectance experiments and in vivo volar forearm experiments. We incorporated the Hilbert transform to reduce the required number of projected patterns per wavelength from three to two per spatial frequency. The 8-tap image sensor has eight charge storage diodes per pixel; therefore, simultaneous image acquisition of eight images based on multi-exposure is possible. Taking advantage of this feature, the sensor simultaneously acquires images for planar illumination, sinusoidal pattern projection at three wavelengths, and ambient light. The ambient light bias is eliminated by subtracting the ambient light image from the others. Motion artifacts are suppressed by reducing the exposure and projection time for each pattern while maintaining sufficient signal levels by repeating the exposure. The system is compared to a conventional SFDI system in tissue phantom experiments and then in vivo measurements of human volar forearms. The 8-tap image sensor-based SFDI system achieved an acquisition rate of 9.4 frame sets per second, with three repeated exposures during each accumulation period. The diffuse reflectance maps of three different tissue phantoms using the conventional SFDI system and the 8-tap image sensor-based SFDI system showed good agreement except for high scattering phantoms. For the in vivo volar forearm measurements, our system successfully measured total hemoglobin concentration, tissue oxygen saturation, and reduced scattering coefficient maps of the subject during motion (16.5cm/s) and under ambient light (28.9lx), exhibiting fewer motion artifacts compared with the conventional SFDI. We demonstrated the potential for motion-resistant three-wavelength SFDI system with ambient light suppression using an 8-tap CIS.

Hydrogen Termination Effect on SiO2/Si Interface State Density in CH3O-Molecular-Ion-Implanted Silicon Epitaxial Wafer for CMOS Image Sensors

The reduction in SiO2/Si interface state density (Dit) at the SiO2/Si interface region is important to improve the performance of complementary metal-oxide semiconductor (CMOS) image sensors. The CH3O-ion-implanted region stores hydrogen and releases the stored hydrogen during the subsequent heat treatment. This study demonstrates that a CH3O-ion-implanted epitaxial silicon wafer can reduce the Dit and Pb0 center density in SiO2/Si interface regions, as analyzed by quasi-static capacitance–voltage and electron spin resonance measurements, respectively. Both Dit and Pb0 center density in the CH3O-implanted wafer decreased with increasing heat treatment temperature. Moreover, the activation energy is estimated to be 1.57 eV for the hydrogen termination reactions induced by the CH3O-ion-implanted wafer. The activation energy is close to those of hydrogen molecules and Si dangling bonds at the SiO2/Si interface. This result means that Dit can be reduced by hydrogen from inside the silicon wafer, regardless of the heat treatment atmosphere. It has unique characteristics not found in conventional silicon wafers. The termination effect of the CH3O-molecular-ion-implanted epitaxial silicon wafers can contribute to the high electrical performance of CMOS image sensors.

The development and application of CMOS image sensor

This paper focuses on the development of Complementary metal-oxide semiconductor (CMOS) image sensor and its applications in aerospace, medical and automotive fields. Firstly, the representative events in history and the contributions of some companies to CMOS image sensor are described. Subsequently, some characteristics of CMOS image sensor are analyzed in the image field involved. In order to evaluate the performance of CMOS image sensor, single even effect and electronic endoscope structures are analyzed and active and passive range finder experiments are carried out. The results show that the imaging based on CMOS sensor can fully meet the requirements of imaging applications in many fields.

A 1/f noise optimized correlated multiple sampling technique for complementary metal oxide semiconductor image sensor

SummaryCorrelated multiple sampling (CMS) technique is often used to reduce random noise in complementary metal oxide semiconductor (CMOS) image sensor, but as the number of samples increases, the suppression of 1/f noise by standard CMS based on averaging sampling gradually decreases. Therefore, in this paper, a 1/f noise optimized correlated multiple sampling (NOCMS) technique based on differentiated sampling weights is proposed. Transfer functions of standard CMS and NOCMS for analyzing the suppression effect of random noise respectively are derived based on the Fourier transform theory. And NOCMS shows a dramatic advantage in the suppression of 1/f noise. In addition, a circuit structure based on single‐slope analog‐to‐digital converter (SSADC) for implementing NOCMS is suggested. Ramp generator provides multiple sets of ramps with different slopes to quantify the reset and signal voltages of pixel output. Sampling weights are increased with the decrease of ramp slopes. The last reset and first signal values are weighted more due to their potentially higher correlations which highlights the principle of assigning weights. Simulation results under 110 nm CMOS technology illustrate that the input‐referred random noise is 142.9 V under standard CMS and 120.9 V under NOCMS when the number of samples equals 8. The noise reduction effect is improved by 15%. NOCMS makes it possible to further reduce 1/f noise of CMOS image sensor.

An improved evanescent fluorescence scanner suitable for high-resolution glycome mapping of formalin-fixed paraffin-embedded tissue sections.

Lectin microarray (LMA) is a high-throughput platform that enables the rapid and sensitive analysis of N- and O-glycans attached to glycoproteins in biological samples, including formalin-fixed paraffin-embedded (FFPE) tissue sections. Here, we evaluated the sensitivity of the advanced scanner based on the evanescent-field fluorescence principle, which is equipped with a 1× infinity correction optical system and a high-end complementary metal-oxide semiconductor (CMOS) image sensor in digital binning mode. Using various glycoprotein samples, we estimated that the mGSR1200-CMOS scanner has at least fourfold higher sensitivity for the lower limit of linearity range than that of a previous charge-coupled device scanner (mGSR1200). A subsequent sensitivity test using HEK293T cell lysates demonstrated that cell glycomic profiling could be performed with only three cells, which has the potential for the glycomic profiling of cell subpopulations. Thus, we examined its application in tissue glycome mapping, as indicated in the online LM-GlycomeAtlas database. To achieve fine glycome mapping, we refined the laser microdissection-assisted LMA procedure to analyze FFPE tissue sections. In this protocol, it was sufficient to collect 0.1 mm2 of each of the tissue fragments from 5-μm-thick sections, which differentiated the glycomic profile between the glomerulus and renal tubules of a normal mouse kidney. In conclusion, the improved LMA enables high-resolution spatial analysis, which expands the possibilities of its application classifying cell subpopulations in clinical FFPE tissue specimens. This will be used in the discovery phase for the development of novel glyco-biomarkers and therapeutic targets, and to expand the range of target diseases.

An Area-Efficient up/down Double-Sampling Circuit for a LOFIC CMOS Image Sensor.

A lateral overflow integration capacitor (LOFIC) complementary metal oxide semiconductor (CMOS) image sensor can realize high-dynamic-range (HDR) imaging with combination of a low-conversion-gain (LCG) signal for large maximum signal electrons and a high-conversion-gain (HCG) signal for electron-referred noise floor. However, LOFIC-CMOS image sensor requires a two-channel read-out chain for LCG and HCG signals whose polarities are inverted. In order to provide an area-efficient LOFIC-CMOS image sensor, a one-channel read-out chain that can process both HCG and LCG signals is presented in this paper. An up/down double-sampling circuit composed of an inverting amplifier for HCG signals and a non-inverting attenuator for LCG signals can reduce the area of the read-out chain by half compared to the conventional two-channel read-out chain. A test chip is fabricated in a 0.18 μm CMOS process with a metal-insulator-metal (MIM) capacitor, achieving a readout noise of 130 μVrms for the HCG signal and 1.19 V for the LCG input window. The performance is equivalent to 103 dB of the dynamic range with our previous LOFIC pixel in which HCG and LCG conversion gains are, respectively, 160 μV/e- and 10 μV/e-.

Watermark Detection in CMOS Image Sensors Using Cosine-Convolutional Semantic Networks

This article proposes a deep learning approach for automatic segmentation of low-contrast watermark defects on Complementary Metal Oxide Semiconductor (CMOS) Image Sensor (CIS) surfaces. The task of surface defect detection has been tackled by machine vision or deep learning methods. Traditional machine vision methods require expert domain knowledge for feature extraction, and they are computationally intensive. Deep learning methods based on Convolutional Neural Network (CNN) are extremely fast, but can be affected by illumination variations and perform poorly for low-contrast images. In this study, a U-Net type of encoder-decoder architecture is proposed. The Convolutional operation is substituted with the proposed Cosine-Convolutional operation in the encoder layers of the U-Net model with the objective of minimizing the effects of uneven illumination and highlighting the defect region. To further reduce the computation complexity and improve defect segmentation, an attention-like mechanism is used in the semantic encoder-decoder network. Pixel-wise multiplication is applied in the skip connections between encoder and decoder layers of the U-Net to replace the classical concatenation, thus reducing the parameter count of the model. The proposed methods are computationally fast, with an evaluation time of less than 10-ms for an image of 384×256 pixels on a single low-end GPU. Experimental results show that the proposed Cosine-Convolutional encoder-decoder architectures can effectively and accurately be applied for defect detection and segmentation of watermarks in low-contrast CIS images.

Variable Angular Rate Measurement for a Spacecraft Based on the Rolling Shutter Mode of a Star Tracker

Angular rate is a piece of useful information for the attitude control of a spacecraft. The star tracker as a space optical sensor can be used to measure the angular rate of a spacecraft. In this paper, a novel approach is proposed to improve the measurement accuracy of the angular rate during spacecraft rotation. The electronic rolling shutter (RS) imaging mode of the complementary metal-oxide semiconductor (CMOS) image sensor in a star tracker is applied to obtain much higher sampling frequency for reducing the change of the angular rate between the sampling interval. The optic flow vector on the imaging plane is approximated within the second order using three successive star images to reflect the nonlinear effect from the variable angular rate. The experiment is performed to demonstrate the advantage of the new approach for variable angular rate measurement.

Recent Advances in Artificial Intelligence Sensors

AbstractSignificant growth in the development and deployment of artificial intelligence (AI) is being witnessed. Driven by the great versatility of emerging computer science and material science, various AI sensors provide cost‐effective approaches for a wide range of monitoring applications toward the realization of smart homes and personal healthcare. Advanced AI sensors have multiple sensors capable of detecting multidimensional information and human‐brain‐like computation device for data processing. Herein, this review outlines the recent advances in the development of AI sensors. This review first introduces the materials, fabrication methods, and algorithms of current AI sensors and their applications, i.e., complementary metal oxide semiconductor image sensors for computer vision, microelectromechanical systems, microphone sensors for voice recognition, and wearable sensors for gesture recognition. Then, the recent advances in AI wearables sensors and self‐powered sensor systems are highlighted. Next, the current developments of neuromorphic computing systems, multimodality, and digital twins are reviewed. Last, a perspective on future directions for further research development is also provided. In summary, the trend of advanced AI sensors is the complementary between edge computing and cloud computing, which will show great potential in the applications of smart buildings, individual healthcare, the Internet of things, etc.

4T Complementary Metal Oxide Semiconductor Image Sensor Charge Transfer Efficiency Optimization

4T CMOS image sensors are widely used in various imaging fields by virtue of low noise, high integration and low cost, but their pixel cells are manufactured with non-ideal effects causing trap energy levels and potential deviations, resulting in incomplete charge transfer of photogenerated signals, thus producing image trailing and affecting the imaging effect. Traditionally, the improvement of charge transfer efficiency is only limited to the optimization of one or two parameters or the optimization of the working state. In this paper, we propose a more systematic research method to optimize the charge transfer efficiency through five aspects: TG channel threshold voltage injection adjustment, PPD N-type impurity injection dose and injection angle, as well as TG operating voltage and FD reset voltage, respectively. The optimal process state and operating conditions are obtained: TG channel injection dose of 8.0 e12 cm−2, PPD N-type impurity injection dose of 3.0 e12 cm−2, injection angle of −4°, TG operating voltage of 2.7 V and FD reset voltage of 3.9 V, where the residual charge in the signal transfer path is minimal, and the design method in this paper has some guidance for the design of CIS pixels.

Single microhole per pixel for thin Ge-on-Si complementary metal-oxide semiconductor image sensor with enhanced sensitivity up to 1700 nm

We present a germanium “Ge-on-Si” CMOS image sensor with backside illumination for the near-infrared (NIR) electromagnetic waves (wavelength range 300 to 1700 nm) detection essential for optical sensor technology. The microholes help to enhance the optical efficiency and extend the range to the 1.7-μm wavelength. We demonstrate an optimization for the width and depth of the microholes for maximal absorption in the NIR. We show a reduction in the crosstalk by employing thin SiO2 deep trench isolation in between the pixels. Finally, we show a 26 to 50% reduction in the device capacitance with the introduction of a microhole. Such CMOS-compatible Ge-on-Si sensors will enable high-density, ultrafast, and efficient NIR imaging.

A gamma radiation scene image clarification method based on spatiotemporal combination

For the problems of complex noise and color cast in the images captured by complementary metal oxide semiconductor image sensors in the gamma radiation environment, this paper proposed a joint temporal correlation denoising and color cast correction approach for the gamma radiation scene image clarification. Concretely, we first adaptively stretch the difference between noise and background to improve the salience of weak noise, making noise detection results more accurate. Secondly, the temporal transient characteristics of gamma radiation noise are utilized for noise detection, and these noisy pixels are repaired by exploiting the temporal correlation characteristics of background pixels. Lastly, the Bayer-Guided auto white balance method is proposed to solve the problem of dysregulated color in the gamma radiation scene image, and the color level remapping approach is applied to eliminate the problem of insufficient dynamic range. Extensive experiments are carried out on the images captured from the Co60 gamma radiation scene. Compared with the source images, our method improves the peak signal to noise ratio by 6.3(dB) and structural similarity by 0.34. Experimental results demonstrate that the proposed method enjoys state-of-the-art performance in improving the clarity of gamma radiation scene images.

Human motion recognition based on Nano-CMOS Image sensor.

Human motion recognition is of great value in the fields of intelligent monitoring systems, driver assistance system, advanced human-computer interaction, human motion analysis, image and video processing. However, the current human motion recognition methods have the problem of poor recognition effect. Therefore, we propose a human motion recognition method based on Nano complementary metal oxide semiconductor (CMOS) image sensor. First, using the Nano-CMOS image sensor to transform and process the human motion image, and combines the background mixed model of pixels in the human motion image to extract the human motion features, and feature selection is conducted. Second, according to the three-dimensional scanning features of Nano-CMOS image sensor, the human joint coordinate information data is collected, the state variables of human motion are sensed by the sensor, and the human motion model is constructed according to the measurement matrix of human motions. Finally, the foreground features of human motion images are obtained by calculating the feature parameters of each motion gesture. According to the posterior conditional probability of human motion images, the recognition objective function of human motion is obtained to realize human motion recognition. The results show that the human motion recognition effect of the proposed method is good, the extraction accuracy is high, the average human motion recognition rate is 92%, the classification accuracy is high, and the recognition speed is up to 186 frames/s.

Fast and Easy Sensor Adaptation With Self-Training

Object detectors based on deep neural networks have the disadvantage that new labels should be acquired whenever the complementary metal-oxide semiconductor (CMOS) image sensor (CIS) is changed. In this study, we propose a fast and easy two-step sensor-adaptation method without labels for the target domain; 1) simple adaptation, and 2) self-training. The simple-adaptation process transfers the knowledge of the source model to the target model by updating the batch normalization parameters, and matches the feature distributions of the source domain and those of target domain. In the self-training process, we employ the ensemble model strategy to mitigate the over-fitting problem using noisy pseudo labels generated by the simple-adaptation model. Quantitative and qualitative experiments show that the proposed method can transfer the knowledge from one CIS model to another, even if the data format of the target domain is different from that of the source CIS domain.

Laser De-Bayering of Low-Cost Image Sensors

Building upon prior work, a novel laser stripping method for removing the Bayer layer from an inexpensive back-illuminated complementary metal–oxide semiconductor image sensor (Raspberry Pi camera) is disclosed. The laser stripping process provides an attractive alternative to chemical stripping, negating the requirement for the processing of such sensors with harmful and environmentally damaging chemicals. Additionally, the method disclosed, though limited in speed, lays the groundwork toward a potentially fast and scalable process. It is demonstrated that such sensors, when stripped of the Bayer layer, exhibit enhanced spectral resolution and range from the ultraviolet to the infrared. Such inexpensive camera sensors, once modified, may be suitable for a variety of scientific applications where a high-resolution, broad spectral response is required, for example, in compact optical spectrometers.

A CMOS Double-Demodulation Lock-In Amplifier for Stimulated Raman Scattering Signal Detection

In typical stimulated Raman scattering (SRS) signal extraction, the photodetector and lock-in amplifier are often based on separate platforms, rendering the system cumbersome and non-scalable. This paper proposes an SRS double-demodulation lock-in amplifier implemented with a complementary metal-oxide semiconductor (CMOS) image sensor technology that integrates two-stage 1/f noise and offset reduction circuits with a high-speed lateral electric field modulation (LEFM) photo-demodulator. A weak SRS signal is buried in a large offset with a ratio of 10−4 to 10−6; boosting such signals in a CMOS device requires an extremely high offset and noise reduction capability. The double-modulation two-stage lock-in amplifier demodulates at 40 MHz with a sampling frequency of 20 MHz, can suppress the laser and circuit’s 1/f noise to achieve higher detection sensitivity. A prototype chip fabricated using 0.11 μm CMOS image sensor technology is evaluated. Both simulation and measurement results are presented to verify the functionality and show that the differential readout structure can successfully reject laser common mode components while emphasizing its differences. The measurement results show that the double-modulation lock-in amplifier effectively suppresses the circuit’s 1/f noise by a factor of nearly two decades.

Investigation of the Hue–Wavelength Response of a CMOS RGB-Based Image Sensor

In this study, a non-linear hue-wavelength (H-W) curve was investigated from 400 to 650 nm. To date, no study has reported on H-W relationship measurements, especially down to the 400 nm region. A digital camera mounted with complementary metal oxide semiconductor (CMOS) image sensors was used. The obtained digital images of the sample were based on an RGB-based imaging analysis rather than multispectral imaging or hyperspectral imaging. In this study, we focused on the raw image to reconstruct the H-W curve. In addition, several factors affecting the digital image, such as exposure time or international organization for standardization (ISO), were investigated. In addition, cross check of the H-W response using laser was performed. We expect that our method will be useful as an auxiliary method in the future for obtaining the fluor emission wavelength information.