Complementary Metal Oxide Semiconductor Image Research Articles

Advancements in CMOS image sensors for enhanced vehicle assistance technologies

The burgeoning adoption of new energy vehicles has precipitated a surge in advanced vehicular assistant technologies, encompassing systems such as assisted driving and parking. The efficacy of these technologies hinges critically on the selection of suitable image sensors, which are integral to fulfilling the stringent image quality requisites necessary for the implementation of these technologies. Predominantly, these assistant systems are reliant on sophisticated image processing techniques, necessitating high-quality images as a fundamental input for efficacious analysis and recognition. Currently, the Complementary Metal-Oxide-Semiconductor (CMOS) image sensor is predominantly employed in vehicular systems, a choice underpinned by its advantageous attributes such as resistance to interference, cost-effectiveness, and high degree of integration. This paper aims to elucidate the developmental trajectory and fundamental principles of CMOS technology and its application in image sensors. It delves into an analysis of the salient features of CMOS image sensors, especially in their application to parking assistance, autonomous driving support, and fatigue detection systems. Conclusively, the paper offers an exhaustive overview of the current state of CMOS image sensor development, alongside a projection of future trends and an exploration of the challenges that must be surmounted to advance this field further.

Miniature computational spectrometer with a plasmonic nanoparticles-in-cavity microfilter array

Optical spectrometers are essential tools for analysing light‒matter interactions, but conventional spectrometers can be complicated and bulky. Recently, efforts have been made to develop miniaturized spectrometers. However, it is challenging to overcome the trade-off between miniaturizing size and retaining performance. Here, we present a complementary metal oxide semiconductor image sensor-based miniature computational spectrometer using a plasmonic nanoparticles-in-cavity microfilter array. Size-controlled silver nanoparticles are directly printed into cavity-length-varying Fabry‒Pérot microcavities, which leverage strong coupling between the localized surface plasmon resonance of the silver nanoparticles and the Fabry‒Pérot microcavity to regulate the transmission spectra and realize large-scale arrayed spectrum-disparate microfilters. Supported by a machine learning-based training process, the miniature computational spectrometer uses artificial intelligence and was demonstrated to measure visible-light spectra at subnanometre resolution. The high scalability of the technological approaches shown here may facilitate the development of high-performance miniature optical spectrometers for extensive applications.

Reliability modelling and assessment of CMOS image sensor under radiation environment

The Complementary Metal-Oxide Semiconductor (CMOS) image sensor is a critical component with the function of providing accurate positioning in many space application systems. Under long-time operation in space environments, there are radiation related degradation and various uncertainties affecting the positioning accuracy of CMOS image sensors, which further leads to a reliability reduction of CMOS image sensors. Obviously, the reliability of CMOS image sensors is related to their specified function, degradation, and uncertainties; however, current research has not fully described this relationship. In this paper, a comprehensive approach to reliability modelling of CMOS image sensors is proposed based on the reliability science principles. Firstly, the performance margin modelling of centroid positioning accuracy is conducted. Then, the degradation model of CMOS image sensors is derived considering the dark current increase induced by the total ionizing dose effects. Finally, various uncertainties are analyzed and quantified, and the measurement equation of reliability is proposed. A case study of a CMOS image sensor is conducted to apply the proposed method, and the sensitivity analysis can provide suggestions for design and use of CMOS image sensors to ensure reliability. A simulation study is conducted to present the advantages of the proposed comprehensive approach.

Color arrestor pixels for high-fidelity, high-sensitivity imaging sensors

Abstract Silicon is the dominant material in complementary metal-oxide-semiconductor (CMOS) imaging devices because of its outstanding electrical and optical properties, well-established fabrication methods, and abundance in nature. However, with the ongoing trend toward electronic miniaturization, which demands smaller pixel sizes in CMOS image sensors, issues, such as crosstalk and reduced optical efficiency, have become critical. These problems stem from the intrinsic properties of Si, particularly its low absorption in the long wavelength range of the visible spectrum, which makes it difficult to devise effective solutions unless the material itself is changed. Recent advances in optical metasurfaces have offered new possibilities for solving these problems. In this study, we propose color arrestor pixels (CAPs) as a new class of color image sensors whose composite spectral responses directly mimic those of the human eye. The key idea is to employ linearly independent combinations of standardized color matching functions. These new basis functions allow our device to reproduce colors more accurately than the currently available image sensors with red-green-blue filters or other metasurface-based sensors, demonstrating an average CIEDE2000 color difference value of only 1.79 when evaluating 24 colors from the Gretag-Macbeth chart under standard illuminant D65. Owing to their high fidelity to the human eye response, CAPs consistently exhibit exceptional color reproduction accuracy under various spectral illumination compositions. With a small footprint of 860 nm height and 221 nm full-color pixel pitch, the CAPs demonstrated high absorption efficiencies of 79 %, 81 %, and 63 % at wavelengths of 452 nm, 544 nm, and 603 nm, respectively, and good angular tolerance. With such a high density of pixels efficiently capturing accurate colors, CAPs present a new direction for optical image sensor research and their applications.

Counting alpha particles produced by radon daughters using commercial off-the-shelf complementary metal oxide semiconductor (COTS CMOS) image sensors

Radon (Rn-222) is the main source of radiation exposure to human beings from natural radiation; it is of great significance to study the measuring methods and measuring instruments of radon for natural radiation protection. In recent years, the application of pixel detectors in radiation detection has attracted attention. In this paper, a Commercial Off-the-Shelf Complementary Metal Oxide Semiconductor (COTS CMOS) image sensor which replaces the glass for protection covering in front of the sensors was used to perform a series of measurements to identify alpha particles. During the experiment, the CMOS image sensor was used to record a video during the sampling period, and then use the FFmpeg software to take screenshots of the video by frame. MATLAB was used to count bright spots from the image frames. A measurement chamber was designed to measure radon concentrations, and when the relative humidity was constant, the count of alpha particles by the CMOS image sensor increased along with the increase of the concentration. The experiments verified the feasibility of the low-cost COTS CMOS image sensors to monitor radon.

Analysis of the Interference Effects in CMOS Image Sensors Caused by Strong Electromagnetic Pulses

With the electromagnetic environment becoming increasingly complex, it is crucial to address the risk posed by electromagnetic pulse, which critically impairs the performance and reliability of electronic systems based on complementary metal oxide semiconductor (CMOS) image sensors. In this context, research on the failure types of CMOS image sensors in a high-power electromagnetic environment, caused by strong electromagnetic pulses and the rapid evaluation method of interference immunity, has garnered significant interest. This paper conducts electromagnetic pulse simulation experiments on CMOS image sensors to first study their failure types, such as image abnormalities and functional interruption, and then identify the corresponding failure criteria. Furthermore, this study builds on the small sample test evaluation method to investigate the interference threshold of functional interruptions in CMOS image sensors by calculating the failure probability at different field strengths. The obtained data were combined with the Weibull distribution function for fitting, the results of which found the interference threshold to be at 40.4 kV/m. The findings of this study provide a basis for evaluating the survivability of CMOS image sensors and their associated reinforcement technology in high-power electromagnetic environments.

Motion-resistant three-wavelength spatial frequency domain imaging system with ambient light suppression using an 8-tap CMOS image sensor.

We present a motion-resistant three-wavelength spatial frequency domain imaging (SFDI) system with ambient light suppression using an 8-tap complementary metal-oxide semiconductor (CMOS) image sensor (CIS) developed at Shizuoka University. The system addresses limitations in conventional SFDI systems, enabling reliable measurements in challenging imaging scenarios that are closer to real-world conditions. Our study demonstrates a three-wavelength SFDI system based on an 8-tap CIS. We demonstrate and evaluate the system's capability of mitigating motion artifacts and ambient light bias through tissue phantom reflectance experiments and in vivo volar forearm experiments. We incorporated the Hilbert transform to reduce the required number of projected patterns per wavelength from three to two per spatial frequency. The 8-tap image sensor has eight charge storage diodes per pixel; therefore, simultaneous image acquisition of eight images based on multi-exposure is possible. Taking advantage of this feature, the sensor simultaneously acquires images for planar illumination, sinusoidal pattern projection at three wavelengths, and ambient light. The ambient light bias is eliminated by subtracting the ambient light image from the others. Motion artifacts are suppressed by reducing the exposure and projection time for each pattern while maintaining sufficient signal levels by repeating the exposure. The system is compared to a conventional SFDI system in tissue phantom experiments and then in vivo measurements of human volar forearms. The 8-tap image sensor-based SFDI system achieved an acquisition rate of 9.4 frame sets per second, with three repeated exposures during each accumulation period. The diffuse reflectance maps of three different tissue phantoms using the conventional SFDI system and the 8-tap image sensor-based SFDI system showed good agreement except for high scattering phantoms. For the in vivo volar forearm measurements, our system successfully measured total hemoglobin concentration, tissue oxygen saturation, and reduced scattering coefficient maps of the subject during motion (16.5cm/s) and under ambient light (28.9lx), exhibiting fewer motion artifacts compared with the conventional SFDI. We demonstrated the potential for motion-resistant three-wavelength SFDI system with ambient light suppression using an 8-tap CIS.

Hydrogen Termination Effect on SiO2/Si Interface State Density in CH3O-Molecular-Ion-Implanted Silicon Epitaxial Wafer for CMOS Image Sensors

The reduction in SiO2/Si interface state density (Dit) at the SiO2/Si interface region is important to improve the performance of complementary metal-oxide semiconductor (CMOS) image sensors. The CH3O-ion-implanted region stores hydrogen and releases the stored hydrogen during the subsequent heat treatment. This study demonstrates that a CH3O-ion-implanted epitaxial silicon wafer can reduce the Dit and Pb0 center density in SiO2/Si interface regions, as analyzed by quasi-static capacitance–voltage and electron spin resonance measurements, respectively. Both Dit and Pb0 center density in the CH3O-implanted wafer decreased with increasing heat treatment temperature. Moreover, the activation energy is estimated to be 1.57 eV for the hydrogen termination reactions induced by the CH3O-ion-implanted wafer. The activation energy is close to those of hydrogen molecules and Si dangling bonds at the SiO2/Si interface. This result means that Dit can be reduced by hydrogen from inside the silicon wafer, regardless of the heat treatment atmosphere. It has unique characteristics not found in conventional silicon wafers. The termination effect of the CH3O-molecular-ion-implanted epitaxial silicon wafers can contribute to the high electrical performance of CMOS image sensors.

High-speed readout system of X-ray CMOS image sensor for time domain astronomy

We developed an FPGA-based high-speed readout system for a complementary metal-oxide-semiconductor (CMOS) image sensor to observe soft X-ray transients in future satellite missions, such as HiZ-GUNDAM. Our previous research revealed that the CMOS image sensor has low-energy X-ray detection capability (0.4–4 keV) and strong radiation tolerance, which satisfies the requirements of the HiZ-GUNDAM mission. However, CMOS sensors typically have small pixel sizes (e.g., ∼10 µm), resulting in large volumes of image data. GSENSE400BSI has 2048×2048 pixels, producing 6 Mbyte per frame. These large volumes of observed raw image data cannot be stored in a satellite bus system with a limited storage size. Therefore, only X-ray photon events must be extracted from the raw image data. Furthermore, the readout time of CMOS image sensors is approximately ten times faster than that of typical X-ray CCDs, requiring faster event extraction on a timescale of ∼0.1 s. To address these issues, we have developed an FPGA-based image signal processing system capable of high-speed X-ray event extraction onboard without storing raw image data. The developed compact system enabled mounting on a CubeSat mission, facilitating early in-orbit operation demonstration. Here, we present the design and results of the performance evaluation tests of the proposed FPGA-based readout system. Utilizing X-ray irradiation experiments, the results of the X-ray event extraction with the onboard and offline processing methods were consistent, validating the functionality of the proposed system.

Analysis and mitigation of an oscillating background on hybrid complementary metal-oxide semiconductor (hCMOS) imaging sensors at the National Ignition Facility

Nanosecond-gated hybrid complementary metal-oxide semiconductor imaging sensors are a powerful tool for temporally gated and spatially resolved measurements in high energy density science, including inertial confinement fusion, and in laser diagnostics. However, a significant oscillating background excited by photocurrent has been observed in image sequences during testing and in experiments at the National Ignition Facility (NIF). Characterization measurements and simulation results are used to explain the oscillations as the convolution of the pixel-level sensor response with a sensor-wide RLC circuit ringing. Data correction techniques are discussed for NIF diagnostics, and for diagnostics where these techniques cannot be used, a proof-of-principle image correction algorithm is presented.

Development of a Charge-Multiplication CMOS Image Sensor Based on Capacitive Trench for Low-Light-Level Imaging

This paper presents an electron multiplication charge coupled device (EMCCD) based on capacitive deep trench isolation (CDTI) and developed using complementary metal oxide semiconductor (CMOS) technology. The CDTI transfer register offers a charge transfer inefficiency lower than 10−4 and a low dark current o 0.11nA/cm2 at room temperature. In this work, the timing diagram is adapted to use this CDTI transfer register in an electron multiplication mode. The results highlight some limitations of this device in such an EM configuration: for instance, an unexpected increase in the dark current is observed. A design modification is then proposed to overcome these limitations and rely on the addition of an electrode on the top of the register. Thus, this new device preserves the good transfer performance of the register while adding an electron multiplication function. Technology computer-aided design (TCAD) simulations in 2D and 3D are performed with this new design and reveal a very promising structure.

Investigating the Influence of Morphine and Cocaine on the Mesolimbic Pathway Using a Novel Microimaging Platform.

Dopamine (DA)'s relationship with addiction is complex, and the related pathways in the mesocorticolimbic system are used to deliver DA, regulating both behavioral and perceptual actions. Specifically, the mesolimbic pathway connecting the ventral tegmental area (VTA) and the nucleus accumbens (NAc) is crucial in regulating memory, emotion, motivation, and behavior due to its responsibility to modulate dopamine. To better investigate the relationship between DA and addiction, more advanced mapping methods are necessary to monitor its production and propagation accurately and efficiently. In this study, we incorporate dLight1.2 adeno-associated virus (AAV) into our latest CMOS (complementary metal-oxide semiconductor) imaging platform to investigate the effects of two pharmacological substances, morphine and cocaine, in the NAc using adult mice. By implanting our self-fabricated CMOS imaging device into the deep brain, fluorescence imaging of the NAc using the dLight1.2 AAV allows for the visualization of DA molecules delivered from the VTA in real time. Our results suggest that changes in extracellular DA can be observed with this adapted system, showing potential for new applications and methods for approaching addiction studies. Additionally, we can identify the unique characteristic trend of DA release for both morphine and cocaine, further validating the underlying biochemical mechanisms used to modulate dopaminergic activation.

Modeling Signal-to-Noise Ratio of CMOS Image Sensors with a Stochastic Approach under Non-Stationary Conditions.

A stochastic model for characterizing the conversion gain of Active Pixel Complementary metal-oxide-semiconductor (CMOS) image sensors (APS), assuming stationary conditions was recently presented in this journal. In this study, we extend the stochastic approach to non-stationary conditions. Non-stationary conditions occur in gated imaging applications. This new stochastic model, which is based on fundamental physical considerations, enlightens us with new insights into gated CMOS imaging, regardless of the sensor. The Signal-to-Noise Ratio (SNR) is simulated, allowing optimized performance. The conversion gain should be determined under stationary conditions.

CMOS Image Sensor for Broad Spectral Range with >90% Quantum Efficiency.

Even though the recent progress made in complementary metal-oxide-semiconductor (CMOS) image sensors (CIS) has enabled numerous applications affecting our daily lives, the technology still relies on conventional methods such as antireflective coatings and ion-implanted back-surface field to reduce optical and electrical losses resulting in limited device performance. In this work, these methods are replaced with nanostructured surfaces and atomic layer deposited surface passivation. The results show that such surface nanoengineering applied to a commercial backside illuminated CIS significantly extends its spectral range and enhances its photosensitivity as demonstrated by >90% quantum efficiency in the 300-700nm wavelength range. The surface nanoengineering also reduces the dark current by a factor of three. While the photoresponse uniformity of the sensor is seen to be slightly better, possible scattering from the nanostructures can lead to increased optical crosstalk between the pixels. The results demonstrate the vast potential of surface nanoengineering in improving the performance of CIS for a wide range of applications.

The development and application of CMOS image sensor

This paper focuses on the development of Complementary metal-oxide semiconductor (CMOS) image sensor and its applications in aerospace, medical and automotive fields. Firstly, the representative events in history and the contributions of some companies to CMOS image sensor are described. Subsequently, some characteristics of CMOS image sensor are analyzed in the image field involved. In order to evaluate the performance of CMOS image sensor, single even effect and electronic endoscope structures are analyzed and active and passive range finder experiments are carried out. The results show that the imaging based on CMOS sensor can fully meet the requirements of imaging applications in many fields.

A Novel Method for Developing Thin Resin Scintillator Screens and Application in an X-ray CMOS Imaging Sensor.

Scintillating screens for X-ray imaging applications are prepared with various methods. Among them, the classic sedimentation method presents certain weak points. In this context, a novel fabrication process was developed that offers simplicity, economy of resources and time, while the screens exhibit adequate durability and image quality performance. The proposed technique involves a resin mixture that contains the phosphor in powder form (Gd2O2S:Tb in the present work) and graphite. The novel method was optimized and validated by coupling the screens to a complementary metal oxide semiconductor (CMOS) X-ray sensor. Indicatively, screens of two surface densities were examined; 34 mg/cm2 and 70 mg/cm2. Various established image quality metrics were calculated following the IEC 62220-1 international standard, including the detective quantum efficiency (DQE). Comparisons were carried out under the same conditions, with a sedimentation screen reported previously and a screen of wide commercial circulation (Carestream Min-R 2190). The novel screens exhibit has comparable or even better performance in image-quality metrics. The 34 mg/cm2 screen achieves a DQE 15-20% greater than its comparison counterpart, and its limiting resolution was 5.3 cycles/mm. The detector coupled to the 70 mg/cm2 screen achieved a DQE 10-24% greater than its own counterpart, and its limiting resolution was found to be 5.4 cycles/mm.

A 1/f noise optimized correlated multiple sampling technique for complementary metal oxide semiconductor image sensor

SummaryCorrelated multiple sampling (CMS) technique is often used to reduce random noise in complementary metal oxide semiconductor (CMOS) image sensor, but as the number of samples increases, the suppression of 1/f noise by standard CMS based on averaging sampling gradually decreases. Therefore, in this paper, a 1/f noise optimized correlated multiple sampling (NOCMS) technique based on differentiated sampling weights is proposed. Transfer functions of standard CMS and NOCMS for analyzing the suppression effect of random noise respectively are derived based on the Fourier transform theory. And NOCMS shows a dramatic advantage in the suppression of 1/f noise. In addition, a circuit structure based on single‐slope analog‐to‐digital converter (SSADC) for implementing NOCMS is suggested. Ramp generator provides multiple sets of ramps with different slopes to quantify the reset and signal voltages of pixel output. Sampling weights are increased with the decrease of ramp slopes. The last reset and first signal values are weighted more due to their potentially higher correlations which highlights the principle of assigning weights. Simulation results under 110 nm CMOS technology illustrate that the input‐referred random noise is 142.9 V under standard CMS and 120.9 V under NOCMS when the number of samples equals 8. The noise reduction effect is improved by 15%. NOCMS makes it possible to further reduce 1/f noise of CMOS image sensor.

An improved evanescent fluorescence scanner suitable for high-resolution glycome mapping of formalin-fixed paraffin-embedded tissue sections.

Lectin microarray (LMA) is a high-throughput platform that enables the rapid and sensitive analysis of N- and O-glycans attached to glycoproteins in biological samples, including formalin-fixed paraffin-embedded (FFPE) tissue sections. Here, we evaluated the sensitivity of the advanced scanner based on the evanescent-field fluorescence principle, which is equipped with a 1× infinity correction optical system and a high-end complementary metal-oxide semiconductor (CMOS) image sensor in digital binning mode. Using various glycoprotein samples, we estimated that the mGSR1200-CMOS scanner has at least fourfold higher sensitivity for the lower limit of linearity range than that of a previous charge-coupled device scanner (mGSR1200). A subsequent sensitivity test using HEK293T cell lysates demonstrated that cell glycomic profiling could be performed with only three cells, which has the potential for the glycomic profiling of cell subpopulations. Thus, we examined its application in tissue glycome mapping, as indicated in the online LM-GlycomeAtlas database. To achieve fine glycome mapping, we refined the laser microdissection-assisted LMA procedure to analyze FFPE tissue sections. In this protocol, it was sufficient to collect 0.1 mm2 of each of the tissue fragments from 5-μm-thick sections, which differentiated the glycomic profile between the glomerulus and renal tubules of a normal mouse kidney. In conclusion, the improved LMA enables high-resolution spatial analysis, which expands the possibilities of its application classifying cell subpopulations in clinical FFPE tissue specimens. This will be used in the discovery phase for the development of novel glyco-biomarkers and therapeutic targets, and to expand the range of target diseases.

Unmanned-aerial-vehicle based optical camera communication system using light-diffusing fiber and rolling-shutter image-sensor.

We put forward and demonstrate a light-diffusing fiber equipped unmanned-aerial-vehicle (UAV) to provide a large field-of-view (FOV) optical camera communication (OCC) system. The light-diffusing fiber can act as a bendable, lightweight, extended and large FOV light source for the UAV-assisted optical wireless communication (OWC). During UAV flying, the light-diffusing fiber light source could be tilted or bended; hence, offering large FOV as well as supporting large receiver (Rx) tilting angle are particularly important for the UAV-assisted OWC systems. To improve the transmission capacity of the OCC system, one method based on the camera shutter mechanism, which is known as rolling-shuttering is utilized. The rolling-shuttering method makes use of the feature of complementary-metal-oxide-semiconductor (CMOS) image sensor to extract signal pixel-row by pixel-row. The data rate can be significantly increased since the capture start time for each pixel-row is different. As the light-diffusing fiber is thin and occupies only a few pixels in the CMOS image frame, Long-short-term-memory neural-network (LSTM-NN) is used to enhance the rolling-shutter decoding. Experimental results show that the light-diffusing fiber can satisfactorily act as an "omnidirectional optical antenna" providing wide FOVs and 3.6 kbit/s can be achieved, accomplishing the pre-forward error correction bit-error-rate (pre-FEC BER = 3.8 × 10-3).

An Area-Efficient up/down Double-Sampling Circuit for a LOFIC CMOS Image Sensor.

A lateral overflow integration capacitor (LOFIC) complementary metal oxide semiconductor (CMOS) image sensor can realize high-dynamic-range (HDR) imaging with combination of a low-conversion-gain (LCG) signal for large maximum signal electrons and a high-conversion-gain (HCG) signal for electron-referred noise floor. However, LOFIC-CMOS image sensor requires a two-channel read-out chain for LCG and HCG signals whose polarities are inverted. In order to provide an area-efficient LOFIC-CMOS image sensor, a one-channel read-out chain that can process both HCG and LCG signals is presented in this paper. An up/down double-sampling circuit composed of an inverting amplifier for HCG signals and a non-inverting attenuator for LCG signals can reduce the area of the read-out chain by half compared to the conventional two-channel read-out chain. A test chip is fabricated in a 0.18 μm CMOS process with a metal-insulator-metal (MIM) capacitor, achieving a readout noise of 130 μVrms for the HCG signal and 1.19 V for the LCG input window. The performance is equivalent to 103 dB of the dynamic range with our previous LOFIC pixel in which HCG and LCG conversion gains are, respectively, 160 μV/e- and 10 μV/e-.