CMOS Image Research Articles

Characterization of a CMOS camera based film digitization platform for gated x-ray imaging diagnostics at the National Ignition Facility.

Hardened gated x-ray detectors use photographic film as the data recording medium due to its low sensitivity to the high-yield neutron environments at the National Ignition Facility (NIF). The photographic film is digitized with a Photometric Data Systems (PDS) microdensitometer, which measures the film's optical density. The PDS scanner is able to measure a dynamic range of 0-5 OD; however, raster scanning the film is time consuming and maintenance of the instrument is challenging due to legacy technology. Since film usage at NIF is expected to continue in the foreseeable future, a digitization platform that is faster and more maintainable would benefit the NIF's current and future operations. This work presents the characterization of the digital transitions (DT) atom, a CMOS camera-based digitization platform that records film data in a single image capture very quickly and has widely available user support. The preliminary results suggest that the DT atom is able to reconstruct exposures accurately enough to be a competitive alternative to the PDS Scanner.

Fisheye Night Sky Imager: A Calibrated Tool to Measure Night Sky Brightness

The Night Skies Team of the U.S. National Park Service (NPS) has developed a new camera system equipped with a fisheye lens to measure night sky brightness. This NPS Fisheye Night Sky Imager comprises a back-illuminated Sony IMX455 CMOS sensor housed in the ZWO ASI6200MM camera, a Johnson V filter, and a Sigma 8 mm F3.5 fisheye lens. All of the components are commercially available. The fisheye lens allows us to capture the entire sky in a single frame. The wide-angle view also introduces calibration challenges. To address this, we have customized tools and devised innovative methods for calibrating the system, including flat-field correction, pixel scale determination, and field of view characterization. Additionally, we use standard stars for precise positional and photometric calibration. Furthermore, we create an open-source Python pipeline to process these fisheye images and leverage Git and GitHub for source code version control and distribution. The resulting images provide a positionally and brightness-calibrated fisheye view of the night sky, with a photometric calibration uncertainty of 0.12 mag. When comparing our Fisheye Night Sky Imager measurements to images taken with other calibrated systems, we reliably achieve consistent results under both dark and bright skies. Our Fisheye Night Sky Imager not only enables rapid assessment of sky quality but also supports the feasibility of establishing permanent monitoring locations. Recognizing that the natural dark sky is a critical element for natural, cultural, educational, and visitor experiences, this new camera system enhances our ability to assess resource conditions and fulfill the NPS mission of conserving resources unimpaired for the enjoyment of this and future generations.

A real-time and energy-efficient SRAM with mixed-signal in-memory computing near CMOS sensors

In-memory computing (IMC) represents a promising approach to reducing latency and enhancing the energy efficiency of operations required for calculating convolution products of images. This study proposes a fully differential current-mode architecture for computing image convolutions across all four quadrants, intended for deep learning applications within CMOS imagers utilizing IMC near the CMOS sensor. This architecture processes analog signals provided by a CMOS sensor without the need for analog-to-digital conversion. Furthermore, it eliminates the necessity for data transfer between memory and analog operators as convolutions are computed within modified SRAM memory. The paper suggests modifying the structure of a CMOS SRAM cell by incorporating transistors capable of performing multiplications between binary (−1 or +1) weights and analog signals. Modified SRAM cells can be interconnected to sum the multiplication results obtained from individual cells. This approach facilitates connecting current inputs to different SRAM cells, offering highly scalable and parallelized calculations. For this study, a configurable module comprising nine modified SRAM cells with peripheral circuitry has been designed to calculate the convolution product on each pixel of an image using a 3×3\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$3 \ imes 3$$\\end{document} mask with binary values (−1 or 1). Subsequently, an IMC module has been designed to perform 16 convolution operations in parallel, with input currents shared among the 16 modules. This configuration enables the computation of 16 convolutions simultaneously, processing a column per cycle. A digital control circuit manages both the readout or memorization of digital weights, as well as the multiply and add operations in real-time. The architecture underwent testing by performing convolutions between binary masks of 3 × 3 values and images of 32 × 32 pixels to assess accuracy and scalability when two IMC modules are vertically integrated. Convolution weights are stored locally as 1-bit digital values. The circuit was synthesized in 180 nm CMOS technology, and simulation results indicate its capability to perform a complete convolution in 3.2 ms, achieving an efficiency of 11,522 1-b TOPS/W (1-b tera-operations per second per watt) with a similarity to ideal processing of 96%.

Portable single-shot lens-free tomographic microscope for imaging dynamic specimens

We present a portable single-shot lens-free tomographic microscope, based on spatial multiplexing in-line digital holography, for three-dimensional (3D) imaging of dynamic specimens. The simplified system is realized by only a laser diodes array, a rectangular aperture, and a CMOS image sensor with a global shutter, which enables the recording of all the complex fields of a dynamic specimen from different illumination angles without any scanning mechanism by a multiplexing hologram, in a single camera exposure. Using our proposed data processing method, high-quality 3D tomograms, with a lateral resolution of 3.36μm and an axial resolution of 6.39μm within an imaging field-of-view of 5.06 mm2, can be eventually reconstructed from a multiplexing hologram. Experimental results of free-swimming paramecium in water demonstrates the systems potential in 3D imaging living or rapidly moving specimens in resource-limited environments.

A versatile and fast pixel matrix read-out architecture for MAPS

A digital asynchronous logic is proposed as a generic matrix readout for Monolithic active pixel sensors. The architecture is implemented for pixel pitch ranging from 18 to 30µm. Post-layout simulations with realistic hit shapes and rates up to 200MHz/cm2 show that time stamping at the 20ns level can be achieved for a digital power cost below 10mW/cm2.

Applications of LIF to Document Natural Variability of Chlorophyll Content and Cu Uptake in Moss.

Chlorophyll has long been used as a natural indicator of plant health and photosynthetic efficiency. Laser-induced fluorescence (LIF) is an emerging technique for understanding broad spectrum organic processes and has more recently been used to monitor chlorophyll response in plants. Previous work has focused on developing a LIF technique for imaging moss mats to identify metal contamination with the current focus shifting toward application to moss fronds and aiding sample collection for chemical analysis. Two laser systems (CoCoBi a Nd:YGa pulsed laser system and Chl-SL with two blue continuous semiconductor diodes) were used to collect images of moss fronds exposed to increasing levels of Cu (1, 10, and 100 nmol/cm2) using a CMOS camera. The best methods for the preprocessing of images were conducted before the analysis of fluorescence signatures were compared to a control. The Chl-SL system performed better than the CoCoBi, with dynamic time warping (DTW) proving the most effective for image analysis. Manual thresholding to remove lower decimal code values improved the data distributions and proved whether using one or two fronds in an image was more advantageous. A higher DTW difference from the control correlated to lower chlorophyll a/b ratios and a higher metal content, indicating that LIF, with the aid of image processing, can be an effective technique for identifying Cu contamination shortly after an event.

Local and Global Context-Enhanced Lightweight CenterNet for PCB Surface Defect Detection.

Printed circuit board (PCB) surface defect detection is an essential part of the PCB manufacturing process. Currently, advanced CCD or CMOS sensors can capture high-resolution PCB images. However, the existing computer vision approaches for PCB surface defect detection require high computing effort, leading to insufficient efficiency. To this end, this article proposes a local and global context-enhanced lightweight CenterNet (LGCL-CenterNet) to detect PCB surface defects in real time. Specifically, we propose a two-branch lightweight vision transformer module with local and global attention, named LGT, as a complement to extract high-dimension features and leverage context-aware local enhancement after the backbone network. In the local branch, we utilize coordinate attention to aggregate more powerful features of PCB defects with different shapes. In the global branch, Bi-Level Routing Attention with pooling is used to capture long-distance pixel interactions with limited computational cost. Furthermore, a Path Aggregation Network (PANet) feature fusion structure is incorporated to mitigate the loss of shallow features caused by the increase in model depth. Then, we design a lightweight prediction head by using depthwise separable convolutions, which further compresses the computational complexity and parameters while maintaining the detection capability of the model. In the experiment, the LGCL-CenterNet increased the mAP@0.5 by 2% and 1.4%, respectively, in comparison to CenterNet-ResNet18 and YOLOv8s. Meanwhile, our approach requires fewer model parameters (0.542M) than existing techniques. The results show that the proposed method improves both detection accuracy and inference speed and indicate that the LGCL-CenterNet has better real-time performance and robustness.

Research on precise identification and localization methods for static small targets based on multimodal data fusion

This investigation introduces a multimodal fusion-based methodology for the precise detection and accurate localization of static small targets against uniform backgrounds. Small targets often present fewer pixels in imagery, which significantly reduces the detectable features and complicates accurate identification and localization. Our approach integrates visual data from CMOS cameras, distance readings from laser rangefinders, and orientation data from angle sensors. The image processing involves non-local means denoising and Canny edge detection to enhance small target detection. We map two-dimensional images to three-dimensional spaces, combining depth and orientation to precisely determine the target’s coordinates, aiding in three-dimensional localization. Empirical results confirm the method’s high accuracy, with a localization error under 2.5%. Such conclusive evidence highlights the approach’s efficacy, presenting a cost-efficient alternative for the nuanced detection and localization of small entities.

Soft X-ray performance of an optimised charge-coupled device for astronomy

For the upcoming European Space agency (ESA)/Chinese Academy of Sciences (CAS) SMILE mission, launching in 2025, large-format soft X-ray optimised CCDs have been manufactured by Teledyne e2v, named the CCD370. The CCDs are approximately 8 cm × 8 cm and are comprised of 4510 × 4510 18 μm pitch pixels, with a store shield covering approximately 1/7th of the active imaging area to facilitate frame-transfer operation mode. To optimise quantum efficiency within the soft X-ray energy band, the device is 16 μm thick, back illuminated, and has an additional back surface passivation process. The focal plane of the soft X-ray imager (SXI) on the SMILE spacecraft will be comprised of 2 CCD370s, operating with 6 × 6 on-chip binning to mitigate against CTI-induced charge transfer losses and charge spreading throughout the 3-year mission lifetime.As part of the pre-flight testing and calibration, a CCD370 was characterised at the PTB beamline at the BESSY 2 synchrotron in Berlin, and key metrics such as quantum efficiency and energy resolution in the 0.2–1.8 keV energy band were assessed. The quantum efficiency measurements show expected performance within specification for the instrument, and also match a transmission-layer QE model. The energy resolution is 68 ± 2 eV FWHM @ 1.2 keV, which is an improvement over the current generation of X-ray telescopes such as XMM-Newton, Chandra, and the Swift X-ray Telescope. Although competing technologies such as DEPFETs and scientific CMOS image sensors now have similar performance to CCDs, the performance shown here can still easily satisfy requirements for novel X-ray instruments, and can still be useful in future astronomy missions given the rich heritage, high technology-readiness-level, and maturity of the technology.

Snapshot spectropolarimeter based on a six-fold separating prism operating from 360 nm to 1 µm.

We describe a new type of spectropolarimeter in which light is separated to simultaneously measure six spectra carrying polarimetric information on a 2D CMOS camera. The polarization separation along one of the camera axes was obtained using a novel prism, and the spectral dispersion along the other camera axis was obtained using an imaging spectrometer. An ideal version of the six-fold separating prism is first described, in which polarimetric separation is performed along the canonical polarization states used to define the Stokes vector, and it can be explained without any math. The real version is then presented, with math that is simple for those familiar with polarization. The operation of the spectropolarimeter is described. Experimental results show that the polarimetric accuracy is a few 10-3, and noise (mainly due to shot noise) is in the same range for a single acquisition. The spectral resolution depends on the entrance pinhole width and can be as low as 2 nm. Several examples are presented that feature how informative snapshots, high spectral resolution, spectropolarimetric measurements can be. The anticipated applications of this spectropolarimeter and, more broadly, for this novel polarization-separating prism are discussed. Technical details, such as the calibration procedure, noise levels, and consistency checks, are presented as supplementary material.

Linear Photodiode Arrays For Time-Domain FLEET Velocimetry In Hypersonic Flows

Linear photodiode arrays are applied to Femtosecond Laser Electronic Excitation Tagging (FLEET) to determine the stream-wise velocity of flow at Mach 8 in the Sandia Hypersonic Wind Tunnel (HWT). Incident light emitted by nitrogen gas upon FLEET excitation is collected by intensifying optics that amplify and transmit the light to the photodiode elements as the gas advects downstream. Two photodiode array configurations are considered: one aligned with the flow direction and one that is perpendicular to it. Conventional FLEET using a CMOS image sensor is also performed for quantitative comparison. The configurations are tested for purely free-stream cases as well as the wake of a seven-degree half-angle cone model at various Reynolds number. The influence of the bi-exponential decay of the FLEET line and the transfer function imposed by the intensifying optics are identified as key considerations on the time-domain method accuracy. Overcoming these effects leads to promising application spaces due to the compact nature of photodiodes and low data overhead.

High count rate alpha particle spectroscopy by a digital optical camera: A comparison study on edge detection algorithms

Alpha particle spectroscopy is highly significant in technologies related to radioactivity, particularly in radiation safety and the study of natural radioactivity. On the other hand, tools capable of identifying and determining the energies of alpha particles are of great interest. Among these tools, Commercial Off-The-Shelf (COTS) instruments, which can be used for alpha particle detection, have gained significant importance due to their ease of use and low cost. In this study, alpha particle spectroscopy of a Ra-226 source has been performed using a commercial digital camera based-on a CMOS sensor. In order to determine energy spectrum of alpha particles emitted from the source, the performance of various edge detection algorithms was examined. An algorithm based-on local pixel changes was introduced and implemented. By comparing the run times of the algorithms, it was shown that the algorithm proposed in this article exhibited the shortest run-time. Additionally, a method for rejecting pile-up events to correct their destructive effects on the alpha particle energy spectrum was introduced and implemented.

Unique algorithm for the evaluation of embryo photon emission and viability

Living cells have spontaneous ultraweak photon emission derived from metabolic reactions associated with physiological conditions. The ORCA-Quest CMOS camera (Hamamatsu Photonics, Japan) is a highly sensitive and essential tool for photon detection; its use with a microscope incubator (Olympus) enables the detection of photons emitted by embryos with the exclusion of harmful visible light. With the application of the second law of thermodynamics, the low-entropy energy absorbed and used by embryos can be distinguished from the higher-entropy energy released and detectable in their environment. To evaluate higher-entropy energy data from embryos, we developed a unique algorithm for the calculation of the entropy-weighted spectral fractal dimension, which demonstrates the self-similar structure of the energy (photons) released by embryos. Analyses based on this structure enabled the distinction of living and degenerated mouse embryos, and of frozen and fresh embryos and the background. This novel detection of ultra-weak photon emission from mouse embryos can provide the basis for the development of a photon emission embryo control system. The ultraweak photon emission fingerprints of embryos may be used for the selection of viable specimens in an ideal dark environment.

Inverse design of color routers in CMOS image sensors: toward minimizing interpixel crosstalk

Abstract Over the past decade, significant advancements in high-resolution imaging technology have been driven by the miniaturization of pixels within image sensors. However, this reduction in pixel size to submicrometer dimensions has led to decreased efficiency in color filters and microlens arrays. The development of color routers that operate at visible wavelengths presents a promising avenue for further miniaturization. Despite this, existing color routers often encounter severe interpixel crosstalk, around 70 %, due to the reliance on periodic boundary conditions. Here, we present interpixel crosstalk-minimized color routers that achieve an unprecedented in-pixel optical efficiency of 87.2 % and significantly reduce interpixel crosstalk to 2.6 %. The color routers are designed through adjoint optimization, incorporating customized incident waves to minimize interpixel crosstalks. Our findings suggest that our color router design surpasses existing color routing techniques in terms of in-pixel optical efficiency, representing a crucial step forward in the push toward commercializing the next generation of solid-state image sensors.

Broad spectral response of large array CIS with BSI-PD from visible light to near ultraviolet

The narrow spectral response range of traditional CMOS image sensors (CISs) limits their application such as weak-light and ultraviolet (UV) scenes. In the paper, a novel method of broadening spectral response range of traditional CISs is proposed by the design of a wide-PN photodiode (PD), which can be achieved by both back-illuminated (BSI) and gradient-doping structure using traditional process. Theoretical research shows that the wide-PN junction near illuminated surface can broaden the spectral range of the BSI-PD from visible light to near UV, which is conducive to weak-light imaging due to the generation of more photogenerated electrons. Moreover, the gradient potential in the BSI-PD with gradient-doping can inhibit the recombination of photogenerated electron-hole pairs, leading to the improvement of QE. The above BSI-PD is used for a large array CIS with 8520 × 9448 pixels, which was taped out by 110 nm CMOS process. The testing results indicate the large array CIS has a good imaging effect in different scenes.

Design, fabrication and testing of Al/p-Si Schottky and pn junctions for radiation studies

Strip and pixels sensors, fabricated on high resistivity silicon substrate, normally of p-type, are used in detectors for High Energy Physics (HEP) typically in a hybrid detector assembly. Furthermore, and owing to their inherent advantages over hybrid sensors, Monolithic Active Pixel Sensors (MAPS) fabricated in CMOS technology have been increasingly implemented in HEP experiments. In all cases, their use in higher radiation areas (HL-LHC and beyond) will require options to improve their radiation hardness and time resolution. These aspects demand a deep understanding of their radiation damage and reliable models to predict their behaviours at high fluences. As a first step, we fabricated several Schottky and n-on-p diodes, to allow a comparison of results and provide a backup solution for test devices, on 6 or 4-inch p-type silicon wafers with 50 μm epitaxial thickness and of doping concentration as they are normally used in HEP detectors and CMOS MAPS devices. In this paper, details of the design and fabrication process, along with test results of the fabricated devices before irradiation, will be provided. Additional test results on irradiated devices will be provided in subsequent publications.

Autofluorescence lifetime flow cytometry with time-correlated single photon counting.

Autofluorescence lifetime imaging microscopy (FLIM) is sensitive to metabolic changes in single cells based on changes in the protein-binding activities of the metabolic co-enzymes NAD(P)H. However, FLIM typically relies on time-correlated single-photon counting (TCSPC) detection electronics on laser-scanning microscopes, which are expensive, low-throughput, and require substantial post-processing time for cell segmentation and analysis. Here, we present a fluorescence lifetime-sensitive flow cytometer that offers the same TCSPC temporal resolution in a flow geometry, with low-cost single-photon excitation sources, a throughput of tens of cells per second, and real-time single-cell analysis. The system uses a 375 nm picosecond-pulsed diode laser operating at 50 MHz, alkali photomultiplier tubes, an FPGA-based time tagger, and can provide real-time phasor-based classification (i.e., gating) of flowing cells. A CMOS camera produces simultaneous brightfield images using far-red illumination. A second PMT provides two-color analysis. Cells are injected into the microfluidic channel using a syringe pump at 2-5 mm/s with nearly 5 ms integration time per cell, resulting in a light dose of 2.65 J/cm2 that is well below damage thresholds (25 J/cm2 at 375 nm). Our results show that cells remain viable after measurement, and the system is sensitive to autofluorescence lifetime changes in Jurkat T cells with metabolic perturbation (sodium cyanide), quiescent versus activated (CD3/CD28/CD2) primary human T cells, and quiescent versus activated primary adult mouse neural stem cells, consistent with prior studies using multiphoton FLIM. This TCSPC-based autofluorescence lifetime flow cytometer provides a valuable label-free method for real-time analysis of single-cell function and metabolism with higher throughput than laser-scanning microscopy systems.

Millimeter-Wave Band Electro-Optical Imaging System Using Polarization CMOS Image Sensor and Amplified Optical Local Oscillator Source.

In this study, we developed and demonstrated a millimeter-wave electric field imaging system using an electro-optic crystal and a highly sensitive polarization measurement technique using a polarization image sensor, which was fabricated using a 0.35-µm standard CMOS process. The polarization image sensor was equipped with differential amplifiers that amplified the difference between the 0° and 90° pixels. With the amplifier, the signal-to-noise ratio at low incident light levels was improved. Also, an optical modulator and a semiconductor optical amplifier were used to generate an optical local oscillator (LO) signal with a high modulation accuracy and sufficient optical intensity. By combining the amplified LO signal and a highly sensitive polarization imaging system, we successfully performed millimeter-wave electric field imaging with a spatial resolution of 30×60 µm at a rate of 1 FPS, corresponding to 2400 pixels/s.

Performance study of the JadePix-3 telescope from a beam test

We present the results of a beam test conducted on a telescope using the JadePix-3 pixel sensor, developed with TowerJazz 180 nm CMOS imaging technology. The telescope is composed of five planes, each equipped with a JadePix-3 sensor with pitches of 26 × 16 µm2 and 23.11×16 µm2. In addition, it features an FPGA-based synchronous readout system. The telescope underwent testing using an electron beam with energy ranging from 4 to 6GeV. At an electron energy of 5.4GeV, the telescope demonstrated superior spatial resolutions of 2.6 and 2.3µm in two dimensions. By designating the central plane as the device under test, we evaluated the JadePix-3 sensor’s spatial resolutions as 5.2 and 4.6µm in two dimensions, achieving a detection efficiency of over 99.0%.

Terahertz-based optoelectronic properties of ZnS quantum dot-polymer composites: For device applications

Terahertz (THz) technology integration with nanomaterials is receiving excellent attention for next-generation applications, including enhanced imaging and communication. The excellent optical properties in THz domain can lead to preparation of low-cost CMOS camera which can convert THz radiation into optical signal in very efficient manner. In the present study, we have studied the properties of Zinc Sulfide quantum dots (ZnS QDs) embedded with Polyvinyl Alcohol (PVA) composites films using THz Signal at room temperature. The optical characterizations such as refractive index, absorption coefficients and dielectric constants of these samples were measured in the 0.1–2.0 THz range. Additionally, optical impedance, surface roughness, and reflection coefficient in TE and TM mode between 0.1 and 2.0 THz range were determined for these samples based on surface roughness-based reflection and scattering properties. The surface roughness factor was used to measure the optical impedance of the ZnS QDs based polymer films. The measured values of the absorption coefficient at 266 nm are compared with THz radiation, and the refractive indices of these samples range from 1.75 to 2.0. Finally, these samples were subjected to UV light excitation (λexe = 266 nm) of 0.15 ns duration and 400 nm for the fluorescence and corresponding life time measurements. We observed two numbers of fluorescence lines in nanosecond based excited domain whereas 400 nm excitation-based fluorescence life time lies between 13.8–11.39 ns range along with shift in fluorescence lines between 538.7 to 560.7 nm, respectively.