High-speed Image Sensor Research Articles

High-speed bipolar phototransistors in a 180 nm CMOS process

Several high-speed pnp phototransistors built in a standard 180nm CMOS process are presented. The phototransistors were implemented in sizes of 40×40μm2 and 100×100μm2. Different base and emitter areas lead to different characteristics of the phototransistors. As starting material a p+ wafer with a p− epitaxial layer on top was used. The phototransistors were optically characterized at wavelengths of 410, 675 and 850nm. Bandwidths up to 92MHz and dynamic responsivities up to 2.95A/W were achieved. Evaluating the results, we can say that the presented phototransistors are well suited for high speed photosensitive optical applications where inherent amplification is needed. Further on, the standard silicon CMOS implementation opens the possibility for cheap integration of integrated optoelectronic circuits. Possible applications for the presented phototransistors are low cost high speed image sensors, opto-couplers, etc.

Construction of a 100[Hz] Sensing System of Textured Range Images and Its Application to Three Dimensional Mapping

In this paper, a compact high-speed range image sensor that can acquire a range image mapped with a color texture is constructed. The sensor consists of only commercially available products: a monochrome CCD camera, a color CCD camera, a laser projector, and two optical filters. The laser projector projects 19×19 multi-spot lights. Using the laser projector and the monochrome CCD camera with the optical filters, range images with 19×19, totally 361 points are measured. As fast as 100[Hz] measurement of a range image mapped with a color texture is achieved. Additionally, a three-dimensional (3D) mapping method using the constructed sensor is introduced. The performance of the constructed sensor and the 3D mapping method are evaluated by experiments. It is shown that the constructed sensing system can acquire 100[Hz] real-time textured range images and that it can be applied to 3D mapping by using range and color information.

Toward real-time particle tracking using an event-based dynamic vision sensor

Optically based measurements in high Reynolds number fluid flows often require high-speed imaging techniques. These cameras typically record data internally and thus are limited by the amount of onboard memory available. A novel camera technology for use in particle tracking velocimetry is presented in this paper. This technology consists of a dynamic vision sensor in which pixels operate in parallel, transmitting asynchronous events only when relative changes in intensity of approximately 10% are encountered with a temporal resolution of 1 μs. This results in a recording system whose data storage and bandwidth requirements are about 100 times smaller than a typical high-speed image sensor. Post-processing times of data collected from this sensor also increase to about 10 times faster than real time. We present a proof-of-concept study comparing this novel sensor with a high-speed CMOS camera capable of recording up to 2,000 fps at 1,024 × 1,024 pixels. Comparisons are made in the ability of each system to track dense (ρ >1 g/cm3) particles in a solid–liquid two-phase pipe flow. Reynolds numbers based on the bulk velocity and pipe diameter up to 100,000 are investigated.

Optical-Cavity-Based Multiwavelength Sensor for Spectral Discrimination and Object Position Detection

An optical-cavity-based multiwavelength sensor is developed for object discrimination and position finding. The working principle of this device employs the multiple laser beam triangulation method to determine object position in addition to its ability to recognize them. The multiwavelength sensor employs five different identically polarized and overlapped laser light beams that are sequentially pulsed and launched through a custom-made curved optical cavity to generate multiple laser spots for each laser. The intensities of the reflected light beams from the different spots are detected by a high-speed area scan image sensor. The discrimination between five different objects, namely, brick, cement sheet, roof tile, cotton, and leather is accomplished by calculating the slopes of the objects' reflectance spectra at the employed wavelengths. The object position (coordinates) are determined using the triangulation method, which is based on the projection of laser spots along determined angles on the objects and the measurements of the objects' reflectance spectra using an image sensor. Experimental results demonstrate the ability of the multiwavelength spectral reflectance sensor to simultaneously discriminate between different objects and predict their positions over a 6 m range with an accuracy exceeding 92%.

Universal bridge interface for DVP-compatible image sensors

In this paper, an efficient design of a digital-video-port (DVP) compatible universal bridge architecture to randomly access image pixels of a high speed image sensor is presented. Most commercially available CMOS image sensors send image data at high speed where the pixel values can only be accessed sequentially. Random access of pixels is not possible because the pixel data are sent row-by-row. On the other hand, commercial microcontrollers run at slower speed compared to the high data-rate of the image sensors and many embedded system applications need random access of pixel values. Moreover, commercial microcontrollers may not have sufficient internal memory to store a complete image of high resolution. The proposed bridge interface addresses these problems and provides a novel solution. By using the proposed bridge, the image processor or microcontroller can capture and store an image data of any resolution in the bridge’s internal memory, which is then accessed in a random fashion at any desired speed. This is particularly useful in image compression applications that use transform coding. The proposed design is modeled in VHDL and tested in FPGA to verify the functionality. The design is later implemented using CMOS 0.18 μm Artisan library cells. The synthesis results show that the bridge can support a data rate up to 254 MHz and suitable for rapid prototyping in different high-speed and low-power embedded system applications.

High speed column-parallel CDS/ADC circuit with nonlinearity compensation for CMOS image sensors

A high speed column-parallel CDS/ADC circuit with nonlinearity compensation is proposed in this paper. The correlated double sampling (CDS) and analog-to-digital converter (ADC) functions are integrated in a three-phase column-parallel circuit based on two floating gate inverters and switched-capacitor network. The conversion rate of traditional single-slope ADC is speeded up by dividing quantization to coarse step and fine step. A storage capacitor is used to store the result of coarse step and locate the section of ramp signal of fine step, which can reduce the clock step from 2 n to 2(n/2+1). The floating gate inverters are implemented to reduce the power consumption. Its induced nonlinear offset is cancelled by introducing a compensation module to the input of inverter, which can equalize the coupling path in three phases of the proposed circuit. This circuit is designed and simulated for CMOS image sensor with 640×480 pixel array using Chartered 0.18 μm process. Simulation results indicate that the resolution can reach 10-bit and the maximum frame rate can reach 200 frames/s with a main clock of 10 MHz. The power consumption of this circuit is less than 36.5 μW with a 3.3 V power supply. The proposed CDS/ADC circuit is suitable for high resolution and high speed image sensors.

Experimental study on the behaviors and shape changes of premixed hydrogen–air flames propagating in horizontal duct

The behaviors and shape changes of premixed hydrogen–air flames at various equivalence ratios propagating in half-open and closed horizontal ducts are experimentally investigated using high-speed schlieren imaging and pressure sensors. The study shows that the premixed hydrogen–air flame undergoes more complex shape changes and exhibits more distinct characteristics than that of other gaseous fuels. One of the outstanding findings is that obvious distortion happens to tulip flame after its full formation when equivalence ratio ranges from 0.84 to 4.22 in the closed duct. The salient tulip flame distortions are specially scrutinized and distinguished from the classical tulip collapse and disappearance. The dynamics of distorting tulip flame is different from that of classical tulip flame. The normal tulip flame can be reproduced after the first distortion followed by another distortion. The initiation of flame shape changes coincides with the deceleration both of pressure rise and flame front speed for flames with tulip distortions. And the formation and dynamics of tulip/distorting tulip flames depend on the mixture composition.

面向实时视觉芯片的高速CMOS图像传感器

A high-speed CMOS image sensor for real-time vision chip is proposed. The high-speed CMOS image sensor consists of CMOS photodiode array, correlated double sampling(CDS) array, programmable gain amplifier(PGA) array, area-efficient single-slope analog-to-digital converter(ADC) array and controller circuit. It can perform the image capturing and row-parallel signal processing. It outputs digital signal or digital image at a frame rate of over1000 frame/s. It can reduce the fixed pattern noise(FPN) and amplify(or shrink) the output signals of the photodiode array to maintain the amplitude of the signal in row-parallel fashion. It can continuously perform8-bit ADC conversion in row-parallel. A128 pixel×128 pixel image sensor with128 rows of CDS, PGA and single-slope ADC is fabricated by using0.18μm1P6M CMOS process. The chip size is2.2 mm×2.6 mm. The measured results demonstrate that the designed chip can perform high-speed real-time optical signal capturing and processing. It can be applied to the real-time vision chip system.

High-speed image detector appliance in free space optical communication

The application of very fast image detectors to the fine tracking system of free space optical (FSO) communication has been demonstrated in several projects. In the near surface of the earth, the influence of the atmosphere on the optical communication system must be considered. The optical communication system must be designed to be able to suppress the influence of atmospheric turbulence. The authors have used the high-speed CMOS image sensor for the fine tracking system of 15 km FSO communication demonstration. In this study, the authors present the atmospheric influence on the fine tracking detector, and find out the tracking algorithm for the fine tracking detector to detect the centre of the received spot. On analysis of atmosphere turbulence influenced and the experiment data, the authors find that using centroid algorithm in the fine tracking system is better than the centre of mass algorithm. They proposed a new centroid algorithm in the fine tracking system and use this algorithm in the demonstration system. They found it can minimise the atmosphere turbulence influence.

A DVP-Based Bridge Architecture to Randomly Access Pixels of High-Speed Image Sensors

A design of a novel bridge is proposed to interface digital-video-port (DVP) compatible image sensors with popular microcontrollers. Most commercially available CMOS image sensors send image data at high speed and in a row-by-row fashion. On the other hand, commercial microcontrollers run at relatively slower speed, and many embedded system applications need random access of pixel values. Moreover, commercial microcontrollers may not have sufficient internal memory to store a complete image of high resolution. The proposed bridge addresses these problems and provides an easy-to-use and compact way to interface image sensors with microcontrollers. The proposed design is verified in FPGA and later implemented using CMOS 0.18 um Artisan library cells. The design costs 4,735 gates and 0.12 mm2 silicon area. The synthesis results show that the bridge can support a data rate up to 254 megasamples/sec. Its applications may include pattern recognition, robotic vision, tracking system, and medical imaging.

A Backside-Illuminated Image Sensor With 200 000 Pixels Operating at 250 000 Frames per Second

In this paper, a high-speed image sensor with very high sensitivity is developed. The high sensitivity is achieved by introduction of backside illumination and charge-carrier multiplication (CCM). The high frame rate is guaranteed by installing the in situ storage image sensor (ISIS) structure on the front side. A test sensor of the BSI-ISIS has been developed and evaluated. It is shown that an image with a very low signal level embedded under the noise floor is recognizable by activating the CCM.

Real-time digital signal processing for live electro-optic imaging

We present an imaging system that enables real-time magnitude and phase detection of modulated signals and its application to a Live Electro-optic Imaging (LEI) system, which realizes instantaneous visualization of RF electric fields. The real-time acquisition of magnitude and phase images of a modulated optical signal at 5 kHz is demonstrated by imaging with a Si-based high-speed CMOS image sensor and real-time signal processing with a digital signal processor. In the LEI system, RF electric fields are probed with light via an electro-optic crystal plate and downconverted to an intermediate frequency by parallel optical heterodyning, which can be detected with the image sensor. The artifacts caused by the optics and the image sensor characteristics are corrected by image processing. As examples, we demonstrate real-time visualization of electric fields from RF circuits.

A High-Speed CMOS Image Sensor With Column-Parallel Two-Step Single-Slope ADCs

This paper proposes a column-parallel two-step single-slope (SS) ADC for high-speed CMOS image sensors. Error correction scheme to improve the linearity is proposed as well. A prototype sensor of 320 times 240 pixels has been fabricated with a 0.35-mum CMOS process. Measurement results demonstrate that the proposed ADC can achieve the conversion time of 4 mus , which is ten times faster than the conventional SS ADC. The proposed error correction effectively removes the dead band problem and yields DNL of +0.53/ -0.78 LSB and INL of +1.42/ -1.61 LSB. The power consumption is 36 mW from a supply voltage of 2.8 V.

3D-LSI technology for image sensor

Recently, the development of three-dimensional large-scale integration (3D-LSI) technologies has accelerated and has advanced from the research level or the limited production level to the investigation level, which might lead to mass production. By separating 3D-LSI technology into elementary technologies such as (1) through silicon via (TSV) formation, (2) bump formation, (3) wafer thinning, (4) chip/wafer alignment, and (5) chip/wafer stacking and reconstructing the entire process and structure, many methods to realize 3D-LSI devices can be developed. However, by considering a specific application, the supply chain of base wafers, and the purpose of 3D integration, a few suitable combinations can be identified. In this paper, we focus on the application of 3D-LSI technologies to image sensors. We describe the process and structure of the chip size package (CSP), developed on the basis of current and advanced 3D-LSI technologies, to be used in CMOS image sensors. Using the current LSI technologies, CSPs for 1.3 M, 2 M, and 5 M pixel CMOS image sensors were successfully fabricated without any performance degradation. 3D-LSI devices can be potentially employed in high-performance focal–plane-array image sensors. We propose a high-speed image sensor with an optical fill factor of 100% to be developed using next-generation 3D-LSI technology and fabricated using micro(μ)-bumps and micro(μ)-TSVs.

A High-Speed CMOS Image Sensor with Global Electronic Shutter Pixels Using Pinned Diodes

A High-Speed CMOS Image Sensor with Global Electronic Shutter Pixels Using Pinned Diodes

1A1-D19 高速距離画像センサとカラーカメラを用いたテクスチャ付距離画像の実時間計測システムの構築

This paper discusses a high-speed range image sensor which can obtain textured range images in real time. One of the important applications of this system is an outside sensing for mobile robots. The proposed system consists of a commercially available laser projector and two CCD cameras. The number of pixels of a range image is 361 and the measurement range is 800-2000mm. Experimental results show 40Hz sensing of textured range images. The effectiveness of the proposed system is verified by experiments. As a major conclusion, the constructed sensing system obtains real-time textured range images with commercially available devices.

Real-time surface profile measurement using a feedback type of sinusoidal phase modulating interferometer

In this paper, we propose a sinusoidal phase modulating (SPM) interferometer that is insensitive to external disturbances, and its measuring principle is analyzed theoretically. In the SPM interferometer, the interference signal is detected by a high-speed image sensor based on a low-speed CCD and a signal processing circuit is used to obtain the phase of each point on the surface. Therefore, the surface profile can be measured real-time. The experiments measuring the surface profile of a wedge-shaped optical flat show that the measurement time of the SPM interferometer is less than 10 ms, the repetitive measurement accuracy is 4.2 nm. The results show that the impacts of nonlinear distortion of the piezoelectric transducer (PZT) and part external disturbance are removed.

Photonic-based spectral reflectance sensor for ground-based plant detection and weed discrimination

A bench prototype photonic-based spectral reflectance sensor architecture for use in selective herbicide spraying systems performing non-contact spectral reflectance measurements of plants and soil is described and experimental data obtained with simulated farming vehicle traveling speeds of 7 and 22 km/h is presented. The sensor uses a three-wavelength laser diode module that sequentially emits identically-polarized laser light beams through a common aperture, along one optical path. Each laser beam enters a multi-spot beam generator which produces up to 14 parallel laser beams over a 210mm span. The intensity of the reflected light from each spot is detected by a high-speed line scan image sensor. Plant discrimination is based on calculating the slope of the spectral response between the 635nm to 670nm and 670nm to 785nm laser wavelengths. The use of finely spaced and collimated laser beam array, instead of an un-collimated light source, allows detection of narrow leaved plants with a width as small as 12mm.

A 200Hz Compact Range Image Sensor Using a Multi-Spot Laser Projector

In this paper, a high-speed range image sensor using a multi-spot laser projector is constructed. Several high-speed range image sensors have been developed recently. Their sampling rate is around the video rate (30 Hz or so) and a faster sensor is required such as the 1 ms vision developed by Ishikawa for measurement of ordinary images. The proposed sensor has achieved 200 Hz measurement. It consists of a commercially available laser projector and a high-speed CCD camera. The distance to each projected spot is measured by triangulation. To achieve fast measurement, each spot image is searched in one line (epipolar line). The laser projector is rotated by 14 deg. and consequently, the number of pixels assigned to each spot is magnified by more than 4 times. The number of pixels of the constructed sensor is 361 and its measurement range is 800-2000 mm. Although the acquired range image is sparse, the proposed sensor is thought to be adequate for several applications such as robot vision because of its high-speed imaging and compactness. Some characteristics such as measurement errors are discussed, and the effectiveness of the proposed sensor is verified by experiments.

Live Electrooptic Imaging System Based on Ultraparallel Photonic Heterodyne for Microwave Near-Fields

We report on 100 x 100 pixel live electrooptic imaging (LEI), which is real-time imaging of electrical signals on a microwave circuit. The circuit's electric near-fields can be depicted in real time on a display screen at video frame rates as high as 30 frames/s. This system is based on photonics technology in which electric near-fields are applied to an electrooptic crystal plate and a sensing light beam is modulated there. The frequency of modulation is down-converted by using a photonic heterodyne of large-scale parallelism, and the spatial pattern of the electrooptic modulation is detected with a high-speed image sensor. A digital signal processor is used to extract the frequency component of interest and to animate it on the display screen. As examples, we present live pictures of electric field pattern data acquired by the LEI system in a real-time manner.