Image Sensor Module Research Articles

BLEscope: A Bluetooth Low Energy (BLE) Microscope for Wireless Multicontrast Functional Imaging.

Recent advances in low-power wireless-capable system-on-chips (SoCs) have accelerated diverse Internet of Things (IoT) applications, encompassing wearables, asset monitoring, and more. Concurrently, the field of neuroimaging has experienced escalating demand for lightweight, untethered, low-power systems capable of imaging in small animals. This article explores the feasibility of using a low-power asset monitoring system as the basis of a new architecture for fluorescence and hemodynamic contrast-based wireless functional imaging. The core system architecture hinges on the fusion of a Bluetooth Low Energy (BLE) 5.2 SoC and a low-power 560×560, 8-bit monochrome CMOS image sensor module. Successful integration of a multicontrast optical front-end consisting of a fluorescence channel (FL) and an intrinsic optical signal (IOS) channel resulted in the creation of a wireless microscope called 'BLEscope'. Next, we developed a wireless (i.e. BLE) protocol to remotely operate the BLEscope via a laptop and acquire in vivo images at 1 frame per second (fps). We then conducted a comprehensive characterization of the BLEscope to assess its optical capabilities and power consumption. We report a new benchmark for continuous wireless imaging of ∼1.5 hours with a 100 mAh battery. Via the FL channel of the BLEscope, we successfully tracked the kinetics of an intravenously injected fluorescent tracer and acquired images of fluorescent brain tumor cells in vivo. Via the IOS channel, we characterized the differential response of normal and tumor-associated blood vessels to a carbogen gas inhalation challenge. When miniaturized, the BLEscope will result in a new class of low-power, implantable or wireless microscopes that could transform preclinical and clinical neuroimaging applications.

Optofluidic zoom system with increased field of view and less chromatic aberration.

Imaging systems are widely used in many fields. However, there is an inherent compromise between field of view (FOV) and resolution. In this paper, we propose an optofluidic zoom system with increased FOV and less chromatic aberration, which can realize switching between large FOV and high resolution. The proposed system consists of a liquid prism, a zoom objective, an image sensor and image processing module, which can realize optical zoom and deflection. The proposed system achieves non-mechanical optical zoom from f = 40.5 mm to f = 84.0 mm. Besides, the angular resolution of zoom objective is up to 26"18 at f = 84.0 mm. The deflection range is ±10°, and the whole FOV of proposed system can reach up to 30.3°. The proposed system is compact and easy to machine. In addition, we reduce chromatic aberration produced by the liquid prism significantly. The proposed system can be used in monitor system, target tracking system, telescope system and so on.

Development of thermally-stable NIR absorbing films based on heptamethine cyanine dyes with bistriflimide anion

Near-infrared (NIR) absorbing film is an essential component of CMOS image sensor modules. In order to exploit the intense NIR absorbing performance of heptamethine cyanine dyes for this application, their thermal stability must be improved. In this study, a series of heptamethine cyanine dyes were synthesized in which the meso-substituent and the counter anion were varied. Furthermore, the effects of the substituent and the counter anion on the material properties were investigated. The molar extinction coefficients of the cyanine dyes were enhanced by replacing the iodide ion with the bulky bistriflimide (TFSI) anion. The TFSI anion, in which the negative charge is dispersed over a large volume, stabilized the cyanine cation by charge delocalization and greatly improved the thermal stability. NIR absorbing films with outstanding transmittance in the visible region were fabricated using the synthesized cyanine dyes and a transparent polysulfone (PSU) binder. The thermal stability was evaluated by post-baking the prepared films at a temperature of 150 °C for 3 h. The films made with TFSI-substituted cyanine dyes showed excellent thermal stability, and they maintained their initial absorbance even after the baking process compared to films made with unsubstituted cyanine-based dyes.

Integrating Neural Network for Pest Detection in Controlled Environment Vertical Farm

Background: An integrated system for creating and maintaining controlled environment ideal for vertical farming prototype is demonstrated. The requirement of optimal artificial light for different growth stages of tomato and chilli plants is studied in detail and CNN model-based method for detection and classification of Leaf disease is also developed. Methods: The artificial environment ensuring adequate artificial lighting, moisture, and minerals was create by implanting various sensors and actuators to the plant beds and connected in network through a cloud based remote server. A CMOS image sensor module was used to monitor the various stages of plant growth. Findings: The duration and intensity requirement for germination, vegetation and flowering of both tomato and chilli plants are relatively lesser with artificial light condition than with sunlight. At the end of fifth epoch the developed convolution neural network model for detection and classification of leaf disease produced training and validation accuracies of 84.8% and 67.2%, respectively. Novelty: For different growth stages of tomato and chilli plants in north eastern India, the requirement of optimal artificial light is studied by exposing them to different light intensities. The study was conducted during summer (May-June) when the average sun exposure in eastern India was ~130- 190 hours. The captured images and data generated were used to monitor the status of the crops and identifying diseases with the application of Deep Learning models. Convolutional Neural Network (CNN) model-based method for detection and classification of leaf disease is presented. Keywords: Convolution Neural Network; Vertical Farming; Artificial Light; Pest Detection; Light Emitting Diode

Image Sensor Module for Detecting Spatial Color Temperature in Indoor Environment

Image Sensor Module for Detecting Spatial Color Temperature in Indoor Environment

Image Sensor Module for Detecting Space Illuminance in Indoor Environment

Image Sensor Module for Detecting Space Illuminance in Indoor Environment

Enhanced Image Sensor Module for Head-Mounted Microscopes.

Several research groups have developed head-mounted fluorescence microscopes as a modality for recording neural activity in freely behaving mice. The current designs have shown exciting results from in vivo imaging of the bright dynamics of genetically encoded calcium indicators (GECI). However, despite their potential, head-mounted microscopes are not in use with genetically encoded voltage indicators (GEVI) or bioluminescence indicators. Due to its ability to match the temporal resolution of neuron spiking, GEVIs offer great benefits to experiments designed to provide feedback after real-time detection of specific neural activity such as the less than 250ms replay events that can occur in the hippocampus. Orthogonally, the emerging bioluminescence activity reporters have the potential to eliminate autofluorescence and photobleaching that can occur in fluorescence imaging. There are two important properties of the head-mounted microscope's image sensor affecting the ability to image GEVIs and bioluminescence indicators. First, the low signal to noise ratio (SNR) characteristics of GEVIs and bioluminescent indicators make signal detection difficult. Second, in order to take advantage of the GEVIs faster fluorescence kinetics, the image sensor must be capable of matching frame rates. Here, we present the design of a new imaging module for head-mounted microscopes incorporating the latest CMOS sensor technology aimed at increasing image sensor sensitivity and frame rates for use in real-time detection experiments. The design builds off an existing open-source project and can integrate into the existing data acquisition hardware and microscope housing.

Multipath routing technique for responding to sniffing attacks in wireless multimedia sensor network environment

As the image sensor modules such as complementary metal oxide semiconductor (CMOS) camera sensors and digital microphones are developed, security is required such as data storage and transmission in wireless multimedia sensor network technology and application. Data transmitting through existing sensor network routing path has the problem of high vulnerability to information leakage by sniffing attacks. Therefore, the study proposes one of the protection technique from a sniffing attack, which protects from attacking. The proposed technique is based on the movement of risk factors that sets priority of the geographically proximal sensor nodes according to defense readiness condition (DEFCON) and data importance. According to the simulation results, it achieved approximately 5.8% transmission delay time compared with existing sensor network path but the security was confirmed as eavesdropping rate where risk factors can attack and eavesdrop decreased by approximately 18.5%.

医療用CMOS撮像モジュールへの応用を目指した撮像素子WL-CSPプロセスの開発

医療用CMOS撮像モジュールへの応用を目指した撮像素子WL-CSPプロセスの開発

이미지 센서 모듈의 광학적 특성 테스트를 위한 표준화된 기술 방법

이미지 센서와 렌즈를 모듈 상에 고정할 때, 기계적인 오차로 인해 취득된 영상의 기울임 또는 회전이 발생하기도 하고 화각이 좁아지기도 한다. 따라서 테스트 장비에서 이미지 센서 모듈의 광학적 특성을 테스트하여야 한다. 본 논문에서는 이미지 센서 모듈의 광학적 특성을 테스트하는 방법을 설명하고, 이를 영상 취득 특성과 유사한 방식으로 표준화한 기술 방법을 제안한다. 제안된 방법은 테스트 장비가 영상 취득 특성과 광학적 특성을 함께 테스트하는데 도움이 된다. When image sensor and lens are fixed on the module, mechanical errors often induce tilt, rotation, or narrow field-of-view of the acquired image. Therefore, the optical characteristics of image sensor modules should be tested by test equipments. This paper explains how to test the optical characteristics of images sensors. It also proposes the standardized description methods of optical characteristics tests which are similar with those of image acquisition characteristics tests. The proposed method helps the test equipments to perform image acquisition characteristics tests and optical characteristics tests together.

An optical multi-sensing system for detection of cardiovascular toxicity

A mini-microscope-based system for multisite detection of cardiovascular toxicity was developed. The mini-microscope consisted of an image sensor and lens module extracted from an inexpensive webcam. The flipped lens module enabled cells to be magnified and monitored during testing. The portability and compactness of this system enables short-term and potential long-term experimentation inside a conventional incubator. The toxicity test results demonstrated that the normalized beating rates of cardiac muscle cells selected from multiple regions increased over time when treated with 100 nM isoprenaline. The presented system could be a promising cost-effective cell-based testing tool for discovering and screening drugs.

Implementation of Video Capture and Playback in Mobile Systems

Mobile phones are backbone of the mobile communications and have experienced fastest growing segment in the consumer market. In the 3G and 4G mobiles, video recording is an essential entertainment features so the user can record and play the video within the mobile phone. In this paper, video capture and playback implementation had been carried out by integrating the camera module to the GSM phone. An alaysis of the video processing had been carried out. Image sensor and LCD module had been interfaced with the base band processor through the video processor. To interface the different module necessary PCB schematic diagram had been arrived. Software implementation had been carried out for the Human Machine Interface for various settings of the camera module, which is integrated with the video processor. The developed playback system had been tested for the various senerious. In this video has been captured through the camera sensor and at the time of playback measured the resolution, power consumption and image transfer rates of the various modules used in the developed camera phone.

Surface Plasmon Resonance Image Sensor Module of Spin-Coated Silver Film with Polymer Layer

Prism modules of 20 nm-, 40 nm-, and 60 nm-thick spin-coated silver films both without and with an upper 100 nm-thick spin-coated polymer layer were fabricated for surface plasmon resonance (SPR) image sensor applications. The prism modules were applied to an SPR image sensor system. The coefficients of determination (R2s) for the 20 nm-, 40 nm- and 60 nm-thick silver films without the polymer layer were 0.9231, 0.9901, and 0.9889, respectively, and with the polymer layer 0.9228, 0.9951, and 0.9880, respectively when standard ethanol solutions with 0.1% intervals in the range of 20.0% to 20.5% were applied. The upper polymer layer has no effect on the R2. The prism modules of the 40-nm-thick spin-coated silver films had the highest R2 value of approximately 0.99. The durability of the 40 nm-thick spin-coated silver film with the 100 nm-thick polymer layer is much better than that without the upper low-loss polymer layer. The developed SPR image sensor module of the 40 nm-thick spin-coated silver film with the upper 100 nm-thick low-loss polymer film is expected to be a very cost-effective and robust solution because the films are formed at low temperatures in a short period of time without requiring a vacuum system and are very durable.

Very Thin Spin-Coated Silver Films via Transparent Silver Ink for Surface Plasmon Resonance Sensor Applications

We fabricated very thin silver films with thicknesses of 20 nm, 40 nm, and 60 nm on a prism using a spin coating method for surface plasmon resonance (SPR) image sensor module applications. An aqueous silver ionic complex solution was spin-coated and then thermally cured for 10 minutes at 150 degrees C in an oven. The spin-coated solid silver films possessed silver crystallinity. The prism modules with the 20-nm-, 40-nm- and 60-nm-thick thin silver films were applied to an SPR image sensor system. The coefficients of determination for the 20-nm-, 40-nm- and 60-nm-thick silver films were 0.923, 0.990 and 0.989, respectively when standard ethanol solutions with 0.1% intervals in the range of 20.0% to 20.5% were applied. The correlation is high-performed and the coefficients of determination are as close as 1. The spin coating method of very thin silver films for SPR image sensor modules is expected to be a very cost-effective solution because the films can be formed at a low temperature in a short period of time without requiring a vacuum system.

A method for evaluation of an optical image stabilizer in an image sensor module

A figure of merit called the image clarity index is proposed for quantifying the performance of an optical image stabilizer (OIS) subjected to hand jitter. The image clarity index is defined as the slope angle of the normalized edge spread intensity of a captured image. A two-dimensional map of the index can offer clues as to the design limit on shake angle and frequency. Because of its relative simplicity over the existing metrics such as the modulation transfer function, the proposed index is especially suitable for summary evaluation of OISs and rapid benchmarking of OIS-equipped image sensor modules.

Adaptive Sensing and Optimal Power Allocation for Wireless Video Sensors With Sigma-Delta Imager

We consider optimal power allocation for wireless video sensors (WVSs), including the image sensor subsystem in the system analysis. By assigning a power-rate-distortion (P-R-D) characteristic for the image sensor, we build a comprehensive P-R-D optimization framework for WVSs. For a WVS node operating under a power budget, we propose power allocation among the image sensor, compression, and transmission modules, in order to minimize the distortion of the video reconstructed at the receiver. To demonstrate the proposed optimization method, we establish a P-R-D model for an image sensor based upon a pixel level sigma-delta (Σ∆) image sensor design that allows investigation of the tradeoff between the bit depth of the captured images and spatio-temporal characteristics of the video sequence under the power constraint. The optimization results obtained in this setting confirm that including the image sensor in the system optimization procedure can improve the overall video quality under power constraint and prolong the lifetime of the WVSs. In particular, when the available power budget for a WVS node falls below a threshold, adaptive sensing becomes necessary to ensure that the node communicates useful information about the video content while meeting its power budget.

The reliability of COF Joints with Tin Bumps and Non-Conductive Adhesives for Image Sensor Module Application

We developed a low cost and low temperature chip-on flex(COF) bonding technology by using Sn bumps and non-conductive adhesive(NCA) for image sensor device. Two types of Sn bumps, square and hemispherical bumps, were fabricated. Sn bumps were formed sequentially by electroplating and then were reflowed to form the hemispherical bumps. Device with Sn bumps was bonded with flexible-printed circuit board(F-PCB) using NCA at 180°C after the NCA was dispensed. To evaluate the reliability of the COF joints, a thermal cycling (TC) test, high temperature storage (HTS) test, and temperature and humidity (T&H) test have been performed. The bondability and reliability of joints were evaluated by measuring the contact resistance of each bump. The electrical test showed that COF bonding was successful with no failed bumps, and the contact resistance of COF joints was very low about 10mΩ per bump. After aging treatment at 150°C slightly increased the contact resistance. But failed COF joints were not observed before and after aging treatment. The reliability of COF joints fabricated using hemispherical bumps was good more than that of joint with square Sn bump. NCA trapping in the interface between bump and pad had an influence on the reliability of COF joints. The results of our experiments were successfully performed in application for the temperature sensitive devices such as image sensor module.

Advances in Wafer Level Processing and Integration for CIS Module Manufacturing

This article presents the advances in wafer-level processing and integration techniques for CMOS image sensor module manufacturing. CMOS image sensors gave birth to the low-cost, high-volume camera phone market and are being adopted for various high-end applications. The backside illumination technique has significant advantages over the front-side illumination due to separation of the optical path from the metal interconnects. Wafer bonding plays a key role in manufacturing backside illuminated sensors. The cost-effective integration of miniaturized cameras in various handheld devices becomes realized through the introduction of CMOS image sensor modules or camera modules manufactured with wafer-level processing and integration techniques. We developed various technologies enabling wafer-level processing and integration, such as (a) wafer-to-wafer permanent bonding with oxide or polymer layers for manufacturing backside illuminated sensor wafers, (b) wafer-level lens molding and stacking based on UV imprint lithography for making wafer-level optics, (c) conformal coating of various photoresists within high aspect ratio through-silicon vias, and (d) advanced backside lithography for various metallization processes in wafer-level packaging. Those techniques pave the way to the future growth of the digital imaging industry by improving the electrical and optical aspects of devices as well as the module manufacturability.

이미지 센서 모듈을 위한 자동-초점 기능의 전력-효율적인 구동 방법에 대한 연구

We present a power-efficient driving scheme that consists of piezoelectric actuator and driver IC for AF (Auto-Focus) on ISM (Image Sensor Module). The piezoelectric actuator is more power-efficient than conventional voice coil motor actuator. And high power-efficiency driver IC is designed. So the proposed driving scheme using designed piezoelectric actuator and driver IC is more close to recent trend of green IT. The diver IC should guarantee fast and accurate performance. So, the optimum driving method and high accurate frequency synthesizer are proposed. The die area of designed driver IC is <TEX>$2.0{\times}1.6mm^2$</TEX> and power consumption is 2.8mW.

13-7 in vivo脳神経イメージング用CMOSイメージングデバイスへの光源内蔵と低侵襲化(第13部門 情報センシング2)

13-7 in vivo脳神経イメージング用CMOSイメージングデバイスへの光源内蔵と低侵襲化(第13部門 情報センシング2)