Lock-in Image Research Articles

Random access vision: an imaging method to observe arbitrary and multiple gaze directions in frame-by-frame manner

In robot vision, it is often desired to measure an area larger than the field of view of the camera, so the camera tends to be mounted on a mechanical pan/tilt platform. However, such mechanisms have a non-negligible response time compared to the frame rate of the camera. In this paper, we describe what we believe to be a new method that allows arbitrary and multiple gaze directions to be observed in a frame-by-frame manner based on a resonant mirror and a lock-in pixel image sensor. In the proposed method, while the gaze direction oscillates due to the resonance mirror, the image sensor makes an exposure of several hundreds of nanoseconds every time the gaze passes through the direction to be captured, and accumulates the captured signals. A prototype system was developed using a lock-in pixel image sensor with four image storage units called taps and a resonant mirror with a resonant frequency of 12 kHz. The system achieved both arbitrary control of the gaze direction in a frame-by-frame manner, and simultaneous capturing of four images with arbitrary gaze directions also in a frame-by-frame manner.

Quasi-simultaneous imaging of multiple shooting angles using Resonant Mirror and lock-in pixel image sensor

Quasi-simultaneous imaging of multiple shooting angles using Resonant Mirror and lock-in pixel image sensor

Quasi-simultaneous multi-focus imaging using a lock-in pixel image sensor and TAG lens.

In this paper, a quasi-simultaneous multi-focus imaging technique named simulfocus imaging is reported. This technique was developed for measuring an entire object distributed in the depth direction beyond the depth of field (DOF) with high resolution in a single shot. Simulfocus imaging can acquire multiple focal planes in one shot by synchronizing a tunable acoustic gradient index (TAG) lens and a lock-in pixel image sensor. The TAG lens is a tunable-focus lens whose focal position can be changed at a high speed of several tens to several hundreds of kilohertz. The lock-in pixel image sensor is a special image sensor that can execute multiple exposures at an arbitrary timing during a single shooting. The sensor includes a number of photoelectron storage units in each pixel, and the units where the photoelectrons generated by each exposure are stored can be freely selected. Since an image can be acquired for a single storage unit, and the lock-in pixel image sensor has a number of storage units, the lock-in pixel image sensor can acquire multiple images in one shot. By assigning a specific exposure timing to each unit and synchronizing the exposure timing with the focus fluctuation of the TAG lens, it is possible to simultaneously acquire images in different focal planes. To evaluate the system, we conducted experiments to show the effectiveness of simulfocus imaging in microscope and telescope configurations. From the experimental results, it was confirmed that simulfocus was effective in both configurations.

Two-stream convolutional neural network for non-destructive subsurface defect detection via similarity comparison of lock-in thermography signals

Active infrared thermography is a safe, fast, and low-cost solution for subsurface defects inspection, providing quality control in many industrial production tasks. In this paper, we explore deep learning-based approaches to analyze lock-in thermography image sequences for non-destructive testing and evaluation (NDT&E) of subsurface defects. Different from most existing Convolutional Neural Network (CNN) models that directly classify individual regions/pixels as defective and non-defective ones, we present a novel two-stream CNN architecture to extract/compare features in a pair of 1D thermal signal sequences for accurate classification/differentiation of defective and non-defective regions. In this manner, we can significantly increase the size of the training data by pairing two individually captured 1D thermal signals, thereby greatly easing the requirement for collecting a large number of thermal sequences of specimens with defects to train deep CNN models. Moreover, we experimentally investigate a number of network alternatives, identifying the optimal fusion scheme/stage for differentiating the thermal behaviors of defective and non-defective regions. Experimental results demonstrate that our proposed method, directly learning how to construct feature representations from a large number of real-captured thermal signal pairs, outperforms the well-established lock-in thermography data processing techniques on specimens made of different materials and at various excitation frequencies.

Crack growth analysis in welded and non-welded T-joints based on lock-in digital image correlation and thermoelastic stress analysis

The method based on the digital image correlation (DIC) technique and thermoelastic stress analysis (TSA) is proposed to monitor the crack propagation process of T-joint specimens during the fatigue test. The lock-in amplifier is used to process DIC speckle pattern images while the specimen is dynamically loaded. The lock-in algorithm uses the fact that lock-in amplifier can be able to detect very small signal change within the measurement noise, which is often used in TSA. Using appropriate post-processing method, both the crack lengths and the stress intensity factors (SIF) can be evaluated in function of the number of fatigue cycles. The tests on non-welded T-joint specimens and welded T-joint specimens in two types of test rigs will be presented in the paper. All of the achieved results were validated by employing extended finite element method (XFEM) performed with ANSYS software.

Active Infrared Thermography for Visualizing Subsurface Micro Voids in an Epoxy Molding Compound

This paper presents an automated subsurface micro void detection technique based on pulsed infrared thermography for inspecting epoxy molding compounds (EMC) used in electronic device packaging. Subsurface micro voids are first detected and visualized by extracting a lock-in amplitude image from raw thermal images. Binary imaging follows to achieve better visualization of subsurface micro voids. A median filter is then applied for removing sparse noise components. The performance of the proposed technique is tested using 36 EMC samples, which have subsurface (below 150 μm ~ 300 μm from the inspection surface) micro voids (150 μm ~ 300 μm in diameter). The experimental results show that the subsurface micro voids can be successfully detected without causing any damage to the EMC samples, making it suitable for automated online inspection.

Ambiguity of mapping the relative phase of blood pulsations.

Blood pulsation imaging (BPI) is a non-invasive optical method based on photoplethysmography (PPG). It is used for the visualization of changes in the spatial distribution of blood in the microvascular bed. BPI specifically allows measurements of the relative phase of blood pulsations and using it we detected a novel type of PPG fast waveforms, which were observable in limited areas with asynchronous regional blood supply. In all subjects studied, these fast waveforms coexisted with traditional slow waveforms of PPG. We are therefore presenting a novel lock-in image processing technique of blood pulsation imaging, which can be used for detailed temporal characterization of peripheral microcirculation.

Investigation on stress distribution of multilayered composite structure (MCS) using infrared thermographic technique

In this paper, the thermoelastic stress analysis (TSA) on a multilayered composite structure (MCS) was investigated by means of lock-in thermographic image technique (LITI). The application of thermoelastic stress analysis on MCS becomes particularly complicated due to consisting of different material components, which determines the different thermoelastic coupling response depended on material thermal-physical property. The thermoelastic coupling constants (TCC) of GFRP, medium-carbon steel and foam were obtained through thermomechanical calibration experiments, respectively. An artificial neural network was proposed to determine the component of MCS. Comparisons between finite element analysis (FEA) and LITI measurement are reported. It is found that the stress distribution of MCS can be evaluated with good accuracies using LITI measurement.

High-Resolution, Low-Noise Imaging in THz Polaritonics

Time-resolved imaging of propagating electromagnetic waves at terahertz (THz) frequencies provides deep insights into waves and their interaction with a variety of photonic elements. As new components for THz control are developed, such as metamaterial microstructures that display deep sub-wavelength $E$ -field localization, finer spatial resolution and more sensitive imaging techniques are required to study them. Here we introduce key advances in the optical design and lock-in image acquisition at 500 Hz for the complementary imaging techniques of phase contrast and polarization gating. Compared to other methods, this leads to a 4-fold improvement in resolution and up to 5-fold reduction in noise through suppression of low frequency laser fluctuations. With a resolution better than 1.5 $\mu{\hbox{m}}$ ( $\lambda/100$ at 0.5 THz) and a noise floor of 0.2%, phase contrast imaging presents new opportunities for studying very fine structures and near-fields in the THz regime. For most other experiments, polarization gating imaging is preferred because its noise floor is lower at 0.12% and its $ resolution is typically more than sufficient.

Statistical theory and applications of lock-in carrierographic image pixel brightness dependence on multi-crystalline Si solar cell efficiency and photovoltage

A solar cell lock-in carrierographic image generation theory based on the concept of non-equilibrium radiation chemical potential was developed. An optoelectronic diode expression was derived linking the emitted radiative recombination photon flux (current density), the solar conversion efficiency, and the external load resistance via the closed- and/or open-circuit photovoltage. The expression was shown to be of a structure similar to the conventional electrical photovoltaic I-V equation, thereby allowing the carrierographic image to be used in a quantitative statistical pixel brightness distribution analysis with outcome being the non-contacting measurement of mean values of these important parameters averaged over the entire illuminated solar cell surface. This is the optoelectronic equivalent of the electrical (contacting) measurement method using an external resistor circuit and the outputs of the solar cell electrode grid, the latter acting as an averaging distribution network over the surface. The statistical theory was confirmed using multi-crystalline Si solar cells.

Solar Cell Defects Detection Using Lock-In Amplifier

In this research we present a Lock-In amplifier photoluminescence image (PLI) detection system. Using a commercially available CCD camera, combined with a modulated excitation light source system, Lock-In amplifier technique is employed to enhance the sensitivity and signal-to-noise ratio of the image. Using a high resolution camera, we developed a high spatial resolution non-contact inspection system for solar cell. Each solar cell can be examined within 2s relevant for inline processing control.

초음파 적외선 열화상을 이용한 접합부의 미세균열 검출 연구

본 논문은 초음파 적외선 열화상과 위상잠금 방법을 이용하여 이종금속용접(STS304 and SA106 Gr. b)된 파이프의 응력부식균열 결함을 검출하였다. 초음파 가진장치는 출력 250 Watt, 주파수 20 kHz이었다. 실험 결과 이종금속용접부의 파이프 내부에 위치한 균열을 적외선 열화상을 이용하여 검출할 수 있었다. 실제 PT검사를 통하여 배관 내부의 균열이 하나가 아닌 일정한 범위 안에 하나 이상의 크랙이 존재하여 열화상 이미지 상에 넓은 범위의 hot spot 이미지를 만들어 냈음을 파악할 수 있었다. 또한 기존 기술로 검출이 용이하지 못한 마이크로 폭의 미세균열을 검출할 수 있었다. 또한, 초음파탐상기술은 <TEX>$10\;{\mu}m$</TEX> 미세크랙의 폭을 갖은 균열을 쉽게 검출하지 못하였다. 그러나 초음파 적외선 열화상 기술은 결함 검출하였다. This study detected SCC defects of dissimilar metal welded(STS304 and SA106 Gr. b) pipes using the ultrasonic infrared thermography method and the lock-in image treatment method among infrared thermography method. The infrared excitement equipment has 250 Watt of output and 20 kHz of frequency. By using the ultrasound infrared thermography method, the internal defects of dissimilar metal weld joints of pipes used at nuclear power plants could get detected. By an actual PT test, it was observed that the cracks inside the pipe existed not as a single crack but rather as a multiple cracks within a certain area and generated a hot spot image of a broad area on the thermography image. In addition, UT technology could not easily defects detected by the width of <TEX>$10\;{\mu}m$</TEX> fine hair cracks. but, ultrasound infrared thermography technique was defect detected.

Infrared lock-in carrierography (photocarrier radiometric imaging) of Si solar cells

Modulated photocarrier radiometric (PCR) imaging (lock-in carrierography) of multicrystalline (mc) Si solar cells is introduced using a near-infrared (NIR) InGaAs camera and a spread superband gap laser beam as an optoelectronic source at low modulation frequencies (&amp;lt;10 Hz) or point-by-point scanning PCR imaging with a focused laser beam at high (kilohertz) frequencies. PCR images are supplemented by quantitative PCR frequency scans and compared to NIR optical reflectance, modulated electroluminescence (MEL) and modulated photovoltage (MPV) images. Noncontact PCR imaging is controlled by the photoexcited carrier diffusion wave and exhibits very similar images to contacting MEL and MPV. Among these methods it exhibits the highest contrast and sensitivity to mechanical and crystalline defects in the substrate at lock-in image frequencies in the range of the inverse recombination lifetime in the quasineutral region (bulk).

Real-time lock-in imaging by a newly developed high-speed image-processing charge coupled device video camera

Real-time lock-in imaging with high-frequency modulation (up to 16 kHz) was successfully performed at high spatial resolution (640×480 pixels) using a newly developed high-speed image-processing charge coupled device (CCD) video camera. To achieve high-resolution lock-in imaging, the high-speed image-processing CCD video camera incorporates a high-definition TV image sensor that uses a high-definition frame-interline-transfer (FIT) CCD architecture. A novel FIT-CCD driving method, in which vertical CCD shift registers are utilized as a temporary frame memory, enables lock-in imaging at a modulation frequency in excess of the video frame rate (30 Hz). Furthermore, since the high-speed image-processing CCD video camera has a function for subtracting images taken with high-frequency modulation, a lock-in image with no background contrast can be observed in real time. The phase detection function for lock-in imaging, when coupled with real-time image processing using the high-speed image-processing CCD video camera, makes it possible to dynamically observe the phase distribution in a two-dimensional area. As a demonstration of the real-time lock-in imaging, the wave pattern that appears on the surface of water was successfully visualized by a real-time lock-in imaging system that uses the high-speed image-processing CCD video camera.

Development of a New Crack Identification Technique Based on Near-Tip Singular Electrothermal Field Measured by Lock-in Infrared Thermography.

A new thermographic NDT technique was proposed, in which singular electrothermal field near crack tips under the application of periodically modulated electric current was measured using an infrared thermography combined with lock-in data processing technique. Experimental investigations were made on the resolution and the applicability in the identification of through-thickness artificial cracks and fatigue cracks embedded in steel and aluminum alloy plate samples. Modulated electric current was applied to the cracked sample by an induction coli. Lock-in thermal images synchronized to the reference current modulation signal were taken by the lock-in thermography. Significant singular electrothermal field was observed at the crack tip in the lock-in thermal image. The fatigue cracks as well as artificial cracks were found to be sensitively identified by the proposed technique in good resolution compared with the singular method using a conventional thermographic temperature measurement.

An instrument for in-situ stress measurement in thin films during growth

An improved stress-measuring technique based on the cantilever bending principle is presented. Any thermal shift of the sample holder position that results in errors in the stress data is minimised by evaluating the difference in the deflection of two laser beams. The film thickness is calculated from the reflected laser intensity, or from ellipsometry data recorded simultaneously during film growth. Without using a lock-in technique or image processing, the resolution in the bending force per unit width is 0.02 N/m for a 50-μm thick cantilever. The system represents an easy, versatile and cheap technique to measure intrinsic and thermal stresses in thin films. The system has been employed to record the instantaneous stress during the ion beam-assisted deposition of BN films, and the global stress during the ion-induced amorphisation of silicon.

Correlation between Noise-after-Write and Magnetic Domain in Thin-Film Heads by Electron Microscopy

Noise-after-write has been reported in thin film heads, although its source or sources have not yet been identified. To discover these noise sources, I observed conversions of magnetic domain structure after writing with an electron microscope. The magnetic domain observations involved lock-in image processing for back-scattered electron contrast (type II), using a JEOL 2000FX II 200 kV electron microscope. Thin-film heads were excited by a 10 mA <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p-p</sub> , 100 kHz driving current. The magnetic domains in the upper yokes of the heads were observed after 60 mA <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p-p</sub> write operations. The head with the highest noise probability, at 5.2×10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-2</sup> , exhibited walls radiating outward from the back-gap closure, which were due to stress. The magnetic domain structure was severely deformed after writing. The head with the lowest noise probability, at 1.6×10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> , exhibited a normal closure domain configuration, with very little conversion. These results indicate that noise-after-write is generated mainly near the backgap closure.

電子顕微鏡による薄膜磁気ヘッドの磁区構造変化と記録後ノイズの相関

Noise-after-write has hen reported in thin-film heads, although the sources have not yet been identified. To discover these noise sources, I observed the manetic domain structum conversions after writing with an electron microscope, The magnetic domain observation consisted of lock-in image processing for back-scattered electron contraat (Type II), using a JEOL 2000FX II 200 kV electron microscope. Thin-film heads were excited with a 10 mAp-p, 100 kHz drive current. The magnetic domains in the upper yokes of the heads were obgerved after 60 mAp-p write operations. The head that had the highest noise probability of 5.2 × 10-2 exhibited radial walls oentered at the backgap closure because af stress. Its magnetic domain configuration after writing was severely converted. The head with the lowest noise probability of 1.6 × 10-4 exhibited a normal closure domain configuration, with very little conversion. These results indicate that noise-after-write is generated mainly near the backgap closure.

Magnetic domains in thin-film recording heads as observed in the SEM by a lock-in technique

In a thin-film magnetic recording head, a magnetic circuit made from two Permalloy thin films, one above the other, is magnetized by a spixally plated thin-film copper coil which is located (together with insulating layers) between those two films. The magnetic domains in a thin Permalloy film can be studied by type-2 magnetic constrast using backscattered electrons (BSE) in the scanning electron microscope (SEM) provided that the film has a smooth surface and is deposited on a flat substrate. In practice, the upper Permalloy film follows the contours of the underlying layers. This gives a topographic signal which is large enough to mask the type-2 magnetic contrast signal in its simple form. The domain walls can, however, be seen if the magnetic recording head is excited with a sinusoidal current and if the video waveform is processed with a lock-in amplifier referenced either to the fundamental or to the second harmonic of the excitation frequency. This lock-in image processing technique has now been applied to obtain images of the magnetic domains both from the upper Permalloy film of an incompletely fabricated head and also from the exposed cross sections of the Permalloy films in an operational thin-film head.