Real-time Pixel Research Articles

Dark background correction with multivariate model fitting for infrared hyperspectral remote sensing

Dark currents in HgCdTe devices are the main factor that limit detector performance in infrared imaging systems, which may result in a reduced dynamic range and poor signal-to-noise ratio (SNR). In this paper, the theoretical limit of dark background correction is defined, and the correction effects of different methods are quantitatively evaluated. Based on accurate measurements of the focal plane array (FPA) temperature, the relationship between the FPA temperature and dark background is evaluated. The fluctuations are highly consistent; thus, the FPA temperature can be used to estimate the dark background of different pixels in real time. Moreover, the temperature of the opto-mechanical components is stabilized at 120 K to suppress instrument radiation, and the temperature of the electrical box is applied to correct gradient changes in the dark background at large time scales. As a result, the dark backgrounds of different pixels can be accurately corrected in real time.

Inversion method of target gas cloud transmittance based on atmospheric profile synthesis background

The sky infrared background radiation varies greatly with spatial distribution and time. When scanning Fourier transform infrared remote sensing imaging system scans the target gas cloud with the sky as the background, the background radiation corresponding to each scanned pixel is different, and the background does not have a constant baseline. It is extremely difficult to obtain the background spectrum of each pixel in real time, which affects the inversion accuracy of the target gas cloud transmittance. An inversion method of target gas cloud transmittance based on atmospheric profile synthesis background is proposed in this work. The temperature, humidity, pressure, and ozone profiles of the measured locations and the atmospheric model are used to generate the sky infrared background in order to solve the problem, i.e. the difficulty in measuring the clean sky infrared background spectrum in the chemical industry park. This paper proposes that there is a continuous derivable relationship between the sky infrared background spectrum and the cosine of zenith angle at each wavenumber, so a small amount of sky infrared background spectrum with a zenith angle gradient can quickly generate a sky infrared background spectrum at any elevation angle. The proposed method is verified by the moderate resolution atmospheric radiative transfer model (MODTRAN) software simulation and the remote sensing imaging experiment of SF<sub>6</sub> gas. The proposed method can quickly generate the sky infrared background spectrum corresponding to any angle within a gradient elevation angle and accurately invert the target gas cloud transmittance at each pixel. The results show that the distribution trend of the column concentration of the SF<sub>6</sub> gas cloud is consistent with the actual distribution, and the correlation is 0.99979.

Real-time pixel merging for weak light detection

Real-time pixel merging for weak light detection

Foreground Segmentation Using Adaptive 3 Phase Background Model

An extensive collection of algorithms have been proposed over the years to identify the foreground from a video scene, but none of them considers the classification history of previous frames for discovering moving objects. All the existing algorithms focus on a single background model to cope with all types of challenging and complex video environments. In this paper, a real-time pixel level classification method is proposed that uses its previous output history to update its parameters. The model has three components designed to handle various types of challenging background environments. For each pixel, three sub-parts, namely the Adaptive Background Model, the Neighborhood Background Model, and the Change Detection Background Model are constructed to detect various types of complex background changes. A pixel level model updating method uses the previous foreground/background binary classification results to periodically refresh all the three background models. This update method helps to recognize the foreground accurately by adjusting the algorithmic parameters to efficiently detect complex background changes. This novel method shows a significant improvement in performance for a variety of complex video scenes.

A New Dynamic Emissivity-retrieval Scheme Based on Wavelet Atlas Constraints and Its Application into Land Surface Satellite Data Assimilation

Accurate description of surface information, including land surface emissivity, is a prerequisite for assimilating satellite observations over land. In this study, we used an emissivity atlas, essentially an image matrix containing noise from unknown sources, to improve the assimilation of observational data. First, we proposed a wavelet method for advanced image processing to denoise the emissivity atlas and developed a wavelet emissivity atlas suitable for the temperature sounder onboard the Chinese FY-3C satellite. Second, based on a real-time pixel dynamic emissivity retrieval scheme and using the wavelet atlas as the constraint set, we constrained dynamic retrieval emissivity values to eliminate retrieval errors. Finally, we designed and optimized a new assimilation system that can successfully utilize low-level microwave sounding channels of the FY-3C satellite. The results showed that the first-guess departures of the middle- and low-level channels were significantly improved as a result of the improved surface emissivity values. Compared with the emissivity atlas scheme, over-land observational data of the middle- and low-level channels could be increased by approximately 4–8%. Moreover, the addition of a large amount of new observational data did not negatively affect the analysis fields but rather improved short-term prediction results.

Fast depth densification for occlusion-aware augmented reality

Current AR systems only track sparse geometric features but do not compute depth for all pixels. For this reason, most AR effects are pure overlays that can never be occluded by real objects. We present a novel algorithm that propagates sparse depth to every pixel in near realtime. The produced depth maps are spatio-temporally smooth but exhibit sharp discontinuities at depth edges. This enables AR effects that can fully interact with and be occluded by the real scene. Our algorithm uses a video and a sparse SLAM reconstruction as input. It starts by estimating soft depth edges from the gradient of optical flow fields. Because optical flow is unreliable near occlusions we compute forward and backward flow fields and fuse the resulting depth edges using a novel reliability measure. We then localize the depth edges by thinning and aligning them with image edges. Finally, we optimize the propagated depth smoothly but encourage discontinuities at the recovered depth edges. We present results for numerous real-world examples and demonstrate the effectiveness for several occlusion-aware AR video effects. To quantitatively evaluate our algorithm we characterize the properties that make depth maps desirable for AR applications, and present novel evaluation metrics that capture how well these are satisfied. Our results compare favorably to a set of competitive baseline algorithms in this context.

A High-Speed Vision System with Multithread Automatic Exposure Control for High-Dynamic-Range Imaging

In this study, we develop a real-time high-frame-rate vision system with frame-by-frame automatic exposure (AE) control that can simultaneously synthesize multiple images with different exposure times into a high-dynamic-range (HDR) image for scenarios with dynamic change in illumination. By accelerating the video capture and processing for time-division multithread AE control at the millisecond level, the proposed system can virtually function as multiple AE cameras with different exposure times. This system can capture color HDR images of 512 × 512 pixels in real time at 500 fps by synthesizing four 8-bit color images with different exposure times at consecutive frames, captured at an interval of 2 ms, with pixel-level parallel processing accelerated by a GPU (Graphic Processing Unit) board. Several experimental results for scenarios with a large change in illumination are demonstrated to confirm the performance of the proposed system for real-time HDR imaging.

Recognition of a Virtual Maze Scene Using Simulated Prosthetic Vision.

A real-time psychological physics experimental platform was built to aid in discovering the best real-time image processing strategy for the limited number of electrodes, and in providing the most useful prosthetic implant visual information. A number of maze pathfinder tasks were performed at low resolution to simulate use by blind individuals in performing daily visual activities. In this study, simple (5 rows × 5 columns), medium (8 × 8), and complex (11 × 11) maze models were created in 3DMAX. The models were constructed by using Unity to build virtual scenes for real-time pixel processing, including binarization, color inversion, and matching to templates with three different resolutions (16 × 16, 24 × 24, 32 × 32). Subjects completed maze pathfinding tasks using 45° and 60° views of the labyrinth at the 32 × 32 resolution to determine the optimal viewing angle. The time required to find the maze entrance and complete the maze were analyzed along with the rate of maze completion (accuracy) at various resolutions. In the first experiment, the average time required to find the entrance and the average maze pathfinding duration were significantly longer at 60°. Therefore, the 45° view provided the best perspective. In the second experiment, the angle was fixed to 45°. As the maze difficulty increased, the average time needed to find the maze entrance decreased, but the average time required to complete the maze increased. When the difficulty of the maze was fixed, the time required to find the maze entrance and solve the maze decreased when the resolution increased. The accuracy with which the maze path was identified increased as well. The average maze pathfinding time at 24 × 24 was significantly less than at 16 × 16. A similar trend was observed when the average maze pathfinding times at 32 × 32 and 24 × 24 were compared. At 32 × 32, the average pathfinding accuracy was 100%. This indicates that 32 × 32 is an effective resolution for maze pathfinding.

Recognition of a Virtual Scene via Simulated Prosthetic Vision.

In order to effectively aid the blind with optimal low-resolution vision and visual recovery training, pathfinding and recognition tests were performed using a simulated visual prosthetic scene. Simple and complex virtual scenes were built using 3DMAX and Unity, and pixelated to three different resolutions (32 × 32, 64 × 64, and 128 × 128) for real-time pixel processing. Twenty subjects were recruited to complete the pathfinding and object recognition tasks within the scene. The recognition accuracy and time required were recorded and analyzed after the trials. In the simple simulated prosthetic vision (SPV) scene, when the resolution was increased from 32 × 32 to 48 × 48, the object recognition time decreased from 92.19 ± 6.97 to 43.05 ± 6.08 s, and the recognition accuracy increased from 51.22 ± 8.53 to 85.52 ± 4.93%. Furthermore, the number of collisions decreased from 10.00 ± 2.31 to 3.00 ± 0.68. When the resolution was increased from 48 × 48 to 64 × 64, the object recognition time further decreased from 43.05 ± 6.08 to 19.46 ± 3.71 s, the recognition accuracy increased from 85.52 ± 4.93 to 96.89 ± 2.06%, and the number of collisions decreased from 3.00 ± 0.68 to 1.00 ± 0.29. In complex scenes, the time required to recognize the room type decreased from 115.00 ± 23.02 to 68.25 ± 17.23 s, and object recognition accuracy increased from 65.69 ± 9.61 to 80.42 ± 7.70% when the resolution increased from 48 × 48 to 64 × 64. When the resolution increased from 64 × 64 to 128 × 128, the time required to recognize the room type decreased from 68.25 ± 17.23 to 44.88 ± 9.94 s, and object recognition accuracy increased from 80.42 ± 7.71 to 85.69 ± 7.39%. Therefore, one can conclude that there are correlations between pathfinding and recognition. When the resolution increased, the time required for recognition decreased, the recognition accuracy increased, and the number of collisions decreased. Although the subjects could partially complete the recognition task at a resolution of 32 × 32, the recognition time was too long and recognition accuracy was not good enough to identify simple scenes. Complex scenes required a resolution of at least 48 × 48 for complete recognition. In addition, increasing the resolution shortened the time required to identify the type of room, and improved the recognition accuracy.

MoEDAL – a new light on the high-energy frontier

In 2010, the MoEDAL (MOnopole and Exotics Detector at the LHC) experiment at the Large Hadron Collider (LHC) was unanimously approved by European Centre for Nuclear Research’s Research Board to start data taking in 2015. MoEDAL is a pioneering experiment designed to search for highly ionising manifestations of new physics such as magnetic monopoles or massive (pseudo-)stable charged particles. Its groundbreaking physics programme defines a number of scenarios that yield potentially revolutionary insights into such foundational questions as: are there extra dimensions or new symmetries; does magnetic charge exist; what is the nature of dark matter; and, how did the Big Bang develop. MoEDAL’s purpose is to meet such far-reaching challenges at the frontier of the field. The innovative MoEDAL detector employs unconventional methodologies tuned to the prospect of discovery physics. The largely passive MoEDAL detector, deployed at Point 8 on the LHC ring, has a dual nature. First, it acts like a giant camera, comprised of nuclear track detectors – analysed offline by ultra fast scanning microscopes – sensitive only to new physics. Second, it is uniquely able to trap the particle messengers of physics beyond the Standard Model for further study. MoEDAL’s radiation environment is monitored by a state-of-the-art real-time TimePix pixel detector array. A new MoEDAL sub-detector designed to extend MoEDAL reach to mini-charged, minimally ionising particles is under study.

Real-time pixel based early apple bruise detection using short wave infrared hyperspectral imaging in combination with calibration and glare correction techniques

High speed data processing for online food quality inspection using hyperspectral imaging (HSI) is challenging as over hundred spectral images have to be analyzed simultaneously. In this study, a real-time pixel based early apple bruise detection system based on HSI in the shortwave infrared (SWIR) range has been developed. This systems consists of a novel, homogeneous SWIR illumination unit and a line scan camera. The system performance was tested on Jonagold apples bruised less than two hours before scanning. Partial least squares-discriminant analysis was used to discriminate bruised pixel spectra from sound pixel spectra. As the glossiness of many fruit and vegetables limits the accuracy in the detection of defects, several reflectance calibrations and pre-processing techniques were compared for glare correction and maximizing the signal to noise ratio. With the best combination of first derivative and mean centering, followed by image post-processing, this system was able to detect fresh bruises in thirty apples with 98% accuracy at the pixel level with a processing time per apple below 200 ms.

The MoEDAL experiment at the LHC. Searching beyond the standard model

MoEDAL is a pioneering experiment designed to search for highly ionizing avatars of new physics such as magnetic monopoles or massive (pseudo-)stable charged particles. Its groundbreaking physics program defines a number of scenarios that yield potentially revolutionary insights into such foundational questions as: are there extra dimensions or new symmetries; what is the mechanism for the generation of mass; does magnetic charge exist; what is the nature of dark matter; and, how did the big-bang develop. MoEDAL’s purpose is to meet such far-reaching challenges at the frontier of the field.The innovative MoEDAL detector employs unconventional methodologies tuned to the prospect of discovery physics. The largely passive MoEDAL detector, deployed at Point 8 on the LHC ring, has a dual nature. First, it acts like a giant camera, comprised of nuclear track detectors - analyzed offline by ultra fast scanning microscopes - sensitive only to new physics. Second, it is uniquely able to trap the particle messengers of physics beyond the Standard Model for further study. MoEDAL’s radiation environment is monitored by a state-of-the-art real-time TimePix pixel detector array. A new MoEDAL sub-detector to extend MoEDAL’s reach to millicharged, minimally ionizing, particles (MMIPs) is under study Finally we shall describe the next step for MoEDAL called Cosmic MoEDAL, where we define a very large high altitude array to take the search for highly ionizing avatars of new physics to higher masses that are available from the cosmos.

Real-time pixel luminance optimization for dynamic multi-projection mapping

Using projection mapping enables us to bring virtual worlds into shared physical spaces. In this paper, we present a novel, adaptable and real-time projection mapping system, which supports multiple projectors and high quality rendering of dynamic content on surfaces of complex geometrical shape. Our system allows for smooth blending across multiple projectors using a new optimization framework that simulates the diffuse direct light transport of the physical world to continuously adapt the color output of each projector pixel. We present a real-time solution to this optimization problem using off-the-shelf graphics hardware, depth cameras and projectors. Our approach enables us to move projectors, depth camera or objects while maintaining the correct illumination, in realtime, without the need for markers on the object. It also allows for projectors to be removed or dynamically added, and provides compelling results with only commodity hardware.

Efficient and low-latency pixel data transmission module for adaptive optics wavefront processor based on field-programmable gate array

An efficient and low-latency pixel data transmission module for the adaptive optics wavefront processor is presented. A fiber-based custom real-time pixel data transfer protocol is developed, which has the characteristics of long transmission distance, low-latency, and low-protocol overhead. The hardware part of the suite has been verified on different circuit boards with different field-programmable gate arrays. Obtained results demonstrate that the transmission and protocol processing delay are only 413.5 ns, the transmission bandwidth is 3.125 Gbps, and the error rate is less than 10-12. Under the same conditions of 156.25 MHz clock frequencies and length of transmission line, compared with the serial front panel data port protocol adopted by the thirty meter telescope and the european southern observatory, the transmission delay is significantly reduced by 2.82 times and has a remarkably low logic occupancy rate through our solution. Also, the method has been applied in several actual projects. (C) 2015 Society of Photo-Optical Instrumentation Engineers (SPIE)

Design of Sliding Window Based Corner Detection Algorithm and Architecture for Image Mosaicing

A new Sliding Window Based Corner Detection Algorithm has been developed for image mosaicing. Using this algorithm, a new architecture suitable for VLSI implementation has also been designed. The proposed design incorporates a high degree of pipelining and parallelism and hence offers high throughputs. The color image mosaicing is achieved by first convolving an original image using a 3x3 sliding window followed by smoothening the image using median filters. The design has been coded in Verilog as per RTL coding guidelines. The algorithm has also been coded in Matlab in order to validate the hardware results. The Verilog design of the proposed architecture for image mosaicing has been implemented on Xilinx Spartan 6 xc6slx45-3fgg676 FPGA device. The design utilizes about 144,858 gates and the operating frequency is about 100 MHz. The design is capable of processing high resolution color pictures of sizes of up to 1600× 1200 pixels in real time.

The MoEDAL experiment - a new light on LHC physics

In 2010 the MoEDAL experiment at the Large Hadron Collider (LHC) was unanimously approved by CERN’s Research Board to start data taking in 2015. MoEDAL is a pioneering experiment designed to search for highly ionizing messengers of new physics such as magnetic monopoles or massive (pseudo-)stable charged particles. Its groundbreaking physics program defines a number of scenarios that yield potentially revolutionary insights into foundational questions. MoEDAL’s purpose is to meet such far-reaching challenges at the frontier of the field. The innovative MoEDAL detector is tuned to the prospect of discovery physics. The largely passive MoEDAL detector, deployed at Point 8 on the LHC ring, has a dual nature. First, it acts like a giant camera, comprised of nuclear track detectors - analyzed offline by ultra fast scanning microscopes – sensitive only to new physics. Second, it is uniquely able to trap the particle harbingers of new physics beyond the Standard Model for further study. MoEDAL’s radiation environment is monitored by a state-of-the-art real-time TimePix pixel detector array.

A Multi-Core FPGA-Based 2D-Clustering Implementation for Real-Time Image Processing

A multi-core FPGA-based 2D-clustering implementation for real-time image processing is presented in this paper. The clustering algorithm is using a moving window technique to reduce the time and data required for the cluster identification process. The implementation is fully generic, with an adjustable detection window size. A fundamental characteristic of the implementation is that multiple clustering cores can be instantiated. Each core can work on a different identification window that processes data of independent “images” in parallel, thus, increasing performance by exploiting more FPGA resources. The algorithm and implementation are developed for the Fast TracKer processor for the trigger upgrade of the ATLAS experiment but their generic design makes them easily adjustable to other demanding image processing applications that require real-time pixel clustering.

Modulation Method Including Noise Model for Minimizing the Wiggling Error of TOF Cameras

Registered depth and intensity data at a high frame rate, a compact design, low weight, and a reduced power consumption have motivated the increasing usage of lock-in time-of-flight (TOF) cameras in research areas such as computer graphics, machine vision, and robotics. The state-of-the-art practice is to use continuous wave modulation with square waves for TOF cameras. Square waves can be easily generated digitally with a high accuracy and stability using programmable logic devices. In this paper, a modulation method based on sine waves is proposed, which significantly reduces the depth distortion offset (wiggling effect) of lock-in TOF cameras. Furthermore, a model for the noise distribution in the depth image is derived. This model can predict the performance of the proposed modulation method for each pixel in real time.

Digitally synthesized beat frequency multiplexing for sub-millisecond fluorescence microscopy

Fluorescence imaging is the most widely used method for unveiling the molecular composition of biological specimens. However, the weak optical emission of fluorescent probes and the tradeoff between imaging speed and sensitivity is problematic for acquiring blur-free images of fast phenomena, such as sub-millisecond biochemical dynamics in live cells and tissues, and cells flowing at high speed. We report a solution that achieves real-time pixel readout rates one order of magnitude faster than a modern electron multiplier charge coupled device (EMCCD) - the gold standard in high-speed fluorescence imaging technology. Deemed fluorescence imaging using radiofrequency-multiplexed excitation (FIRE), this approach maps the image into the radiofrequency spectrum using the beating of digitally synthesized optical fields. We demonstrate diffraction-limited confocal fluorescence imaging of stationary cells at a frame rate of 4.4 kHz, as well as fluorescence microscopy in flow at a throughput of approximately 50,000 cells per second.

적응적 휘도 감소를 이용한 OLED 패널의 저전력 디스플레이 방법 및 하드웨어 구현

OLED has good efficiency of power consumption by having no power consumption from black color as different with LCD. when it has white color, all RGB pixel should be glowing with high power consumption and that can make it has short life time. This paper suggest the way of low power consumption for OLED panel using adaptive luminance enhancement with color compensation and implement it as hardware. This way which is based on luminance information of input image makes converted luminance value from each pixel in real time. There is with using the basic idea of chromaticity reduction algorithm, showing new algorithm of color correction. And performance of proposed method was confirmed by comparing the conventional method in experiments about 48.43% current reduction. The proposed method was designed by Verilog HDL and was verified by using OpenCV and Windows Program.