Single Image Sensor Research Articles

Calibration of Kinect for Xbox One and Comparison between the Two Generations of Microsoft Sensors.

In recent years, the videogame industry has been characterized by a great boost in gesture recognition and motion tracking, following the increasing request of creating immersive game experiences. The Microsoft Kinect sensor allows acquiring RGB, IR and depth images with a high frame rate. Because of the complementary nature of the information provided, it has proved an attractive resource for researchers with very different backgrounds. In summer 2014, Microsoft launched a new generation of Kinect on the market, based on time-of-flight technology. This paper proposes a calibration of Kinect for Xbox One imaging sensors, focusing on the depth camera. The mathematical model that describes the error committed by the sensor as a function of the distance between the sensor itself and the object has been estimated. All the analyses presented here have been conducted for both generations of Kinect, in order to quantify the improvements that characterize every single imaging sensor. Experimental results show that the quality of the delivered model improved applying the proposed calibration procedure, which is applicable to both point clouds and the mesh model created with the Microsoft Fusion Libraries.

Double-field-of-view, quasi-common-path interferometer using Fourier domain multiplexing.

We present a quasi-common-path interferometer with a double field of view (FOV). The laser beam of an imaging system is separated into three parts using three mirrors; the first and second beams are used to image two different areas of a sample, while the third beam functions as a reference beam. The reference beam is prepared by making clear area in a sample and projecting it on an image sensor. A double FOV is obtained by Fourier domain multiplexing, whereby two interferometric images corresponding to two different areas of a sample are modulated with two different spatial carrier frequencies. The feasibility of this technique is experimentally demonstrated by imaging two different areas of a test target with a single image sensor.

Detecting lines and building intersection correspondences by computing edge oriented histogram on multi-sensor images

This paper proposes an approach to establishing correspondences for line intersections on multi-sensor images containing lines. Keypoints have been widely applied in a variety of computer vision fields. On multi-sensor images the number of keypoints may be much smaller than on single-sensor images due to the lack of texture. In addition, the multimodality often causes the incorrect assignment of main orientation to keypoints. Observing this, this paper proposes extracting line intersections as keypoints and utilizing one line as the main orientation to compute edge oriented histogram (EOH) descriptor. For EOH descriptor, gradient orientation is employed to compute the filter responses. Experimental results show that the proposed method can build more reliable keypoint matchings on challenging image pairs such as visible image and middle/long-wave infrared images.

Wavelet and Curvelet Transformation based Image Fusion with ANFIS and SVM

The Image fusion is a data fusion innovation which keeps images as main research substance which refers to the strategies that integrate multi-images of the same scene from multiple image sensor data or integrate multi images of the same scene at different times from single image sensor. In this paper we describes a novel image fusion method, is suitable for pan-sharpening of multispectral (MS) bands which are based on multi-resolution analysis. The low-resolution MS bands are sharpened by injecting high-pass directional details extracted from the high-resolution panchromatic (Pan) image by means of the Wavelet and Curvelet transform, which is a non-separable MRA, whose basis function are directional edges with progressively increasing resolution. We introduce a new method based on the Wavelet and Curvelet transform using Neural Network which represents edges better than wavelets in this paper. Therefore, edges play a fundamental role in image understanding and one important way to enhance spatial resolution is to enhance the edges. Wavelet and Curvelet-based image fusion method provides richer information in the spatial and spectral domains simultaneously General Terms Image Fusion, Image processing, pattern recognition

A practical one-shot multispectral imaging system using a single image sensor.

Single-sensor imaging using the Bayer color filter array (CFA) and demosaicking is well established for current compact and low-cost color digital cameras. An extension from the CFA to a multispectral filter array (MSFA) enables us to acquire a multispectral image in one shot without increased size or cost. However, multispectral demosaicking for the MSFA has been a challenging problem because of very sparse sampling of each spectral band in the MSFA. In this paper, we propose a high-performance multispectral demosaicking algorithm, and at the same time, a novel MSFA pattern that is suitable for our proposed algorithm. Our key idea is the use of the guided filter to interpolate each spectral band. To generate an effective guide image, in our proposed MSFA pattern, we maintain the sampling density of the G -band as high as the Bayer CFA, and we array each spectral band so that an adaptive kernel can be estimated directly from raw MSFA data. Given these two advantages, we effectively generate the guide image from the most densely sampled G -band using the adaptive kernel. In the experiments, we demonstrate that our proposed algorithm with our proposed MSFA pattern outperforms existing algorithms and provides better color fidelity compared with a conventional color imaging system with the Bayer CFA. We also show some real applications using a multispectral camera prototype we built.

Proof-of-concept and evaluation of a dual function visible/NIR camera for iris authentication in smartphones

As a robust method of person authentication, iris biometrics is making its way in to consumer devices such as smartphones. Current iris image acquisition devices typically work under controlled environment and constrained acquisition conditions. In this paper, the adaption of iris biometrics for unconstrained, hand-held devices such as smartphones is investigated. A prototype device is presented with full system description. This device is equipped with a single image sensor with both visible and NIR sensing capabilities. The device is analysed in terms of its optical properties and iris imaging capabilities. Preliminary results indicate that there are challenges to achieve a reliable recognition performance from the images captured using this device. Current system acquires images with marginal optical quality and spatial resolution in an unconstrained acquisition scenario for iris recognition. Nevertheless, the analyses presented in this paper indicate a similar camera module with improved optics and sensor could combine iris biometrics with conventional front camera functions such as video call and the capture of selfie images.

Performance improvement of indoor positioning using light-emitting diodes and an image sensor for light-emitting diode communication

Light-emitting diodes (LEDs) are expected to replace existing lighting technologies in the near future because of the potential dual function of LED light (i.e., wireless communication and lighting) in the context of visible light communication (VLC). We propose a highly precise indoor positioning algorithm using lighting LEDs, an image sensor, and VLC. In the proposed algorithm, three LEDs transmit their three-dimensional coordinate information, which is received and demodulated by a single image sensor at an unknown position. The unknown position is then calculated from the geometrical relations of the LED images created on the image sensor plane. We describe the algorithm in detail. A simulation of the proposed algorithm is presented in this paper. We also compare the performance of this algorithm with that of our previously proposed algorithm. The comparison indicates significant improvement in positioning accuracy because of the simple algorithmic structure and low computational complexity. This technique does not require any angular measurement, which is needed in the contemporary positioning algorithms using LEDs and image sensor. The simulation results show that the proposed system can estimate the unknown position to an accuracy of 0.001 m inside the approximate positioning area when the pixel value is >3000.

A Study on Various Color Filter Array based Techniques

A demosaicing algorithm is just a digital image process used to reconstruct full color image from the incomplete color samples output from a picture sensor overlaid with a color filter array (CFA). It is also known as CFA interpolation or color reconstruction. Most contemporary digital camera models acquire images using a single image sensor overlaid with a CFA, so demosaicing is the main processing pipeline required to render these images into a viewable format. Many modern digital camera models can save images in a natural format allowing the consumer to demosaicit using software, as opposed to utilizing the camera's built-in firmware. Thus demosaicing becomes and major area of research in vision processing applications. The key objective of the paper is to examine and analyze various image demosaicing techniques. The entire aim would be to explore various limitations of the earlier techniques. This paper ends up with the suitable gaps in earlier techniques.

English

Image fusion has found many applications in computer vision, remote sensing, intelligent robots and military purposes. The use of different image fusion algorithms gives precise resultant images.In many remote sensing applications, the quantity of image data from the satellite sensors has been increasing because of advanced sensor technology.To avoid the limitations of single sensor images, multisensory image fusion provides the data that is suitable for further applications by eliminating the problem of lack of information. In this paper, a literature review has been made based on different techniques for combining multispectral images available. It includes IHS transform, High Pass filtering, PCA analysis, ANN, Wavelet transform and DCT. One of the effective techniques to get a good quality image is by using the Fuzzylet Fusion Algorithm in which the advantages of both Stationary Wavelet Transform and Fuzzy logic are combined.

Sequential filtering for color image acquisition.

We present an alternative method of color imaging. It utilizes a single standard Si-based image sensor and a clear aperture, tunable, color filter. Within this scheme, three monochromatic frames are taken; each is acquired with a single color and with a fraction of the total acquisition duration. This scheme offers several advantages compared to standard Bayer-demosaicing schemes. Among them - significantly higher resolution, flexibility in exposure durations, improved noise performance and in sum - better image quality. This paper presents both color filter design and the adjusted image pipeline. Lastly, the proposed imaging scheme and the Bayer scheme are compared in terms of MSE.

육각형 구조의 베이어 컬러 필터 배열에 대한 디모자익킹

This paper describes a demosaicking method for hexagonally-structured color filter array. Demosaicking is essential to acquire color images using color filter array (CFA) in single sensor imaging. Thus, CFA patterns have been discussed in order to improve image quality in single sensor imaging after the Bayer pattern are introduced. Advancements in imaging sensor technology recently introduce a hexagonal CFA pattern. The hexagonal CFA can be considered to be a 45-degree rotational version of the Bayer pattern, thus demosaicking can be implemented by an existing method with backward and forward 45-degree rotations. However, this approach requires heavy computing power and memory in image sensing devices because of the image rotations. To overcome this problem, we proposes a demosaicking method for a hexagonal Bayer CFA without rotations. In addition, we introduce a weighting parameter in our demosaicking method to improve image quality and to unifying exiting method with our method. Experimental results indicate that the proposed method is superior to conventional methods in terms of PSNR. In addition, some optimized values for the weighting parameter are provided experimentally.

UDP – a New Universal Demosaicing Pipeline Algorithm for RGB Color Filter Arrays

Abstract A new universal demosaicing method suitable for imaging pipelines using a RGB color filter array (CFA) is proposed. The proposed algorithm can reconstruct single-sensor images registered by a digital camera equipped with any RGB CFA currently in use. The algorithm includes a spatio-spectral interpolation model, a new edge-detection technique, and a post processing to reduce the color shifts and artifacts of the reconstructed image. Experimental results over various test CFA patterns and real images demonstrate the efficiency and feasibility of a universal demosaicing algorithm proposed in the paper. Two methods - objective metrics (mean squared error - MSE and peak signal to noise ratio - PSNR), and subjective evaluation are used to analyze the results

A new tracking approach for visible and infrared sequences based on tracking-before-fusion

Traditional target tracking algorithms based on single sensor images are not applicable in some circumstances such as cross-background, similar appearance object, shadow and occlusion with nonlinear and random characteristic. Previous image fusion tracking approach has requirement for precise registration. The traditional fusion-before-tracking strategy suffers from several serious drawbacks such as mutual influence between different single sensor resulting in the loss of the own advantages. The current proposed approach using tracking-before-fusion strategy operates in five ways. Firstly, the specified target is tracked using improved particle filter algorithm in visible sequence. Secondly, tracking in infrared sequence is based on improved template matching scheme. Thirdly, an un-registered treatment scheme is applied for un-registered sequences. Fourthly, the tracking results from visible and infrared sequences are fused by the new fusion rule. Finally, each result is fed back and updated for tracking improvement. The tracking experiments indicate that the proposed approach has higher efficient, robustness and real-time capability.

Digital sensing and sizing of vesicular stomatitis virus pseudotypes in complex media: a model for Ebola and Marburg detection.

Rapid, sensitive, and direct label-free capture and characterization of nanoparticles from complex media such as blood or serum will broadly impact medicine and the life sciences. We demonstrate identification of virus particles in complex samples for replication-competent wild-type vesicular stomatitis virus (VSV), defective VSV, and Ebola- and Marburg-pseudotyped VSV with high sensitivity and specificity. Size discrimination of the imaged nanoparticles (virions) allows differentiation between modified viruses having different genome lengths and facilitates a reduction in the counting of nonspecifically bound particles to achieve a limit-of-detection (LOD) of 5 × 10(3) pfu/mL for the Ebola and Marburg VSV pseudotypes. We demonstrate the simultaneous detection of multiple viruses in a single sample (composed of serum or whole blood) for screening applications and uncompromised detection capabilities in samples contaminated with high levels of bacteria. By employing affinity-based capture, size discrimination, and a "digital" detection scheme to count single virus particles, we show that a robust and sensitive virus/nanoparticle sensing assay can be established for targets in complex samples. The nanoparticle microscopy system is termed the Single Particle Interferometric Reflectance Imaging Sensor (SP-IRIS) and is capable of high-throughput and rapid sizing of large numbers of biological nanoparticles on an antibody microarray for research and diagnostic applications.

Low Light CMOS Contact Imager with Integrated Dual Band Emission Filters for Fluorescence Detection

In recent times, the scope of optical imaging systems has been widened substantially to address the need for inexpensive, portable and miniaturized sensory system for applications such as in situ biomedical and environmental monitoring. Contact imaging is a relatively new and low-cost technique that offers significant advantages over conventional imaging. Conventional imaging systems often require bulky and expensive magnifying objectives along with photodetector devices. In contrast, contact imaging lacks the need of optical elements such as lenses between the sample and the sensor array, and therefore provides better light collection efficiency without optical loss and a significant improvement in sensitivity. Such unique features are achieved by contact imaging systems mainly due to direct coupling of the sensor array with the sample of interest via a thin-film filter.Fluorescence imaging is a powerful and highly sensitive tool that is widely used in diverse areas including diagnostic purposes, bioassays, pharmaceutical research and sensor development. Recently, contact image sensors fabricated using complementary metal-oxide semiconductor (CMOS) technology have been foreseen as an alternative to conventional fluorescence detection systems. Owing to the ability of CMOS imagers to integrate possibly all functions required for timing, exposure control, analog-to-digital (ADC) conversion, color processing, image enhancement and image compression on the same platform, they are considered key components of future micro-total-analysis-system (µTAS). CMOS imagers consume low power and require low voltage as well as facilitate a vast range of flexibility. However, fabrication of a high quality emission filter, also known as absorption filter, remains one of the major challenges in developing high sensitive CMOS-based fluorescence detection system. In particular, the integration of a CMOS image sensor with different types of filters that possess discrete cut-off wavelengths has so far been unexplored.Here, we demonstrate the fabrication of polyvinyl acetate (PVAc) based two distinct emission filters integrated with a single CMOS contact imager for fluorescence detection of two different analytes. PVAc was used as an adhesive for the emission filter due to its optical transparency, inexpensiveness and biocompatiblity properties. The fabrication process of emission filters involve dissolving PVAc in methanol, mixing it with absorbing specimens and finally drying up in air to harden the filter. Two distinct emission filters that possess discrete spectra are integrated onto a single CMOS image sensor. The integrated sensory system thus incorporates dual bandpass emission filters that are capable of selectively isolating fluorescence emission from two probes simultaneously. The thickness of the filters is optimized by calculating the desired signal-to-noise ratio (SNR) using Beer-Lambert’s law for liquids, Quantum Yield of the fluorophore and the Quantum Efficiency of the sensor array. Finally, the efficacy of the sensory system is tested by examining two different analytes i.e., calcium and pH. Overall, CMOS image sensor that combines dual bandpass emission filters holds much potential for low-cost and on-site deployable sensor development.

Dual-coded compressive hyperspectral imaging

This Letter presents a new snapshot approach to hyperspectral imaging via dual-optical coding and compressive computational reconstruction. We demonstrate that two high-speed spatial light modulators, located conjugate to the image and spectral plane, respectively, can code the hyperspectral datacube into a single sensor image such that the high-resolution signal can be recovered in postprocessing. We show various applications by designing different optical modulation functions, including programmable spatially varying color filtering, multiplexed hyperspectral imaging, and high-resolution compressive hyperspectral imaging.

A sharp edge-preserving joint color demosaicking and zooming algorithm using integrated gradients and an iterative back-projection technique

Following the advances in single-sensor imaging techniques, interest in producing a zoomed full-color image from a Bayer mosaic data has been increased. Almost all of the recent approaches identified, with respect to the demosaicking step in the imaging pipeline, have chiefly focused on misguidance problems. However, in regions consisting of sharp edges or fine textures, these approaches are prone to large blurring effects. This paper proposes a new joint solution to overcome the above problems associated with demosaicking and zooming operations. On the basis of an enhanced soft-decision framework, we estimate the edge features by computing the integrated gradients. This allows the extraction of gradient information from both color intensity and color difference domains, simultaneously. Then, the edge guidance is incorporated in the interpolation of various stages to preserve edge consistency and improve computational efficiency. In addition, an edge-adaptive, iterative, back-projection technique is developed to compensate for image blurring as well as to further suppress color artifacts. Experimental results indicate that the new algorithm produces outstanding objective performances and sharp, visually pleasing color outputs, when compared to numerous other single-sensor image zooming solutions.

An Interferometric Reflectance Imaging Sensor for Point of Care Viral Diagnostics

The use of in vitro diagnostic devices is transitioning from the laboratory to the primary care setting to address early disease detection needs. Time critical viral diagnoses are often made without support due to the experimental time required in today's standard tests. Available rapid point of care (POC) viral tests are less reliable, requiring a follow-on confirmatory test before conclusions can be drawn. The development of a reliable POC viral test for the primary care setting would decrease the time for diagnosis leading to a lower chance of transmission and improve recovery. The single particle interferometric reflectance imaging sensor (SP-IRIS) has been shown to be a sensitive and specific-detection platform in serum and whole blood. This paper presents a step towards a POC viral assay through a SP-IRIS prototype with automated data acquisition and analysis and a simple, easy-to-use software interface. Decreasing operation complexity highlights the potential of SP-IRIS as a sensitive and specific POC diagnostic tool. With the integration of a microfluidic cartridge, this automated instrument will allow an untrained user to run a sample-to-answer viral assay in the POC setting.

Infrared Image and Visible Image Fusion Based on Wavelet Transform

The same scene, the infrared image and visible image fusion can concurrently take advantage of the original image information can overcome the limitations and differences of a single sensor image in terms of geometric, spectral and spatial resolution, to improve the quality of the image , which help to locate, identify and explain the physical phenomena and events. Put forward a kind of image fusion method based on wavelet transform. And for the wavelet decomposition of the frequency domain, respectively, discussed the principles of select high-frequency coefficients and low frequency coefficients, highlight the contours of parts and the weakening of the details section, fusion, image fusion has the characteristics of two or multiple images, more people or the visual characteristics of the machine, the image for further analysis and understanding, detection and identification or tracking of the target image.

Multiwavelength parallel phase-shifting digital holography using angular multiplexing

We propose a single-shot digital holography for recording multiwavelength and complex amplitude information by using a single monochromatic image sensor. The zeroth-order wave and conjugate image in each wavelength are removed from a recorded single hologram by applying parallel phase-shifting interferometry. Angular multiplexing is utilized to record the complex amplitude of an object wave in each wavelength separately, and no color filter is required. The effectiveness of the proposed technique was experimentally verified.