Multiple Input Images Research Articles

User-Guided Segmentation of Multi-modality Medical Imaging Datasets with ITK-SNAP.

ITK-SNAP is an interactive software tool for manual and semi-automatic segmentation of 3D medical images. This paper summarizes major new features added to ITK-SNAP over the last decade. The main focus of the paper is on new features that support semi-automatic segmentation of multi-modality imaging datasets, such as MRI scans acquired using different contrast mechanisms (e.g., T1, T2, FLAIR). The new functionality uses decision forest classifiers trained interactively by the user to transform multiple input image volumes into a foreground/background probability map; this map is then input as the data term to the active contour evolution algorithm, which yields regularized surface representations of the segmented objects of interest. The new functionality is evaluated in the context of high-grade and low-grade glioma segmentation by three expert neuroradiogists and a non-expert on a reference dataset from the MICCAI 2013 Multi-Modal Brain Tumor Segmentation Challenge (BRATS). The accuracy of semi-automatic segmentation is competitive with the top specialized brain tumor segmentation methods evaluated in the BRATS challenge, with most results obtained in ITK-SNAP being more accurate, relative to the BRATS reference manual segmentation, than the second-best performer in the BRATS challenge; and all results being more accurate than the fourth-best performer. Segmentation time is reduced over manual segmentation by 2.5 and 5 times, depending on the rater. Additional experiments in interactive placenta segmentation in 3D fetal ultrasound illustrate the generalizability of the new functionality to a different problem domain.

Fusion of Color Images Based on Fuzzy Transform and Spatial Frequency

Multiple images of a scene, captured using different imaging devices and containing complementary information, are required to be combined into a single fused image. The fused image, so obtained, should ideally contain all important features of individual input images. Further, color image processing is also gaining importance in most of the real-life applications. Color images provide better visual information and are highly suitable for human visual perception. Motivated by the gaining importance of color image processing, fusion of color images is proposed. The fusion algorithm is proposed in HSV color space where the luminance components of multiple input images, based on fuzzy transform and spatial frequency, are fused. The chrominance information of input images provides color to the fused image. The fused image contains all important information and possesses natural color appearance. Experiments performed on various pairs of visible–infrared and multifocus images demonstrate the superiority of the proposed algorithm over recent state-of-the-art image fusion algorithms.

Super-resolution for biometrics: A comprehensive survey

The lack of resolution of imaging systems has critically adverse impacts on the recognition and performance of biometric systems, especially in the case of long range biometrics and surveillance such as face recognition at a distance, iris recognition and gait recognition. Super-resolution, as one of the core innovations in computer vision, has been an attractive but challenging solution to address this problem in both general imaging systems and biometric systems. However, a fundamental difference exists between conventional super-resolution motivations and those required for biometrics. The former aims to enhance the visual clarity of the scene while the latter, more significantly, aims to improve the recognition accuracy of classifiers by exploiting specific characteristics of the observed biometric traits. This paper comprehensively surveys the state-of-the-art super-resolution approaches proposed for four major biometric modalities: face (2D+3D), iris, fingerprint and gait. We approach the super-resolution problem in biometrics from several different perspectives, including from the spatial and frequency domains, single and multiple input images, learning-based and reconstruction-based approaches. Especially, we highlight two special categories: feature-domain super-resolution which performs super-resolution directly on the feature space to purposely improve the recognition performance, and deep-learning super-resolution which discusses the most recent advances in deep learning for the super-resolution task. Finally, we discuss the current and open research challenges and provide recommendations into the future for the improved use of super-resolution with biometrics.

Partitioning Compute Units in CNN Acceleration for Statistical Memory Traffic Shaping

The design complexity of CNNs has been steadily increasing to improve accuracy. To cope with the massive amount of computation needed for such complex CNNs, the latest solutions utilize blocking of an image over the available dimensions and batching of multiple input images to improve data reuse in the memory hierarchy. While there has been numerous works on maximizing data reuse, only a few studies have focused on the memory bottleneck caused by limited bandwidth. Bandwidth bottleneck can easily occur in CNN acceleration as CNN layers have different sizes with varying computation needs and as batching is typically performed over each CNN layer for an ideal data reuse. In this case, the data transfer demand for a layer can be relatively low or high compared to the computation requirement of the layer, and hence temporal fluctuations in memory access can be induced eventually causing bandwidth problems. In this paper, we first show that there exists a high degree of fluctuation in memory access to computation ratio depending on CNN layers and functions in the layer being processed by the compute units (cores), where the units are tightly synchronized to maximize data reuse. Then we propose a strategy of partitioning the compute units where the cores within each partition process a batch of input data synchronously to maximize data reuse but different partitions run asynchronously. As the partitions stay asynchronous and typically process different CNN layers at any given moment, the memory access traffic sizes of the partitions become statistically shuffled. Thus, the partitioning of compute units and asynchronous use of them make the total memory access traffic size be smoothened over time. We call this smoothing statistical memory traffic shaping, and we show that it can lead to 8.0 percent of performance gain on a commercial 64-core processor when running ResNet-50.

A Proposed Futuristic Framework for utilizing image Super Resolution Algorithms over mobile cloud Computing Platform

Super resolution imaging is to get high quality picture details either from multiple input images or from a single image. Currently researchers have developed several super resolution techniques to improve image quality. With evolution of broadband wireless technology and mobile device industry, the Mobile image processing has attracted researchers concern towards itself. In this paper we proposed a framework which focuses on utilizing Mobile Cloud Computing platform for the use of super resolution algorithms. Both super resolution imaging and mobile cloud computing are the most spotlighted research areas nowadays, so we intend to merge them in a way that researchers can get benefit from both of them. Super resolution algorithms (POCS, IBP, MAP, etc) are executed on the Cloud server, the mobile user will only to send the request (low resolution input images) to the cloud server, which will process his request accordingly and then send the high resolution image to the mobile user. The experimental results performed show the realization of our proposed framework.

A fast single-image super-resolution via directional edge-guided regularized extreme learning regression

The goal of super-resolution (SR) is to increase the spatial resolution of a low-resolution (LR) image by a certain factor using either single or multiple LR input images. This paper presents a machine learning-based approach to reconstruct a high-resolution (HR) image from a single LR image. Inspired by the human visual cortex system, which is sensitive to high-frequency (HF) components in an image, we aim to model this concept by training a neural network to estimate the missing HF components that contain structural details. In our method, various directional edge responses at each pixel are considered to obtain more complete HF information and then a regularized extreme learning regression model is trained using a set of LR and HR images. Finally, the trained system is applied to a LR image to generate HR image. The experimental results confirm the effectiveness and efficiency of the proposed scheme in comparison with the state-of-the-art SR methods.

Spotsizer: High-throughput quantitative analysis of microbial growth.

Microbial colony growth can serve as a useful readout in assays for studying complex genetic interactions or the effects of chemical compounds. Although computational tools for acquiring quantitative measurements of microbial colonies have been developed, their utility can be compromised by inflexible input image requirements, non-trivial installation procedures, or complicated operation. Here, we present the Spotsizer software tool for automated colony size measurements in images of robotically arrayed microbial colonies. Spotsizer features a convenient graphical user interface (GUI), has both single-image and batch-processing capabilities, and works with multiple input image formats and different colony grid types. We demonstrate how Spotsizer can be used for high-throughput quantitative analysis of fission yeast growth. The user-friendly Spotsizer tool provides rapid, accurate, and robust quantitative analyses of microbial growth in a high-throughput format. Spotsizer is freely available at https://data.csiro.au/dap/landingpage?pid=csiro:15330 under a proprietary CSIRO license.

Multi-focal image fusion using degree of focus and fuzzy logic

The continuous advancement in the field of imaging sensor necessitates the development of an efficient image fusion technique. The multi-focal image fusion extracts the in-focus information from the source images to construct a single composite image with increased depth-of-field. Traditionally, the information in multi-focal images is divided into two categories: in-focus and out-of-focus data. Instead of using a binary focus map, in this work we calculate the degree of focus for each source image using fuzzy logic. The fused image is then generated based on the weighted sum of this information. An initial focus tri-state map is built for each input image by using spatial frequency and a proposed focus measure named as alternate sum modified Laplacian. For the cases where these measures indicate different source images to contain focused pixel or have equal strength, another focus measure based on sum of gradient is employed to calculate the degree of focus in a fuzzy inference system. Finally, the fused image is computed from the weights determined by the degree of focus map of each image. The proposed algorithm is designed to fuse two source images, whereas fusion of multiple input images can be performed by fusing a source image with the fusion output of the previous input group. The comparison of the proposed method with several transform and pixel domain based techniques are conducted in terms of both subjective visual assessment and objective quantitative evaluation. Experimental results demonstrate that our method can be competitive with or even outperforms the methods in comparison.

Pixel-level image fusion: A survey of the state of the art

Pixel-level image fusion is designed to combine multiple input images into a fused image, which is expected to be more informative for human or machine perception as compared to any of the input images. Due to this advantage, pixel-level image fusion has shown notable achievements in remote sensing, medical imaging, and night vision applications. In this paper, we first provide a comprehensive survey of the state of the art pixel-level image fusion methods. Then, the existing fusion quality measures are summarized. Next, four major applications, i.e., remote sensing, medical diagnosis, surveillance, photography, and challenges in pixel-level image fusion applications are analyzed. At last, this review concludes that although various image fusion methods have been proposed, there still exist several future directions in different image fusion applications. Therefore, the researches in the image fusion field are still expected to significantly grow in the coming years.

Visual parameter optimisation for biomedical image processing.

BackgroundBiomedical image processing methods require users to optimise input parameters to ensure high-quality output. This presents two challenges. First, it is difficult to optimise multiple input parameters for multiple input images. Second, it is difficult to achieve an understanding of underlying algorithms, in particular, relationships between input and output.ResultsWe present a visualisation method that transforms users' ability to understand algorithm behaviour by integrating input and output, and by supporting exploration of their relationships. We discuss its application to a colour deconvolution technique for stained histology images and show how it enabled a domain expert to identify suitable parameter values for the deconvolution of two types of images, and metrics to quantify deconvolution performance. It also enabled a breakthrough in understanding by invalidating an underlying assumption about the algorithm.ConclusionsThe visualisation method presented here provides analysis capability for multiple inputs and outputs in biomedical image processing that is not supported by previous analysis software. The analysis supported by our method is not feasible with conventional trial-and-error approaches.

Removing Camera Shake via Weighted Fourier Burst Accumulation.

Numerous recent approaches attempt to remove image blur due to camera shake, either with one or multiple input images, by explicitly solving an inverse and inherently ill-posed deconvolution problem. If the photographer takes a burst of images, a modality available in virtually all modern digital cameras, we show that it is possible to combine them to get a clean sharp version. This is done without explicitly solving any blur estimation and subsequent inverse problem. The proposed algorithm is strikingly simple: it performs a weighted average in the Fourier domain, with weights depending on the Fourier spectrum magnitude. The method can be seen as a generalization of the align and average procedure, with a weighted average, motivated by hand-shake physiology and theoretically supported, taking place in the Fourier domain. The method's rationale is that camera shake has a random nature, and therefore, each image in the burst is generally blurred differently. Experiments with real camera data, and extensive comparisons, show that the proposed Fourier burst accumulation algorithm achieves state-of-the-art results an order of magnitude faster, with simplicity for on-board implementation on camera phones. Finally, we also present experiments in real high dynamic range (HDR) scenes, showing how the method can be straightforwardly extended to HDR photography.

Multi-scale pixel-based image fusion using multivariate empirical mode decomposition.

A novel scheme to perform the fusion of multiple images using the multivariate empirical mode decomposition (MEMD) algorithm is proposed. Standard multi-scale fusion techniques make a priori assumptions regarding input data, whereas standard univariate empirical mode decomposition (EMD)-based fusion techniques suffer from inherent mode mixing and mode misalignment issues, characterized respectively by either a single intrinsic mode function (IMF) containing multiple scales or the same indexed IMFs corresponding to multiple input images carrying different frequency information. We show that MEMD overcomes these problems by being fully data adaptive and by aligning common frequency scales from multiple channels, thus enabling their comparison at a pixel level and subsequent fusion at multiple data scales. We then demonstrate the potential of the proposed scheme on a large dataset of real-world multi-exposure and multi-focus images and compare the results against those obtained from standard fusion algorithms, including the principal component analysis (PCA), discrete wavelet transform (DWT) and non-subsampled contourlet transform (NCT). A variety of image fusion quality measures are employed for the objective evaluation of the proposed method. We also report the results of a hypothesis testing approach on our large image dataset to identify statistically-significant performance differences.

Comparative Study and Analysis of Medical Image Fusion Techniques

The term fusion that means in general an approach of extraction of information acquired from several domains. Basically, Image Fusion can be defines as a process of combining information from multiple input images in such way that final fused image having good quality information then individual image. Medical imaging field demands images with high resolution and higher information contents for necessary disease diagnosis and visualization. Therefore, in practical scenario we required more complementary information for necessary disease diagnosis purpose. In this paper, Image Fusion techniques are broadly classified as spatial domain fusion techniques and Transform domain fusion techniques. Various techniques in each class are discussed in short description. In this study Performance of various techniques are evaluated using quality measures such as Entropy, Peak Signal to Noise Ratio, Signal to Noise Ratio, and Structural Similarity Index. For analysis purpose one can considered CT and MRI images of same scene. General Terms Medical image fusion, Image fusion Techniques, Fuzzy Inference System

Asymmetric multiple-image hiding using phase retrieval technique based on amplitude- and phase-truncation in fractional Fourier domain

We propose a multiple-image hiding scheme based on the amplitude- and phase-truncation approach, and phase retrieval iterative algorithm in the fractional Fourier domain. The proposed scheme offers multiple levels of security with asymmetric keys. Multiple input images multiplied with random phase masks are independently fractional Fourier transformed with different orders. The individual keys and common keys are generated by using phase and amplitude truncation of fractional spectrum. After using two fractional Fourier transform, the resultant encrypted image is hided in a host image with phase retrieval iterative algorithm. Using the correct universal keys, individual keys, and fractional orders, one can recover the original image successfully. Computer simulation results with four gray-scale images support the proposed method. To measure the validity of the scheme, we calculated the mean square error between the original and the decrypted images. In this scheme, the encryption process and generation of decryption keys are complicated and should be realized using computer. For decryption, an optoelectronic setup has been suggested.

Image Fusion Using Pca in Cs Domain

Compressive sampling (CS), also called Compressed Sensing, has generated a tremendous amount of excitement in the image processing community. It provides an alternative to Shannon/Nyquist sampling when the signal under acquisition is known to be sparse or compressible. In this paper, we propose a new efficient image fusion method for compressed sensing imaging. In this method, we calculate the twodimensional discrete cosine transform of multiple input images, these achieved measurements are multiplied with sampling filter, so compressed images are obtained. we take inverse discrete cosine transform of them. Finally, fused image achieves from these results by using PCA fusion method. This approach also is implemented for multi-focus and noisy images. Simulation results show that our method provides promising fusion performance in both visual comparison and comparison using objective measures. Moreover, because this method does not need to recovery process the computational time is decreased very much.

MULTI TIMES IMAGES FUSION BASED ON WAVELET THEORY

Abstract. The development of new monitoring systems and the increasing interest of researchers in obtaining reliable measurements have leaded to the development of automatic monitoring moving objects. One way to ensure monitoring object is to use multi time's image fusion. Image fusion is a sub area of the more general topic of data fusion. Image fusion can be roughly defined as the process of combining multiple input images into an image, which contains the "relevant" information from the inputs. The aim of image fusion is to integrate complementary and redundant information from multiple images to create a composite that contains a better fused image than any of the individual source images. Main purpose of the former is to increase both the spectral and spatial resolution of images by combining multiple images. In this paper we tried to use this theory for moving object tracking, so with the usage of multi images that are obtained in different times and combination of them with this theory we identify the path of movement of moving object so this result could help us to implement automatic systems that that could monitor objects automatically without human interventation. So in this paper first we will discuss the principal of fusion and its famous method (wavelet theory) and all process that involved for doing a fusion process.

Image encryption based on interference that uses fractional Fourier domain asymmetric keys

We propose an image encryption technique based on the interference principle and phase-truncation approach in the fractional Fourier domain. The proposed scheme offers multiple levels of security with asymmetric keys and is free from the silhouette problem. Multiple input images bonded with random phase masks are independently fractional Fourier transformed. Amplitude truncation of obtained spectrum helps generate individual and universal keys while phase truncation generates two phase-only masks analytically. For decryption, these two phase-only masks optically interfere, and this results in the phase-truncated function in the output. After using the correct random phase mask, universal key, individual key, and fractional orders, the original image is retrieved successfully. Computer simulation results with four gray-scale images validate the proposed method. To measure the effectiveness of the proposed method, we calculated the mean square error between the original and the decrypted images. In this scheme, the encryption process and decryption keys formation are complicated and should be realized digitally. For decryption, an optoelectronic scheme has been suggested.

Frontal face synthesizing according to multiple non-frontal inputs and its application in face recognition

A multi-to-one frontal view face synthesizing strategy, and how it could be utilized to improve traditional face recognition algorithms on pose variant problems, is introduced in this paper. The word multi-to-one means more than one input source images and one output synthetic image, and this is an information selection procedure. Through picking up the gray intensity most similar with that of frontal view face from multiple non-frontal input images, proposed algorithm tries to simulate real natural pose variance of human face. The similarity is evaluated according to the magnitude of non-rigid bending deformation involved during synthesizing, the underlying observation of which is, the more the bending deformation are utilized, the less natural the synthesized image looks like. The specific approach is realized based on Moving Least Squares (MLS). Besides synthesizing frontal faces, our Minimum Bending Synthesizing (MBS) strategy could also be utilized to unify the poses of both gallery and probe images, and hence effectively reduce the influence of variant pose to 2D face recognition. From experiments on the CMU PIE and FERET databases, it could be observed that the frontal view faces synthesized by MBS could effectively approximate the real ground truth frontal faces, and MBS could greatly improve the performance of classic face recognition algorithms, PCA and LDA, on pose variant problems. Apart from specific algorithms, the idea of synthesizing frontal face according to more than one input images, is much valuable as well.

CT-PET weighted image fusion for separately scanned whole body rat.

The limited resolution and lack of spatial information in positron emission tomography (PET) images require the complementary anatomic information from the computed tomography (CT) and/or magnetic resonance imaging (MRI). Therefore, multimodality image fusion techniques such as PET/CT are critical in mapping the functional images to structural images and thus facilitate the interpretation of PET studies. In our experimental situation, the CT and PET images are acquired in separate scanners at different times and the inherent differences in the imaging protocols produce significant nonrigid changes between the two acquisitions in addition to dissimilar image characteristics. The registration conditions are also poor because CT images have artifacts due to the limitation of current scanning settings, while PET images are very blurry (in transmission-PET) and have vague anatomical structure boundaries (in emission-PET). The authors present a new method for whole body small animal multimodal registration. In particular, the authors register whole body rat CT image and PET images using a weighted demons algorithm. The authors use both the transmission-PET and the emission-PET images in the registration process emphasizing particular regions of the moving transmission-PET image using the emission-PET image. After a rigid transformation and a histogram matching between the CT and the transmission-PET images, the authors deformably register the transmission-PET image to the CT image with weights based on the intensity-normalized emission-PET image. For the deformable registration process, the authors develop a weighted demons registration method that can give preferences to particular regions of the input image using a weight image. The authors validate the results with nine rat image sets using the M-Hausdorff distance (M-HD) similarity measure with different outlier-suppression parameters (OSP). In comparison with standard methods such as the regular demons and the normalized mutual information (NMI)-based nonrigid free-form deformation (FFD) registration, the proposed weighted demons registration method shows average M-HD errors: 3.99 ± 1.37 (OSP = 10), 5.04 ± 1.59 (OSP = 20) and 5.92 ± 1.61 (OSP = ∞) with statistical significance (p < 0.0003) respectively, while NMI-based nonrigid FFD has average M-HD errors: 5.74 ± 1.73 (OSP = 10), 7.40 ± 7.84 (OSP = 20) and 9.83 ± 4.13 (OSP = ∞), and the regular demons has average M-HD errors: 6.79 ± 0.83 (OSP = 10), 9.19 ± 2.39 (OSP = 20) and 11.63 ± 3.99 (OSP = ∞), respectively. In addition to M-HD comparisons, the visual comparisons on the faint-edged region between the CT and the aligned PET images also show the encouraging improvements over the other methods. In the whole body multimodal registration between CT and PET images, the utilization of both the transmission-PET and the emission-PET images in the registration process by emphasizing particular regions of the transmission-PET image using an emission-PET image is effective. This method holds promise for other image fusion applications where multiple (more than two) input images should be registered into a single informative image.

A Variational Method for Multiple-Image Blending

The main aim of this paper is to develop an algorithm for blending of multiple images in the image-stitching process. Our idea is to propose a variational method containing an energy functional to determine both a stitched image and weighting mask functions of multiple input images for image blending. The existence of the solution of the proposed energy functional is shown. We also present an alternative minimizing algorithm to solve the proposed model numerically and show the convergence of this algorithm. Experimental results show that the proposed model works effectively and efficiently and that the proposed method is competitive with the tested existing methods under noisy conditions.