Unknown Pixels Research Articles

Efficient Image Inpainting Using Dimensional Space Reduction via Adaptive Patch-based Concept and Rank Lowering Technique

In this paper, we present a new algorithm for image inpainting using low dimensional feature space. In our method, projecting a low dimensional space from the original space is accomplished firstly using SVD, which is named low rank component, and then the missing pixels are filled in the new space. Finally, the original image is inpainted so that adaptive patch size is considered by quad-tree based on the previous step. In our algorithm, the missing pixels in the target region are estimated twice, one in low dimension feature space and another in the original space. It is noticeable that both processes estimate the unknown pixels using patch-based idea and rank lowering concept. Experimental results of this algorithm show better consistency in comparison with state-of-the-art methods.

Improvement of thin retinal vessel extraction using mean matting method

AbstractIn this paper, a new mean matting method based on mean correlation is proposed to extract thin blood vessels precisely. The proposed algorithm is the combination of both supervised and unsupervised method. The supervised methodology performs well in extracting thick blood vessels; however, thin vessels are not precisely extracted. Even the capability of unsupervised method is better in extracting thin vessels; it has some artifacts in the output. The proposed method combines the advantages of both supervised and unsupervised method to extract vessel regions more precisely irrespective of their thickness. Using supervised methodology, thick blood vessels are extracted by training the classifier with the significant features describing the vessel regions. Trimap is generated on the unsupervised output and mean correlation is computed for all unknown pixels in the trimap to classify those pixels into vessels or background. The proposed matting method has less computational complexity compared to other existing matting methods. The performance of the proposed method is evaluated in detail on DRIVE and STARE datasets.

Susceptibility to Soil Erosion and Risk Assessment at Hilly Farms Using Geospatial Techniques

Soil erosion is a serious environmental challenge which persistently diminishes available land resources globally.The impact of soil erosion is more severeat hilly regions where various techniques are deployed to evaluateits risk levels. However, the traditional approach of estimating the magnitude of erosion is tedious, costly, and considerably time consuming. This study was intended to assess the risk associated with soil erosion at hilly areas of Cameron Highlands through geospatial means. The Digital Elevation Model (DEM) with 5m resolution from Interferometric Synthetic Aperture Radar (IfSAR) was utilized to generate the slope inthe highlands. Soil erosion rates was estimated using Universal Soil Loss Equation (USLE), while information about land use and cover were sourced from relevant government agencies. Inversed Distance Weighted (IDW) method of spatial interpolation was applied to predict the values of unknown pixels. The analysis shows that, there is 217.5 km2of the highlands which is greater than 45-degreeaccounted for about 30.5% of the total land area. Moreover, erosion risk assessment indicated that 66.3%, 11.4%, 11.7% and 10.8% are respectively classified as very low, law, moderate and high susceptible to soil erosion. In general, the risk of soil erosion is relatively low and could be attributed to den sevegetation coverage within the study watershed despite the steep slopes where it was found to be at very high risk to soil erosion susceptibility. However, there is need to deploy best management practices to reduce the effect of soil disturbances at hilly areas and prevent excessive soil loss in future.

Super-resolution of License-plates Using Weighted Interpolation of Neighboring Pixels from Video Frames

Recognizing the license plate from a set of low-resolution video frames using an Optical Character Recognition system (OCR) is a very challenging task. OCR systems fail to properly work in this condition. The use of high-quality cameras is a costly solution to this situation. To overcome this problem, we propose a weighted interpolation method that enhances the resolution of the license plate, using consecutive frames of a video. For this purpose, first, we register the low-resolution video frames of the license plate to the reference license plate in two steps. In the first step, a coarse registration is performed by matching the SURF features. Then a fine registration on the license plate region is performed using the phase correlation technique. After registration, the reference image of the license plate is up-sampled to the desired scale. We propose a method for estimating the intensity of pixels in the up-sampled image with an unknown value. In this method, we use a weighted averaging strategy to estimate the intensity of unknown pixels using the neighboring pixels from video frames. The obtained super-resolution is suitable for OCR. Experimental results show that applying the proposed method on low-resolution frames of the license plate, improves the quality of the license plate significantly.

Image Interpolation via Gradient Correlation-Based Edge Direction Estimation

This paper introduces an image interpolation method that provides performance superior to that of the state-of-the-art algorithms. The simple linear method, if used for interpolation, provides interpolation at the cost of blurring, jagging, and other artifacts; however, applying complex methods provides better interpolation results, but sometimes they fail to preserve some specific edge patterns or results in oversmoothing of the edges due to postprocessing of the initial interpolation process. The proposed method uses a new gradient-based approach that makes an intelligent decision based on the edge direction using the edge map and gradient map of an image and interpolates unknown pixels in the predicted direction using known intensity pixels. The input image is subjected to the efficient hysteresis thresholding-based edge map calculation, followed by interpolation of low-resolution edge map to obtain a high-resolution edge map. Edge map interpolation is followed by classification of unknown pixels into obvious edges, uniform regions, and transitional edges using the decision support system. Coefficient-based interpolation that involves gradient coefficient and distance coefficient is applied to obvious edge pixels in the high-resolution image, whereas transitional edges in the neighborhood of an obvious edge are interpolated in the same direction to provide uniform interpolation. Simple line averaging is applied to pixels that are not detected as an edge to decrease the complexity of the proposed method. Applying line averaging to smooth pixels helps to control the complexity of the algorithm, whereas applying gradient-based interpolation preserves edges and hence results in better performance at reasonable complexity.

Deep learning-based reconstruction of the structure of heterogeneous composites from their temperature fields

Inverse problems involving transport phenomena are ubiquitous in engineering practice, but their solution is often challenging. In this work, we build a data-driven deep learning model to predict the heterogeneous distribution of circle-shaped fillers in two-dimensional thermal composites using the temperature field in the composite as an input. The deep learning model is based on convolutional neural networks with a U-shape architecture and encoding–decoding processes. The temperature field is cast into images of 128 × 128 pixels. When the true temperature at each pixel is given, the trained model can predict the distribution of fillers with an average accuracy of over 0.979. When the true temperature is only available at 0.88% of the pixels inside the composite, the model can predict the distribution of fillers with an average accuracy of 0.94, if the temperature at the unknown pixels is obtained through the Laplace interpolation. Even if the true temperature is only available at pixels on the boundary of the composite, the average prediction accuracy of the deep learning model can still reach 0.80; the prediction accuracy of the model can be improved by incorporating true temperature in regions where the model has low prediction confidence.

Pyramid Matting: A Resource-Adaptive Multi-Scale Pixel Pair Optimization Framework for Image Matting

Image matting is an important problem in computer vision with significant theoretical interest and diverse practical applications, including image/video editing, target tracking, and object recognition. Pixel-pair-optimization-based image matting approaches have been shown very successful in estimating the opacity of the foreground by searching for the best pair of foreground and background pixels for each unknown pixel. However, extant approaches encounter difficulties in adapting to the changes of available computing resources, which limits the application of image matting. This drawback has motivated the present study, as a part of which a multi-scale evolutionary pixel pair optimization framework named pyramid matting framework (PMF) was developed. In this framework, the large-scale pixel pair optimization problem is transformed to multiple pixel pair optimization problems of different scales using image pyramid. The resulting problems are solved level by level, starting from the problem at the small scale. Pixel pair heuristic information obtained from solving low-scale problems are iteratively propagated to the spatially-related pixel pairs in the larger-scale problem. PMF can adapt to changes in available computing resources due to its capability of transforming a small-scale problem solution to the large-scale problem solution through the heuristic information propagation. Experimental results show that the PMF-based image matting approach not only provides high-quality alpha mattes with sufficient computing resources, but also works well when computing resources are scarce.

Spectral–Spatial Hyperspectral Image Classification Using a Multiscale Conservative Smoothing Scheme and Adaptive Sparse Representation

Spatial information has been demonstrated to be useful for hyperspectral images (HSIs) classification. The challenge is that spatial properties are often present at various spatial scales instead of a single fixed scale. A multiscale conservative smoothing algorithm is proposed in this paper to reduce noise and extract spatial structure information from coarse to fine levels. Over-smoothing is prevented automatically by imposing a weighting scheme on the neighboring pixels used for smoothing, where dissimilar neighbors' contributions are suppressed. Motived by multitask learning, an adaptive sparse representation is introduced to integrate different characteristics from the series of enhanced HSIs. The sparse coefficients of a given unknown pixel can be obtained from this representation and then used for classification. Experiments conducted on three benchmark data sets demonstrate that the proposed methodology leads to superior classification performance when compared to several well-known classifiers.

Multiresolution neural networks for tracking seismic horizons from few training images

Detecting a specific horizon in seismic images is a valuable tool for geologic interpretation. Because hand picking the locations of the horizon is a time-consuming process, automated computational methods were developed starting three decades ago. Until now, most networks have been trained on data that were created by cutting larger seismic images into many small patches. This limits the networks ability to learn from large-scale geologic structures. Moreover, currently available networks and training strategies require label patches that have full and continuous horizon picks (annotations), which are also time-consuming to generate. We have developed a projected loss function that enables training on labels with just a few annotated pixels and has no issue with the other unknown label pixels. We use this loss function for training convolutional networks with a multiresolution structure, including variants of the U-net. Our networks learn from a small number of large seismic images without creating patches. Training uses all seismic data without reserving some for validation. Only the labels are split into training/testing. We validate the accuracy of the trained network using the horizon picks that were never shown to the network. Contrary to other work on horizon tracking, we train the network to perform nonlinear regression, not classification. As such, we generate labels as the convolution of a Gaussian kernel and the known horizon locations that communicate uncertainty in the labels. The network output is the probability of the horizon location. We examine the new method on two different data sets, one for horizon extrapolation and another data set for interpolation. We found that the predictions of our methodology are accurate even in areas far from known horizon locations because our learning strategy exploits all data in large seismic images.

Airborne hyperspectral data for mineral mapping in Southeastern Rajasthan, India

Geology majorly deals with the structures, features, minerals, rocks, etc. of the planet Earth. Geologists are using the technology of remote sensing for structural interpretation and regional mapping. Ores and minerals identification are also done with the help of remote sensing. This paper is mainly focused on the identification of minerals and mapping with the help of various algorithms such as Spectral Angle Mapper (SAM), Spectral Feature Fitting (SFF) and Mixture Tuned Matched Filtering (MTMF) using airborne hyperspectral data. Minerals are identified on the basis of the visible and near-infrared spectral reflectance. Spectral reflectance is having absorption features at different positions and absorption peaks are used for the analysis of imagery. This technique provides surface mineralogical details. SAM algorithm mainly computes the angle between the unknown pixel spectrum and unique pixel spectrum. SFF algorithm matched the continuum removed the spectrum of the imagery pixel from the continuum removed reference spectra. MTMF algorithm detects the abundance of the minerals and removes the erroneous positive. Total 13 endmembers (minerals) were identified in the study area. These minerals are grouped into clay minerals, iron minerals, carbonate minerals, and other minerals. These endmembers are used for the mineral map creation from different algorithms. Algorithms produce the diverse kind of mineral map and these are compared with each other on the basis of the mineralogy and discrimination of mineralized region from settlements. Mineral map produced by MTMF algorithm provides convenient results with better accuracy than other algorithms.

Multiobjective Evolutionary Optimization Based on Fuzzy Multicriteria Evaluation and Decomposition for Image Matting

Image matting is evolving for a wide range of applications including image/video editing. Sampling-based image matting aims to estimate the opacity of foreground objects by properly selecting a pair of foreground and background pixels for every unknown pixel. Sampling-based image matting is essentially an uncertain multicriteria optimization problem (UMCOP). It shows unique advantages in parallelization and handling spatially disconnected regions. However, sampling-based approaches encounter difficulty in accurately evaluating pixel pairs and efficiently optimizing the large-scale UMCOP. To address these two problems, a fuzzy multicriteria evaluation (FMCE) and a multiobjective evolutionary algorithm based on multicriteria decomposition (MOEA-MCD) are proposed. We model three fuzzy membership functions for three selection criteria and aggregate them by Einstein and averaging operators providing FMCE for pixel pairs. MOEA-MCD uses the heuristic information for each criterion by multicriteria decomposition that divides the single objective into multiple objectives and optimizes them simultaneously using a multiobjective optimizer with neighborhood grouping strategy. Experimental results show that FMCE accurately evaluates pixel pairs even in uncertain cases with low satisfaction degree of some evaluation criteria, and the heuristic information for each criterion enhances the population diversity of MOEA-MCD. MOEA-MCD outperforms state-of-the-art large-scale optimization approaches and sampling-based image matting approaches.

Multiview‐coherent disocclusion synthesis using connected regions optimization

AbstractHandling of missing areas is a key step for depth‐based rendering to synthesize virtual views. Existing methods usually consider finding candidate pixels from only one reference view to fill missing areas. However, the information provided by one reference view is restricted by the position of the view. By utilizing two reference views located on both the left and right sides of the virtual view, we propose to synthesize the missing areas at hole level with connected regions optimization. To avoid the appearance of ghost boundary, we apply morphological operations to generate a boundary band map for the depth map, which restricts the warping of the virtual view. We use a binary map to mark unknown pixels in a hole, label the connected unknown regions, and count the area of each connected region, which decide the order in our enhanced inpainting synthesis. Besides, we separate the foreground and background regions of the depth map to constrain the searching of candidate pixels. Multiple experiments on virtual view synthesis have shown the effectiveness and high quality of our multiview‐coherent disocclusion synthesis.

Image interpolation using convolutional neural networks with deep recursive residual learning

Recent developments of image super-resolution often utilize the deep convolutional neural network (CNN) and residual learning to relate the observed low-resolution pixels and unknown high-resolution pixels. However, image interpolation assumes that the observed image was directly down-sampled without low-pass filtering, such that the aliased down-sampled low-resolution image exhibits jags and chaos that cannot be easily modeled by conventional residual learning in super-resolution. In this paper, we propose a new framework to exploit the residual dense network using hierarchical levels of recursive residual learning and densely connected convolutional layers for image interpolation. The proposed deep recursive network iteratively reconstructs hierarchical levels of image details for aliased and discontinuous residual of interpolated pixels. Experimental results on popular Set16, Set18, and Urban12 image datasets show that the proposed method outperforms state-of-the-art image interpolation methods using local and nonlocal autoregressive models, random forests and deep CNN, in terms of PSNR (0.27–1.57 dB gain), SSIM and subjective evaluations. More importantly, model parameters of the proposed method are significantly less than that of existing deep CNN for image interpolation.

Image super-resolution for heterogeneous embedded smart devices and displays in smart global village

The smart city has become the key factor of the progression of the future city, and the smart device sensors are widely used in real-time applications to achieve smart city. Especially, super-resolution techniques have resulted in embedded video surveillance turning into a principal method of protecting public security. In this paper, we present a super-resolution method for embedded system which interpolates unknown pixel components at the zero-padded signals. This work is for a smart global village by applying real-time embedded monitoring system (such as traffic purpose) using visual sensor networks. In general, images are obtained from a number of visual sensor nodes and cameras. The visual sensor node conveys them to the computer devices for processing. To this end, a process called image super-resolution is needed. In this work, a filter design method is researched by using a regression approach. There are two main stages of this work. First, we study filter designing method training dataset. Second, we implement super-resolution process by applying pre-designed filters. Experimental results are given to show the superiority of the proposed method compared to the conventional methods. This paper also demonstrates super-resolution results obtained by designed filter and its implementation.

Pixel-Level Discrete Multiobjective Sampling for Image Matting.

In sampling-based matting methods, the alpha is estimated by choosing the best pair of foreground and background color samples. The lack of true samples is the major obstacle in obtaining high-quality alpha mattes. Regrettably, several proposed approaches did not address the conflicts among multiple sampling criteria and the effects of incomplete sample spaces. To address this issue, we propose a pixel-level discrete multiobjective sampling (PDMS) method. The color sampling process at each unknown pixel is formalized as a multiobjective optimization problem (MOP). The strength of PDMS includes its ability to minimize both color difference and spatial distance between unknown and known pixels, and its capacity to adaptively make trade-offs among conflicting sampling criteria. To mitigate the effects of incomplete sample spaces, the sample space is extended to complete known regions in PDMS, which means that the colors of all known pixels can be sampled, instead of mean colors of superpixels. Our experimental results show that PDMS collects a small set of samples while achieving smaller minimum absolute difference in alpha estimation. Moreover, PDMS implements pixel-level sampling by using the proposed multiobjective optimization algorithm to efficiently solve sampling MOPs. The PDMS-based matting method provides high-quality alpha mattes with sharp boundaries and thus outperforms those prior image matting methods in terms of gradient error.

Identification and mapping of winter wheat by integrating temporal change information and Kullback–Leibler divergence

Crop acreage and its spatial distribution are a base for agriculture related works. Current research combining medium and low spatial resolution images focuses on data fusion and unmixing methods. The purpose of the former is to generate synthetic fine spatial resolution data instead of directly solving the problem. In the latter, high-resolution data is only used to provide endmembers and the result is usually an abundance map rather than the true spatial distribution data. To solve this problem, this paper designs a conceptual model which divides the study area into different types of pixels at a MODIS 250 m scale. Only three types of pixels contain winter wheat, i.e., pure winter wheat pixels (PA), the mixed pixels comprising winter wheat and other vegetation (MA) and the mixed pixels comprising winter wheat and other crops (MB). Different strategies are used in processing them. (1) Within the pure cultivated land pixels, the Kullback–Leibler (KL) divergence is employed to analyze the similarity between unknown pixels and the pure winter wheat samples on the temporal change characteristics of NDVI. Further PA is identified. (2) For MA, a proposed reverse unmixing method is firstly used to extract the temporal change information of cultivated land components, after which winter wheat is identified from the cultivated land components as previously described. (3) For MB which only appears on the border of PA, a mask is created by expanding the PA and temporal difference is utilized to identify winter wheat under the mask. Finally, these three results are integrated at a TM scale with the aid of 25 m resolution land use data. We applied the proposed solution and obtained a good result in the main agricultural area of the Yiluo River Basin. The identified winter wheat planting acreage is 161,050.00 hm2. The result is validated based on the five-hundred random validation points. Overall accuracy is 94.80% and Kappa coefficient is 0.85. This demonstrates that the temporal information reflecting crop growth is also an important indicator, and the KL divergence makes it more convenient in identifying winter wheat. This research provided a new perspective for the combination of low and medium spatial resolution remote sensing images. The proposed solution can also be effectively applied in other places and countries for the crop which has a clear temporal change characteristic that is different from others.

Guided depth image reconstruction from very sparse measurements

Depth images captured from modern depth cameras generally suffer from low spatial resolution, noise, and missing regions. These kinds of images cannot be used directly in applications related to depth images, e.g., robot navigation, 3DTV, and augmented reality, which basically need high-resolution input images with no noise o missing regions to function properly. To address the problem of low spatial resolution, noise degradation, and missing regions in depth images, we propose methods based on a guidance color image for depth reconstruction (DR) from sparse depth inputs and depth image super-resolution (SR). We also suggest a scenario wherein these problems can be integrated and addressed simultaneously. Further, we also demonstrate applications of the proposed approach for depth image denoising and depth image inpainting. In our approach, the guidance color image is used for obtaining the segment cues by applying mean-shift (MS) or simple linear iterative clustering (SLIC) segmentation on it. These strong segment cues help in aiding the DR and SR problems by considering the corresponding segments in the input depth image, and estimate the unknown pixels by either plane fitting or median filling approaches. Furthermore, we explore both direct and pyramidal (hierarchical) approaches for SR and DR-SR for higher upsampling factor. As such, our approaches are relatively simpler than some of the contemporary methods, yet the experimental results of the proposed methods show superior performance as compared with some other state-of-the-art DR and SR methods.

Image Matting Based Multi-Focus Image Fusion With Optimal Cluster Size

Image fusion is used to intensify the quality of images by combining two images of same scene obtained from different techniques. The present work deals with the effective extraction of pixel information from the source images that hold the key to multi focus image fusion. A solely vicinity-based image matting algorithm that relies on the close pixel clusters in the input images and their trimap, is presented in this article. The pixel cluster size, N plays a significant role in deciding the identity of the unknown pixel. The distance between each unknown pixel from foreground and background pixel clusters has been computed based on minimum quasi Euclidean distance. The minimum distance ratio gives the alpha value of each unknown pixel in the image. Finally, the focus regions are blend together to obtain the resultant fused image. On perceiving the results visually and objectively, it is concluded that proposed method works better in extracting the focused pixels and improving fusion quality, compared with other existing fusion methods.

Interactive stereo image segmentation via adaptive prior selection

Interactive stereo image segmentation (i.e., cutting out objects from stereo pairs with limited user assistance) is an important research topic in computer vision. Given a pair of images, users mark a few foreground/background pixels, based on which prior models are formulated for labeling unknown pixels. Note that color priors might not help if the marked foreground and background have similar colors. However, integrating multiple types of priors, e.g., color and disparity in segmenting stereo pairs, is not trivial. This is because differing pairs of images and even differing pixels in the same image might require different proportions of the priors. Besides, disparities of natural images are too noisy to be directly used. This paper presents a method that can adaptively determine the proportion of the priors (color or disparity) for each pixel. Specifically speaking, the segmentation problem is defined in the framework of MRF (Markov Random Field). We formulate an MRF energy function which is composed of clues from the two types of priors, as well as neighborhood smoothness and stereo correspondence constraints. The weights of the color and disparity priors at each pixel are treated as variables which are optimized together with the label (foreground or background) of the pixel. In order to overcome the noise problem, the weight of the disparity prior is controlled by a confidence value learned from data. The energy function is optimized by using multi-label graph cut. Experimental results show that our method performs well.

Image Interpolation Scheme based on SVM and Improved PSO

In order to obtain visually pleasing images, a support vector machines (SVM) based interpolation scheme is proposed, in which the improved particle swarm optimization is applied to support vector machine parameters optimization. Training samples are constructed by the pixels around the pixel to be interpolated. Then the support vector machine with optimal parameters is trained using training samples. After the training, we can get the interpolation model, which can be employed to estimate the unknown pixel. Experimental result show that the interpolated images get improvement PNSR compared with traditional interpolation methods, which is agrees with the subjective quality.