Pixel-wise Error Research Articles

Adversarial Spatio-Temporal Learning for Video Deblurring.

Camera shake or target movement often leads to undesired blur effects in videos captured by a hand-held camera. Despite significant efforts having been devoted to video-deblur research, two major challenges remain: 1) how to model the spatio-temporal characteristics across both the spatial domain (i.e., image plane) and the temporal domain (i.e., neighboring frames) and 2) how to restore sharp image details with respect to the conventionally adopted metric of pixel-wise errors. In this paper, to address the first challenge, we propose a deblurring network (DBLRNet) for spatial-temporal learning by applying a 3D convolution to both the spatial and temporal domains. Our DBLRNet is able to capture jointly spatial and temporal information encoded in neighboring frames, which directly contributes to the improved video deblur performance. To tackle the second challenge, we leverage the developed DBLRNet as a generator in the generative adversarial network (GAN) architecture and employ a content loss in addition to an adversarial loss for efficient adversarial training. The developed network, which we name as deblurring GAN, is tested on two standard benchmarks and achieves the state-of-the-art performance.

A large-area, spatially continuous assessment of land cover map error and its impact on downstream analyses.

Land cover maps increasingly underlie research into socioeconomic and environmental patterns and processes, including global change. It is known that map errors impact our understanding of these phenomena, but quantifying these impacts is difficult because many areas lack adequate reference data. We used a highly accurate, high-resolution map of South African cropland to assess (1) the magnitude of error in several current generation land cover maps, and (2) how these errors propagate in downstream studies. We first quantified pixel-wise errors in the cropland classes of four widely used land cover maps at resolutions ranging from 1 to 100km, and then calculated errors in several representative "downstream" (map-based) analyses, including assessments of vegetative carbon stocks, evapotranspiration, crop production, and household food security. We also evaluated maps' spatial accuracy based on how precisely they could be used to locate specific landscape features. We found that cropland maps can have substantial biases and poor accuracy at all resolutions (e.g., at 1km resolution, up to ∼45% underestimates of cropland (bias) and nearly 50% mean absolute error (MAE, describing accuracy); at 100km, up to 15% underestimates and nearly 20% MAE). National-scale maps derived from higher-resolution imagery were most accurate, followed by multi-map fusion products. Constraining mapped values to match survey statistics may be effective at minimizing bias (provided the statistics are accurate). Errors in downstream analyses could be substantially amplified or muted, depending on the values ascribed to cropland-adjacent covers (e.g., with forest as adjacent cover, carbon map error was 200%-500% greater than in input cropland maps, but ∼40% less for sparse cover types). The average locational error was 6km (600%). These findings provide deeper insight into the causes and potential consequences of land cover map error, and suggest several recommendations for land cover map users.

Error assessment of satellite-derived lead fraction in the Arctic

Abstract. Leads within consolidated sea ice control heat exchange between the ocean and the atmosphere during winter, thus constituting an important climate parameter. These narrow elongated features occur when sea ice is fracturing under the action of wind and currents, reducing the local mechanical strength of the ice cover, which in turn impact the sea ice drift pattern. This creates a high demand for a high-quality lead fraction (LF) data set for sea ice model evaluation, initialization, and for the assimilation of such data in regional models. In this context, an available LF data set retrieved from satellite passive microwave observations (Advanced Microwave Scanning Radiometer – Earth Observing System, AMSR-E) is of great value, which has been providing pan-Arctic light- and cloud-independent daily coverage since 2002. In this study errors in this data set are quantified using accurate LF estimates retrieved from Synthetic Aperture Radar (SAR) images employing a threshold technique. A consistent overestimation of LF by a factor of 2–4 is found in the AMSR-E LF product. It is shown that a simple adjustment of the upper tie point used in the method to estimate the LF can reduce the pixel-wise error by a factor of 2 on average. Applying such an adjustment to the full data set may thus significantly increase the quality and value of the original data set.

Straight to the Source: Detecting Aggregate Objects in Astronomical Images With Proper Error Control

The next generation of telescopes, coming online in the next decade, will acquire terabytes of image data each night. Collectively, these large images will contain billions of interesting objects, which astronomers call sources. One critical task for astronomers is to construct from the image data a detailed source catalog that gives the sky coordinates and other properties of all detected sources. The source catalog is the primary data product produced by most telescopes and serves as an important input for studies that build and test new astrophysical theories. To construct an accurate catalog, the sources must first be detected in the image. A variety of effective source detection algorithms exist in the astronomical literature, but few, if any, provide rigorous statistical control of error rates. A variety of multiple testing procedures exist in the statistical literature that can provide rigorous error control over pixelwise errors, but these do not provide control over errors at the level of sources, which is what astronomers need. In this article, we propose a technique that is effective at source detection while providing rigorous control on sourcewise error rates. We demonstrate our approach with data from the Chandra X-ray Observatory Satellite. Our method is competitive with existing astronomical methods, even finding two new sources that were missed by previous studies, while providing stronger performance guarantees and without requiring costly follow up studies that are commonly required with current techniques.

A study on the impact of AL-FEC techniques on TV over IP Quality of Experience

In this contribution, an evaluation of the effectiveness of Application Layer-Forward Error Correction (AL-FEC) scheme in video communications over unreliable channels is presented. In literature, several AL-FEC techniques for reducing the effect of noisy transmission on multimedia communication have been adopted. Recently, their use has been proposed for inclusion in TV over IP broadcasting international standards. The objective of the analysis performed in this paper is to verify the effectiveness of AL-FEC techniques in terms of perceived Quality of Service (QoS) and more in general of Quality of Experience (QoE), and to evaluate the trade-off between AL-FEC redundancy and video quality degradation for a given packet loss ratio. To this goal, several channel error models are investigated (random i.i.d. losses, burst losses, and network congestions) on test sequences encoded at 2 and 4 Mbps. The perceived quality is evaluated by means of three quality metrics: the full-reference objective quality metric NTIA-VQM combined with the ITU-T Rec. G.1070, the full-reference DMOS-KPN metric, and the pixel-wise error comparison performed by using the PSNR distortion measure. A post-processing synchronization between the original and the reconstructed stream has also been designed for improving the fidelity of the performed quality measures. The experimental results show the effectiveness and the limits of the Application Layer protection schemes.

Modeling the Error Statistics in Support Vector Regression of Surface Temperature From Infrared Data

Land and sea surface temperatures are important input parameters for many hydrological and meteorological models. Satellite infrared remote sensing is an effective tool for mapping these variables on regional and global scales. A supervised approach, based on support vector machines (SVMs), has recently been developed to estimate surface temperature from satellite radiometry. However, in order to integrate temperature estimates into hydrological or meteorological data-assimilation schemes (e.g., in flood-prevention applications), a further critical input is often required in the form of pixelwise error statistics. This information is important because it quantifies inaccuracies in the temperature estimate computed for each pixel. This letter proposes two novel methods to model the statistics of the SVM regression error on a pixelwise basis. Both approaches take into account the nonstationary behavior of the error itself. This problem has been only recently explored in the SVM literature through the use of Bayesian reformulations of SVM regressions. The methods proposed in this letter extend this approach by integrating it with either maximum-likelihood or confidence-interval supervised estimators. In both cases, the goal is improved modeling of the error contribution due to intrinsic random variability in the data (e.g., noise). The methods are experimentally validated on Advanced Very High Resolution Radiometer (AVHRR) and Meteosat Second Generation-Spinning Enhanced Visible and Infrared Imager (MSG-SEVIRI) images.

Non-monotonic behaviour in relaxation dynamics of image restoration

We have investigated the relaxation dynamics of image restoration through a Bayesian approach. The relaxation dynamics is much faster at zero temperature than at the Nishimori temperature where the pixel-wise error rate is minimized in equilibrium. At low temperature, we observed non-monotonic development of the overlap. We suggest that the optimal performance is realized through premature termination in the relaxation processes in the case of the infinite-range model. We also performed Markov chain Monte Carlo simulations to clarify the underlying mechanism of non-trivial behaviour at low temperature by checking the local field distributions of each pixel.

Microarray image compression: SLOCO and the effect of information loss

Microarray image technology is a powerful tool for monitoring the expression of thousands of genes simultaneously. Each microarray experiment produces immense amounts of image data, and efficient storage and transmission require compression that takes advantage of microarray image structure. In this paper we develop a compression scheme for microarray images which can be either lossless or lossy with successive refinements. Existing measures of distortion such as mean squared pixel-wise error and visual fidelity are not appropriate for microarray images. We introduce a new measure of distortion for lossy compression: the sensitivity of microarray information extraction to compression loss. Furthermore, our scheme has a coded data structure that allows fast decoding and reprocessing of image sub-blocks, and includes summary statistics and image segmentation information. The average lossless compression ratio is 1.83:1 for our cDNA test images and 2.43:1 for our inkjet test images, comparable or better than state-of-the-art lossless schemas, yet with additional structure and information. At an average lossy compression ratio of 8:1 for cDNA microarrays, we find that our scheme minimizes the effects of compression loss compared to other algorithms. We show that the variability in differential gene expression levels extracted from lossily vs. losslessly compressed microarray images is less than both the variability between different arrays and the variability between different extraction algorithms. In fact, lossy compression can improve the estimation of gene expression levels for cDNA images.

Two new neural network approaches to two-dimensional CT image reconstruction

Recently, we developed two neural network techniques for reconstructing two-dimensional CT images from a small number of projection data. They are simulated annealing and back propagation reconstruction techniques. The two techniques have been developed independent of each other. In this paper we present a comparative evaluation study on the two techniques. We start with introducing the two new approaches one by one, and then present simulation results. Reconstruction results by a well known conventional method — Algebraic Reconstruction Technique (ART) — is also presented for the sake of comparison. A quantitative evaluation among the three reconstruction methods is presented. A pixel-wise error estimator is used to calculate the overall error in the reconstructed images. The estimator reveals the effectiveness of the new neural network techniques compared to the conventional technique ART.