Degraded Image Sequences Research Articles

Intelligent Spot Detection for Degraded Image Sequences Based on Machine Vision

Intelligent Spot Detection for Degraded Image Sequences Based on Machine Vision

Annotation and Benchmarking of a Video Dataset under Degraded Complex Atmospheric Conditions and Its Visibility Enhancement Analysis for Moving Object Detection

Detection of moving objects in outdoor environments is an extremely researched topic. However, studies on moving object detection in complex atmospheric/weather conditions are limited, primarily because of the absence of any relevant benchmark dataset. To address this disparity, we introduce a novel benchmark video dataset entitled “Extended Tripura University Video Dataset (E-TUVD)” which is a diverse dataset of complex atmospheric/weather conditions. Currently, E-TUVD is the largest video dataset for moving object detection under degraded atmospheric/weather conditions. The dataset comprises 147 video clips spanning 1-5 minutes in duration of each video clips. Because of the requirement of evaluating any object detection model, this study emphasizes on generation of ground-truth images of salient moving objects on E-TUVD. Using this dataset, we assessed the performance of several state-of-the-art algorithms, considering both the ability to detect moving objects and visibility enhancement under such complex conditions. The method with the best performance was used to investigate the effectiveness of visibility enhancement of atmospheric/weather degraded image sequences for accurate moving object detection. Results and analysis reveal that effective enhancement can significantly improve the ability of detection algorithms under degraded atmospheric/weather conditions to resemble the true properties of moving objects in terms of pixel oriented binary masks.

Removing Atmospheric Turbulence Effects in Unified Complex Steerable Pyramid Framework

Simultaneously removing atmospheric turbulence-induced geometric distortion and blurry degradation is a challenging task. In this paper, we propose an effective method to remove or at least reduce turbulence effects in unified complex steerable pyramid (CSP) framework. The proposed method first decomposes the degraded image sequence by CSP. Then, the local motion and the energy information of the image sequence can be represented by multiscale and multidirectional phases and amplitudes. To mitigate turbulence-induced random oscillation, we use temporal average phase as the initial reference phase. Then, the reference phase is iteratively corrected, using the proposed phase correction method which is capable of correcting the large displacement. To reduce blurry degradation, optimal amplitude selection and fusion methods are proposed to reduce blur variation and CSP reconstruction errors. Finally, the corrected phase and fused amplitude can be synthesized to generate a reconstructed image. To further enhance the image quality, a blind deconvolution approach is adopted to deblur the reconstructed image. Through a variety of experiments on the simulated and real data, experimental results show that the proposed method can effectively alleviate the turbulence effects, recover image details, and significantenhance visual quality.

Diffraction-limited image reconstruction with SURE for atmospheric turbulence removal

In order to remove the atmospheric turbulence effects in long-distance imaging systems, a novel method of diffraction-limited image reconstruction from multiframe image based on Stein’s unbiased risk estimate (SURE) is proposed. A diffraction-limited image reconstruction objective function is designed by combining the temporal and spatial information. The SURE is employed to optimize the objective function. The experimental results on real data demonstrate that a single high-quality image with more details of the scene can be restored from a degraded image sequence. The performance of the proposed algorithm is superior to the method of near-diffraction-limited image reconstruction in visual quality.

잡음이 있고 흐릿한 영상의 블라인드 디컨벌루션을 위한 유전 프로그래밍 기법

영상의 디컨벌루션은 보통 감시 시스템에서 모션 블러 (motion blur)나 초점이 맞지 않아 발생하는 블러 (out-of-focus blur) 문제를 줄이기 위해 전처리 과정에서 사용된다. 본 논문에서는 유전 프로그래밍 (Genetic Programming, GP)에 기반한 새로운 블라인드 영상 디컨벌루션 필터링 방법을 제안한다. GP 진화 과정을 통해 영상 복원을 위한 수학적 표현이 만들어지는데, 이것은 블러 영상의 특징들 사이의 관계를 최적으로 조합하여 원래 화소 값을 복원할 수 있는 추정자 함수가 된다. 이를 위해, 먼저 각 화소의 작은 이웃으로부터 특징 벡터를 만들고 추정자 함수를 학습시키는데, 이러한 GP 진화 과정을 통해 지정한 적합성 기준에 따라 유용한 정보들이 자동으로 조합된다. 개발된 함수는 훼손된 영상의 각 화소에 적용하여 원래의 화소 값을 복원하게 된다. 개발된 함수는 다양한 방법으로 훼손된 영상에 적용하여 실험하였는데, 실험 결과 제안된 방법이 기존 알고리즘에 비해 좋은 결과를 나타내는 것을 알 수 있었다. Usually, image deconvolution is applied as a preprocessing step in surveillance systems to reduce the effect of motion or out-of-focus blur problem. In this paper, we propose a blind-image deconvolution filtering approach based on genetic programming (GP). A numerical expression is developed using GP process for image restoration which optimally combines and exploits dependencies among features of the blurred image. In order to develop such function, first, a set of feature vectors is formed by considering a small neighborhood around each pixel. At second stage, the estimator is trained and developed through GP process that automatically selects and combines the useful feature information under a fitness criterion. The developed function is then applied to estimate the image pixel intensity of the degraded image. The performance of developed function is estimated using various degraded image sequences. Our comparative analysis highlights the effectiveness of the proposed filter.

Vision-based place recognition: how low can you go?

In this paper we use the algorithm SeqSLAM to address the question, how little and what quality of visual information is needed to localize along a familiar route? We conduct a comprehensive investigation of place recognition performance on seven datasets while varying image resolution (primarily 1 to 512 pixel images), pixel bit depth, field of view, motion blur, image compression and matching sequence length. Results confirm that place recognition using single images or short image sequences is poor, but improves to match or exceed current benchmarks as the matching sequence length increases. We then present place recognition results from two experiments where low-quality imagery is directly caused by sensor limitations; in one, place recognition is achieved along an unlit mountain road by using noisy, long-exposure blurred images, and in the other, two single pixel light sensors are used to localize in an indoor environment. We also show failure modes caused by pose variance and sequence aliasing, and discuss ways in which they may be overcome. By showing how place recognition along a route is feasible even with severely degraded image sequences, we hope to provoke a re-examination of how we develop and test future localization and mapping systems.

Genetic programming based blind image deconvolution for surveillancesystems

Image acquisition, segmentation, object detection and tracking are essential parts of surveillance systems. Usually, image filtering approaches are employed as preprocessing step to reduce the effect of motion or out-of-focus blur problem. In this paper, we propose genetic programming (GP) based blind-image deconvolution filter. A GP based numerical expression is developed for image restoration which optimally combines and exploits dependencies among features of the blurred image. In order to develop such function, first, a set of feature vectors is formed by considering a small neighborhood around each pixel. At second stage, the estimator is trained and developed through GP process that automatically selects and combines the useful feature information under a fitness criterion. The developed function is then applied to estimate the image pixel intensity of the degraded images. The performance of filter function is estimated using various degraded image sequences. Our comparative analysis highlight the effectiveness of GP based proposed filter.

Archive Film Defect Detection and Removal: An Automatic Restoration Framework

In this paper, we present an automatic restoration system targeting on dirt and blotches in digitized archive films. The system is composed of mainly two modules: defect detection and defect removal. In defect detection, we locate the defects by combing temporal and spatial information across a number of frames. An HMM is trained for normal observation sequences and then applied within a framework to detect defective pixels. The resulting defect maps are refined in a two-stage false alarm elimination process and then passed over to the defect removal procedure. A labelled (degraded) pixels is restored in a multiscale framework by first searching the optimal replacement in its dynamically generated, random walk based region of candidate pixel-exemplars and then updating all its features (intensity, motion and texture). Finally, the proposed system is compared against state-of-the-art methods to demonstrate improved accuracy in both detection and restoration using synthetic and real degraded image sequences.

Fast and Efficient MRF-Based Detection Algorithm of Missing Data in Degraded Image Sequences

This paper proposes a fast, efficient detection algorithm of missing data (also referred to as blotches) based on Markov Random Field (MRF) models with less computational load and a lower false alarm rate than the existing MRF-based blotch detection algorithms. The proposed algorithm can reduce the computational load by applying fast block-matching motion estimation based on the diamond searching pattern and restricting the attention of the blotch detection process to only the candidate bloch areas. The problem of confusion of the blotches is frequently seen in the vicinity of a moving object due to poorly estimated motion vectors. To solve this problem, we incorporate a weighting function with respect to the pixels, which are accurately detected by our moving edge detector and inputed into the formulation. To solve the blotch detection problem formulated as a maximum a posteriori (MAP) problem, an iterated conditional modes (ICM) algorithm is used. The experimental results show that our proposed method results in fewer blotch detection errors than the conventional blotch detectors, and enables lower computational cost and the more efficient detecting performance when compared with existing MRF-based detectors.

Enhancing a sequence of facial images by combining multiple undersampled and compressed images

AbstractThe enhancement of a sequence of low‐resolution images of a human face into a single higher resolution composite poses a real problem in the fields of forensics and security. This is due to the substantial variations in appearance that faces undergo with changing illumination, orientation, scale, facial expressions and occlusions. In addition, faces often appear small in surveillance imagery because of the wide fields of view that are typically used and the relatively large distance between the cameras and the scene. A device‐independent algorithm for the estimation of an enhanced resolution image from multiple low‐resolution and non‐coplanar compressed images having arbitrary motion is proposed in this paper. The basic algorithm has previously been described but its application to enhancing images of objects, in this case a human face, with a range of scales and orientations is the extension detailed here.The task of obtaining a super‐resolved image from a dynamic, undersampled and degraded image sequence can take advantage of the additional spatio‐temporal data available in the image sequence. In particular, camera and scene motion lead to frames containing similar, but not identical information. While the images may look similar to the human eye, the grey‐scale values of the individual pixels are sufficiently different to allow reconstruction of a super‐resolved image with wider bandwidth than that of any of the individual low‐resolution frames. The arbitrarily distorted images must be warped, firstly, to a common orientation so that a rigorous least squares area‐based matching technique can then compute the registration parameters needed for their accurate combination. The warping is an iterative process relying on manual intervention. Once all the low‐resolution images are brought into registration and comply with pre‐established image correlation criteria, they are combined using the algorithm to recover the desired high‐resolution composite. In addition to improving the spatial resolution of the undersampled sequence, the method may also attenuate compression artefacts.

Reconstruction of degraded image sequences. Application to film restoration

A suitable detection–reconstruction approach is proposed for removing impulsive distortion and other types of deterioration from degraded image sequences. The main application that has motivated this work is the problem of digital film restoration for the movie industry, which has only very recently been explored. Line artifacts, which are prominent degradations in motion picture films, are also considered here. The detection procedure consists of two steps. First, a morphological filter provides impulsive distortions and line scratch candidates. Unlike impulsive distortions, which appear randomly in an image, line artifacts persist in nearby or the same location across several frames. Furthermore, the detection process is complicated by the fact that lines occur as natural part in interesting scenes. Therefore, we add a validation step for separating possible line defects from false detections. It consists in tracking the potential line artifacts over the frames using a Kalman filter. An interpolation technique, dealing with both low and high frequencies around the detected deteriorations, is investigated to achieve a nearly invisible reconstruction of damaged areas.

Evidence for the view that temporospatial integration in vision is temporally anisotropic.

In two experiments low-pass and high-pass spatially filtered versions of a base image were prepared and the effect of the order of delivery of sequences of filtered and base images investigated. A task that required subjects to discriminate 120 ms presentations of a full-bandwidth base image and degraded sequences that contained sets of three different spatially filtered versions, or mixtures of spatially filtered and full-bandwidth versions of the image, were used. Each set of images used in the degraded sequences was presented either so that within the 120 ms presentation window the spatial content swept from low to high spatial frequencies or from high to low. In experiment 1 twenty subjects discriminated between a base image and degraded sequences of an urban scene. Results showed both a significant overall effect of image order, with low-to-high spatial-frequency information delivery being mistaken more often for the full-bandwidth presentation than high-to-low, and that different sets of degraded image sequences varied significantly in the frequency with which they were mistaken for the full-bandwidth presentation. In experiment 2 a base and filtered versions of a human face were used in an identical task with twenty different subjects and a very similar pattern of significant results was obtained, although imposed on a lower overall error frequency than that obtained in experiment 1. It was concluded that the results of both experiments provide evidence for an anisotropic temporospatial integration mechanism in which availability of spatial information in a low-to-high spatial-frequency sequence results in more efficient integration than a high-to-low.