Recursive Temporal Filtering Research Articles

Solar-blind ultraviolet object detection based on the ultraviolet and visible bispectral airborne imaging system

The solar-blind ultraviolet (UV) and visible (VIS) imaging system provides a valuable tool for search and rescue missions. However, due to atmospheric scattering and absorption effects, the UV images are significantly degraded, even missing the target in some frames. A framework based on a weighted mask, with three schemes suitable for various imaging conditions is proposed. Compared with traditional methods, this framework not only preserves low-intensity target regions but also highlights and tracks any suspicious target. Scheme 1 enhances the signal-to-noise ratio (SNR) by computing the accumulating weights of sequential frames, supporting temporal and average weighting means. The temporal weighting serves as a traditional recursive temporal filtering method, which has an effect similar to that of average weighting. Scheme 2 mitigates small platform drifts by introducing a Kalman filter. Scheme 3 mitigates large platform perturbations by eliminating interference from a moving background, which is achieved by determining the warping relationship from adjacent VIS frames. The experiments are designed to cover as many situations as possible, including low-SNR imaging on a static platform, high-SNR imaging on a flat-flying small drone, and strong/weak complex target imaging on a hovering platform. The experiments assess the proposed methods and validate their predicted performance.

A Novel Video Denoising Method Based on Total Variation and Recursive Temporal Filtering

A Novel Video Denoising Method Based on Total Variation and Recursive Temporal Filtering

A Combination of Spatial and Recursive Temporal Filtering for Noise Reduction when Using Region of Interest (ROI) Fluoroscopy for Patient Dose Reduction in Image Guided Vascular Interventions with Significant Anatomical Motion.

Because x-ray based image-guided vascular interventions are minimally invasive they are currently the most preferred method of treating disorders such as stroke, arterial stenosis, and aneurysms; however, the x-ray exposure to the patient during long image-guided interventional procedures could cause harmful effects such as cancer in the long run and even tissue damage in the short term. ROI fluoroscopy reduces patient dose by differentially attenuating the incident x-rays outside the region-of-interest. To reduce the noise in the dose-reduced regions previously recursive temporal filtering was successfully demonstrated for neurovascular interventions. However, in cardiac interventions, anatomical motion is significant and excessive recursive filtering could cause blur. In this work the effects of three noise-reduction schemes, including recursive temporal filtering, spatial mean filtering, and a combination of spatial and recursive temporal filtering, were investigated in a simulated ROI dose-reduced cardiac intervention. First a model to simulate the aortic arch and its movement was built. A coronary stent was used to simulate a bio-prosthetic valve used in TAVR procedures and was deployed under dose-reduced ROI fluoroscopy during the simulated heart motion. The images were then retrospectively processed for noise reduction in the periphery, using recursive temporal filtering, spatial filtering and a combination of both. Quantitative metrics for all three noise reduction schemes are calculated and are presented as results. From these it can be concluded that with significant anatomical motion, a combination of spatial and recursive temporal filtering scheme is best suited for reducing the excess quantum noise in the periphery. This new noise-reduction technique in combination with ROI fluoroscopy has the potential for substantial patient-dose savings in cardiac interventions.

Curvelet based contrast enhancement in fluoroscopic sequences.

Image guided interventions have seen growing interest in recent years. The use of X-rays for the procedure impels limiting the dose over time. Image sequences obtained thereby exhibit high levels of noise and very low contrasts. Hence, the development of efficient methods to enable optimal visualization of these sequences is crucial. We propose an original denoising method based on the curvelet transform. First, we apply a recursive temporal filter to the curvelet coefficients. As some residual noise remains, a spatial filtering is performed in the second step, which uses a magnitude-based classification and a contextual comparison of curvelet coefficients. This procedure allows to denoise the sequence while preserving low-contrasted structures, but does not improve their contrast. Finally, a third step is carried out to enhance the features of interest. For this, we propose a line enhancement technique in the curvelet domain. Indeed, thin structures are sparsely represented in that domain, allowing a fast and efficient detection. Quantitative and qualitative evaluations performed on synthetic and real low-dose sequences demonstrate that the proposed method enables a 50% dose reduction.

SU‐E‐I‐83: Parallel Programming Upgrades for the Control Acquisition, Processing and Image Display System (CAPIDS) of the Micro Angiographic Fluoroscope (MAF)

Purpose:CAPIDS is a unique software platform designed to control and acquire images from the high‐resolution MAF detector, process them and display them in a clinical environment. The images are then stored for optional playback at a later stage. CAPIDS also acquires and records the exposure parameters from the x‐ray unit. We present new parallel programming modifications using the host computer system's Graphics Processing Unit (GPU) and Central Processing unit (CPU) to improve the system performance for the various MAF imaging tasks.Methods:Multicore CPU's allow for concurrent tasks to be executed at the same time in parallel. During runtime, CAPIDS has three concurrent tasks: image acquisition and processing, image saving, and exposure parameter acquisition. Parallel programming constructs from LabVIEW allow each tasks to be executed on a separate core.GPU's allow for the same task to be performed on independent data sets in parallel. During runtime, all the image processing including flat‐field correction, digital image subtraction, image averaging, and temporal recursive filtering are performed on the GPU.Results:The new version of CAPIDS with all the parallel programming updates was successfully used for the first time to control the MAF, acquire the images, process the images and display the images during an actual clinical intervention. The images were acquired under fluoroscopy, digital subtraction angiography, and roadmap modes.Conclusion:Distributing concurrent tasks to different cores of a multicore CPU results in an efficient utilization of resources, efficient power management and increases operation speed. Use of GPU's for image processing further enhances the speed of operation.Supported by NIH Grant: 2R01EB002873 and an equipment grant from Toshiba Medical Systems Corporation

Dose Reduction in Fluoroscopic Interventions Using a Combination of a Region of Interest (ROI) X-Ray Attenuator and Spatially-Different, Temporally-Variable Temporal Filtering.

A novel dose reduction technique for fluoroscopic interventions involving a combination of a material x-ray region of interest (ROI) attenuator and spatially different, temporally variable ROI temporal recursive filter, was used to guide the catheter to the ROI in three live animal studies, two involving rabbits and one involving a sheep. In the two rabbit studies presented, a catheter was guided to the entrance of the carotid artery. With the added ROI attenuator the image under the high attenuation region is very noisy. By using temporal filtering with a filter weight of 0.6 on previous frames, the noise is reduced. In the sheep study the catheter was guided to the descending aorta of the animal. The sheep offered a relatively higher attenuation to the incident x-rays and thus a higher temporal filter weight of 0.8 on previous frames was used during the procedure to reduce the noise to levels acceptable by the interventionalist. The image sequences from both studies show that significant dose reduction of 5-6 times can be achieved with acceptable image quality outside the ROI by using the above mentioned technique. Even though the temporal filter weighting outside the ROI is higher, the consequent lag does not prevent perception of catheter movement.

A stereoscopic video conversion scheme based on spatio-temporal analysis of MPEG videos

Abstract In this article, an automatic stereoscopic video conversion scheme which accepts MPEG-encoded videos as input is proposed. Our scheme is depth-based, relying on spatio-temporal analysis of the decoded video data to yield depth perception cues, such as temporal motion and spatial contrast, which reflect the relative depths between the foreground and the background areas. Our scheme is shot-adaptive, demanding that shot change detection and shot classification be performed for tuning of algorithm or parameters that are used for depth cue combination. The above-mentioned depth estimation is initially block-based, followed by a locally adaptive joint trilateral upsampling algorithm to reduce the computing load significantly. A recursive temporal filter is used to reduce the possible depth fluctuations (and also artifacts in the synthesized images) resulting from wrong depth estimations. The traditional Depth-Image-Based-Rendering algorithm is used to synthesize the left- and right-view frames for 3D display. Subjective tests show that videos converted by our scheme provide comparable perceived depth and visual quality with those converted from the depth data calculated by stereo vision techniques. Also, our scheme is shown to outperform the well-known TriDef software in terms of human’s perceived 3D depth. Based on the implementation by using “OpenMP” parallel programming model, our scheme is capable of executing in real-time on a multi-core CPU platform.

Integration of Recursive Temporal LMMSE Denoising Filter Into Video Codec

The presence of noise can dramatically affect the efficiency of video compression systems. For performance improvement, most practical video compression systems adopt a denoising filter as a pre-processing module for the video encoder, or as a post-processing module for the video decoder, but the complexity introduced by denoising can be very high. This paper first presents a recursive temporal linear minimum mean squared error (LMMSE) filter for video denoising. Based on the analysis of the hybrid video compression process, two novel schemes are presented, one for video encoding and the other for video decoding, in which the proposed recursive temporal LMMSE filter is seamlessly integrated into the encoding and the decoding processes, respectively. For both of these two schemes, the denoising is implemented with nearly no extra computation introduced. Experimental results validate the effectiveness of the proposed schemes on encoding and decoding noisy video sequences.

Amortized supersampling

We present a real-time rendering scheme that reuses shading samples from earlier time frames to achieve practical antialiasing of procedural shaders. Using a reprojection strategy, we maintain several sets of shading estimates at subpixel precision, and incrementally update these such that for most pixels only one new shaded sample is evaluated per frame. The key difficulty is to prevent accumulated blurring during successive reprojections. We present a theoretical analysis of the blur introduced by reprojection methods. Based on this analysis, we introduce a nonuniform spatial filter, an adaptive recursive temporal filter, and a principled scheme for locally estimating the spatial blur. Our scheme is appropriate for antialiasing shading attributes that vary slowly over time. It works in a single rendering pass on commodity graphics hardware, and offers results that surpass 4x4 stratified supersampling in quality, at a fraction of the cost.

SU‐DD‐A4‐04: Micro Angiographic and Fluoroscopic Real‐Time Image Data Handling Using Parallel Coding Techniques in LabVIEW

Purpose: Multicore programming coding techniques have been designed to meet the high‐speed requirements of a High‐Sensitivity, Micro Angiographic‐Fluoroscope (HSMAF) detector for 30 fps acquisition, image‐processing, display, and rapid frame transfer of high‐resolution, region‐of‐interest (ROI) images. Method and Materials: The HSMAF detector was built by our group using a CsI(Tl) phosphor, a light image intensifier, and a fiber‐optic taper coupled to a charge‐coupled device (CCD) camera which provides real‐time 12 bit, 1k × 1k images capable of greater than 10 lp/mm resolution. A graphical user interface (GUI) was developed to control the system and enable real‐time acquisition, image‐processing, display, and rapid storage of high‐resolution images. To accommodate all these processes working in real‐time, parallel coding methods, such as instruction pipelining and task parallelism, were designed and used to take advantage of the available multicore processors (dual and quad cores). Results: The parallel coding techniques of the GUI can handle radiographic procedures that require on‐the‐fly image processing at 30fps such as Roadmapping, Digital‐Subtraction‐Acquisition (DSA), and rotational DSA. On a 2.4 GHz dual processor a high resolution image is acquired, processed and stored in less than 30ms. Recursive temporal filtering, as well as gain correction can be added to the above procedures and still maintain the required high frame rates when a quad‐core processor is used. Moreover, the overall system design offers virtually unlimited memory for acquisition and huge, expandable storage capacity. Conclusion: The ability of high frame‐rate acquisition, image processing and display for this unique high‐resolution detector along with the user friendly implementation of the interface should provide angiographers and interventionalists with a new capability for visualizing details of small vessels and endovascular devices such as stents. Such capability should enable more accurate diagnoses and image guided interventions. (Support from NIH Grants R01NS43924, R01EB002873 and Toshiba Medical Systems Corporation).

Synthesis of Nonseparable 3-D Spatiotemporal Bandpass Filters on Analog Networks

Linear cellular neural networks (CNNs) are capable of performing efficient spatiotemporal filtering operations as recursive infinite impulse response (IIR) filters. Particularly, linear CNNs can be characterized as a spatial frequency-dependent recursive temporal filter with complex coefficients. Based on a modified version of the CNN paradigm recently proposed by the authors, nonseparable spatiotemporal bandpass filters with tunable spatiotemporal passband volumes are synthesized. The filters reported here qualitatively resemble spatiotemporal receptive field models for the primary visual cortex. Numerical simulation results confirm the bandpass characteristics of our filtering network.

Temporal Video Denoising Based on Multihypothesis Motion Compensation

Denoising module is required by any practical video processing systems. Most existing denoising schemes are spatio-temporal filters which operate on data over three dimensions. However, to limit the number of inputs, these filters only utilize one reference frame and cannot fully exploit temporal correlation. In this paper, a recursive temporal denoising filter named multihypothesis motion compensated filter (MHMCF) is proposed. To fully exploit temporal correlation, MHMCF performs motion estimation in a number of reference frames to construct multiple hypotheses (temporal predictions) of the current pixel. These hypotheses are combined by weighted averaging to suppress noise and estimate the actual current pixel value. Based on the multihypothesis motion compensated residue model presented in this paper, we investigate the efficiency of MHMCF, and some numerical evaluations are revealed. Experimental results show that MHMCF demonstrates quite good denoising performance while the inputs are much fewer than spatio-temporal filters. Moreover, as a purely temporal filter, it can well preserve spatial details and achieve satisfactory visual quality.

Digital subtraction peripheral angiography using image stacking: initial clinical results.

Using clinically acquired x-ray angiography image sequences, we compared three algorithms for creating a single diagnostic quality image that combined input images containing flowing contrast agent. These image-stacking algorithms were: maximum opacity with the minimum gray-scale value across time recorded at each spatial location, (REC) recursive temporal filtering followed by a maximum opacity operation, and (AMF) an approximate matched filter consisting of a convolution with a kernel approximating the matched filter followed by a maximum opacity operation. Eighteen clinical exams of the peripheral arteries of the legs were evaluated. AMF gave 2.7 times greater contrast to noise ratio than the single best subtraction image and 1.3 times improvement over REC, the second best stacking algorithm. This is consistent with previous simulations showing that AMF performs nearly equal to the optimal result from matched filtering without the well-known limitations. For example, unlike matched filtering, AMF filter coefficients were obtained automatically using an image-processing algorithm. AMF effectively brought out small collateral arteries, otherwise difficult to see, without degrading artery sharpness or stenosis grading. Comparing results using reduced and full contrast agent volumes demonstrated that contrast agent load could be reduced to one-third of the conventional amount with AMF processing. By simulating reduced x-ray exposures on clinical exams, we determined that x-ray exposure could be reduced by 80% with AMF processing. We conclude that AMF is a promising, potential technique for reducing contrast agent load and for improving vessel visibility, both very important characteristics for vascular imaging.

Low-latency temporal filter design for real-time MRI using UNFOLD.

To improve real-time control of interventional procedures such as guidance of catheters, monitoring of ablation therapy, or control of dosage during drug delivery, the image acquisition and reconstruction must be high speed and have low latency (small time delay) in processing. A number of different methods have been demonstrated which increase the speed of MR acquisition by decreasing the number of sequential phase-encodes. A design and implementation of the UNFOLD method which achieves the desired low latency with a recursive temporal filter is presented. The recursive filter design is characterized for this application and compared with more commonly used moving average filters. Experimental results demonstrate low-latency UNFOLD for two applications: 1) high-speed, real-time imaging of the heart to be used in conjunction with cardiac interventional procedures; and 2) the injection of drugs into muscle tissue with contrast enhancement, i.e., monitoring needle insertion and injection of a drug with contrast enhancement properties. Proof-of-concept was demonstrated by injecting a contrast agent. In both applications the UNFOLD technique was used to double the frame rate.

Perception adaptive temporal TV-noise reduction using contour preserving prefilter techniques

The reduction of Gaussian noise is still an important task in video systems. For this purpose a spatio-temporal noise reduction scheme for interlaced video is presented. It consists mainly of a subband based temporal recursive filter which makes use of some special properties of the human visual system. This temporal system is supported by a preceding detail preserving spatial filter with low hardware expense, which consists of an image analysing highpass filter bank and an adaptive lowpass FIR-filter for noise reduction. Both the spatial and temporal noise reduction have been evaluated with a large amount of simulations which result in a very good objective and subjective efficiency. Furthermore the chain of both temporal and spatial noise reduction may even yield results which are better than the sum of pure spatial and temporal techniques.

Point target detection in consecutive frame staring infrared imagery with evolving cloud clutter

The problem of detection of aircraft at long range in a background of evolving cloud clutter is treated. A staring infrared camera is favored for this application due to its passive nature, day/night operation, and rapid frame rate. The rapid frame rate increases the frame-to-frame correlation of the evolving cloud clutter; cloud-clutter leakage is a prime source of false alarms. Targets of opportunity in daytime imagery were used to develop and compare two algorithm approaches: banks of spatio-temporal velocity filters followed by dynamic-programming-based stage-to-stage association, and a simple recursive temporal filter arrived at from a singular-value decomposition analysis of the data. To quantify the relative performance of the two approaches, we modify conventional metrics for signal-to-clutter gains in order to make them more germane to consecutive frame real data processing. The temporal filter, in responding preferentially to pixels influenced by moving point targets over those influenced by drifting clouds, achieves impressive cloud-clutter suppression without requiring subpixel frame registration. The velocity filter technique is roughly half as effective in clutter suppression but is twice as sensitive to weak targets in white noise (close to blue sky conditions). The real-time hardware implementation of the temporal filter is far more practical.

X-ray fluoroscopy spatio-temporal filtering with object detection

One potential way to reduce patient and staff X-ray fluoroscopy dose is to reduce the quantum exposure to the detector and compensate the additional noise with digital filtering. A new filtering method, spatio-temporal filtering with object detection, is described that reduces noise while minimizing motion and spatial blur. As compared to some conventional motion-detection filtering schemes, this object-detection method incorporates additional a priori knowledge of image content; i.e. much of the motion occurs in isolated long thin objects (catheters, guide wires, etc.). We create object-likelihood images and use these to control spatial and recursive temporal filtering such as to reduce blurring the objects of interest. We use automatically computed receiver operating characteristic (ROC) curves to optimize the object-likelihood enhancement method and determine that oriented matched filter kernels with 4 orientations are appropriate. The matched filter kernels are simple projected cylinders. We demonstrate the method on several representative X-ray fluoroscopy sequences to which noise is added to simulate very low dose acquisitions. With processing, we find that noise variance is significantly reduced with slightly less noise reduction near moving objects. We estimate an effective exposure reduction greater than 80%

Recursive Temporal Filtering and Frame Rate Reduction for Image Coding

The effect of temporal recursive filtering and frame rate reduction on a sequence of moving images is investigated. Following a well-known first-order Markov model for the temporal domain data, the effect of filtering and frame rate reduction on the values of variance, correlation coefficient, filtering distortion, and interframe prediction error are mathematically derived. The validity of these derivations is verified by the experimental results using a CCITT standard sequence which closely follows the first-order Markov model. The effect of temporal filtering and frame rate reduction to the coding rate is examined by encoding the CCITT standard sequence using a combined interframe and intraframe Scene Adaptive Coding system. At a frame rate of 30 frames per second, a 2:1 reduction in coding rate is obtained with a filtering coefficient of 0.5. At a frame rate of 10 frames per second, an additional reduction factor of nearly 2 is obtained using the same filtering coefficient. Selected images are presented to demonstrate the subjective effect.

Estimation Of Ship Profiles From A Time Sequence Of Forward-Looking Infrared Images

This paper addresses the problem of estimating the profile, or silhouette, of a ship target from a sequence of forward-looking infrared (FLIR) imagery obtained at video rates. We are specifically interested in the performance of multiframe processing techniques compared with single-frame methods. Our approach to multiframe signal processing combines spatial and temporal processing in two stages: first, a target profile is extracted from each image frame by a segmentation algorithm; second, the resulting sequence of profiles is temporally filtered in order to increase the signal-to-noise ratio. Long sequences of FLIR imagery tend to exhibit several characteristic forms of non-Gaussian noise (random speckle, occlusions, and flaring) caused by various atmospheric and background phenomena, as well as instrument noise. Simple temporal averaging is inadequate in this environment. In this paper we develop a combination of spatial and temporal processing algorithms that may be used to overcome the signal-to-noise ratio problems associated with these noise effects. A recursive temporal median filter in conjunction with a simple spatial segmentation algorithm is proposed for this purpose. Experimental results based on a database of FLIR ship imagery are presented in support of the spatiotemporal processor. Although the approach is developed for ship targets, we feel it could be adapted for a broader class of FLIR applications.

Novel structure for television picture noise reduction

A new method for noise reduction in image sequences is described which is based on a modified form of the well known first-order recursive temporal filter. The new features of the system are the improved separation of signal and noise and subsequently the reduction of noise in moving areas. This is accomplished without increasing the complexity of the whole system very much.