Temporal Filtering Techniques Research Articles

Fixed-Wing Unmanned Aerial Vehicle 3D-Model-Based Tracking for Autonomous Landing

The vast increase in the available computational capability has allowed the application of Particle-Filter (PF)-based approaches for monocular 3D-model-based tracking. These filters depend on the computation of a likelihood function that is usually unavailable and can be approximated using a similarity metric. We can use temporal filtering techniques between filter iterations to achieve better results when dealing with this suboptimal approximation, which is particularly important when dealing with the Unmanned Aerial Vehicle (UAV) model symmetry. The similarity metric evaluation time is another critical concern since we usually want a real-time implementation. We explored, tested, and compared with the same dataset two different types of PFs, (i) an Unscented Bingham Filter (UBiF) and (ii) an Unscented Bingham–Gauss Filter (UBiGaF), using pose optimization in both implementations. Using optimization steps between iterations increases the convergence capability of the filter and decreases the obtained error. A new tree-based similarity metric approach is also explored based on the Distance Transform (DT), allowing a faster evaluation of the possibilities without losing accuracy. The results showed that the obtained pose estimation error is compatible with the automatic landing requirements.

The 3D Position Estimation and Tracking of a Surface Vehicle Using a Mono-Camera and Machine Learning

The ability to obtain the 3D position of target vehicles is essential to managing and coordinating a multi-robot operation. We investigate an ML-backed object localization and tracking system to estimate the target’s 3D position based on a mono-camera input. The passive vision-only technique provides a robust field awareness in challenging conditions such as GPS-denied or radio-silent environments. Our processing pipeline utilizes a YOLOv5 neural network as the back-end detection module and a temporal filtering technique to improve detection and tracking accuracy. The filtering process effectively removes false positive labels to improve tracking accuracy. We propose a piecewise projection model to predict the target 3D position from the estimated 2D bounding box. Our projection model utilizes the co-plane property of ground vehicles to calculate 2D–3D mapping. Experimental results show that the piecewise model is more accurate than existing methods when the training dataset is not evenly distributed in the sampling space. Our piecewise model outperforms the singular RANSAC-based and the 6DPose methods by 28% in location errors. A less than 10-m error is observed for most near-to-mid-range cases.

Adaptive reconstruction of the heterogeneous scan line ETM+ correction technique

ETM+ is a land-imaging sensor with great and wide use in many fields, however, after May 2003, because of a technical defect in the sensor´s system scan line corrector SLC, it started giving images of earth containing black gap lines at a 22% rate. These gaps made the process of analyzing and extracting accurate information from these images difficult and complicated. Therefore, scientists have developed many techniques to remove the gap lines from all ETM+ band-images and complete the missing data. In this study, three different ETM+ time images with a 16-day interval between them were used to fill gap lines, complementing the images, and minimize the effect of the gaps. However, the adopted temporal filtration method was not able to completely remove the gaps. An added algorithm is used, which is based on using one of three filters (mean, median, and max.) to work on a vertical linear window whose center is on the points of the gap lines in the images. The quality of the resulting images was evaluated using the Jaccard metric standard to find the degree of similarity between the original images and the images that resulted from the implementation of the algorithm, the results showed good quality images, and the gaps were completely removed from the resulting images. A higher degree of similarity was obtained from implementing the (mean and median), ranging between 0.6568 and 0.9007, compared with the (max), where the similarity ranged between 0.6246 and 0.8820. This study aims to fill the images gaps using the temporal filtering technique.

Colorblind-shareable videos by synthesizing temporal-coherent polynomial coefficients

To share the same visual content between color vision deficiencies (CVD) and normal-vision people, attempts have been made to allocate the two visual experiences of a binocular display (wearing and not wearing glasses) to CVD and normal-vision audiences. However, existing approaches only work for still images. Although state-of-the-art temporal filtering techniques can be applied to smooth the per-frame generated content, they may fail to maintain the multiple binocular constraints needed in our applications, and even worse, sometimes introduce color inconsistency (same color regions map to different colors). In this paper, we propose to train a neural network to predict the temporal coherent polynomial coefficients in the domain of global color decomposition. This indirect formulation solves the color inconsistency problem. Our key challenge is to design a neural network to predict the temporal coherent coefficients, while maintaining all required binocular constraints. Our method is evaluated on various videos and all metrics confirm that it outperforms all existing solutions.

Snow Cover Evolution in the Gran Paradiso National Park, Italian Alps, Using the Earth Observation Data Cube

Mountainous regions are particularly vulnerable to climate change, and the impacts are already extensive and observable, the implications of which go far beyond mountain boundaries and the environmental sectors. Monitoring and understanding climate and environmental changes in mountain regions is, therefore, needed. One of the key variables to study is snow cover, since it represents an essential driver of many ecological, hydrological and socioeconomic processes in mountains. As remotely sensed data can contribute to filling the gap of sparse in-situ stations in high-altitude environments, a methodology for snow cover detection through time series analyses using Landsat satellite observations stored in an Open Data Cube is described in this paper, and applied to a case study on the Gran Paradiso National Park, in the western Italian Alps. In particular, this study presents a proof of concept of the preliminary version of the snow observation from space algorithm applied to Landsat data stored in the Swiss Data Cube. Implemented in an Earth Observation Data Cube environment, the algorithm can process a large amount of remote sensing data ready for analysis and can compile all Landsat series since 1984 into one single multi-sensor dataset. Temporal filtering methodology and multi-sensors analysis allows one to considerably reduce the uncertainty in the estimation of snow cover area using high-resolution sensors. The study highlights that, despite this methodology, the lack of available cloud-free images still represents a big issue for snow cover mapping from satellite data. Though accurate mapping of snow extent below cloud cover with optical sensors still represents a challenge, spatial and temporal filtering techniques and radar imagery for future time series analyses will likely allow one to reduce the current cloud cover issue.

Magnifying Subtle Facial Motions for Effective 4D Expression Recognition

In this paper, an effective approach is proposed for automatic 4D Facial Expression Recognition (FER). It combines two growing but disparate ideas in the domain of computer vision, i.e., computing spatial facial deformations using a Riemannian method and magnifying them by a temporal filtering technique. Key frames highly related to facial expressions are first extracted from a long 4D video through a spectral clustering process, forming the Onset-Apex-Offset flow. It is then analyzed to capture the spatial deformations based on Dense Scalar Fields (DSF), where registration and comparison of neighboring 3D faces are jointly led. The generated temporal evolution of these deformations is further fed into a magnification method to amplify facial activities over time. The proposed approach allows revealing subtle deformations and thus improves the emotion classification performance. Experiments are conducted on the BU-4DFE and BP-4D databases, and competitive results are achieved compared to the state-of-the-art.

Enhanced Background Oriented Schlieren (EBOS)

The Background Orientend Schlieren (BOS) is a line of sight optical technique that exploits light refraction arising in inhomogeneous density field to visualize or either measure density gradients. The working principle of BOS is quite simple: a background patter appears distorted when observed through an optical inhomogeneous media, by comparing the distorted and undistorted images the apparent local displacement of the background pattern can be measured. The images comparison is carried out using either cross-correlation or optical flow techniques. In the framework of geometric optics and for a 2D refractive index field the measured displacements is proportional to the refractive index gradient. When applicable, the Gladstone-Dale relationship provides the link between the refractive index and the fluid density. The uncertainty in the measured displacements affects the qualitative and quantitative results extracted from BOS images. Spatial and temporal filtering techniques can reduce the uncertainty at the expenses of the spatial and the temporal resolutions. This work describes a modified BOS technique, named Enhanced BOS (EBOS), which is able to reduce the displacement uncertainty without using any spatial or temporal filter. The EBOS technique uses N undistorted images of a grey-scale background pattern taken at slightly different potions, the N reference images are paired with a single distorted background image. Eventually displacement maps obtained from the N image couples are averaged together. The mean displacement map is an improved estimate of the actual displacement field. The EBOS technique can also be used with optical flow techniques anyhow we limited our investigation to image analysis based on cross-correlation technique. The paper reports the results obtained by applying EBOS and BOS techniques to the same test case. The comparison evidence the capability of EBOS to reduce displacement uncertainty respect to BOS by using a limited number of undistorted background images. EBOS and BOS techniques achieve almost the same uncertainty when the number of particles contained in an interrogation area is high, thus EBOS is most advantageous to use when small interrogation area are required.

Visual-speech-pass filtering for robust automatic lip-reading

This paper proposes a temporal filtering technique used in extraction of visual features for improved robustness of automatic lip-reading, called visual-speech-pass filtering. A band-pass filter is applied to the pixel value sequence of the images containing the speaker's lip region to remove unwanted variations that are not relevant to the speech information. The filter is carefully designed based on psychological, spectral, and experimental analyses. Experimental results on two speaker-independent and one speaker-dependent recognition tasks demonstrate that the proposed technique significantly improves recognition performance in both clean and visually noisy conditions.

Channel Parameter Estimation for Scatter Cluster Model Using Modified MUSIC Algorithm

Recently, the scatter cluster models which precisely evaluate the performance of the wireless communication system have been proposed in the literature. However, the conventional SAGE algorithm does not work for these scatter cluster-based models because it performs poorly when the transmit signals are highly correlated. In this paper, we estimate the time of arrival (TOA), the direction of arrival (DOA), and Doppler frequency for scatter cluster model by the modified multiple signal classification (MUSIC) algorithm. Using the space-time characteristics of the multiray channel, the proposed algorithm combines the temporal filtering techniques and the spatial smoothing techniques to isolate and estimate the incoming rays. The simulation results indicated that the proposed algorithm has lower complexity and is less time-consuming in the dense multipath environment than SAGE algorithm. Furthermore, the estimations’ performance increases with elements of receive array and samples length. Thus, the problem of the channel parameter estimation of the scatter cluster model can be effectively addressed with the proposed modified MUSIC algorithm.

A novel approach for flip chip solder joint inspection based on pulsed phase thermography

Surface mount components have been extensively used in microelectronic packaging. However, it brings great challenge for defect inspection with the development of solder bumps towards ultra-fine pitch and high density. Traditional nondestructive detection methods are insufficient for solder joint assessment due to their own disadvantages. Therefore, it is necessary to explore new methods for solder joint inspection. A novel approach based on the pulsed phase thermography was investigated for identifying the defects of solder joints. In this approach, the test chip was stimulated by a thermal pulse, and the consequent transient response was captured by a commercial thermal imager. The spacial and temporal filtering techniques were adopted to improve the signal to noise ratio. The recorded thermograms were input to an improved median filter with a 5×5 mask, and the temperature evolution of each pixel was extracted and smoothed by the moving average operation. Then the temperature–time curve was fitted with an exponential function. To eliminate emissivity variations and heating non-uniformity, we converted the fitted temperature values in time domain to the phase information in frequency domain using the fast Fourier transform. In low frequency range, the phase–frequency curve of the defect area was differentiated from that of the sound area. The results demonstrate that this approach is effective for identification of the missing bumps, and can be used in the solder joint inspection in high density packaging.

Robust Filtering Technique for Reduction of Temporal Fluctuation in H.264 Video Sequences

The H.264/advanced video coding standard applies a spatial loop-filter in both encoder and decoder to reduce the blocking artifact. However, the temporal fluctuation artifact is still noticeable between intra-coded frames or between an intra (I)-frame and its preceding or subsequent inter-prediction (P) frames. This letter proposes a temporal postfilter using a robust statistical model to reduce the temporal fluctuation and to preserve the motion sharpness of the reconstructed video. The novelty is the implementation of a simplified robust statistical model as well as the Huber statistical model for the temporal fluctuation reduction. Performance evaluations of fluctuation reduction have been conducted using the proposed robust statistical temporal filter (RSTF), the Huber filter, and three other temporal filtering techniques, in terms of a temporal impairment metric (i.e., the sum of squared differences), and a spatial quality metric (i.e., the peak-signal-to-noise ratio). Consistent improvements of the temporal fluctuation reduction and the perceptual picture quality produced by the RSTF are demonstrated using the aforementioned performance criteria, and can be best appreciated by viewing the sequences on professional broadcasting television monitors.

Daylight operation of a free space, entanglement-based quantum key distribution system

Many quantum key distribution (QKD) implementations using a free space transmission path are restricted to operation at night time in order to distinguish the signal photons used for a secure key establishment from the background light. Here, we present a lean entanglement-based QKD system overcoming that limitation. By implementing spectral, spatial and temporal filtering techniques, we establish a secure key continuously over several days under varying light and weather conditions.

Cardiac phase-correlated image reconstruction and advanced image processing in pulmonary CT imaging

Image quality in pulmonary CT imaging is commonly degraded by cardiac motion artifacts. Phase-correlated image reconstruction algorithms known from cardiac imaging can reduce motion artifacts but increase image noise and conventionally require a concurrently acquired ECG signal for synchronization. Techniques are presented to overcome these limitations. Based on standard and phase-correlated images that are reconstructed using a raw data-derived synchronization signal, image-merging and temporal-filtering techniques are proposed that combine the input images automatically or interactively. The performance of the approaches is evaluated in patient and phantom datasets. In the automatic approach, areas of strong motion and static areas were well detected, providing an optimal combination of standard and phase-correlated images with no visible border between the merged regions. Image noise in the non-moving regions was reduced to the noise level of the standard reconstruction. The application of the interactive filtering allowed for an optimal adaptation of image noise and motion artifacts. Noise content after interactive filtering decreased with increasing temporal filter width used. We conclude that a combination of our motion-free merging approach and a dedicated interactive filtering procedure can highly improve pulmonary imaging with respect to motion artifacts and image noise.

System and process for locating a speaker using 360 degree sound source localization

A system and process is described for estimating the location of a speaker using signals output by a microphone array characterized by multiple pairs of audio sensors. The location of a speaker is estimated by first determining whether the signal data contains human speech components and filtering out noise attributable to stationary sources. The location of the person speaking is then estimated using a time-delay-of-arrival based SSL technique on those parts of the data determined to contain human speech components. A consensus location for the speaker is computed from the individual location estimates associated with each pair of microphone array audio sensors taking into consideration the uncertainty of each estimate. A final consensus location is also computed from the individual consensus locations computed over a prescribed number of sampling periods using a temporal filtering technique.

The Development of an Automated Method for Analyzing Communication Rate in Augmentative and Alternative Communication

A significant barrier to evidence-based practice in Augmentative and Alternative Communication (AAC) is the lack of validated performance measures that can be used by speech-language pathologists and rehabilitation engineers to evaluate the communication and device use of AAC consumers. Recently an effort has been made to develop automated data-logging techniques to facilitate the transcription and analysis of the AAC speaker's device use. A major source of error for the automated measurement of communication rate is the presence of excessive Inter-Selection Intervals (ISIs) (i.e., pause times), for which no communicative activity is occurring. The goal of this study was to develop an automated technique to filter out extreme ISIs, while preserving true communication rate performance. AAC data log files were obtained from seven individuals participating in a 1-month field trial of an AAC technology. Two temporal filtering techniques (arbitrary and individual) were compared for their ability to eliminate excessive ISIs. Results indicated that use of an individualized temporal filter was more sensitive to performance variability than use of an arbitrary temporal filter. Further, the individualized temporal filter elevated the participants' communication rate (measured by words per minute) by a factor of 1.8 to 34.5 greater than that of the unfiltered communication rate estimate. In addition, the first AAC communication rate performance estimates taken from the field are presented. Implications for further research and the valid use of automated data logging and analysis are discussed.

Extraction of blood vessel through new temporal filtering in angiographic sequence

A new method is proposed for a digital angiography in which only blood vessels are extracted from an angiographic sequence. Extraction is accomplished by the removal of both noise and static objects, such as diaphragm, which often disturb the examination of the coronary arteries. The method utilizes both a spatial and new temporal filtering technique. Temporal filtering is based on the a priori knowledge that blood vessels move synchronously with the heartbeat. The solutions proposed here for the practical problems caused by temporal sampling make it possible to estimate the frequency components over the Nyquist frequency which is defined by the sampling ratio. Experimental results show that the method is suitable for a practical usage in clinical applications.

Effects of statistical quality, sampling rate and temporal filtering techniques on the extraction of functional parameters from the left ventricular time-activity curves.

Several techniques for the processing of a series of curves derived from two left ventricular time-activity curves acquired at rest and during exercise with a nuclear stethoscope were evaluated. They were three and five point time smoothing, Fourier filtering preserving one to four harmonics (H), truncated curve Fourier filtering, and third degree polynomial curve fitting. Each filter's ability to recover, with fidelity, systolic and diastolic function parameters was evaluated under increasingly 'noisy' conditions and at several sampling rates. Third degree polynomial curve fittings and truncated Fourier filters exhibited very high sensitivity to noise. Three and five point time smoothing had moderate sensitivity to noise, but were highly affected by sampling rate. Fourier filtering preserving 2H or 3H produced the best compromise with high resilience to noise and independence of sampling rate as far as the recovery of these functional parameters is concerned.

Spatial and temporal filtering technique for processing lidar photocount data

Shot noise places a practical limit on the spatial and temporal resolution of lidar photocount data. A 2-D signal-processing technique that utilizes spatial and temporal filtering to reduce shot noise and increase resolution is described. The technique is applied to sodium lidar data collected during the fall of 1979 over Urbana, Illinois. Temporal filtering is shown to enhance the spatial resolution of the sodium profiles significantly by reducing shot noise by more than 10 dB. The signal-processing technique is applicable to a wide variety of lidar data.

Observation of acoustically induced space charge gratings in CdS

Static space charge gratings, produced via a nonlinear acoustoelectric interaction, have been directly observed in CdS by means of laser diffraction. Spatial and temporal filtering techniques were used to separate the required optical information from extraneous signals due to background and acoustic diffraction effects.