Adaptive Savitzky-Golay Filter Research Articles

State of charge estimation for energy storage lithium-ion batteries based on gated recurrent unit neural network and adaptive Savitzky-Golay filter

State of charge estimation for energy storage lithium-ion batteries based on gated recurrent unit neural network and adaptive Savitzky-Golay filter

A novel deep neural network based marine predator model for effective classification of big data from social Internet of Things

SummarySocial Internet of Things (SIoT) is considered as one of the most recently evolved topics that connects people and object, object and object as well as people and people. The SIoT and big data provide an exact representation of IoT and social system for human progression characterization. Numerous machine learning techniques are employed to classify the data gathered from SIoT in a more powerful way. In this article, a deep neural network based marine predator (DRNN‐MP) algorithm is proposed in classifying big data. Here, an adaptive Savitzky–Golay filter is employed for selecting the subset and to eliminate undesirable data, as well as different noises. The big data databases are reduced using a Hadoop map reducing framework thereby enhancing the performances of the proposed approach. In addition to this, a modified relief technique is employed to select optimal features thereby performing better classification. The testing and training process based on the proposed approach for optimal classification of features from the big data considers four different databases namely coronary illness, GPS trajectories, localization data, and water treatment plant obtained from UCI machine learning repository. In addition to this, the proposed approach is evaluated for diverse performance measures namely accuracy, precision, specificity, sensitivity, throughput, and energy consumption. Finally, the proposed approach is compared for various metrics to illustrate the effectiveness of the system and the results demonstrated that the accuracy of our work is 98.25%.

Highly sensitive methane detection based on light-induced thermoelastic spectroscopy with a 2.33 µm diode laser and adaptive Savitzky-Golay filtering.

In this manuscript, a highly sensitive methane (CH4) sensor based on light-induced thermoelastic spectroscopy (LITES) using a 2.33 µm diode laser with high power is demonstrated for the first time. A quartz tuning fork (QTF) with an intrinsic resonance frequency of 32.768kHz was used to detect the light-induced thermoelastic signal. A Herriot multi-pass cell with an effective optical path of 10 m was adopted to increase the laser absorption. The laser wavelength modulation depth and concentration response of this CH4-LITES sensor were investigated. The sensor showed excellent long term stability when Allan deviation analysis was performed. An adaptive Savitzky-Golay (S-G) filtering algorithm with χ2 statistical criterion was firstly introduced to the LITES technique. The SNR of this CH4-LITES sensor was improved by a factor of 2.35 and the minimum detection limit (MDL) with an integration time of 0.1 s was optimized to 0.5 ppm. This reported CH4-LITES sensor with sub ppm-level detection ability is of great value in applications such as environmental monitoring and industrial safety.

Adaptive Savitzky-Golay filtering and its applications

Noise reduction is a central issue of the theory and practice of signal processing. The Savitzky-Golay (SG) smoothing and differentiation filter is widely acknowledged as a simple and efficient method for denoising. However only few book on signal processing contain this method. As is well known, the performance of the classical SG-filter depends on the appropriate setting of the window length and the polynomial degree, which should match the scale of the signal since, in the case of signals with high rate of change, the performance of the filter may be limited. This paper presents a new adaptive strategy to smooth irregular signals based on the Savitzky-Golay algorithm. The proposed technique ensures high precision noise reduction by iterative multi-round smoothing and correction. In each round the parameters dynamically change due to the results of the previous smoothing. Our study provides additional support for data compression based on optimal resolution of the signal with linear approximation. Here, simulation results validate the applicability of the novel method.

Adaptive Savitzky-Golay filtering and its applications

Noise reduction is a central issue of the theory and practice of signal processing. The Savitzky-Golay (SG) smoothing and differentiation filter is widely acknowledged as a simple and efficient method for denoising. However only few book on signal processing contain this method. As is well known, the performance of the classical SG-filter depends on the appropriate setting of the window length and the polynomial degree, which should match the scale of the signal since, in the case of signals with high rate of change, the performance of the filter may be limited. This paper presents a new adaptive strategy to smooth irregular signals based on the Savitzky-Golay algorithm. The proposed technique ensures high precision noise reduction by iterative multi-round smoothing and correction. In each round the parameters dynamically change due to the results of the previous smoothing. Our study provides additional support for data compression based on optimal resolution of the signal with linear approximation. Here, simulation results validate the applicability of the novel method.

Vegetation Phenological Characterization of Alluvial Plain Shorea robusta-dominated Tropical Moist Deciduous Forest of Northeast India Using MODIS NDVI Time Series Data

The current study attempts to extract the phenological variables of alluvial plain Shorea robusta-dominated tropical moist deciduous forest of Northeast India using threshold-based method. A total of 230 Moderate Resolution Imaging Spectroradiometer (MODIS) MOD13Q1 time series normalized difference vegetation index (NDVI) datasets (year 2006–2015) has been used. Time series NDVI datasets were fitted to an adaptive Savitzky–Golay filter for smoothing. Important phenological matrices, namely start of season (SOS), end of season (EOS), length of season (LOS), peak of season (POS), seasonal amplitude, seasonally integrals (large and small integral), were evaluated. SOS varied from 106 to 120 day of year (DOY) (average of 110 ± 17.6), and EOS varied from 425 to 441 DOY (average of 431 ± 14.33). POS reaches in the month of September and October (average 262 ± 15). In the current study, a mean amplitude of 0.35, lower value of small integral (4.70 ± 0.34) and higher value of large integral (16.24 ± 0.25) signify that the studied forest is highly productive ecosystem, with semievergreen or moist deciduous canopy. Strong linear relationship of NDVI with temperature and rainfall was witnessed, particularly with a 1–2 month time lag. Also, NDVI-temperature correlation was found stronger than NDVI-precipitation correlation, suggesting that the area being a humid subtropical region, temperature plays a greater role in the timing of phenological events than rainfall and can act as a crucial factor for growth of the species under the climate changing scenario.

A Discrete Curvature Estimation Based Low-Distortion Adaptive Savitzky⁻Golay Filter for ECG Denoising.

Electrocardiogram (ECG) sensing is an important application for the diagnosis of cardiovascular diseases. Recently, driven by the emerging technology of wearable electronics, massive wearable ECG sensors are developed, which however brings additional sources of noise contamination on ECG signals from these wearable ECG sensors. In this paper, we propose a new low-distortion adaptive Savitzky-Golay (LDASG) filtering method for ECG denoising based on discrete curvature estimation, which demonstrates better performance than the state of the art of ECG denoising. The standard Savitzky-Golay (SG) filter has a remarkable performance of data smoothing. However, it lacks adaptability to signal variations and thus often induces signal distortion for high-variation signals such as ECG. In our method, the discrete curvature estimation is adapted to represent the signal variation for the purpose of mitigating signal distortion. By adaptively designing the proper SG filter according to the discrete curvature for each data sample, the proposed method still retains the intrinsic advantage of SG filters of excellent data smoothing and further tackles the challenge of denoising high signal variations with low signal distortion. In our experiment, we compared our method with the EMD-wavelet based method and the non-local means (NLM) denoising method in the performance of both noise elimination and signal distortion reduction. Particularly, for the signal distortion reduction, our method decreases in MSE by 33.33% when compared to EMD-wavelet and by 50% when compared to NLM, and decreases in PRD by 18.25% when compared to EMD-wavelet and by 25.24% when compared to NLM. Our method shows high potential and feasibility in wide applications of ECG denoising for both clinical use and consumer electronics.

Accurate and Standardized Coronary Wave Intensity Analysis

Objective: Coronary wave intensity analysis (cWIA) has increasingly been applied in the clinical research setting to distinguish between the proximal and distal mechanical influences on coronary blood flow. Recently, a cWIA-derived clinical index demonstrated prognostic value in predicting functional recovery post-myocardial infarction. Nevertheless, the known operator dependence of the cWIA metrics currently hampers its routine application in clinical practice. Specifically, it was recently demonstrated that the cWIA metrics are highly dependent on the chosen Savitzky–Golay filter parameters used to smooth the acquired traces. Therefore, a novel method to make cWIA standardized and automatic was proposed and evaluated in vivo . Methods: The novel approach combines an adaptive Savitzky–Golay filter with high-order central finite differencing after ensemble-averaging the acquired waveforms. Its accuracy was assessed using in vivo human data. The proposed approach was then modified to automatically perform beatwise cWIA. Finally, the feasibility (accuracy and robustness) of the method was evaluated. Results: The automatic cWIA algorithm provided satisfactory accuracy under a wide range of noise scenarios ( $\leq$ 10% and $\leq$ 20% error in the estimation of wave areas and peaks, respectively). These results were confirmed when beat-by-beat cWIA was performed. Conclusion: An accurate, standardized, and automated cWIA was developed. Moreover, the feasibility of beatwise cWIA was demonstrated for the first time. Significance: The proposed algorithm provides practitioners with a standardized technique that could broaden the application of cWIA in the clinical practice as well as enabling multicenter trials. Furthermore, the demonstrated potential of beatwise cWIA opens the possibility of investigating the coronary physiology in real time.

Application of adaptive Savitzky–Golay filter for EEG signal processing

Summary A Savitzky–Golay filter typically requires pre-determined values of order and frame size for its fabrication. Generally, a random hit-and-trial method or prior experience is required to determine the suitable values of design parameters. However, the proposed adaptive Savitzky–Golay filter aims to provide a generic framework for optimal design of filter vis-a-vis the order and frame size of the filter. The algorithm uses all the possible combinations of these parameters in a certain range and the correlation coefficient is evaluated in each case to measure the filter efficiency. The parameters which provide the highest correlation coefficient are considered for filter design. In this paper the relative advantages of adaptive Savitzky–Golay filter over the standard models are also discussed. The proposed adaptive model of Savitzky–Golay filter is successfully tested for EEG signal processing.

A River Basin over the Course of Time: Multi-Temporal Analyses of Land Surface Dynamics in the Yellow River Basin (China) Based on Medium Resolution Remote Sensing Data

The Yellow River Basin is one of China’s most densely-populated, fastest growing and most dynamic regions, with abundant natural resources and intense agricultural production. Major land policies have recently resulted in remarkable landscape modifications throughout the basin. The availability of precise regional land cover change information is crucial to better understand the prevailing dynamics and underlying factors influencing the current processes in such a complex system and can additionally serve as a valuable component for modeling and decision making. Such comprehensive and detailed information is lacking for the Yellow River Basin so far. In this study, we derived land cover characteristics and dynamics from the complete last decade based on optical high-temporal MODIS Normalized Differenced Vegetation Index (NDVI) time series for the whole Yellow River Basin. After filtering and smoothing for noise reduction with the use of the adaptive Savitzky–Golay filter, the processed time series was used to derive a large variety of phenological and annual metrics. The final classifications for the basin (2003 and 2013) were based on a random forest classifier, trained by reference samples from very high-resolution imagery. The accuracy assessment for all 18 thematic classes, which was based on a 30% reference data split, yielded an overall accuracy of 87% and 84% for 2003 and 2013, respectively. Major land cover and land use changes during the last decade have occurred on the Loess Plateau, where land and conservation reforms triggered large-scale recovery of grassland and shrubland habitat that had been previously covered by agriculture or sparse vegetation. Agricultural encroachment and urban area expansion are other processes influencing the dynamics in the basin. The necessity for regionally-adapted land cover maps becomes obvious when our land cover products are compared to existing global products, where thematic accuracy remains low, particularly in a heterogeneous landscape, such as the Yellow River Basin. The basin-wide novel land cover and land use products of the Yellow River Basin hold a large potential for climate, hydrology and biodiversity modelers, as well as river basin and regional governmental authorities and will be shared upon request.

Real-time infrared gas detection based on an adaptive Savitzky–Golay algorithm

Based on the Savitzky–Golay filter, we have developed in the present study a simple but robust method for real-time processing of tunable diode laser absorption spectroscopy (TDLAS) signals. Our method was developed to resolve the blindness of selecting the input filter parameters and to mitigate potential signal distortion induced in digital signal processing. Application of the developed adaptive Savitzky–Golay filter algorithm to the simulated and experimentally observed signals and comparison with the wavelet-based de-noising technique indicate that the newly developed method is effective in obtaining high-quality TDLAS data for a wide variety of applications including atmospheric environmental monitoring and industrial processing control.

Mapping Crop Cycles in China Using MODIS-EVI Time Series

As the Earth’s population continues to grow and demand for food increases, the need for improved and timely information related to the properties and dynamics of global agricultural systems is becoming increasingly important. Global land cover maps derived from satellite data provide indispensable information regarding the geographic distribution and areal extent of global croplands. However, land use information, such as cropping intensity (defined here as the number of cropping cycles per year), is not routinely available over large areas because mapping this information from remote sensing is challenging. In this study, we present a simple but efficient algorithm for automated mapping of cropping intensity based on data from NASA’s (NASA: The National Aeronautics and Space Administration) MODerate Resolution Imaging Spectroradiometer (MODIS). The proposed algorithm first applies an adaptive Savitzky-Golay filter to smooth Enhanced Vegetation Index (EVI) time series derived from MODIS surface reflectance data. It then uses an iterative moving-window methodology to identify cropping cycles from the smoothed EVI time series. Comparison of results from our algorithm with national survey data at both the provincial and prefectural level in China show that the algorithm provides estimates of gross sown area that agree well with inventory data. Accuracy assessment comparing visually interpreted time series with algorithm results for a random sample of agricultural areas in China indicates an overall accuracy of 91.0% for three classes defined based on the number of cycles observed in EVI time series. The algorithm therefore appears to provide a straightforward and efficient method for mapping cropping intensity from MODIS time series data.

A comparison of methods for smoothing and gap filling time series of remote sensing observations – application to MODIS LAI products

Abstract. Moderate resolution satellite sensors including MODIS (Moderate Resolution Imaging Spectroradiometer) already provide more than 10 yr of observations well suited to describe and understand the dynamics of earth's surface. However, these time series are associated with significant uncertainties and incomplete because of cloud cover. This study compares eight methods designed to improve the continuity by filling gaps and consistency by smoothing the time course. It includes methods exploiting the time series as a whole (iterative caterpillar singular spectrum analysis (ICSSA), empirical mode decomposition (EMD), low pass filtering (LPF) and Whittaker smoother (Whit)) as well as methods working on limited temporal windows of a few weeks to few months (adaptive Savitzky–Golay filter (SGF), temporal smoothing and gap filling (TSGF), and asymmetric Gaussian function (AGF)), in addition to the simple climatological LAI yearly profile (Clim). Methods were applied to the MODIS leaf area index product for the period 2000–2008 and over 25 sites showed a large range of seasonal patterns. Performances were discussed with emphasis on the balance achieved by each method between accuracy and roughness depending on the fraction of missing observations and the length of the gaps. Results demonstrate that the EMD, LPF and AGF methods were failing because of a significant fraction of gaps (more than 20%), while ICSSA, Whit and SGF were always providing estimates for dates with missing data. TSGF (Clim) was able to fill more than 50% of the gaps for sites with more than 60% (80%) fraction of gaps. However, investigation of the accuracy of the reconstructed values shows that it degrades rapidly for sites with more than 20% missing data, particularly for ICSSA, Whit and SGF. In these conditions, TSGF provides the best performances that are significantly better than the simple Clim for gaps shorter than about 100 days. The roughness of the reconstructed temporal profiles shows large differences between the various methods, with a decrease of the roughness with the fraction of missing data, except for ICSSA. TSGF provides the smoothest temporal profiles for sites with a % gap > 30%. Conversely, ICSSA, LPF, Whit, AGF and Clim provide smoother profiles than TSGF for sites with a % gap < 30%. Impact of the accuracy and smoothness of the reconstructed time series were evaluated on the timing of phenological stages. The dates of start, maximum and end of the season are estimated with an accuracy of about 10 days for the sites with a % gap < 10% and increases rapidly with the % gap. TSGF provides more accurate estimates of phenological timing up to a % gap < 60%.

Characterisation of land surface phenology and land cover based on moderate resolution satellite data in cloud prone areas — A novel product for the Mekong Basin

Information on vegetation phenology and land cover for the area of the Mekong Basin are derived for the year 2010, based on optical satellite data from the Moderate Resolution Imaging Spectroradiometer (MODIS). On account of almost persistent cloud cover in the rainy season, the application of optical remote sensing data presents a challenging exercise and demands special data processing and classification methods. An Enhanced Vegetation Index (EVI) time series is produced based on data from both platforms AQUA and TERRA, with dropouts and noise being effectively reduced by applying an adaptive Savitzky–Golay filter. Thereby, different parameterisation is applied dependent on the number of harvest cycles, which are defined by an initial harmonic analysis, based on an EVI time series of 5years. Information on land cover is derived by a multistep unsupervised classification approach optimised for regions with frequent cloud cover, based on multispectral monthly and seasonal composites, amplitude and phase information from a 11-year EVI time series (2001–2011), and phenological metrics from 2010. Moreover, the environmental heterogenic conditions in the region are addressed by a regionally tuned clustering approach based on physiographic subregions and the use of auxiliary geodata. The results obtained demonstrate that the adapted approach performs satisfactory in terms of accuracy under given conditions.

Detecting long-duration cloud contamination in hyper-temporal NDVI imagery

Cloud contamination impacts on the quality of hyper-temporal NDVI imagery and its subsequent interpretation. Short-duration cloud impacts are easily removed by using quality flags and an upper envelope filter, but long-duration cloud contamination of NDVI imagery remains. In this paper, an approach that goes beyond the use of quality flags and upper envelope filtering is tested to detect when and where long-duration clouds are responsible for unreliable NDVI readings, so that a user can flag those data as missing. The study is based on MODIS Terra and the combined Terra-Aqua 16-day NDVI product for the south of Ghana, where persistent cloud cover occurs throughout the year. The combined product could be assumed to have less cloud contamination, since it is based on two images per day. Short-duration cloud effects were removed from the two products through using the adaptive Savitzky–Golay filter. Then for each ‘cleaned’ product an unsupervised classified map was prepared using the ISODATA algorithm, and, by class, plots were prepared to depict changes over time of the means and the standard deviations in NDVI values. By comparing plots of similar classes, long-duration cloud contamination appeared to display a decline in mean NDVI below the lower limit 95% confidence interval with a coinciding increase in standard deviation above the upper limit 95% confidence interval. Regression analysis was carried out per NDVI class in two randomly selected groups in order to statistically test standard deviation values related to long-duration cloud contamination. A decline in seasonal NDVI values (growing season) were below the lower limit of 95% confidence interval as well as a concurrent increase in standard deviation values above the upper limit of the 95% confidence interval were noted in 34 NDVI classes. The regression analysis results showed that differences in NDVI class values between the Terra and the Terra-Aqua imagery were significantly correlated (p<0.05) with the corresponding standard deviation values of the Terra imagery in case of all NDVI classes of two selected NDVI groups. The method successfully detects long-duration cloud contamination that results in unreliable NDVI values. The approach offers scientists interested in time series analysis a method of masking by area (class) the periods when pre-cleaned NDVI values remain affected by clouds. The approach requires no additional data for execution purposes but involves unsupervised classification of the imagery to carry out the evaluation of class-specific mean NDVI and standard deviation values over time.

Sub-paddock scale spatial variability between the pasture and cropping phases of mixed farming systems in Australia

Precision farming techniques are now widely applied within Australian cropping systems. However, the use of spatial monitoring technologies to investigate livestock and pasture interactions in mixed farming systems remains largely unexplored. This paper builds on previously reported work that is using remote sensing technologies to investigate livestock and pasture interactions in the pasture phase and to follow the after-effects of different management strategies into a subsequent cropping phase. Experimental data has been gathered from two dryland cropping sites, one in Western Australia (annual pasture system) and one in north-eastern Victoria (perennial pasture system). Smoothed time series of weekly NDVI composites were constructed by means of the adaptive Savitzky-Golay filter. Eight complete phenological cycles (2004–2011) were processed and eleven phenology metrics were calculated for each cycle. The analysis reinforces previous results that indicated spatial variation in biomass between pasture and cropping phases is consistent and correlated over time. In the annual pasture system, two metrics showed strong correlation between cropping and pasture phases – end of season biomass and peak NDVI. In the perennial pasture system, strong correlations were observed for seven metrics – end of season biomass, time of season onset, end of season, season duration, peak NDVI, season amplitude and rate of NDVI decrease.

TIMESAT—a program for analyzing time-series of satellite sensor data

Three different least-squares methods for processing time-series of satellite sensor data are presented. The first method uses local polynomial functions and can be classified as an adaptive Savitzky–Golay filter. The other two methods are more clear cut least-squares methods, where data are fit to a basis of harmonic functions and asymmetric Gaussian functions, respectively. The methods incorporate qualitative information on cloud contamination from ancillary datasets. The resulting smooth curves are used for extracting seasonal parameters related to the growing seasons. The methods are implemented in a computer program, TIMESAT, and applied to NASA/NOAA Pathfinder AVHRR Land Normalized Difference Vegetation Index data over Africa, giving spatially coherent images of seasonal parameters such as beginnings and ends of growing seasons, seasonally integrated NDVI and seasonal amplitudes. Based on general principles, the TIMESAT program can be used also for other types of satellite-derived time-series data.