Whittaker Smoother Research Articles

Signal detrending in elastographic wave data with large motion artifacts through a 2D Whittaker smoother

A variety of advanced shear wave elastography techniques require the measurement of frequency-resolved phase velocity to properly characterize wave signals where factors such as tissue viscosity or guided wave effects are present. However, when applying these techniques in a dynamic environment where extrinsic motion cannot be limited, nonlinear and nonstationary trends can be expected in the motion signal used for phase velocity estimation. Because such extrinsic motion signals can contaminate a broad range of frequencies, they can produce significant errors in the phase velocity estimates. In this study, we propose a spatiotemporal detrending approach for elastography wave motion signals built from a 2D Whittaker smoother. This smoother has its origins in nonparametric regression and is well suited for fitting nonlinear curves of arbitrary shape. The resulting detrending filter was applied to Lamb wave signals from clinical ultrasound bladder vibrometry measurements. We found that the proposed detrending filter allowed for phase velocity estimates to be made in acquisitions that are otherwise be too corrupted by large motion. This works was supported by a grant from the NIH (DK131685)

Automated tuning of denoising algorithms for noise removal in chromatograms

Different algorithms, such as the Savitzky-Golay filter and Whittaker smoother, have been proposed to improve the quality of experimental chromatograms. These approaches avoid excessive noise from hampering data analysis and as such allow an accurate detection and quantification of analytes. These algorithms require fine-tuning of their hyperparameters to regulate their roughness and flexibility. Traditionally, this fine-tuning is done manually until a signal is obtained that removes the noise while conserving valuable peak information. More objective and automated approaches are available, but these are usually method specific and/or require previous knowledge.In this work, the L-and V-curve, k-fold cross-validation, autocorrelation function and residual variance estimation approach are introduced as alternative automated and generally applicable parameter tuning methods. These methods do not require any previous information and are compatible with a multitude of denoising methods. Additionally, for k-fold cross-validation, autocorrelation function and residual variance estimation, a novel implementation based on median estimators is proposed to handle the specific shape of chromatograms, typically composed of alternating flat baselines and sharp peaks. These tuning methods are investigated in combination with four denoising methods; the Savitsky-Golay filter, Whittaker smoother, sparsity assisted signal smoother and baseline estimation and denoising using sparsity approach.It is demonstrated that the median estimators approach significantly improves the denoising and information conservation performance of relevant smoother-tuner combinations up to a factor 4 for simulated datasets and even up to a factor 10 for an experimental chromatogram. Moreover, the parameter tuning methods relying on residual variance estimation, k-fold cross-validation and autocorrelation function lead to similar small root-mean squared errors on the different simulated datasets and experimental chromatograms. The sparsity assisted signal smoother and baseline estimation and denoising using sparsity approach, which both rely on the use of sparsity, systematically outperform the two other methods and are hence most appropriate for chromatograms.

Cloud-Free Global Maps of Essential Vegetation Traits Processed from the TOA Sentinel-3 Catalogue in Google Earth Engine

Global mapping of essential vegetation traits (EVTs) through data acquired by Earth-observing satellites provides a spatially explicit way to analyze the current vegetation states and dynamics of our planet. Although significant efforts have been made, there is still a lack of global and consistently derived multi-temporal trait maps that are cloud-free. Here we present the processing chain for the spatiotemporally continuous production of four EVTs at a global scale: (1) fraction of absorbed photosynthetically active radiation (FAPAR), (2) leaf area index (LAI), (3) fractional vegetation cover (FVC), and (4) leaf chlorophyll content (LCC). The proposed workflow presents a scalable processing approach to the global cloud-free mapping of the EVTs. Hybrid retrieval models, named S3-TOA-GPR-1.0-WS, were implemented into Google Earth Engine (GEE) using Sentinel-3 Ocean and Land Color Instrument (OLCI) Level-1B for the mapping of the four EVTs along with associated uncertainty estimates. We used the Whittaker smoother (WS) for the temporal reconstruction of the four EVTs, which led to continuous data streams, here applied to the year 2019. Cloud-free maps were produced at 5 km spatial resolution at 10-day time intervals. The consistency and plausibility of the EVT estimates for the resulting annual profiles were evaluated by per-pixel intra-annually correlating against corresponding vegetation products of both MODIS and Copernicus Global Land Service (CGLS). The most consistent results were obtained for LAI, which showed intra-annual correlations with an average Pearson correlation coefficient (R) of 0.57 against the CGLS LAI product. Globally, the EVT products showed consistent results, specifically obtaining higher correlation than R> 0.5 with reference products between 30 and 60° latitude in the Northern Hemisphere. Additionally, intra-annual goodness-of-fit statistics were also calculated locally against reference products over four distinct vegetated land covers. As a general trend, vegetated land covers with pronounced phenological dynamics led to high correlations between the different products. However, sparsely vegetated fields as well as areas near the equator linked to smaller seasonality led to lower correlations. We conclude that the global gap-free mapping of the four EVTs was overall consistent. Thanks to GEE, the entire OLCI L1B catalogue can be processed efficiently into the EVT products on a global scale and made cloud-free with the WS temporal reconstruction method. Additionally, GEE facilitates the workflow to be operationally applicable and easily accessible to the broader community.

Grassland cut detection based on Sentinel-2 time series to respond to the environmental and technical challenges of the Austrian fodder production for livestock feeding

The relationship between yield and quality of grassland fodder is an important factor for livestock farmers to consider when deciding the timing of grassland cuts. Even for experienced farmers, estimating this relationship is difficult because external factors such as weather can affect the growth rate of plants. Satellite remote sensing techniques have good potential for the modelling of fodder biomass and quality. However, more accurate grassland cut dates are needed. Unfortunately, current cut detection methods vary in both form, function and accuracy and a common framework for cut detection does not yet exist, especially for in-season cut estimation. The aim of this study is to develop an improved methodology for Austrian fodder production systems that can be used for both post-season and in-season cut detection. In the first case, cuts are detected at the end of the vegetation period and used for post-season assessment at national and regional levels. In the second case, cuts are detected in near-real-time during the season, and they can be used to model yield and quality. The approach is tested across the Austrian landscape on mown grasslands using freely available Copernicus Sentinel-2 imagery on the Google Earth Engine (GEE). The method uses a fitted NDVI curve obtained by iteratively smoothing the upper envelope of the actual observations using the Whittaker smoother. Cuts are detected by meeting a threshold in the difference between the fitted NDVI curve and the actual observed NDVI values. Additional innovative steps were developed to reduce the impact of cloud cover on the results for timely and accurate cut detection, including a gradient boosting algorithm to detect false positive cuts due to missed clouds. A set of rules for the method was trained on 97 plots and validated on a further 502 plots over three years (2020, 2021 and 2022). The method performed well in plots with two to four cuts per season, but occasionally struggled in more intensively managed systems, most likely due to a rule preventing two cuts within 27 days. A pooled post-season cut date detection f-score of 0.79 suggests that the method can accurately detect cuts, with absolute detection errors of <5 days. The main misclassifications were the results of (thin and semi-transparent) clouds being erroneously detected as cuts. With an f-score of 0.80 and total absolute delayed detection of approximately 6 days, the in-season cut date detection performs similarly to the post-season configuration. This indicates a high potential for near-real-time applications. Our results moreover indicate a general improvement of existing methodologies despite performing the cut detection across the fragmented, mountainous and diverse Austrian landscape. Future work will include grazed pastures and natural grassland with less intensive human activity.

Long-term dynamic monitoring of forest area changes with the aid of Google Earth Engine

Remote sensing has become the most effective tool for monitoring and evaluating forest resources. Owing to the development of big data, current research on forest resource information mainly focuses on a regional, national, or global scale. However, there are still certain limitations on small regional scales. We used supervised classification based on the Google Earth Engine cloud platform to extract county land use information and a Whittaker smoother to extract vegetation indices to monitor forest area and vegetation indices in the study areas from the 1980s to the 2020s. The results show that the classification accuracy is improved for the study area where the image quality is affected by clouds after adding complementary datasets of greenness and water bodies. The overall accuracy (OA) increases from 0.82 to 0.89, and the kappa accuracy increases from 0.77 to 0.85. The OA of the classification results was 0.88, and the forest area increased and decreased with an overall increase of 225.35 km2. The building area continued to increase with an increase of 34.88 km2. The cropland area first decreased and then increased with a complete decrease of 243.6 km2. The bare soil area decreased by 16.2 km2. The area of the water bodies was relatively stable. The vegetation index indicated good forest growth. Jiangle County’s forest area and forest growth stability cannot be achieved without the protection of national and local policies.

Recovering fluorescence spectra hidden by scattering signal: In search of the best smoother

Interpolation of the scattering areas in fluorescence excitation-emission matrices is a useful preprocessing method in fluorescence spectroscopy and data modelling. Commonly used row-by-row interpolation using piecewise cubic Hermite interpolating polynomials smoother (PCHIP), however, frequently leads to artifacts because it does not make any use of the information in the other dimension. We have suggested the way of constructing the penalty matrices for Whittaker smoothing that removed one of the main sources of difference between the axis of multiparametric signal – the grid step size – thus making it possible to reduce the number of parameters to optimize. We have compared Whittaker smoother with various surface interpolation methods, including LOESS, Kriging, multilevel B-spline approximation, and PCHIP for the purpose of data preprocessing before PARAFAC modelling of fluorescence signal on a model dataset. The two leaders by signal reconstruction and reconstruction of PARAFAC loadings are LOESS and Whittaker smoothing; the latter is additionally shown to have fundamentally interpretable parameters, which are easier to optimise for the whole dataset. Moreover, Whittaker keeps the shape of the signal and is resistant to variations in data structure and noise level that is very important in numerous applications. We also tested smoothers performance for Åsmund Rinnan fluorescence dataset and the high performance of Whittaker was proved. We can recommend the Whittaker smoothing as a perfect tool for interpolation of scattering areas in florescence spectra of seawaters with low signal-to-noise ratio.

Time-frequency optimization of RSEI: A case study of Yangtze River Basin

Remote Sensing Ecological Index (RSEI) is one of the most widely used ecological quality assessment indicators. Due to the noise caused by adverse atmoshperic conditions and other factors, the RSEI calculated from the original image usually has the phenomena of lack of information and unstable image quality. Therefore, based on Google Earth Engine (GEE) cloud platform, this study adopts three common data reconstruction algorithms firstly, namely: Savitory-Golay filter (SG), harmonic analysis of time series (HANTS), Whittaker Smoother (WS), which are used to reconstruct the original MODIS time series data in the Yangtze River Basin (YRB) from 2000 to 2020, in order to optimize the calculation process of RSEI. At the same time, three indicators (correlation coefficient (R), standard deviation (STD), root mean square error (RMSE)) are used for the accuracy evaluation. The results show that data reconstruction can fill gaps in RSEI, the reconstruction performance of WS and SG for four parameters is better than HANTS, and the four SG reconstructed sequences have the strongest correlation with the original sequences (R between 0.8 ∼ 1), while the WS reconstruction sequence has the lowest error value (both STD and RMSE are less than 1), both of them can correct the pixel value, which is conducive to maintaining the stability of RSEI in the temporal dimension; the RSEI produced by HANTS has the best accuracy, that is, R, STD, RMSE are respectively 0.898, 0.130, 0.104. As shown by the research, it is necessary to de-noise each parameter before synthesizing RSEI. This study can provide a theoretical basis for applying time-frequency algorithms to optimize the ecological monitoring performance of RSEI.

Deriving drought indices from MODIS vegetation indices (NDVI/EVI) and Land Surface Temperature (LST): Is data reconstruction necessary?

Droughts pose significant economic and ecological concerns, and considering climate change projections, timely monitoring and early warning based on satellite observations must be realized at regional to global scales. Nevertheless, whether data reconstruction is necessary to produce high-quality satellite-based time series data for drought monitoring and the data reconstruction approaches to be applied, if necessary, remain unclear. We attempted to fill this knowledge gap by investigating three widely used data reconstruction approaches, i.e., the Savitzky–Golay filter, harmonic analysis of time series (HANTS) and Whittaker Smoother, across the Lancang–Mekong river basin through the Moderate Resolution Imaging Spectroradiometer (MODIS) MOD13Q1 and MOD11A2 products for 2001–2018 and Google Earth Engine cloud platform. Several remote sensing drought indices based on the unreconstructed and reconstructed vegetation indices (VIs) and Land Surface Temperature (LST) were retrieved for basin-wide drought detection. The performance of the examined reconstruction approaches was evaluated using three statistical indices (coefficient of determination (R2), standard deviation of bias (BIAS(std)) and mean squared error (MSE)), spatial consistency with the reference dataset and capability to characterize the drought events. Data reconstruction considerably enhanced the drought index performances for drought detection; however, reconstruction was not necessary in all situations. The reconstructed drought indices exhibited higher R2 values (by 10.6–10.8%), lower BIAS(std) values (by −1.7– −12.5%), and smaller MSE values (by −5.8– −13.4%) compared to those of unreconstructed indices. For most evaluation indicators, HANTS outperformed the other methods, and Vegetation Condition Index (VCI) and Vegetation Health Index (VHI) outperformed the other drought indices. The findings highlight the importance of data reconstruction to detect and characterize drought events and the dependency of the performance of reconstruction methods on drought indices and evaluation metrics when using MODIS time series data.

Comparison of Remote Sensing Time-Series Smoothing Methods for Grassland Spring Phenology Extraction on the Qinghai–Tibetan Plateau

Accurate evaluation of start of season (SOS) changes is essential to assess the ecosystem’s response to climate change. Smoothing method is an understudied factor that can lead to great uncertainties in SOS extraction, and the applicable situation for different smoothing methods and the impact of smoothing parameters on SOS extraction accuracy are of critical importance to be clarified. In this paper, we use MOD13Q1 normalized difference vegetation index (NDVI) data and SOS observations from eight agrometeorological stations on the Qinghai–Tibetan Plateau (QTP) during 2001–2011 to compare the SOS extraction accuracies of six popular smoothing methods (Changing Weight (CW), Savitzky-Golay (SG), Asymmetric Gaussian (AG), Double-logistic (DL), Whittaker Smoother (WS) and Harmonic Analysis of NDVI Time-Series (HANTS)) for two types of different SOS extraction methods (dynamic threshold (DT) with 9 different thresholds and double logistic (Zhang)). Furthermore, a parameter sensitivity analysis for each smoothing method is performed to quantify the impacts of smoothing parameters on SOS extraction. Finally, the suggested smoothing methods and reference ranges for the parameters of different smoothing methods were given for grassland phenology extraction on the QTP. The main conclusions are as follows: (1) the smoothing methods and SOS extraction methods jointly determine the SOS extraction accuracy, and a bad denoising performance of smoothing method does not necessarily lead to a low SOS extraction accuracy; (2) the default parameters for most smoothing methods can result in acceptable SOS extraction accuracies, but for some smoothing methods (e.g., WS) a parameter optimization is necessary, and the optimal parameters of the smoothing method can increase the R2 and reduce the RMSE of SOS extraction by up to 25% and 331%; (3) The main influencing factor of the SOS extraction using the DT method is the stability of the minimum value in the NDVI curve, and for the Zhang method the curve shape before the peak of the NDVI curve impacts the most; (4) HANTS is the most stable method no matter with (fitness = 35.05) or without parameter optimization (fitness = 33.52), which is recommended for QTP grassland SOS extraction. The findings of this study imply that remote sensing-based vegetation phenology extraction can be highly uncertain, and a careful selection and parameterization of the time-series smoothing method should be taken to achieve an accurate result.

A Comparison of Three Temporal Smoothing Algorithms to Improve Land Cover Classification: A Case Study from NEPAL

Time series land cover data statistics often fluctuate abruptly due to seasonal impact and other noise in the input image. Temporal smoothing techniques are used to reduce the noise in time series data used in land cover mapping. The effects of smoothing may vary based on the smoothing method and land cover category. In this study, we compared the performance of Fourier transformation smoothing, Whittaker smoother and Linear-Fit averaging smoother on Landsat 5, 7 and 8 based yearly composites to classify land cover in Province No. 1 of Nepal. The performance of each smoother was tested based on whether it was applied on image composites or on land cover primitives generated using the random forest machine learning method. The land cover data used in the study was from the years 2000 to 2018. Probability distribution was examined to check the quality of primitives and accuracy of the final land cover maps were accessed. The best results were found for the Whittaker smoothing for stable classes and Fourier smoothing for other classes. The results also show that classification using a properly selected smoothing algorithm outperforms a classification based on its unsmoothed data set. The final land cover generated by combining the best results obtained from different smoothing approaches increased our overall land cover map accuracy from 79.18% to 83.44%. This study shows that smoothing can result in a substantial increase in the quality of the results and that the smoothing approach should be carefully considered for each land cover class.

Comparative assessment on smart pre-processing methods for extracting information in FT-NIR measured data

Fourier transform near infrared (FT-NIR) spectroscopy is a rapid and straightforward technology to analyse soil properties. However, FT-NIR measurement of soil is always accompanied with latent noises coming from masking and overlaps because of systematic bias and manual errors. Several pre-processing methods were investigated as smart pre-modelling modes for extracting data information and getting rid of the noises. Using least squares support vector regression (LSSVR), the FT-NIR calibration models were established for predicting target nutrients (nitrogen, phosphorus and potassium). With the combined investigation on noise-sensing pre-processing methods, the LSSVR models were automatically optimized by tuning its kernel parameters. Results show that the pre-processing methods were optimized in smart ways of polynomial modifications or function derivatives on quasi-continuous data. The effective pre-processing methods for the three target nutrients were originated from optical path length estimation and correction (OPLEC) and Whittaker smoother (WTK), switched to modification of the regulation polynomials and 1st- or 2nd- order derivatives, respectively. Thus we concluded that the modified OPLEC and the derivative WTK had strong potentials to replace the traditional pre-processing methods in FT-NIR analysis of soil nutrients.

Estimation of groundwater level based on the robust training of recurrent neural networks using corrupted data

In the present study, a cost function is developed for the robust training of recurrent neural-network models using groundwater-level data that are corrupted by outliers and noise. The optimal cost function in this study utilizes least trimmed squares (LTS) with asymmetric weighting (AW) and the Whittaker smoother (WS), which have different outlier- or noise-rejecting mechanisms. The developed cost function is benchmarked with other cost functions in the training of a long short-term memory (LSTM) model using data from the Gangjin–Seongjeon and Pohang–Gibuk monitoring wells in South Korea, the results of which are then compared to the validation data. Based on comparisons of the validation results, it is confirmed that the optimal cost function is the most successful in rejecting the influence of outliers during the training process when applied to data from the Gangjin–Seongjeon monitoring well. It is also demonstrated that the estimation results based on this optimal cost function can effectively identify outliers in groundwater-level data. For the Pohang–Gibuk monitoring well data, the optimal cost function without AW exhibits superior regularizing performance by generating the lowest mean estimation error. Using this cost function, the influence of persistent noise is mostly canceled out, and the estimation results reflect the regular changes in the water table level of a shallow aquifer over time. The developed robust cost function can potentially be employed in many hydrogeological applications, such as the monitoring of groundwater resources, the prediction and analysis of water table levels, and the identification of changes in aquifer processes. The cost function is also expected to be useful for many other field applications in which the data are susceptible to external influences.

Parametric-scaling optimization of pretreatment methods for the determination of trace/quasi-trace elements based on near infrared spectroscopy.

This work proposes a parametric-scaling strategy to optimize the pretreatments of near infrared (NIR) spectroscopic data, so as to cope with the difficulty of NIR technology in detecting trace or quasi-trace elements. This novel strategy helps enhancing the signal to noise ratio and contributes to extracting features from the raw spectrum, so that the information corresponding to the trace elements could be detected much easier. However, due to the complexity of NIR data, it is difficult to comprehensively evaluate and compare the performance of different pretreatment methods, especially when multiple target components are determined simultaneously. For this reason, we create some comprehensive model indicators to define the goodness of pretreatments in simultaneous multiple detection of trace elements. In this paper two near infrared data sets have been investigated, one is used to determinate the key indices in the primary screening of thalassemia and the other one is used to detect the heavy metal pollutants in farmland soil. Results show that the proposed parametric-scaling optimization strategy can improve the effect of pretreatments in the determination of trace/quasi-trace elements, and the model performance with the optimized pretreated data is significantly superior to that with the raw data. The optimized Savitzky-Golay smoother (SGS) keeps its merits in the real data examples. Especially, the newly emerged methods optical path length estimation and correction (OPLEC) and Whittaker smoother (WTK), as well as their parametric-scaling modified methods, show their advantages in the comparison with other pretreatments. According to the results of our experiments, they have shown promising potential in the NIR rapid analysis of trace/quasi-trace elements in the field of biomedical science and agricultural science. This is expected to be tested for other analytes with larger variation.

Spectral and chromatographic overall analysis: An insight into chemical equivalence assessment of traditional Chinese medicine

Several issues are associated with the use of high-performance liquid chromatography-diode-array detector (HPLC-DAD) analysis in the evaluation of the chemical equivalence of traditional Chinese medicine (TCM), such as the choice of representative wavelengths, quick and efficient integration, correct matching of different components, and full use of spectral information. These problems can be addressed by handling raw three-dimensional (3D) data in a different manner, which led to the development of the spectral and chromatographic overall analysis (SCOA) method. In SCOA, the Whittaker smoother is used to eliminate the baseline drift caused by solvents, and the projection profiling is extracted from 3D data as the most informative chromatogram. Peak matching takes advantage of spectral similarity to exclude spectrally dissimilar components in the matching group. Subsequently, the exponential-Gaussian hybrid function (EGH) is used to fit and integrate the projection profiling, thus avoiding the cumbersome and time-consuming integration process. Two data sets were used to validate and demonstrate the SCOA. The results show that the SCOA is more comprehensive and reliable than HPLC-DAD analysis in the chemical equivalence assessment of TCM.

Modeling physical activity data using L0 -penalized expectile regression.

In recent years accelerometers have become widely used to objectively assess physical activity. Usually intensity ranges are assigned to the measured accelerometer counts by simple cut points, disregarding the underlying activity pattern. Under the assumption that physical activity can be seen as distinct sequence of distinguishable activities, the use of hidden Markov models (HMM) has been proposed to improve the modeling of accelerometer data. As further improvement we propose to use expectile regression utilizing a Whittaker smoother with an L0 -penalty to better capture the intensity levels underlying the observed counts. Different expectile asymmetries beyond the mean allow the distinction of monotonous and more variable activities as expectiles effectively model the complete distribution of the counts. This new approach is investigated in a simulation study, where we simulated 1,000 days of accelerometer data with 1 and 5 s epochs, based on collected labeled data to resemble real-life data as closely as possible. The expectile regression is compared to HMMs and the commonly used cut point method with regard to misclassification rate, number of identified bouts and identified levels as well as the proportion of the estimate being in the range of of the true activity level. In summary, expectile regression utilizing a Whittaker smoother with an L0 -penalty outperforms HMMs and the cut point method and is hence a promising approach to model accelerometerdata.

Applications of the Whittaker smoother in NMR spectroscopy

This review presents a general survey of the main applications of the Whitttaker Smoother in 1D and 2D NMR spectroscopy, including smoothing, interpolation, solvent suppression and baseline correction. Juan Carlos Cobas Gómez, Ph.D. Dr. Carlos Cobas is co-founder, President of Mestrelab Research and director of the Research and Development department at Mestrelab Research. He was born in Santiago de Compostela, Spain, in 1971. He received his B.Sc. in Chemistry from the University of Santiago de Compostela. After spending a year at the University of Leicester (U.K.) under the supervision of Dr. Paul R. Jenkins, he returned to the University of Santiago, where he obtained a M.Sc. degree in 1995 and a Ph.D. in 2000. At the end of 2004, after 8 years working on the MestReC project, he took the decision to spin off from the Universidad de Santiago de Compostela and launch Mestrelab Research. His research activities cover diverse applications of NMR spectroscopy, from signal processing and analysis algorithms to quantum simulation of NMR data, as well as methodology and applications of automatic structure verification systems by NMR and LC/GC/MS. He currently has a particular focus on the application of machine learning techniques in the fields of spectroscopy and chemoinformatics.

Applications of the Whittaker smoother in NMR spectroscopy.

The Whittaker smoother, a special case of penalized least square, is a multipurpose algorithm that has proven to be very useful in many scientific fields, including image processing, chromatography, and optical spectroscopy. It shares many similarities with the Savitzky-Golay algorithm, but it is significantly faster and easier to automate. Its use in nuclear magnetic resonance, however, is not widespread although several applications have recently been published. In this review, the mathematical background of the method and its main applications in nuclear magnetic resonance spectroscopy will be discussed.

Multi-objective optimization for an automated and simultaneous phase and baseline correction of NMR spectral data

Spectral data preprocessing is an integral and sometimes inevitable part of chemometric analyses. For Nuclear Magnetic Resonance (NMR) spectra a possible first preprocessing step is a phase correction which is applied to the Fourier transformed free induction decay (FID) signal. This preprocessing step can be followed by a separate baseline correction step. Especially if series of high-resolution spectra are considered, then automated and computationally fast preprocessing routines are desirable.A new method is suggested that applies the phase and the baseline corrections simultaneously in an automated form without manual input, which distinguishes this work from other approaches. The underlying multi-objective optimization or Pareto optimization provides improved results compared to consecutively applied correction steps. The optimization process uses an objective function which applies strong penalty constraints and weaker regularization conditions. The new method includes an approach for the detection of zero baseline regions. The baseline correction uses a modified Whittaker smoother. The functionality of the new method is demonstrated for experimental NMR spectra. The results are verified against gravimetric data. The method is compared to alternative preprocessing tools. Additionally, the simultaneous correction method is compared to a consecutive application of the two correction steps.

Two dimensional smoothing via an optimised Whittaker smoother

BackgroundIn many applications where moderate to large datasets are used, plotting relationships between pairs of variables can be problematic. A large number of observations will produce a scatter-plot which is difficult to investigate due to a high concentration of points on a simple graph.In this article we review the Whittaker smoother for enhancing scatter-plots and smoothing data in two dimensions. To optimise the behaviour of the smoother an algorithm is introduced, which is easy to programme and computationally efficient.ResultsThe methods are illustrated using a simple dataset and simulations in two dimensions. Additionally, a noisy mammography is analysed. When smoothing scatterplots the Whittaker smoother is a valuable tool that produces enhanced images that are not distorted by the large number of points. The methods is also useful for sharpening patterns or removing noise in distorted images.ConclusionThe Whittaker smoother can be a valuable tool in producing better visualisations of big data or filter distorted images. The suggested optimisation method is easy to programme and can be applied with low computational cost.

Global evaluation of gap-filling approaches for seasonal NDVI with considering vegetation growth trajectory, protection of key point, noise resistance and curve stability

A variety of approaches are available to fill the gaps in the time series of vegetation parameters estimated from satellite observations. In this paper, a scheme considering vegetation growth trajectory, protection of key point, noise resistance and curve stability was proposed to evaluate the gap-filling approaches. Six approaches for gap filling were globally evaluated pixel-by-pixel based on a reference NDVI generated from MODIS observations during the past 15years. The evaluated approaches include the Fourier-based approach (Fourier), the double logistic model (DL), the iterative interpolation for data reconstruction (IDR), the Whittaker smoother (Whit), the Savitzky-Golay filter (SG) and the locally adjusted cubic spline capping approach (LACC). Considering the five aspects, the ranks of the overall performance are LACC>Fourier>IDR>DL>SG>Whit. The six approaches are similar in filling the gaps and remaining the curve stability but there are large difference in protection of key points and noise resistance. The SG is sensitive to noises and the Whit is poor in protection of key points. In the monsoon regions of India, all evaluated approaches don't work well. This paper provides some new views for evaluating the gap filling approaches that will be helpful in selecting the optimal approach to reconstruct the time series of parameters for data applications.