Iterative Polynomial Fitting Research Articles

An Iterative Curve-Fitting Baseline Correction Method for Raman Spectra Driven by Neural Network.

Baseline correction is a vital part of spectral preprocessing, especially for Raman spectra. Iterative polynomial fitting is an easy but less accurate way to find baselines compared to other methods such as wavelet transform and penalized least squares (PLS) methods. The polynomial fitting methods can also get distorted results in certain conditions. In this paper, a neural network model for detecting the trend of the baseline was proposed to improve the correction accuracy of the fitting methods. The model selects the function basis according to the baseline trend instead of using a fixed polynomial function to match the baseline for a more precise fit. We also propose a way to generate simulation data, these data can be used to train the neural network model. The model provides reliable results for real spectral data with noise. Our method provides a new idea to correct the baseline with a strange shape. In addition, we examine the limitations of conventional iterative polynomial fitting, adaptive iteratively reweighted PLS and explain why our approach surpasses these methods.

Semi-automated Generation of Geometric Digital Twin for Bridge Based on Terrestrial Laser Scanning Data

Digital twins (DTs) have a great potential for bridge operation and maintenance. Geometric digital twins (gDTs) are the key component of DTs. At present, a growing number of researchers are using high-precision 3D laser point clouds to generate gDTs. However, for large bridges, such as arch, cable-stayed, and suspension bridges, comprehensive point-cloud collection stations are difficult to set up due to their large span, narrow site, and limited field of vision. Consequently, the complete point clouds of these bridges cannot be easily obtained. Thus, knowing how to process absence point clouds and generate gDTs is an urgent problem. This study proposes a semiautomatic method of extracting geometric information of a bridge’s components in the absence of point clouds. First, an algorithm based on the combination of the iterative polynomial fitting curve and sliding window is developed to extract the arch ring accurately. Second, an improved random sample consensus (RANSAC) algorithm based on distribution density is adopted to extract the cross sections of the arch bridge components, except the arch ring. For cross sections that lack point clouds, a translation strategy is used to supplement the unknown line segment. Finally, for the T-beam, a model alignment method is proposed to best match the characteristic intersections extracted by the improved RANSAC algorithm and the points corresponding to the design model. The quality of the generated models is gauged using a point cloud deviation chromatogram. In addition, the stressed component piers are compared with its design parameters to verify the accuracy of the proposed method. Results show that our method can efficiently and accurately extract geometric information and generate gDT for the bridge.

Impact of preprocessing methods on the Raman spectra of brain tissue.

Delineating cancer tissue while leaving functional tissue intact is crucial in brain tumor resection. Despite several available aids, surgeons are limited by preoperative or subjective tools. Raman spectroscopy is a label-free optical technique with promising indications for tumor tissue identification. To allow direct comparisons between measurements preprocessing of the Raman signal is required. There are many recognized methods for preprocessing Raman spectra; however, there is no universal standard. In this paper, six different preprocessing methods were tested on Raman spectra (n > 900) from fresh brain tissue samples (n = 34). The sample cohort included both primary brain tumors, such as adult-type diffuse gliomas and meningiomas, as well as metastases of breast cancer. Each tissue sample was classified according to the CNS WHO 2021 guidelines. The six methods include both direct and iterative polynomial fitting, mathematical morphology, signal derivative, commercial software, and a neural network. Data exploration was performed using principal component analysis, t-distributed stochastic neighbor embedding, and k-means clustering. For each of the six methods, the parameter combination that explained the most variance in the data, i.e., resulting in the highest Gap-statistic, was chosen and compared to the other five methods. Depending on the preprocessing method, the resulting clusters varied in number, size, and associated spectral features. The detected features were associated with hemoglobin, neuroglobin, carotenoid, water, and protoporphyrin, as well as proteins and lipids. However, the spectral features seen in the Raman spectra could not be unambiguously assigned to tissue labels, regardless of preprocessing method. We have illustrated that depending on the chosen preprocessing method, the spectral appearance of Raman features from brain tumor tissue can change. Therefore, we argue both for caution in comparing spectral features from different Raman studies, as well as the importance of transparency of methodology and implementation of the preprocessing. As discussed in this study, Raman spectroscopy for in vivo guidance in neurosurgery requires fast and adaptive preprocessing. On this basis, a pre-trained neural network appears to be a promising approach for the operating room.

A Method for Capture and Detection of Crop Airborne Disease Spores Based on Microfluidic Chips and Micro Raman Spectroscopy.

Airborne crop diseases cause great losses to agricultural production and can affect people's physical health. Timely monitoring of the situation of airborne disease spores and effective prevention and control measures are particularly important. In this study, a two-stage separation and enrichment microfluidic chip with arcuate pretreatment channel was designed for the separation and enrichment of crop disease spores, which was combined with micro Raman for Raman fingerprinting of disease conidia and quasi identification. The chip was mainly composed of arc preprocessing and two separated enriched structures, and the designed chip was numerically simulated using COMSOL multiphysics5.5, with the best enrichment effect at W2/W1 = 1.6 and W4/W3 = 1.1. The spectra were preprocessed with standard normal variables (SNVs) to improve the signal-to-noise ratio, which was baseline corrected using an iterative polynomial fitting method to further improve spectral features. Raman spectra were dimensionally reduced using principal component analysis (PCA) and stability competitive adaptive weighting (SCARS), support vector machine (SVM) and back-propagation artificial neural network (BPANN) were employed to identify fungal spore species, and the best discrimination effect was achieved using the SCARS-SVM model with 94.31% discrimination accuracy. Thus, the microfluidic-chip- and micro-Raman-based methods for spore capture and identification of crop diseases have the potential to be precise, convenient, and low-cost methods for fungal spore detection.

A machine learning based method for automatic differential scanning calorimetry signal analysis

Differential scanning calorimetry (DSC) is a powerful technique to study temperature induced phase transitions by monitoring the heat capacity changes. Traditional ways of DSC data analysis require manual baseline subtraction and peak analysis, which inevitably leads to errors caused by human bias. To tackle this long-standing challenge, we propose an automated method for DSC signal extraction and baseline estimation based on semi-supervised machine learning. We implement an exponential modified Gaussian mixture (EMGM) model to identify the signal of interest, and use the expectation–maximization algorithm to optimize the log-likelihood of the model. This method is then combined with the iterative polynomial fitting method for baseline allocation. One advantage of the method is not requiring the knowledge of the signal, as it is learned through matrix factorization. We demonstrate the method’s efficacy using three types of protein data measured by distinctive DSC instruments. It can effectively identify the signals of interest from the raw signal and perform proper baseline subtraction. Furthermore, the program can accurately obtain the thermodynamic parameters from the peak signals for thermal characterization. In summary, this work’s automated signal processing method improves the speed and accuracy of the DSC data interpretation. The code and data for this work can be found at: https://github.com/shuyu-wang/DSC_data_analysis/.

Baseline Correction Based on a Search Algorithm from Artificial Intelligence.

Many spectra have a polynomial-like baseline. Iterative polynomial fitting is one of the most popular methods for baseline correction of these spectra. However, the baseline estimated by iterative polynomial fitting may have a substantial error when the spectrum contains significantly strong peaks or have strong peaks located at the endpoints. First, iterative polynomial fitting uses temporary baseline estimated from the current spectrum to identify peak data points. If the current spectrum contains strong peaks, then the temporary baseline substantially deviates from the true baseline. Some good baseline data points of the spectrum might be mistakenly identified as peak data points and are artificially re-assigned with a low value. Second, if a strong peak is located at the endpoint of the spectrum, then the endpoint region of the estimated baseline might have a significant error due to overfitting. This study proposes a search algorithm-based baseline correction method (SA) that aims to compress sample the raw spectrum to a dataset with small number of data points and then convert the peak removal process into solving a search problem in artificial intelligence to minimize an objective function by deleting peak data points. First, the raw spectrum is smoothened out by the moving average method to reduce noise and then divided into dozens of unequally spaced sections on the basis of Chebyshev nodes. Finally, the minimal points of each section are collected to form a dataset for peak removal through search algorithm. SA selects the mean absolute error as the objective function because of its sensitivity to overfitting and rapid calculation. The baseline correction performance of SA is compared with those of three baseline correction methods, the Lieber and Mahadevan-Jansen method, adaptive iteratively reweighted penalized least squares method, and improved asymmetric least squares method. Simulated and real Fourier transform infrared and Raman spectra with polynomial-like baselines are employed in the experiments. Results show that for these spectra the baseline estimated by SA has fewer error than those by the three other methods.

RPE layer detection and baseline estimation using statistical methods and randomization for classification of AMD from retinal OCT

Background and Objective: Age-related macular degeneration (AMD) is a condition of the eye that affects the aged people. Optical coherence tomography (OCT) is a diagnostic tool capable of analyzing and identifying the disease affected retinal layers with high resolution. The objective of this work is to extract the retinal pigment epithelium (RPE) layer and the baseline (natural eye curvature, particular to every patient) from retinal spectral-domain OCT (SD-OCT) images. It uses them to find the height of drusen (abnormalities) in the RPE layer and classify it as AMD or normal.Methods: In the proposed work, the contrast enhancement based adaptive denoising technique is used for speckle elimination. Pixel grouping and iterative elimination based on the knowledge of typical layer intensities and positions are used to obtain the RPE layer. Using this estimate, randomization techniques are employed, followed by polynomial fitting and drusen removal to arrive at a baseline estimate. The classification is based on the drusen height obtained by taking the difference between the RPE and baseline levels. We have used a patient, wise classification approach where a patient is classified diseased if more than a threshold number of patient images have drusen of more than a certain height. Since all slices of an affected patient will not show drusen, we are justified in adopting this technique.Results: The proposed method is tested on a public data set of 2130 images/slices, which belonged to 30 patient volumes (15 AMD and 15 Normal) and achieved an overall accuracy of 96.66%, with no false positives. In comparison with existing works, the proposed method achieved higher overall accuracy and a better baseline estimate.Conclusions: The proposed work focuses on AMD/normal classification using a statistical approach. It does not require any training. The proposed method modifies the motion restoration paradigm to obtain an application-specific denoising algorithm. The existing RPE detection algorithm is modified significantly to make it robust and applicable even to images where the RPE is not very evident/there is a significant amount of perforations (drusen). The baseline estimation algorithm employs a powerful combination of randomization, iterative polynomial fitting, and pixel elimination in contrast to mere fitting techniques. The main highlight of this work is, it achieved an exact estimation of the baseline in the retinal image compared to the existing methods.

基于改进迭代多项式拟合的红外光谱基线校正方法

基于改进迭代多项式拟合的红外光谱基线校正方法

Hapke-based computational method to enable unmixing of hyperspectral data of common salts

Environmental scientists are currently assessing the ability of hyper-spectral remote sensing to detect, identify, and analyze natural components, including minerals, rocks, vegetation and soil. This paper discusses the use of a nonlinear reflectance model to distinguish multicomponent particulate mixtures. Analysis of the data presented in this paper shows that, although the identity of the components can often be found from diagnostic wavelengths of absorption bands, the quantitative abundance determination requires knowledge of the complex refractive indices and average particle scattering albedo, phase function and size. The present study developed a method for spectrally unmixing halite and gypsum combinations. Using the known refractive indexes of the components, and with the assistance of Hapke theory and Legendre polynomials, the authors develop a method to find the component particle sizes and mixing coefficients for blends of halite and gypsum. Material factors in the method include phase function parameters, bidirectional reflectance, imaginary index, grain sizes, and iterative polynomial fitting. The obtained Hapke parameters from the best-fit approach were comparable to those reported in the literature. After the optical constants (n, the so-called real index of refraction and k, the coefficient of the imaginary index of refraction) are derived, and the geometric parameters are determined, single-scattering albedo (or ω) can be calculated and spectral unmixing becomes possible.

Extraction of digital terrain model based on regular mesh generation in mountainous areas

Airborne LiDAR technology is a popular technique to quickly acquire high-precision information of the ground and the objects above it. Moreover, filtering the data is one of the necessary core processing steps. In recent years, many algorithms have been derived from traditional filtering algorithms. However, when applied to LiDAR data obtained from mountainous regions, most algorithms generate numerous omissions and errors when attempting to retain steep terrain features and filter vegetation information. This paper aims to quickly and accurately extract the digital terrain model (DTM) in mountainous regions. To accomplish this goal, a filtering algorithm based on a regular mesh generation strategy is proposed. The proposed algorithm initially divides the huge amount of point cloud data into strips and selects the appropriate spacing to subdivide each strip into equidistant grid data. The grid data is used as the input to an iterative polynomial fitting process, after which the point cloud is classified based on a controlled threshold. The experimental results show that the proposed method can quickly and efficiently classify data of mountains with different characteristics while retaining terrain feature information better than other algorithms. The average accuracy of recognition is greater than 92%. In addition, this algorithm also applies to mountains with lush vegetation.

Iterative Curve Fitting: A Robust Technique to Estimate the Wavelength Position and Depth of Absorption Features From Spectral Data

In this paper, we introduce a robust method coined iterative curve fitting to estimate the wavelength position and depth from spectral absorption features. The technique iteratively fits a curve to a continuum-removed spectrum and subsets the bands based on the minimum of the previous fit until fulfilling a specified threshold for residual error. The minimum of the latest iteration and its substituted reflectance value are then retrieved as the feature wavelength and depth. Two variants of the technique named iterative Gaussian fitting and iterative polynomial fitting (IPF) are presented. The superiority of these algorithms over current methods is demonstrated using four different absorption features between 400 and 2500 nm collected from an array of sandstones in the laboratory. The methods can achieve rmse values of ±1.0 nm for the wavelength and 1% for the feature depth. The estimated wavelength position in a hyperspectral sensor with less than 10-nm sampling interval is demonstrated to be in error by at most ±3 nm at 95% confidence level. Experiments with varying signal-to-noise ratios (SNRs) indicate the robustness of the technique against noise. The IPF is able to estimate the wavelength of narrow features with an rmse of ±2.7 nm at an SNR of 150 : 1 and broad features with an rmse of ±4.2 nm at an SNR of 400 : 1. The method, which is embedded in a package named Automated Absorption-based Mineral Spectral Analyzer (AMISA), enables the simultaneous calculation of width, area, and asymmetry of spectral data acquired from imaging and nonimaging sensors.

Flexible binding simulation by a novel and improved version of virtual-system coupled adaptive umbrella sampling

Virtual-system coupled adaptive umbrella sampling (VAUS) enhances sampling along a reaction coordinate by using a virtual degree of freedom. However, VAUS and regular adaptive umbrella sampling (AUS) methods are yet computationally expensive. To decrease the computational burden further, improvements of VAUS for all-atom explicit solvent simulation are presented here. The improvements include probability distribution calculation by a Markov approximation; parameterization of biasing forces by iterative polynomial fitting; and force scaling. These when applied to study Ala-pentapeptide dimerization in explicit solvent showed advantage over regular AUS. By using improved VAUS larger biological systems are amenable.

Estimating complicated baselines in analytical signals using the iterative training of Bayesian regularized artificial neural networks

The present work deals with the development of a new baseline correction method based on the comparative learning capabilities of artificial neural networks. The developed method uses the Bayes probability theorem for prevention of the occurrence of the over-fitting and finding a generalized baseline. The developed method has been applied on simulated and real metabolomic gas-chromatography (GC) and Raman data sets. The results revealed that the proposed method can be used to handle different types of baselines with cave, convex, curvelinear, triangular and sinusoidal patterns. For further evaluation of the performances of this method, it has been compared with benchmarking baseline correction methods such as corner-cutting (CC), morphological weighted penalized least squares (MPLS), adaptive iteratively-reweighted penalized least squares (airPLS) and iterative polynomial fitting (iPF). In order to compare the methods, the projected difference resolution (PDR) criterion has been calculated for the data before and after the baseline correction procedure. The calculated values of PDR after the baseline correction using iBRANN, airPLS, MPLS, iPF and CC algorithms for the GC metabolomic data were 4.18, 3.64, 3.88, 1.88 and 3.08, respectively. The obtained results in this work demonstrated that the developed iterative Bayesian regularized neural network (iBRANN) method in this work thoroughly detects the baselines and is superior over the CC, MPLS, airPLS and iPF techniques. A graphical user interface has been developed for the suggested algorithm and can be used for easy implementation of the iBRANN algorithm for the correction of different chromatography, NMR and Raman data sets.

Automatic Baseline Recognition for the Correction of Large Sets of Spectra Using Continuous Wavelet Transform and Iterative Fitting

A new algorithm for the automatic recognition of peak and baseline regions in spectra is presented. It is part of a study to devise a baseline correction method that is particularly suitable for the simple and fast treatment of large amounts of data of the same type, such as those coming from high-throughput instruments, images, process monitoring, etc. This algorithm is based on the continuous wavelet transform, and its parameters are automatically determined using the criteria of Shannon entropy and the statistical distribution of noise, requiring virtually no user intervention. It was assessed on simulated spectra with different noise levels and baseline amplitudes, successfully recognizing the baseline points in all cases but for a few extremely weak and noisy signals. It can be combined with various fitting methods for baseline estimation and correction. In this work, it was used together with an iterative polynomial fitting to successfully process a real Raman image of 40,000 pixels in about 2.5 h.

Modifications of iterative schemes used for curvature correction in noninvasive biomedical imaging

Iterative polynomial fitting along image rows and columns has recently been used to remove curvature bias in multispectral image sets of the human forearm and phantoms. However, this method is only applicable if foreground and background features satisfy strong separation conditions. In this method, we verify that the iterative polynomial approach converges toward bivariate polynomial fitting, and, hence, the resulting fit corresponds to low-pass filtering the image. In contrast to the iterative fitting, the bivariate polynomial fit can be performed on images with missing or excluded parts. Indeed, our observation enables us to modify the scheme and significantly weaken the required assumptions on foreground/background separation allowing a wider range of application.

Baseline correction by improved iterative polynomial fitting with automatic threshold

An iterative polynomial fitting method is proposed for the estimate of the baseline of chemical signal. The new method generates automatic thresholds by comparing the chemical signal with the calculated signal from polynomial fitting in the iterative processes. The signal peaks are cut out consecutively in the iterative processes so the polynomial fitting will finally give a good estimation of the baseline. Simulated data and real data from capillary electrophoresis experiment are used to demonstrate the feasibility of the proposed method.