Nonlinear Regression Algorithm Research Articles

Distributed statistical learning algorithm for nonlinear regression with autoregressive errors

The growing size of modern data brings challenges to statistical learning, and substantial distributed algorithms have been proposed. However, most of them need the homogeneity assumption that the distribution of the local data is the same as that of the global data. This is seldom in practice, and the learning performance deteriorates seriously if this assumption is not satisfied. Moreover, they are only for independent data, and cannot incorporate the serial correlations between data. To solve these issues, we propose a novel distributed statistical learning framework for the nonlinear regression with autoregressive errors, which realizes communication-efficient distributed optimization, and overcomes the homogeneity assumption. The theoretical results also guarantee that the new distributed framework is equivalent to the global one. Numerical experiments also illustrate the good performance of the new method.

Biomolecular Adsorption on Nanomaterials: Combining Molecular Simulations with Machine Learning.

Adsorption free energies of 32 small biomolecules (amino acids side chains, fragments of lipids, and sugar molecules) on 33 different nanomaterials, computed by the molecular dynamics - metadynamics methodology, have been analyzed using statistical machine learning approaches. Multiple unsupervised learning algorithms (principal component analysis, agglomerative clustering, and K-means) as well as supervised linear and nonlinear regression algorithms (linear regression, AdaBoost ensemble learning, artificial neural network) have been applied. As a result, a small set of biomolecules has been identified, knowledge of adsorption free energies of which to a specific nanomaterial can be used to predict, within the developed machine learning model, adsorption free energies of other biomolecules. Furthermore, the methodology of grouping of nanomaterials according to their interactions with biomolecules has been presented.

A continuous cuffless blood pressure measurement from optimal PPG characteristic features using machine learning algorithms

Background and objectiveHypertension is a potentially dangerous health condition that can be detected by measuring blood pressure (BP). Blood pressure monitoring and measurement are essential for preventing and treating cardiovascular diseases. Cuff-based devices, on the other hand, are uncomfortable and prevent continuous BP measurement. MethodsIn this study, a new non-invasive and cuff-less method for estimating Systolic Blood Pressure (SBP), Mean Arterial Pressure (MAP), and Diastolic Blood Pressure (DBP) has been proposed using characteristic features of photoplethysmogram (PPG) signals and nonlinear regression algorithms. PPG signals were collected from 219 participants, which were then subjected to preprocessing and feature extraction steps. Analyzing PPG and its derivative signals, a total of 46 time, frequency, and time-frequency domain features were extracted. In addition, the age and gender of each subject were also included as features. Further, correlation-based feature selection (CFS) and Relief F feature selection (ReliefF) techniques were used to select the relevant features and reduce the possibility of over-fitting the models. Finally, support vector regression (SVR), K-nearest neighbour regression (KNR), decision tree regression (DTR), and random forest regression (RFR) were established to develop the BP estimation model. Regression models were trained and evaluated on all features as well as selected features. The best regression models for SBP, MAP, and DBP estimations were selected separately. ResultsThe SVR model, along with the ReliefF-based feature selection algorithm, outperforms other algorithms in estimating the SBP, MAP, and DBP with the mean absolute error of 2.49, 1.62 and 1.43 mmHg, respectively. The proposed method meets the Advancement of Medical Instrumentation standard for BP estimations. Based on the British Hypertension Society standard, the results also fall within Grade A for SBP, MAP, and DBP. ConclusionThe findings show that the method can be used to estimate blood pressure non-invasively, without using a cuff or calibration, and only by utilizing the PPG signal characteristic features.

An effective Q extraction method via deep learning

Abstract The quality factor (Q) is a parameter reflecting the physical properties of reservoirs. Accurate estimation of the quality factor plays an important role in improving the resolution of seismic data. Spectral-ratio method is a widely used traditional method based on the linear least-squares fitting to extract the quality factor, but is sensitive to noise. This is the main reason preventing this method from being widely used. Some supervised deep-learning methods are proposed to extract Q in which the construction of training labels is a key link. The proposed method is based on the spectral-ratio method to create training labels, avoiding errors in generating them. In contrast to the least-squares method, this paper proposes to use a nonlinear regression algorithm based on a fully connected network to fit the spectral logarithmic ratio and frequency. Meanwhile, the empirical equation is applied to constrain prediction results. The proposed method can effectively overcome the influence of noise and improve the accuracy of prediction results. Tests on the synthesized data of vertical seismic profile and common middle profile show that the proposed method has better generalization ability than the spectral-ratio method. Applying the method to the field vertical seismic profile data successfully extracts the quality factor, which can provide effective information for dividing stratigraphic layers.

Exploring the Potential of Multi-Temporal Crop Canopy Models and Vegetation Indices from Pleiades Imagery for Yield Estimation

In this paper, we demonstrate the capabilities of Pleiades-1a imagery for very high resolution (VHR) crop yield estimation by utilizing the predictor variables from the horizontal-spectral information, through Normalized Difference Vegetation Indices (NDVI), and the vertical-volumetric crop characteristics, through the derivation of Crop Canopy Models (CCMs), from the stereo imaging capacity of the satellite. CCMs captured by Unmanned Aerial Vehicles are widely used in precision farming applications, but they are not suitable for the mapping of large or inaccessible areas. We further explore the spatiotemporal relationship of the CCMs and the NDVI for five observation dates during the growing season for eight selected crop fields in Germany with harvester-measured ground truth crop yield. Moreover, we explore different CCM normalization methods, as well as linear and non-linear regression algorithms, for the crop yield estimation. Overall, using the Extremely Randomized Trees regression, the combination of CCMs and NDVI achieves an R2 coefficient of determination of 0.92.

Incorporation of first-order backscattered power in Water Cloud Model for improving the Leaf Area Index and Soil Moisture retrieval using dual-polarized Sentinel-1 SAR data

The novel modification in the WCM (mWCM) is proposed in this study to simulate total backscattering contribution and to improve Leaf Area Index (LAI) and Soil Moisture (SM) retrieval using Sentinel-1 Single Look Complex (SLC) datasets at VV and VH polarizations for the wheat crop. The intended modification was achieved through two steps; (1) Proposing the scaling constants of vegetation (fveg), soil (fsoil) and vegetation-soil interaction (finter) within the traditional WCM. The scaling constants are dimensionless quantity and were derived utilizing the degree of polarization (which were computed using the Hermitian covariance matrix); (2) Incorporation of the first order scattering component derived from the novel Vegetation-Soil Scattering Model (VSSM) to the total backscattering within the traditional WCM. The aim of including the vegetation-soil interaction in land surface models is to predict the combined effect of vegetation and soil on the total backscattering. The model parameters (i.e., A, B, C, and E) were calibrated using a non-linear least square regression algorithm. The accuracy of the retrieved and measured LAI and SM is evaluated using the different statistical indicators, e.g., coefficient of determination (R2), Root Mean Square Error (RMSE), and Nash Sutcliffe Efficiency (NSE). The retrieval from mWCM produced better accuracy with lower error than traditional WCM. The forward simulation results of mWCM revealed a notably higher accuracy for the total simulated radar backscattered coefficient at the VH polarization (σ0VH). The VH results showed a high R2 = 0.86, a high NSE = 0.85, and a low RMSE = 0.51 dB, outperforming the simulated σ0VV with R2 = 0.84, NSE = 0.84, RMSE = 0.66 dB. Consequently, the inversion of the mWCM yielded significantly improved accuracy in retrieving LAI at the VH polarization. The VH retrieval results exhibited a R2 = 0.80, NSE = 0.78, and RMSE = 0.44 m2m−2, while the VV polarization achieved an R2 = 0.78, NSE = 0.77, and RMSE = 0.53 m2m−2 for LAI estimation. In SM retrieval, higher accuracy was observed at VV polarization with R2 value of 0.77, NSE value = 0.79, and RMSE = 0.048 m3m−3 than the VH polarization with R2 = 0.75, NSE = 0.76, and RMSE = 0.050 m3m−3.

Machine-learning-based predictions of imprinting quality using ensemble and non-linear regression algorithms

The molecularly imprinted polymers are artificial polymers that, during the synthesis, create specific sites for a definite purpose. These polymers due to their characteristics such as stability, easy of synthesis, reproducibility, reusability, high accuracy, and selectivity have many applications. However, the variety of the functional monomers, templates, solvents, and synthesis conditions like pH, temperature, the rate of stirring, and time, limit the selectivity of imprinting. The Practical optimization of the synthetic conditions has many drawbacks, including chemical compound usage, equipment requirements, and time costs. The use of machine learning (ML) for the prediction of the imprinting factor (IF), which indicates the quality of imprinting is a very interesting idea to overcome these problems. The ML has many advantages, for example a lack of human error, high accuracy, high repeatability, and prediction of a large amount of data in the minimum time. In this research, ML was used to predict the IF using non-linear regression algorithms, including classification and regression tree, support vector regression, and k-nearest neighbors, and ensemble algorithms, like gradient boosting (GB), random forest, and extra trees. The data sets were obtained practically in the laboratory, and inputs, included pH, the type of the template, the type of the monomer, solvent, the distribution coefficient of the MIP (KMIP), and the distribution coefficient of the non-imprinted polymer (KNIP). The mutual information feature selection method was used to select the important features affecting the IF. The results showed that the GB algorithm had the best performance in predicting the IF, and using this algorithm, the maximum R2 value (R2 = 0.871), and the minimum mean absolute error (MAE = − 0.982), and mean square error were obtained (MSE = − 2.303).

Predicting emergency health care demands due to respiratory diseases

BackgroundTimely care in the health sector is essential for the recovery of patients, and even more so in the case of a health emergency. In these cases, appropriate management of human and technical resources is essential. These are limited and must be mobilised in an optimal and efficient manner. ObjectiveThis paper analyses the use of the health emergency service in a city, Jaén, in the south of Spain. The study is focused on the most recurrent case in this service, respiratory diseases. MethodsMachine Learning algorithms are used in which the input variables are multisource data and the target attribute is the prediction of the number of health emergency demands that will occur for a selected date. Health, social, economic, environmental, and geospatial data related to each of the emergency demands were integrated and related. Linear and nonlinear regression algorithms were used: support vector machine (SVM) with linear kernel and generated linear model (GLM), and the nonlinear SVM with Gaussian kernel. ResultsPredictive models of emergency demand due to respiratory disseases were generated with am absolute error better than 35 %. ConclusionsThis model helps to make decisions on the efficient sizing of emergency health resources to manage and respond in the shortest possible time to patients with respiratory diseases requiring urgent care in the city of Jaén.

Network-Based Molecular Features Selection to Predict the Drug Synergy in Cancer Cells

Identifying synergistic drug combinations in cancer treatment is challenging due to the complex molecular circuitry of cancer and the exponentially increasing number of drugs. Therefore, computational approaches for predicting drug synergy are crucial in guiding experimental efforts toward finding rational combination therapies. This research selects the molecular features of cancer cells with a diffusion network-based approach. Additionally, a model is developed using non-linear regression algorithms, namely Random Forest, Extremely Randomized Tree, and XGBoost, to predict the synergy score of drug combinations against the selected cancer cell features. The data used are drug combination screening data and cancer cell molecules provided by AstraZeneca-Sanger DREAM Challenge. The feature selection results demonstrate the relevance of cancer cell molecular features selected by the diffusion network. The prediction results indicate that the Random Forest algorithm shows a good correlation value of 0.570 in the model with a small dataset. In contrast, for the model with an instance or row size larger than the number of features or columns, the XGBoost algorithm achieves a good correlation value of 0.932.
 INDEX TERMS cancer, drug combination, drug synergy, network diffusion kernel, non-linear regression.

Study of Regression Algorithms and Influent Factors between Intelligent Compaction Measurement Values and In-Situ Measurement Values

Intelligent compaction (IC) technology have been used for quality control and quality assurance (QC/QA) in subgrade construction. The effective regression correlations between Intelligent Compaction Measurement Values (ICMV) and In-situ Measurement Values (ISMV, including compaction degree and subgrade modulus ELWD) are the essential prerequisite of using IC technology for QC/QA. This paper presents the results from an experimental research study that was conducted from a practical subgrade project of China to explore the regression relationships between ICMV and ISMV. Three types of ICMV, including CMV, CCV and Evib, were collected along with the corresponding positions of the rollers. Two types of ISMV, containing compaction degree and ELWD, were measured by ring sampler method and light weight deflectometer (LWD) at specified test points, respectively. Based on these data, the influences of roller parameters and subgrade properties on the regression relationships of ICMV and ISMV were investigated. In addition, linear regression and 5 nonlinear regression algorithms were compared. The results suggest that ICMV reflect the stiffness of subgrade more than reflecting the density. In the regression of ICMV and ISMV, subgrade properties are more important than roller parameters while both of them should not be neglected. The influences of underlying stiffness and roller amplitude are linear while that of roller speed and moisture content are nonlinear. Nonlinear algorithms, especially the random forest, have the better performance compared to linear algorithm. Moreover, the combination of random forest and linear algorithm can further improve the accuracy of ISMV prediction.

PAMDI: Privacy aware missing data inference scheme for sparse mobile crowd sensing

The ubiquity of mobile devices has birthed one of the most promising IoT applications called Mobile Crowd Sensing (MCS) wherein mobile devices carried around by a crowd are used to sense phenomena of interest. Subsequently, sensed data are collected, aggregated and analysed to extract useful information. Sparse Mobile Crowd Sensing (SMCS) aims at reducing the sensing overhead (e.g., battery consumption, incentive cost, etc.) by lowering the number of sensing tasks performed. Sensed data thus collected are used to infer missing values. However, it must be ensured that user’s private information (e.g., user’s home location) cannot be derived from the sensed data shared by a user. We propose a novel approach entitled ‘Privacy Aware Missing Data Inference Scheme for Sparse Mobile Crowd Sensing (PAMDI)’ which employs the concept of perceptual hash for ensuring privacy while trying to maintain performance guarantees. Simulation results with the help of two real-world data-sets point towards the feasibility of the proposed approach for provisioning user privacy. We use regression algorithms for missing data inference in PAMDI and find that linear regression algorithms work best with the proposed privacy approach as compared to non-linear regression algorithms. Moreover, we observe that inference accuracy is more or less maintained even after introducing privacy with the proposed approach. In particular, for the first data-set (Temperature data-set), the mean absolute error (MAE) and root mean squared error (RMSE) values obtained by the linear algorithms using the proposed approach are about 2.65 ∘ C and 2 . 9 ∘ C respectively. On the other hand, the corresponding MAE and RMSE values generated by the linear algorithms when no privacy is introduced are about 2.25 ∘ C and 2.85 ∘ C respectively. For non-linear algorithms, the corresponding error values are higher. We also observe the same trend in the results of the second data-set.

A Novel Colorimetric Sensor Array Coupled Multivariate Calibration Analysis for Predicting Freshness in Chicken Meat: A Comparison of Linear and Nonlinear Regression Algorithms

As a source of vital nutrients for the normal functioning of the body, chicken meat plays an important role in promoting good health. This study examines the occurrence of total volatile basic nitrogen (TVB-N) as an index for evaluating freshness, using novel colorimetric sensor arrays (CSA) in combination with linear and nonlinear regression models. Herein, the TVB-N was determined by steam distillation, and the CSA was fabricated via the use of nine chemically responsive dyes. The corresponding dyes utilized, and the emitted volatile organic compounds (VOCs) were found to be correlated. Afterwards, the regression algorithms were applied, assessed, and compared, with the result that a nonlinear model based on competitive adaptive reweighted sampling coupled with support vector machines (CARS-SVM) achieved the best results. Accordingly, the CARS-SVM model provided improved coefficient values (Rc = 0.98 and Rp = 0.92) based on the figures of merit used, as well as root mean square errors (RMSEC = 3.12 and RMSEP = 6.75) and a ratio of performance deviation (RPD) of 2.25. Thus, this study demonstrated that the CSA paired with a nonlinear algorithm (CARS-SVM) could be employed for fast, noninvasive, and sensitive detection of TVB-N concentration in chicken meat as a major indicator of freshness in meat.

Cotton farming industry development and policy finance support for demand estimation in Aksu

Using data mining, the purpose of this study is to forecast and analyze the growth of the cotton cultivation industry and the policy financial support demands in the Aksu region. Data mining is a method for maximizing the value of data via the application of numerous algorithms. In contrast to conventional data mining, which adheres to specific algorithms, data mining employs a variety of analysis algorithms to analyze raw data, such as image and panel data, and produce accurate results. In this paper, we propose a data mining method that combines the semantic segmentation algorithm of remote sensing images with various nonlinear regression algorithms to predict the demand for policy-based financial support in a specific region based on a combination of multiple factors, including agricultural crop cultivation area, catastrophe analyses, agricultural price and inflation rates, etc. This paper intends to estimate and analyze actual data pertaining to the cotton cultivation industry in Aksu, and this methodology can further improve the policy-based financial inverse model. The methods presented in this paper can further improve countercyclical regulation of policy finance.

Region Specific Ground-Motion Predictive Models for Shallow Active Regions

ABSTRACT Using a global dataset, we propose a series of ground-motion predictive models for acceleration response spectra. The proposed models include regional adjustments to source, path, and site terms with nonlinear soil behavior. The regression coefficients are computed using linear and nonlinear mixed-effect regression algorithms. Furthermore, we develop heteroscedastic variability models for between-event, site-to-site, and single-site standard deviations. The between-event sigma model depends solely on magnitude, but the single-site standard deviation model depends on both magnitude and distance. Finally, the site-to-site standard deviation model is given in terms of VS30 and spectral acceleration at rock site condition.

Analysis of Biochemical and Inflammatory Markers for Predicting COVID-19 Severity: Insights From a Tertiary Healthcare Institution of Eastern India.

Background Coronavirus disease-19 (COVID-19) patients often deteriorate rapidly based on viral infection-related inflammation and the subsequent cytokine storm. The clinical symptoms were found to be inconsistent with laboratory findings. There is a need to develop biochemical severity score to closely monitor COVID-19 patients. Methods This study was conducted in the department of biochemistry at All India Institute of Medical Sciences (AIIMS) Bhubaneswar in collaboration with the intensive care unit. Laboratory data of 7,395 patients diagnosed with COVID-19 during the first three waves of the pandemic were analyzed. The serum high sensitivity high-sensitivity C-reactive protein (hs-CRP, immuno-turbidity method), lactate dehydrogenase (LDH, modified Wacker et al. method), and liver enzymes (kinetic-UV method) were estimated by fully automated chemistry analyzer. Serum ferritin and interleukin-6 (IL-6) were measured by one-step immunoassay using chemiluminescence technology. Three models were used in logistic regression to check for the predictive potential of biochemical parameters, and a COVID-19 biochemical severity score was calculated using a non-linear regression algorithm. Results The receiver operating characteristic curve found age, urea, uric acid, CRP, ferritin, IL6, and LDH with the highest odds of predicting ICU admission for COVID-19 patients. COVID-19 biochemical severity scores higher than 0.775were highly predictive (odds ratio of 5.925) of ICU admission (AUC=0.740, p<0.001) as compared to any other individual parameter. For the validation, 30% of the total dataset was used as testing data (n=2095) with a sensitivity of 68.3%, specificity of 74.5%, and odds ratio of 6.304. Conclusion Age, urea, uric acid, ferritin, IL6, LDH, and CRP-based predictive probability algorithm calculating COVID-19 severity was found to be highly predictive of ICU admission for COVID-19 patients.

A combination of library search and Levenberg-Marquardt algorithm in optical scatterometry

Optical scatterometry has been widely used for nanostructure metrology in integrated circuit (IC) given the fast, low-cost, non-contact, non-destructive, and in-line technique. Library search is a common solution for optical scatterometry, but the deterministic error in this method is hard to be eliminated due to the limitation of the grid interval in the spectral library. Another solution is the nonlinear regression algorithm, which faces difficulty in practical applications in meeting the in-line real-time requirements as it involves the time-consuming rigorous coupled-wave analysis (RCWA) calculations in the forward modeling. In this paper, we propose a combination of library search and Levenberg-Marquardt (LSLM) algorithm to solve the problem. We apply a presorted K-dimensional tree (KD-tree) algorithm to speed up searching the initial parameters in the spectral library and replace the time-consuming RCWA calculations with the Taylor expansion in the spectral library to compute the Jacobian matrix in the Levenberg-Marquardt (LM) procedure. Simulations and experiments demonstrate that the results in the proposed method are accurate, and in particular, the speed is much faster than that in previous methods.

Non-Linear Regression Algorithms for Selected Growth Models

For the optimization process to start, traditional statistical methods for nonlinear regression models need a starting point (initial parameters or guess values). The parameters must first be predicted using an iterative process after the nonlinear regression model expression has been written, the parameter names have been expressed, and the initial parameter values have been defined. The Gauss-Newton and Levenberg-Marquardt methods (algorithms) for solving non-linear regression models were used in a computer program for predicting three growth models (Logistic, Gompertz, and Weibull models). The multiplicative error terms used to decompose the growth models help specify the best algorithm and model for growth studies. Before using an iterative approach, second-order regression techniques were used to solve the problem of the initial parameters. The final estimate of the parameters, standard errors, p-values, and model adequacy metrics like (R2, Adj. R2, MSE, SSE, AIC, and BIC) that were used to select the best algorithm and growth model are displayed in the result. The Weibull Growth Model with Multiplicative Error Terms was determined to be the ideal growth model by this study. The Gauss-Newton Algorithm was once again found to be the best algorithm for solving nonlinear regression models in this study. Finally, this study suggest/recommend using the Gauss-Newton algorithm to solve non-linear regression models and the Weibull Growth Model for additional growth studies.

Models for magnetospheric mass density and average ion mass including radial dependence

Analytical models for magnetospheric mass density, ρm, and average ion mass, M, were created from a database of ρm and electron density, ne, values from six spacecraft missions by making use of the Eureqa nonlinear genetic regression algorithm. All values of ρm were determined from Alfvén frequencies, and the values of ne were determined from plasma wave or spacecraft potential data. Models of varying complexity are listed. The most complex models appearing in this paper are capable of modeling ρm within a factor of 1.81, and M within a factor of 1.34 if ne is used as an input parameter, or within a factor of 1.45 if ne is not used. The most important parameters for modeling ρm are L, the solar EUV index F10.7, magnetic local time, MLT, the geomagnetic activity index Kp, and the solar wind dynamic pressure, Pdyn. The very simplest model for M depends on Kp. In more complex models for M including ne, the most important parameters are ne with L, F10.7, and Pdyn or Kp. In more complex models for M not including ne, the most important parameters are Kp, MLT, F10.7, L, and the auroral electrojet index, AE. Explanations for most of the dependencies are given. We also demonstrate the danger of calculating spatial dependence without taking account of different conditions sampled in different regions. Here we avoid that problem by using multivariant models.

Determinación de la energía interfacial del sistema ácido acetil salicílico-etanol aplicando la teoría de nucleación 3D y el método integral

In this work, the interfacial energy of the acetylsalicylic acid-ethanol system was evaluated using experimental results of the metastable zone width (MSZW) obtained at different operation conditions: agitation rate (230 and 400 rpm), cooing rate (6, 9 12 and 15 K/h) and saturation temperature (288.15, 293.15, 298.15, 303.15, 308.15 and 313.15 K). A nonlinear regression algorithm and a simple to implement lineal integration approach of the number density of nuclei were used to determine the interfacial energy. The results obtained with the two integral strategies were compared with the values obtained with the 3D nucleation theory. Estimations obtained with the nonlinear regression presented a difference smaller than 6.5 % but estimations determined with the lineal integration approach had an average difference of 48.6 %.

Application of Digitalization in Real-Time Analysis of Drilling Dynamics Using Along-String Measurement (ASM) Data along Wired Pipes

An automated drilling system requires a real-time evaluation of the drilling bit during drilling to optimize operation and determine when to stop drilling and switch bits. Furthermore, in the dynamic modeling of drill strings, it is necessary to take into account the interactions between drilling bits and rock. To address this challenge, a hybrid approach that combines physics-based models with data analytics has been developed to handle downhole drilling measurements in real time. First, experimental findings were used to formulate mathematical models of cutter–rock interaction in accordance with their geometrical characteristics, rock properties, and drilling parameters. Specifically, these models represent the normal and contact forces of polycrystalline diamond compact cutters (PDCs). Experimental data are analyzed utilizing deep learning, nonlinear regression, and genetic algorithms to fit nonlinear equations to data points. Following this, the recursive least square was implemented as a data analytic method to integrate real-time drilling data, drilling bit models, and mathematical models. Drilling data captured by the along-string measurement system (ASM) is implemented to estimate cutting and normal forces, torque, and specific energy at the bit. The unique aspect of this research is our approach in developing a detailed cutter–rock interaction model that takes all design and operation parameters into account. In addition, the applicability of the algorithm is demonstrated by real-time assessments of drilling dynamics, utilizing downhole digital data, that enable the prediction of drilling events and problems related to drilling bits.