Projection Pursuit Regression Research Articles

Application of projection pursuit regression model for blasting vibration velocity peak prediction

Application of projection pursuit regression model for blasting vibration velocity peak prediction

Predicting cow milk quality traits from routinely available milk spectra using statistical machine learning methods

Numerous statistical machine learning methods suitable for application to highly correlated features, as those that exist for spectral data, could potentially improve prediction performance over the commonly used partial least squares approach. Milk samples from 622 individual cows with known detailed protein composition and technological trait data accompanied by mid-infrared spectra were available to assess the predictive ability of different regression and classification algorithms. The regression-based approaches were partial least squares regression (PLSR), ridge regression (RR), least absolute shrinkage and selection operator (LASSO), elastic net, principal component regression, projection pursuit regression, spike and slab regression, random forests, boosting decision trees, neural networks (NN), and a post-hoc approach of model averaging (MA). Several classification methods (i.e., partial least squares discriminant analysis (PLSDA), random forests, boosting decision trees, and support vector machines (SVM)) were also used after stratifying the traits of interest into categories. In the regression analyses, MA was the best prediction method for 6 of the 14 traits investigated [curd firmness at 60 min, αS1-casein (CN), αS2-CN, κ-CN, α-lactalbumin, and β-lactoglobulin B], whereas NN and RR were the best algorithms for 3 traits each (rennet coagulation time, curd-firming time, and heat stability, and curd firmness at 30 min, β-CN, and β-lactoglobulin A, respectively), PLSR was best for pH, and LASSO was best for CN micelle size. When traits were divided into 2 classes, SVM had the greatest accuracy for the majority of the traits investigated. Although the well-established PLSR-based method performed competitively, the application of statistical machine learning methods for regression analyses reduced the root mean square error compared with PLSR from between 0.18% (κ-CN) to 3.67% (heat stability). The use of modern statistical machine learning methods for trait prediction from mid-infrared spectroscopy may improve the prediction accuracy for some traits.

Evaluation and Prediction on the Hydraulic Abrasion Performance of High Belite Cement-Based Concrete

High belite cement (HBC) concrete is increasingly applied to the hydraulic projects, where much attention should be paid to the abrasion performance. In this paper, the mass loss, abrasion rate and morphology indexes such as depth of abrasion, volume loss and fractal dimension of HBC-based concrete at different wear periods were determined through underwater method. Moreover, a new method by transforming the survey area of worn surface was put forward to simulating practical erosion conditions, based on which, the abrasion performance prediction model was built by Projection Pursuit Regression (PPR) theory. Results show that the compounding addition of 30% mineral admixtures had little contribution to the abrasion resistance of HBC concrete after being long-term cured, while reducing water-binder (w/b) ratios was still an effective way for improving that. The worn surface of concrete specimens was divided into three regions for abrade zone, transitional zone and sound zone, and the abrasion evolution was found to be related to the wear duration and space position. Furthermore, there were good correlations between mass loss rate, depth of abrasion, volume loss and fractal dimension, and the fitting formulas were also proposed with good coefficients. Eventually, the PPR model based on the data of transformed survey area was proved to be a promising approach for predicting the abrasion performance.

Regional Flood Frequency Analysis Through Some Machine Learning Models in Semi-arid Regions

The machine learning models (MLMs), including support vector regression (SVR), multivariate adaptive regression spline (MARS), boosted regression trees (BRT), and projection pursuit regression (PPR) are compared to traditional method i.e. nonlinear regression (NLR) in regional flood frequency analysis (RFFA). In this study, the Karun and Karkheh watersheds, which is located in the southwestern of Iran, with the same climatic and physiographic conditions are considered. Fifty-four hydrometric stations with a period of 21 years (1993–2013) are selected based on the instructions of U.S. Federal Agencies Bulletin 17 B were applied for RFFA. The generalized normal (GNO) probability distribution function (PDF) is selected by the L-moment method among five PDFs, including the GNO, generalized Pareto (GP), generalized logistic (GL), generalized extreme value (GEV) and Pearson type 3 (P ІІІ) to estimate flood discharge for the expected return periods. Twenty-five predictor variables, such as physiographic, climatologic, geologic, soil and land use variables are extracted. Follow land, maximum 24-h rainfall, mean watershed slope, compactness coefficient, mean and maximum watershed elevation variables are recognized as the appropriate combination of input using gamma test (GT). The overall results indicate that the SVR, PPR, and MARS models in comparison to the NLR and BRT models have a better performance to estimate flood discharge with the expected return periods. Future, the SVR model based on radial basis function (RBF) kernel is chosen as the best model in terms of the mean of the Nash-Sutcliff coefficient (M-Ef) and the mean of relative root mean squared error (M-RMSEr) (i.e. 0.94 and 63.93, respectively) for different return periods.

Dibenzoylhydrazines as Insect Growth Modulators: Topology-Based QSAR Modelling

Dibenzoylhydrazines Xa-(C6H5)a-CO-N-(t-Bu)-NH-CO-(C6H5)b-Yb are efficient insect growth regulators with high activity and selectivity toward lepidopteran and coleopteran pests. For 123 congeneric molecules, a quantitative structure activity relationship model was built in the framework of the QSARINS package using 2D, Topology-based, PaDEL descriptors. Variable selection by GA-MLR allows building an efficient multilinear regression linking pEC50 values to nine structural variables. Robustness and quality of the model were carefully examined at various levels: data-fitting (recall), leave-one (or some) - out, internal and external validation (including random splitting), points not in depth investigated in previous works. Various Machine Learning approaches (Partial Least Squares Regression, Projection Pursuit Regression, Linear Support Vector Machine or Three Layer Perceptron Artificial Neural Network) confirm the validity of the analysis, giving highly consistent results of comparable quality, with only a slight advantage for the three-layer perceptron.

Sequential Nonlinear Encoding; A Low Dimensional Regression Algorithm with Application to EEG based Driving Fatigue Detection

We introduce a novel Regularized Kernel Projection Pursuit Regression method which is a two-step nonlinearity encoding algorithm tailored for such very low dimensional problems as fatigue detection. This way, the data nonlinearity can be investigated from two different perspectives, first by transforming the data into a high dimensional intermediate space and then by using their spline estimations to the output variables which allows for a hierarchical unfolding of data. Experimental results on the SEED database shows an average RMSE value of 0.1080% and 0.1054% respectively for the temporal and posterior areas of the brain. Our method is also validated by conducting some experiments on Parkinson's disease prediction which further demonstrate the efficiency of our method for low-dimensional regression problems.Traditional off-the-shelf regression methods like SVR, KSVR, and GLM methods all require their link functions to be previously selected which limits their effectiveness for encoding the nonlinearity of a highly complex low dimensional data set. Moreover, conventional PPR does not deal with the very low dimensionality of data. This paper proposes a novel regression algorithm to address the encoding problem of a highly complex low dimensional data, which is usually encountered in bio-neurological prediction tasks like EEG based driving fatigue detection.

Experimental Design for Optimization of Orthogonal Projection Pursuit Models

Bayesian optimization and kernelized bandit algorithms are widely used techniques for sequential black box function optimization with applications in parameter tuning, control, robotics among many others. To be effective in high dimensional settings, previous approaches make additional assumptions, for example on low-dimensional subspaces or an additive structure. In this work, we go beyond the additivity assumption and use an orthogonal projection pursuit regression model, which strictly generalizes additive models. We present a two-stage algorithm motivated by experimental design to first decorrelate the additive components. Subsequently, the bandit optimization benefits from the statistically efficient additive model. Our method provably decorrelates the fully additive model and achieves optimal sublinear simple regret in terms of the number of function evaluations. To prove the rotation recovery, we derive novel concentration inequalities for linear regression on subspaces. In addition, we specifically address the issue of acquisition function optimization and present two domain dependent efficient algorithms. We validate the algorithm numerically on synthetic as well as real-world optimization problems.

Abstract TP417: An Autoregulation-Derived Prediction Model for Midline Shift After Ischemic Stroke

Introduction: Identification of patients likely to develop midline shift (MLS) after large-vessel occlusion (LVO) stroke is essential for appropriate triage and patient disposition. Studies have identified clinical and radiographic predictors of MLS, but with limited accuracy. Using an innovative assessment of cerebral autoregulation, we sought to develop an accurate predictive model for MLS. Methods: We prospectively enrolled 73 patients with LVO stroke. Beat-by-beat cerebral blood flow (transcranial Doppler) and arterial pressure (arterial catheter or finger photoplethysmography) were recorded within 24 hours of the stroke, and a 24-hour brain MRI was obtained to determine infarct volume and MLS. Autoregulatory function was quantified from pressure-flow relation via projection pursuit regression (PPR), allowing for characterization of 5 hemodynamic markers (Figure 1A). We assessed the predictive relation of autoregulatory capacity and radiological and clinical variables to MLS using recursive classification tree models. Results: PPR successfully quantified autoregulatory function in 50/73 (68.5%) patients within 24 hours of LVO ischemic stroke (age 63.9±13.6, 66% F, NIHSS 15.8±6.7). Of these 50 patients, most (78%) underwent endovascular therapy. Thirteen (26%) experienced 24-h MLS; in these patients, infarct volumes were larger (140.2 vs. 48.6 mL, P<0.001 ), and ipsilateral (but not contralateral) falling slopes were steeper (1.1 vs. 0.7 cm·s -1 ·mmHg -1 , P=0.001 ). Among all clinical, demographic, and hemodynamic variables, only two (infarct volume, ipsilateral falling slope) significantly contributed to prediction of MLS (accuracy 94%; Figure 1B). Conclusions: This predictive model of MLS wields translatable potential for triaging level of care in patients suffering from LVO ischemic stroke, but further research, including optimization of the PPR algorithm as well as prospective use of the predictive model, is needed.

Optimization of mixture proportions in ternary low-heat Portland cement-based cementitious systems with mortar blends based on projection pursuit regression

The low hydration heat of low-heat Portland (LHP) cement-based cementitious systems affects their mechanical properties at an early age. Models for predicting mechanical and thermal performances and optimizing the mixture proportion in ternary LHP cement-based cementitious systems with mortar blends will be useful. In this study, the projection pursuit regression (PPR) model, which is a nonassuming modeling technique, is applied to predict the relationship between the mixture proportion and compressive strength and hydration heat of LHP cement-based cementitious systems with mortar. The optimum mixture proportion was determined via analysis of the contour plots compressive strength and hydration heat as functions of fly ash and ground-granulated blast-furnace slag content in accordance with the demand of actual mass concrete engineering. The PPR model was compared with other methods and found to present higher calculation accuracy. The stability and modeling efficiency of the PPR model were determined and verified by proposing the “accuracy consistency test” criterion and modeling sample selection criteria. The multiobjective optimization problem was converted into two single-objective optimization problems by the PPR model and the optimization method. The problems of artificial parameter assignment and assumptions, which exist in traditional multiobjective optimization, were thus avoided. The PPR model is a valuable tool for predicting the properties of cementitious systems and optimizing the mixture proportion in cementitious systems.

Bioavailability Evaluation of Dissolved Organic Matter Derived from Compost-Amended Soils.

In this study, hetero-two-dimensional correlation spectroscopy (hetero-2DCOS) combined with parallel factor analysis (PARAFAC) was employed to reveal the inner changes in the dissolved organic matter (DOM) components derived from soil amended with seven different composts. The dynamics of the four DOM components showed that the fluorescence peaks in each component varied in different directions during mineralization. Structural equation models (SEMs) demonstrated that the compost amendments changed the correlations of the total organic carbon (TOC), total nitrogen (TN), and bacterial community composition with DOM components and strengthened the cooperative function related to transformation of DOM components. The compost sources were further ranked as cabbage waste (CW) > chicken manure (CM), dairy cattle manure (DCM), tomato stem waste (TSW), peat (P) > municipal solid waste (MSW), sewage sludge (SS) by projection pursuit regression (PPR) analysis. It is helpful to improve the bioavailability of compost products to obtain composts with a particular function.

Host bacterial community of MGEs determines the risk of horizontal gene transfer during composting of different animal manures

Mobile genetic elements (MGEs) play critical roles in transferring antibiotic resistance genes (ARGs) among different microorganisms in the environment. This study aimed to explore the fate of MGEs during chicken manure (CM) and bovine manure (BM) composting to assess horizontal transfer risks of ARGs. The results showed that the removal efficiency of MGEs during CM composting was significantly higher than that during BM composting, because the potential host bacteria of MGEs were eliminated largely during CM composting. Meanwhile, these potential host bacterial communities are significantly influenced by pH, NH4+, NO3− and total N, which can be used to regulate host bacterial communities to remove MGEs during composting. Projection pursuit regression further confirmed that composting can effectively reduce the horizontal transfer risk of ARGs, especially for CM composting. These results identified the critical roles of host bacterial communities in MGEs removal during composting of different animal manures.

Impact of measurement errors on the inferred stellar asteroseismic ages

Aims.We aim to perform a theoretical investigation on the direct impact of measurement errors in the observational constraints on the recovered age for stars in main sequence (MS) and red giant branch (RGB) phases. We assumed that a mix of classical (effective temperatureTeffand metallicity [Fe/H]) and asteroseismic (Δνandνmax) constraints were available for the objects.Methods.Artificial stars were sampled from a reference isochrone and subjected to random Gaussian perturbation in their observational constraints to simulate observational errors. The ages of these synthetic objects were then recovered by means of a Monte Carlo Markov chains approach over a grid of pre-computed stellar models. To account for observational uncertainties the grid covers different values of initial helium abundance and mixing-length parameter, that act as nuisance parameters in the age estimation.Results.The obtained differences between the recovered and true ages were modelled against the errors in the observables. This procedure was performed by means of linear models and projection pursuit regression models. The first class of statistical models provides an easily generalizable result, whose robustness is checked with the second method. From linear models we find that no age error source dominates in all the evolutionary phases. Assuming typical observational uncertainties, for MS the most important error source in the reconstructed age is the effective temperature of the star. An offset of 75 K accounts for an underestimation of the stellar age from 0.4 to 0.6 Gyr for initial and terminal MS. An error of 2.5% inνmaxresulted the second most important source of uncertainty accounting for about −0.3 Gyr. The 0.1 dex error in [Fe/H] resulted particularly important only at the end of the MS, producing an age error of −0.4 Gyr. For the RGB phase the dominant source of uncertainty isνmax, causing an underestimation of about 0.6 Gyr; the offset in the effective temperature and Δνcaused respectively an underestimation and overestimation of 0.3 Gyr. We find that the inference from the linear model is a good proxy for that from projection pursuit regression models. Therefore, inference from linear models can be safely used thanks to its broader generalizability. Finally, we explored the impact on age estimates of adding the luminosity to the previously discussed observational constraints. To this purpose, we assumed – for computational reasons – a 2.5% error in luminosity, much lower than the average error in theGaiaDR2 catalogue. However, even in this optimistic case, the addition of the luminosity does not increase precision of age estimates. Moreover, the luminosity resulted as a major contributor to the variability in the estimated ages, accounting for an error of about −0.3 Gyr in the explored evolutionary phases.

Quantitative Structure Activity Relationships for Carboxamides and Related Compounds Active on Aedes aegypti Adult Females

Aedes aegypti mosquitoes are important vectors in the transmission of severe diseases responsible for million deaths per year. Intensive use of insecticides results in environmental damages and induced resistance in mosquitoes. Search for new molecules devoid of detrimental side effects is therefore an urgent need. In this context, we derived QSAR models for evaluating the acute toxicity of 74 carboxamides and related chemicals to females of Ae. aegypti. These models based on PaDEL, 2D topological descriptors or CODESSA, 2D/3D geometrical and quantum variables, involved multilinear regression (MLR), and various machine learning methods namely support vector machine (SVM), projection pursuit regression (PPR) and artificial neural network (ANN). We considered first the full dataset, and then, a more homogeneous, reduced set of 50 compounds with non-conjugated carbonyl. In all cases, for data fitting and leave-one-out cross-validation, satisfactory results were attained. Good performance was also obtained for extended validation sets. Generally speaking, the modeling methods were broadly equivalent. PaDEL 2D descriptors worked better than 2D/3D CODESSA descriptors. A hybrid model combining the two descriptor sets gave improved results. Setting such QSAR models, linking activity to structural features of examined chemicals, will be of interest for prioritizing experimental tests on new candidates, and evaluate their toxicity and potential synergist effects.

Deep Learning: A Bayesian Perspective

Deep learning is a form of machine learning for nonlinear high dimensional pattern matching and prediction. By taking a Bayesian probabilistic perspective, we provide a number of insights into more efficient algorithms for optimisation and hyper-parameter tuning. Traditional high-dimensional data reduction techniques, such as principal component analysis (PCA), partial least squares (PLS), reduced rank regression (RRR), projection pursuit regression (PPR) are all shown to be shallow learners. Their deep learning counterparts exploit multiple deep layers of data reduction which provide predictive performance gains. Stochastic gradient descent (SGD) training optimisation and Dropout (DO) regularization provide estimation and variable selection. Bayesian regularization is central to finding weights and connections in networks to optimize the predictive bias-variance trade-off. To illustrate our methodology, we provide an analysis of international bookings on Airbnb. Finally, we conclude with directions for future research.

A Conjugate Model for Dimensional Analysis

ABSTRACTDimensional analysis (DA) is a methodology widely used in physics and engineering. The main idea is to extract key variables based on physical dimensions. Its overlooked importance in statistics has been recognized recently. However, most literature treats DA as merely a preprocessing tool, leading to multiple statistical issues. In particular, there are three critical aspects: (a) the nonunique choice of basis quantities and dimensionless variables; (b) the statistical representation and testing of DA constraints; (c) the spurious correlations between post-DA variables. There is an immediate need for an appropriate statistical methodology that integrates DA and the quantitative modeling. In this article, we propose a power-law type of “DA conjugate” model that is useful for incorporating dimensional information and analyzing post-DA variables. Adapting the similar idea of “conjugacy” in Bayesian analysis, we show that the proposed modeling technique not only produces flexible and effective results, but also provides good solutions to the above three issues. A modified projection pursuit regression analysis is implemented to fit the additive power-law model. A numerical study on ocean wave speed is discussed in detail to illustrate and evaluate the advantages of the proposed procedure. Supplementary materials for this article are available online.

QSAR models for predicting the toxicity of piperidine derivatives against Aedes aegypti

QSAR models are proposed for predicting the toxicity of 33 piperidine derivatives against Aedes aegypti. From 2D topological descriptors, calculated with the PaDEL software, ordinary least squares multilinear regression (OLS-MLR) treatment from the QSARINS software and machine learning and related approaches including linear and radial support vector machine (SVM), projection pursuit regression (PPR), radial basis function neural network (RBFNN), general regression neural network (GRNN) and k-nearest neighbours (k-NN), led to four-variable models. Their robustness and predictive ability were evaluated through both internal and external validation. Determination coefficients (r2) greater than 0.85 on the training sets and 0.8 on the test sets were obtained with OLS-MLR and linear SVM. They slightly outperform PPR, radial SVM and RBFNN, whereas GRNN and k-NN showed lower performance. The easy availability of the involved structural descriptors and the simplicity of the MLR model make the corresponding model attractive at an exploratory level for proposing, from this limited dataset, guidelines in the design of new potentially active molecules.

Stock return predictability in emerging markets: Does the choice of predictors and models matter across countries?

This study aims to examine return predictability in 24 emerging markets disaggregated in different regions. We propose four specifications, including a benchmark model. Then, an augmented model appropriate for each country, including a large set of potential factors, is evaluated. Furthermore, a dynamic multifactor model is investigated for all countries. Finally, we relax the symmetric hypothesis in asset return predictability based on a non-parametric non-linear approach: the projection pursuit regression model. Our study reveals three main findings. First, we reject all previous findings supporting a standard model of asset return predictability that is valuable for all countries, as we show that each country has specific domestic factors (both macroeconomic and financial) useful to predict future returns. Second, our empirical framework shows that asset return predictability might be robustly modelled based on non-linear specification based on the projection pursuit regression model. Our findings’ explanatory power of out-of-sample estimations is economically relevant. Our results are useful for investors and policy-makers for portfolio diversification and regulation policies.

Prediction of galaxy ellipticities and reduction of shape noise in cosmic shear measurements

The intrinsic scatter in the ellipticities of galaxies about the mean shape, known as noise, is the most important source of noise in weak lensing shear measurements. Several approaches to reducing shape noise have recently been put forward, using information beyond photometry, such as radio polarization and optical spectroscopy. Here we investigate how well the intrinsic ellipticities of galaxies can be predicted using other, exclusively photometric parameters. These parameters (such as galaxy colours) are already available in the data and do not necessitate additional, often expensive observations. We apply two regression techniques, generalized additive models (GAM) and projection pursuit regression (PPR) to the publicly released data catalog of galaxy properties from CFHTLenS. In our simple analysis we find that the individual galaxy ellipticities can indeed be predicted from other photometric parameters to better precision than the scatter about the mean ellipticity. This means that without additional observations beyond photometry the ellipticity contribution to the shear can be measured to higher precision, comparable to using a larger sample of galaxies. Our best-fit model, achieved using PPR, yields a gain equivalent to having 114.3% more galaxies. Using only parameters unaffected by lensing (e.g.~surface brightness, colour), the gain is only ~12%.

Hydrology Forecasting Model of Upper-lower Bound Estimation Based on Projection Pursuit Regression

Due to its simple structure and easy calculation, upper-lower bound estimation method has been applied in more and more researches related to probabilistic hydrology forecast. Upper and lower bounds are usually estimated by artificial neural network method, namely forecast intervals. This thesis proposes an upper- lower bound interval estimation forecasting method based on projection pursuit regression model. Coverage ratio, forecasting interval width, and symmetry shall be set as forecast criterions for upper-lower bound forecast of runoff. This thesis selected the flow data monitored in Yichang Station which is on the upstream of the Yangtze River as research object. It introduced wavelet analysis method to conduct denoising process on input data. Then, this thesis compared the results respectively forecasted by projection pursuit upper-lower bound estimation model after wavelet denoising process, projection pursuit upper-lower bound estimation model, and upper-lower estimation model of neural network model. The results showed that: projection pursuit interval forecasting model after wavelet denoising process has similar effect to BP neural network model and they both have better effect than interval forecasting model only with projection pursuit regression model.

QSPRs for estimating nematic transition temperatures of pyridine-containing liquid crystalline compounds

ABSTRACTQuantitative structure–property relationships were developed to predict the nematic–isotropic transition temperatures of 92 pyridine-containing liquid crystalline compounds using molecular descriptors calculated by CODESSA software and DRAGON software. The descriptors were also analysed by using principal component analysis. Essentials accounting for a reliable model were all considered carefully during model construction and assessment process. Five variables were selected out by stepwise forward regression analysis and were used as inputs to perform the multiple linear regression, support vector machine and projection pursuit regression (PPR) study. All models were validated through two ways, that is, internal cross-validation combined with a test set. Comparatively, the PPR model performs best both in the fitness and in the prediction capacity. For the test set, it gave a predictive correlation coefficient (R) of 0.991, root mean square error of 11.799 and absolute average relative deviation of 5.456, respectively. The relationships between the descriptors and the nematic–isotropic transition temperature of compounds were also discussed. The odd–even effect in the transition temperatures of mesogens in the same homologous series was also discussed.