Bayesian Nonlinear Regression Research Articles

Adaptive Bayesian Inference in Nonlinear Regression Models

There has been continuing interest in Bayesian regressions without imposing any parametric assumption on the error distribution, but the asymptotic efficiency of such procedures has not been fully understood yet. In this article, we consider semiparametric Bayesian nonlinear regression models. We do not impose a parametric form for the likelihood function; rather, we treat the true density function of error terms as an infinite dimensional nuisance parameter and estimate it nonparametrically. Thereafter, we conduct conventional parametric Bayesian inference using MCMC methods. We derive the asymptotic properties of the resulting estimator and identify conditions of adaptive estimation, under which our two-step Bayes estimator enjoys the same asymptotic normality as if we knew the true density. We compare accuracy and coverage of the adaptive Bayesian estimator with the maximum likelihood estimator in empirical studies on simulated and real data. In particular, we observe that the Bayesian inference may be superior in numerical stability for small sample sizes.

Reduced-order modelling of parameter-dependent, linear and nonlinear dynamic partial differential equation models.

In this paper, we develop reduced-order models for dynamic, parameter-dependent, linear and nonlinear partial differential equations using proper orthogonal decomposition (POD). The main challenges are to accurately and efficiently approximate the POD bases for new parameter values and, in the case of nonlinear problems, to efficiently handle the nonlinear terms. We use a Bayesian nonlinear regression approach to learn the snapshots of the solutions and the nonlinearities for new parameter values. Computational efficiency is ensured by using manifold learning to perform the emulation in a low-dimensional space. The accuracy of the method is demonstrated on a linear and a nonlinear example, with comparisons with a global basis approach.

Wavelet-based Bayesian nonlinear regression for the air pollution effects on clinic visits in small areas of Taiwan

ABSTRACTWe revisit the complete clinic visit records and environmental monitoring data at 50 townships and city districts of Taiwan. Extending the earlier analyses, here we consider a Bayesian analysis using Daubechies wavelet. Appropriate model selection is also considered using Bayesian model averaging. Temperature, dew point, and NO2 and CO of the current day and the previous day are identified as the pollutants in different areas of the island following some spatial pattern.

Efficiency and safety of the combination of moxifloxacin, pretomanid (PA-824), and pyrazinamide during the first 8 weeks of antituberculosis treatment: a phase 2b, open-label, partly randomised trial in patients with drug-susceptible or drug-resistant pulmonary tuberculosis

New antituberculosis regimens are urgently needed to shorten tuberculosis treatment. Following on from favourable assessment in a 2 week study, we investigated a novel regimen for efficacy and safety in drug-susceptible and multidrug-resistant (MDR) tuberculosis during the first 8 weeks of treatment. We did this phase 2b study of bactericidal activity--defined as the decrease in colony forming units (CFUs) of Mycobacterium tuberculosis in the sputum of patients with microscopy smear-positive pulmonary tuberculosis-at eight sites in South Africa and Tanzania. We enrolled treatment-naive patients with drug-susceptible, pulmonary tuberculosis, who were randomly assigned by computer-generated sequences to receive either 8 weeks of moxifloxacin, 100 mg pretomanid (formerly known as PA-824), and pyrazinamide (MPa100Z regimen); moxifloxacin, 200 mg pretomanid, and pyrazinamide (MPa200Z regimen); or the current standard care for drug-susceptible pulmonary tuberculosis, isoniazid, rifampicin, PZA, and ethambutol (HRZE regimen). A group of patients with MDR tuberculosis received MPa200Z (DRMPa200Z group). The primary outcome was bactericidal activity measured by the mean daily rate of reduction in M tuberculosis CFUs per mL overnight sputum collected once a week, with joint Bayesian non-linear mixed-effects regression modelling. We also assessed safety and tolerability by monitoring adverse events. This study is registered with ClinicalTrials.gov, number NCT01498419. Between March 24, 2012, and July 26, 2013 we enrolled 207 patients and randomly assigned them to treatment groups; we assigned 60 patients to the MPa100Z regimen, 62 to the MPa200Z regimen, and 59 to the HRZE regimen. We non-randomly assigned 26 patients with drug-resistant tuberculosis to the DRMPa200Z regimen. In patients with drug-susceptible tuberculosis, the bactericidal activity of MPa200Z (n=54) on days 0-56 (0·155, 95% Bayesian credibility interval 0·133-0·178) was significantly greater than for HRZE (n=54, 0·112, 0·093-0·131). DRMPa200Z (n=9) had bactericidal activity of 0·117 (0·070-0·174). The bactericidal activity on days 7-14 was strongly associated with bactericidal activity on days 7-56. Frequencies of adverse events were similar to standard treatment in all groups. The most common adverse event was hyperuricaemia in 59 (29%) patients (17 [28%] patients in MPa100Z group, 17 [27%] patients in MPa200Z group, 17 [29%] patients. in HRZE group, and 8 [31%] patients in DRMPa200Z group). Other common adverse events were nausea in (14 [23%] patients in MPa100Z group, 8 [13%] patients in MPa200Z group, 7 [12%] patients in HRZE group, and 8 [31%] patients in DRMPa200Z group) and vomiting (7 [12%] patients in MPa100Z group, 7 [11%] patients in MPa200Z group, 7 [12%] patients in HRZE group, and 4 [15%] patients in DRMPa200Z group). No on-treatment electrocardiogram occurrences of corrected QT interval more than 500 ms (an indicator of potential of ventricular tachyarrhythmia) were reported. No phenotypic resistance developed to any of the drugs in the regimen. The combination of moxifloxacin, pretomanid, and pyrazinamide, was safe, well tolerated, and showed superior bactericidal activity in drug-susceptible tuberculosis during 8 weeks of treatment. Results were consistent between drug-susceptible and MDR tuberculosis. This new regimen is ready to enter phase 3 trials in patients with drug-susceptible tuberculosis and MDR-tuberculosis, with the goal of shortening and simplifying treatment. Global Alliance for TB Drug Development.

Predicting Peer-to-Peer Loan Rates Using Bayesian Non-Linear Regression

Peer-to-peer lending is a new highly liquid market for debt, which is rapidly growing in popularity. Here we consider modelling market rates, developing a non-linear Gaussian Process regression method which incorporates both structured data and unstructured text from the loan application. We show that the peer-to-peer market is predictable, and identify a small set of key factors with high predictive power. Our approach outperforms baseline methods for predicting market rates, and generates substantial profit in a trading simulation.

A Bayesian Nonlinear Mixed-Effects Regression Model for the Characterization of Early Bactericidal Activity of Tuberculosis Drugs

Trials of the early bactericidal activity (EBA) of tuberculosis (TB) treatments assess the decline, during the first few days to weeks of treatment, in colony forming unit (CFU) count of Mycobacterium tuberculosis in the sputum of patients with smear-microscopy-positive pulmonary TB. Profiles over time of CFU data have conventionally been modeled using linear, bilinear, or bi-exponential regression. We propose a new biphasic nonlinear regression model for CFU data that comprises linear and bilinear regression models as special cases and is more flexible than bi-exponential regression models. A Bayesian nonlinear mixed-effects (NLME) regression model is fitted jointly to the data of all patients from a trial, and statistical inference about the mean EBA of TB treatments is based on the Bayesian NLME regression model. The posterior predictive distribution of relevant slope parameters of the Bayesian NLME regression model provides insight into the nature of the EBA of TB treatments; specifically, the posterior predictive distribution allows one to judge whether treatments are associated with monolinear or bilinear decline of log(CFU) count, and whether CFU count initially decreases fast, followed by a slower rate of decrease, or vice versa.

The use of Bayesian nonlinear regression techniques for the modelling of the retention behaviour of volatile components of Artemisia species

The main aim of this work was to assess the ability of Bayesian multivariate adaptive regression splines (BMARS) and Bayesian radial basis function (BRBF) techniques for modelling the gas chromatographic retention indices of volatile components of Artemisia species. A diverse set of molecular descriptors was calculated and used as descriptor pool for modelling the retention indices. The ability of BMARS and BRBF techniques was explored for the selection of the most relevant descriptors and proper basis functions for modelling. The results revealed that BRBF technique is more reproducible than BMARS for modelling the retention indices and can be used as a method for variable selection and modelling in quantitative structure–property relationship (QSPR) studies. It is also concluded that the Markov chain Monte Carlo (MCMC) search engine, implemented in BRBF algorithm, is a suitable method for selecting the most important features from a vast number of them. The values of correlation between the calculated retention indices and the experimental ones for the training and prediction sets (0.935 and 0.902, respectively) revealed the prediction power of the BRBF model in estimating the retention index of volatile components of Artemisia species.

Bayesian multiple response kernel regression model for high dimensional data and its practical applications in near infrared spectroscopy

Non-linear regression based on reproducing kernel Hilbert space (RKHS) has recently become very popular in fitting high-dimensional data. The RKHS formulation provides an automatic dimension reduction of the covariates. This is particularly helpful when the number of covariates (p) far exceed the number of data points. In this paper, we introduce a Bayesian nonlinear multivariate regression model for high-dimensional problems. Our model is suitable when we have multiple correlated observed response corresponding to same set of covariates. We introduce a robust Bayesian support vector regression model based on a multivariate version of Vapnik’s ϵ-insensitive loss function. The likelihood corresponding to the multivariate Vapnik’s ϵ-insensitive loss function is constructed as a scale mixture of truncated normal and gamma distribution. The regression function is constructed using the finite representation of a function in the reproducing kernel Hilbert space (RKHS). The kernel parameter is estimated adaptively by assigning a prior on it and using the Markov chain Monte Carlo (MCMC) techniques for computation.Practical applications of our model are demonstrated via applications in near-infrared (NIR) spectroscopy and simulation studies. Our Bayesian kernel models are highly accurate in predicting composition of materials based on its near infrared (NIR) spectroscopy signature. We have compared our method with popularly used methodologies in NIR spectroscopy, like partial least square (PLS), principal component regression (PCA), support vector machine (SVM), Gaussian process regression (GPR), and random forest (RF). In all the simulation and real case studies, our multivariate Bayesian RKHS regression model outperforms the standard methods by a substantially large margin. The implementation of our models based on MCMC is fairly fast and straight forward.

Bayesian nonlinear regression models with scale mixtures of skew-normal distributions: Estimation and case influence diagnostics

The purpose of this paper is to develop a Bayesian analysis for nonlinear regression models under scale mixtures of skew-normal distributions. This novel class of models provides a useful generalization of the symmetrical nonlinear regression models since the error distributions cover both skewness and heavy-tailed distributions such as the skew-t, skew-slash and the skew-contaminated normal distributions. The main advantage of these class of distributions is that they have a nice hierarchical representation that allows the implementation of Markov chain Monte Carlo (MCMC) methods to simulate samples from the joint posterior distribution. In order to examine the robust aspects of this flexible class, against outlying and influential observations, we present a Bayesian case deletion influence diagnostics based on the Kullback–Leibler divergence. Further, some discussions on the model selection criteria are given. The newly developed procedures are illustrated considering two simulations study, and a real data previously analyzed under normal and skew-normal nonlinear regression models.

Uncertainty analysis on remoulded undrained shear strength of marine clay

Remoulded undrained shear strength s u−r is a soil property associated with disturbed clay. This parameter is used to indicate the evolution of the post-failure behaviour of seafloor soil, estimate the rheological properties of debris flows and the drag forces on seafloor structures exerted by a submarine slide, and calculate the penetration resistance of skirt walls. This work discusses the influence of uncertainty on the estimate of empirical relationships between s u−r and liquidity index I L as a potential predictor of the remoulded soil strength. Special emphasis is made on assessing the uncertainty inherent to s u−r , when the definition of the empirical models is updated from a global to a local (regional) data set. For this purpose two approaches are compared: a parameterisation in the ‘log space’ (Log-Linear Model, LLM) by transforming least square exponential fitting to log-linear expression, and a parameterisation in the ‘physical space’ using Bayesian Non-Linear Regression (BNLR). The updating of the trend models conditioned on the two available data sets is performed using simple Bayesian principles. Results show similar trend models obtained by the two approaches, and the standard deviation of the trend indicates different values. A thorough discussion is included based on the different hypothesis considered for the estimate of each model.

Bayesian Nonlinear Regression for the Air Pollution Effects on Daily Clinic Visits in Small Areas of Taiwan

We consider the complete clinic visit records and environmental monitoring data at 50 townships and city districts where ambient air monitoring stations of Taiwan Air Quality Monitoring Stations are located. A Bayesian analysis is carried out using regression spline model on principal components obtained from several pollutant covariables. The appropriate model is selected using Bayesian model averaging. A brief account of our results is provided for the elderly patients group.

Bayesian non-linear regression models with skew-elliptical errors: Applications to the classification of longitudinal profiles

Typically, the fundamental assumption in non-linear regression models is the normality of the errors. Even though this model offers great flexibility for modeling these effects, it suffers from the same lack of robustness against departures from distributional assumptions as other statistical models based on the Gaussian distribution. It is of practical interest, therefore, to study non-linear models which are less sensitive to departures from normality, as well as related assumptions. Thus the current methods proposed for linear regression models need to be extended to non-linear regression models. This paper discusses non-linear regression models for longitudinal data with errors that follow a skew-elliptical distribution. Additionally, we discuss Bayesian statistical methods for the classification of observations into two or more groups based on skew-models for non-linear longitudinal profiles. Parameter estimation for a discriminant model that classifies individuals into distinct predefined groups or populations uses appropriate posterior simulation schemes. The methods are illustrated with data from a study involving 173 pregnant women. The main objective in this study is to predict normal versus abnormal pregnancy outcomes from beta human chorionic gonadotropin data available at early stages of pregnancy.

Nonlinear regression modeling of nutrient loads in streams: A Bayesian approach

A Bayesian nonlinear regression modeling method is introduced and compared with the least squares method for modeling nutrient loads in stream networks. The objective of the study is to better model spatial correlation in river basin hydrology and land use for improving the model as a forecasting tool. The Bayesian modeling approach is introduced in three steps, each with a more complicated model and data error structure. The approach is illustrated using a data set from three large river basins in eastern North Carolina. Results indicate that the Bayesian model better accounts for model and data uncertainties than does the conventional least squares approach. Applications of the Bayesian models for ambient water quality standards compliance and TMDL assessment are discussed.

Models in a Bayesian framework for inactivation of pectinesterase in a commercial enzyme formulation by pulsed electric fields

The inactivation of pectinesterase (PE) in a commercial enzyme formulation (CEF) under pulsed electric fields (PEF) was studied. Samples of an aqueous solution of the CEF were exposed to exponential waveform pulses for up to 16 ms at electric field intensities ranging from 5 to 24 kV cm−1. Pulses were delivered in monopolar mode. The observed inactivation of the enzyme was described by several kinetics and regarding the input of electrical energy density (Q) models using Bayesian non-linear regression. Posterior distributions of the characteristic parameters for each kinetic model (based on the Hulsheger or the Weibull equation) and each Q model (based on exponential decay or the Weibull equation) were obtained. Kinetics models based on the Weibull equation showed better goodness and accuracy than the other models.

Bayesian nonlinear model selection and neural networks: a conjugate prior approach

In order to select the best predictive neural-network architecture in a set of several candidate networks, we propose a general Bayesian nonlinear regression model comparison procedure, based on the maximization of an expected utility criterion. This criterion selects the model under which the training set achieves the highest level of internal consistency, through the predictive probability distribution of each model. The density of this distribution is computed as the model posterior predictive density and is asymptotically approximated from the assumed Gaussian likelihood of the data set and the related conjugate prior density of the parameters. The use of such a conjugate prior allows the analytic calculation of the parameter posterior and predictive posterior densities, in an empirical-Bayes-like approach. This Bayesian selection procedure allows us to compare general nonlinear regression models and in particular feedforward neural networks, in addition to embedded models as usual with asymptotic comparison tests.

An application of Bayesian nonlinear regression to hydrologic models

A suite of FORTRAN 77 computer programs implementing Bayesian nonlinear regression is described. These programs, developed to cope with complex hydrologic models, are used to infer model parameters, test model structure and make predictions of future response. The programs are fully interactive with features including: interactive parameter optimization using the Gauss-Marquardt algorithm; run-time editing of user options; availability of a general error model; use of prior information; and joint fitting of multiple-response data.

On Bayesian Nonlinear Regression with an Enzyme Example

On Bayesian Nonlinear Regression with an Enzyme Example

On Bayesian nonlinear regression with an enzyme example

A nonlinear regression model is applied to several sets of enzyme kinetics data, treating the entire regression vector as the parameter of interest. The resulting marginal posterior distributions are presented alongside the usual Student's t posterior distributions implicit in the usual linearized regression. The prior distribution used is that which is minimally informative about the regression vector. In the special case of the linear model this procedure leads to the prior density 1/θ, thereby producing the classical inferences.

Reduction of dimensionality in Bayesian nonlinear regression with a pharmacokinetic application

Application of Bayes's theorem to the analysis of nonlinear regression models is limited by numerical problems associated with calculation of integrals of functions of several variables. For k-parameter models that are linear in l of the parameters, a dimension-reduction procedure is described for factoring the posterior distribution into the product of a multivariate normal density and a function of k- l nonlinear parameters. Integrals can then be calculated with ( k- l)-dimensional numerical integration. A four-parameter, two-compartment pharmacokinetic model of lidocaine disposition is analyzed using a change of variables in order to obtain a model that is linear in two parameters. It is shown that a Bayesian analysis, with reduction of dimensionality, applied to this model produces appropriate results with reasonable CPU-time requirements.