Flexible Regression Models Research Articles

Data-adaptive additive modeling.

In this paper, we consider fitting a flexible and interpretable additive regression model in a data-rich setting. We wish to avoid pre-specifying the functional form of the conditional association between each covariate and the response, while still retaining interpretability of the fitted functions. A number of recent proposals in the literature for nonparametric additive modeling are data adaptive, in the sense that they can adjust the level of flexibility in the functional fits to the data at hand. For instance, the sparse additive model makes it possible to adaptively determine which features should be included in the fitted model, the sparse partially linear additive model allows each feature in the fitted model to take either a linear or a nonlinear functional form, and the recent fused lasso additive model and additive trend filtering proposals allow the knots in each nonlinear function fit to be selected from the data. In this paper, we combine the strengths of each of these recent proposals into a single proposal that uses the data to determine which features to include in the model, whether to model each feature linearly or nonlinearly, and what form to use for the nonlinear functions. We establish connections between our approach and recent proposals from the literature, and we demonstrate its strengths in a simulation study.

Bayesian nonparametric regression models for modeling and predicting healthcare claims

Standard regression models are often insufficient to describe the complex relationships that exist in healthcare claims. A Bayesian nonparametric regression approach is presented as a flexible regression model that relaxes the assumption of Gaussianity. The details for implementation are presented. Bayesian nonparametric regression is applied to a dataset of claims by episode treatment group (ETG) with a specific focus on prediction of new observations. It is shown that the predictive accuracy improves when compared both to standard linear model assumptions and the more flexible Generalized Beta regression. Of the 347 different ETGs, the nonparametric regression outperformed both the standard linear and generalized beta regression on all but 11. By studying Conjunctivitis and Lung Transplants specifically, it is shown that this approach can handle complex characteristics of the regression error distribution such as skewness, thick tails, outliers, and bimodality.

A flexible quantile regression model for medical costs with application to Medical Expenditure Panel Survey Study.

Medical costs are often skewed to the right and heteroscedastic, having a sophisticated relation with covariates. Mean function regression models with low-dimensional covariates have been extensively considered in the literature. However, it is important to develop a robust alternative to find the underlying relationship between medical costs and high-dimensional covariates. In this paper, we propose a new quantile regression model to analyze medical costs. We also consider variable selection, using an adaptive lasso penalized variable selection method to identify significant factors of the covariates. Simulation studies are conducted to illustrate the performance of the estimation method. We apply our method to the analysis of the Medical Expenditure Panel Survey dataset.

A new generalized odd log-logistic flexible Weibull regression model with applications in repairable systems

We define and study a four-parameter model called the generalized odd log-logistic flexible Weibull distribution. The new distribution can be used effectively in the context of reliability since it accommodates different hazard rate forms such as monotone, unimodal, bathtub-shaped, increasing-decreasing-increasing, among possible others. We provide an extensive study of the quantile function. Further, we present a parametric regression model based on the new distribution as an alternative to the location-scale regression model. An important property of this new regression model is that it does not need the assumption of proportional risks. We use the method of maximum likelihood for estimating the model parameters and perform various simulations for different parameter settings, sample sizes and censoring percentages. Applications in real engineering data sets illustrate the flexibility of the proposed models.

The relationship between energy intensity and economic growth: New evidence from a multi-country multi-sectorial dataset

This paper revisits the relationship between energy intensity and economic growth, using a flexible piecewise linear regression model. Based on a panel data set of 137 economies during 1990-2014, the analysis identifies a threshold effect of income growth on energy intensity change: although energy intensity is negatively correlated with income growth throughout the entire sample and study period, the declining rate significantly slows by more than 30 percent after the level of per capita income reaches $5,000. Based on index decomposition, the analysis also finds that although structural change is important for intensity levels in all countries, the efficiency effect is more important in higher-income countries. The results suggest that when countries move beyond lower-middle-income levels, energy efficiency policies become far more critical for sustaining the rate of improvement in energy efficiency.

Estimating Logistic Regressions with Two Stage Least Squares

I develop an algorithm to estimate a flexible binary regression model with endogeneity by repeatedly solving a two-stage least squares problem; the algorithm is numerically stable and guaranteed to converge regardless of starting value. The method is numerically stable even when a successful outcome is rare because it has a uniformly small condition number, unlike Newton methods with maximum likelihood estimation whose condition number is unbounded across potential parameter values. The instrumental variable method does not require choosing a special regressor or making assumptions on the first stage relationship between covariates and instruments other than a rank restriction to ensure the instruments are relevant enough.

Additive Nonlinear Functional Concurrent Model.

We propose a flexible regression model to study the association between a functional response and multiple functional covariates that are observed on the same domain. Specifically, we relate the mean of the current response to current values of the covariates by a sum of smooth unknown bivariate functions, where each of the functions depends on the current value of the covariate and the time point itself. In this framework, we develop estimation methodology that accommodates realistic scenarios where the covariates are sampled with or without error on a sparse and irregular design, and prediction that accounts for unknown model correlation structure. We also discuss the problem of testing the null hypothesis that the covariate has no association with the response. The proposed methods are evaluated numerically through simulations and two real data applications.

Further econometric evidence on the extent and sources of cost savings in competitively tendered contracts

By estimating a flexible nonlinear regression model of savings on an original dataset of service procurements conducted by the Spanish Armed Forces, this paper provides robust and precise novel econometric evidence on the extent and sources of cost savings in public procurement. The net effect on savings of the policy-amenable and economically advantageous variables that we estimate, such as the size of the procured function, the importance of price in the contract award criteria, and the number of bidders who participate in the tendering, may help contracting agencies to select management practices and to forecast the price paid out. We find that savings increase proportionally to the size of the procured function, that an increase of 10 percentage points in the importance of price increases savings by approximately 2% of the function’s size, and that savings are generally reduced by restricting the number of bidders. A comparison with estimates reported in previous studies is also made.

Relationship between season of birth, temperature exposure, and later life wellbeing

We study how exposure to extreme temperatures in early periods of child development is related to adult economic outcomes measured 30 y later. Our analysis uses administrative earnings records for over 12 million individuals born in the United States between 1969 and 1977, linked to fine-scale, daily weather data and location and date of birth. We calculate the length of time each individual is exposed to different temperatures in utero and in early childhood, and we estimate flexible regression models that allow for nonlinearities in the relationship between temperature and long-run outcomes. We find that an extra day with mean temperatures above 32 °C in utero and in the first year after birth is associated with a 0.1% reduction in adult annual earnings at age 30. Temperature sensitivity is evident in multiple periods of early development, ranging from the first trimester of gestation to age 6-12 mo. We observe that household air-conditioning adoption, which increased dramatically over the time period studied, mitigates nearly all of the estimated temperature sensitivity.

Implementing Generalized Additive Models to Estimate the Expected Value of Sample Information in a Microsimulation Model: Results of Three Case Studies.

The expected value of sample information (EVSI) can help prioritize research but its application is hampered by computational infeasibility, especially for complex models. We investigated an approach by Strong and colleagues to estimate EVSI by applying generalized additive models (GAM) to results generated from a probabilistic sensitivity analysis (PSA). For 3 potential HIV prevention and treatment strategies, we estimated life expectancy and lifetime costs using the Cost-effectiveness of Preventing AIDS Complications (CEPAC) model, a complex patient-level microsimulation model of HIV progression. We fitted a GAM-a flexible regression model that estimates the functional form as part of the model fitting process-to the incremental net monetary benefits obtained from the CEPAC PSA. For each case study, we calculated the expected value of partial perfect information (EVPPI) using both the conventional nested Monte Carlo approach and the GAM approach. EVSI was calculated using the GAM approach. For all 3 case studies, the GAM approach consistently gave similar estimates of EVPPI compared with the conventional approach. The EVSI behaved as expected: it increased and converged to EVPPI for larger sample sizes. For each case study, generating the PSA results for the GAM approach required 3 to 4 days on a shared cluster, after which EVPPI and EVSI across a range of sample sizes were evaluated in minutes. The conventional approach required approximately 5 weeks for the EVPPI calculation alone. Estimating EVSI using the GAM approach with results from a PSA dramatically reduced the time required to conduct a computationally intense project, which would otherwise have been impractical. Using the GAM approach, we can efficiently provide policy makers with EVSI estimates, even for complex patient-level microsimulation models.

Regression learning based on incomplete relationships between attributes

In recent years, machine learning researchers have focused on methods to construct flexible and interpretable regression models. However, the method of obtaining complete knowledge from incomplete and fuzzy prior knowledge and the trade-off between the generalization performance and the interpretability of the model are very important factors to consider. In this paper, we propose a new regression learning method. Complete relationships are obtained from the incomplete fuzzy relationships between attributes by using Markov logic networks [29]. The complete relationships are then applied to constrain the shape of the regression model in the optimization procedure to solve the trade-off problem. Finally, the benefits of our approach are illustrated on benchmark data sets and in real-world experiments.

A flexible semiparametric regression model for bimodal, asymmetric and censored data

ABSTRACTIn this paper, we propose a new semiparametric heteroscedastic regression model allowing for positive and negative skewness and bimodal shapes using the B-spline basis for nonlinear effects. The proposed distribution is based on the generalized additive models for location, scale and shape framework in order to model any or all parameters of the distribution using parametric linear and/or nonparametric smooth functions of explanatory variables. We motivate the new model by means of Monte Carlo simulations, thus ignoring the skewness and bimodality of the random errors in semiparametric regression models, which may introduce biases on the parameter estimates and/or on the estimation of the associated variability measures. An iterative estimation process and some diagnostic methods are investigated. Applications to two real data sets are presented and the method is compared to the usual regression methods.

Inferring catchment flow path responses using a data-driven model: an exploratory study based on a generalized additive model

ABSTRACTIdentifying physical catchment processes from streamflow data, such as quick- and slow-flow paths, remains challenging. This study is designed to explore whether a flexible nonparametric regression model (generalized additive model, GAM) can be used to infer different flow paths. This assumes that the data relationship in data-driven models is also a reflection of catchment physical processes. The GAM, using time-lagged flow covariates, was fitted to synthetic rainfall–runoff data simulated using simple linear reservoirs. Partial plots of the time-lagged covariates show that the model could differentiate simple and more complex flow paths in simulated synthetic data with short and long memory systems and varying between dry and wet climates. Further analysis of data from real catchments showed that the model could differentiate catchments dominated by slow flow and by quick flow. Therefore, this study indicates that GAM can be used to identify catchment storages and delay processes from streamflow data.

Flexible Tweedie regression models for continuous data

ABSTRACTTweedie regression models (TRMs) provide a flexible family of distributions to deal with non-negative right-skewed data and can handle continuous data with probability mass at zero. Estimation and inference of TRMs based on the maximum likelihood (ML) method are challenged by the presence of an infinity sum in the probability function and non-trivial restrictions on the power parameter space. In this paper, we propose two approaches for fitting TRMs, namely quasi-likelihood (QML) and pseudo-likelihood (PML). We discuss their asymptotic properties and perform simulation studies to compare our methods with the ML method. We show that the QML method provides asymptotically efficient estimation for regression parameters. Simulation studies showed that the QML and PML approaches present estimates, standard errors and coverage rates similar to the ML method. Furthermore, the second-moment assumptions required by the QML and PML methods enable us to extend the TRMs to the class of quasi-TRMs in Wedderburn's style. It allows to eliminate the non-trivial restriction on the power parameter space, and thus provides a flexible regression model to deal with continuous data. We provide an R implementation and illustrate the application of TRMs using three data sets.

Robust functional regression model for marginal mean and subject-specific inferences.

We introduce flexible robust functional regression models, using various heavy-tailed processes, including a Student t-process. We propose efficient algorithms in estimating parameters for the marginal mean inferences and in predicting conditional means as well as interpolation and extrapolation for the subject-specific inferences. We develop bootstrap prediction intervals (PIs) for conditional mean curves. Numerical studies show that the proposed model provides a robust approach against data contamination or distribution misspecification, and the proposed PIs maintain the nominal confidence levels. A real data application is presented as an illustrative example.

Development of spatial regression models for predicting summer river temperatures from landscape characteristics: Implications for land and fisheries management

There is increasing demand for models that can accurately predict river temperature at the large spatial scales appropriate to river management. This paper combined summer water temperature data from a strategically designed, quality controlled network of 25 sites, with recently developed flexible spatial regression models, to understand and predict river temperature across a 3000 km2 river catchment. Minimum, mean and maximum temperatures were modelled as a function of nine potential landscape covariates that represented proxies for heat and water exchange processes. Generalised Additive Models were used to allow for flexible responses. Spatial structure in the river network data (local spatial variation) was accounted for by including river network smoothers. Minimum and mean temperatures decreased with increasing elevation, riparian woodland and channel gradient. Maximum temperatures increased with channel width. There was greater between-river and between-reach variability in all temperature metrics in lower order rivers indicating that increased monitoring effort should be focussed at these smaller scales. The combination of strategic network design and recently developed spatial statistical approaches employed in this study have not been used in previous studies of river temperature. The resulting catchment scale temperature models provide a valuable quantitative tool for understanding and predicting river temperature variability at the catchment scales relevant to land use planning and fisheries management and provide a template for future studies.

Regression Models for Multivariate Count Data

ABSTRACTData with multivariate count responses frequently occur in modern applications. The commonly used multinomial-logit model is limiting due to its restrictive mean-variance structure. For instance, analyzing count data from the recent RNA-seq technology by the multinomial-logit model leads to serious errors in hypothesis testing. The ubiquity of overdispersion and complicated correlation structures among multivariate counts calls for more flexible regression models. In this article, we study some generalized linear models that incorporate various correlation structures among the counts. Current literature lacks a treatment of these models, partly because they do not belong to the natural exponential family. We study the estimation, testing, and variable selection for these models in a unifying framework. The regression models are compared on both synthetic and real RNA-seq data. Supplementary materials for this article are available online.

You Drink, You Drive, You Die? The Dynamics of Youth Risk Taking in Response to a Change in the Legal Drinking Age

This paper exploits the reduction in the legal drinking age in New Zealand from 20 to 18 to study the dynamics of youth risk taking. Using administrative data on the universe of road accidents over a fifteen year period spanning the law change, we undertake three complimentary analyses to examine the dynamics of alcohol-related and total vehicular accidents among youth. First, using an event history approach, we find no evidence that changing the drinking age from 20 to 18 led to more vehicular accidents or alcohol-related accidents among teens. This is true both in the short-run following the law change and when examining cumulative accidents for the affected cohorts.Next, using an age-based regression discontinuity design (RDD), we find that accidents do increase after one's 18th birthday, but this appears to be a short-run phenomenon. Finally, estimating flexible parametric regression models suggests that reducing the drinking age led to a decline in risky driving by youth who were already 15 at the time of the change but had no longer-run impacts. Overall, our results support the argument that the legal drinking age can be lowered without increasing detrimental outcomes for youth and call into question previous studies that have made policy recommendations by extrapolating from results identified using age-based RDDs.

Breastfeeding patterns and its determinants among mothers living with Human Immuno-deficiency Virus -1 in four African countries participating in the ANRS 12174 trial.

BackgroundHIV-1 transmission rates have been reduced over the last decade, an estimated 2 million new infections per year arise, including 220,000 paediatric cases. The main post-natal HIV exposure is through breastfeeding, where both its duration and modality (exclusive or not) are associated with postnatal transmission. The ANRS 12174 trial compared HIV-1 postnatal transmission of 2 prophylaxis drugs for infants during lactation (lamivudine and lopinavir-ritonavir). Our objective has been to examine the feeding practices and the determinants of exclusive/ predominant (EPBF) or any breastfeeding among the participants of this trial in Burkina Faso, South Africa, Uganda and Zambia.MethodsMothers infected with HIV-1 and their uninfected offspring were followed from day 7 after birth for 50 weeks, keeping monthly records of their feeding patterns. Feeding was classified into 3 categories: 1) exclusive breastfeeding during the first six months, only breast-milk being given to infant for 6 months, 2) predominant breastfeeding, breast-milk with liquid-based items being given, and 3) mixed feeding, other non-breast milk or solid food being given in addition to breast milk with or without liquid-based items. The categories were merged into 2 groups: EPBF applying to infants aged <6 months and mixed feeding applying to infants of any age. The feeding patterns have been given as Kaplan-Meier curves. A flexible parametric multiple regression model was used to identify the determinants of the mothers’ feeding behaviour.ResultsA total of 1,225 mother-infant pairs provided feeding data from Burkina Faso (N = 204), South Africa (N = 213), Uganda (N = 274) and Zambia (N = 534) between November 2009 and March 2013. The mean maternal age was 27.4 years and the mean BMI was 24.5. 57.7 and 93.9% of mothers initiated breastfeeding within the first hour and first day, respectively. Overall, the median durations of any form of breastfeeding and EPBF were 40.6, and 20.9 weeks, respectively. Babies randomized to the lopinavir/ritonavir group in South Africa tended to do less EPBF than those in the lamivudine group.Overall the group of mothers aged between 25 and 30 years, those married, employed or multiparous tended to stop early EPBF. Mothers living in Uganda or Zambia, those aged between 25 -30 years, better educated (at least secondary school level), employed or having undergone C-section stopped any breastfeeding early.ConclusionsThere is a need to improve breastfeeding and complementary feeding practices of children, particularly those exposed to HIV and anti-retrovirals, taking into account context and socio-demographic factors.Trial registrationClinical trial registration: NCT00640263.Electronic supplementary materialThe online version of this article (doi:10.1186/s13006-017-0112-2) contains supplementary material, which is available to authorized users.

No ‘cure’ within 12 years of diagnosis among breast cancer patients who are diagnosed via mammographic screening: women diagnosed in the West Midlands region of England 1989–2011

We have previously reported that there is little evidence of population 'cure' among two populations of women diagnosed with invasive breast cancer. 'Cure' has not yet been examined in the context of screen-detection. We examined cancer registry data on 19 800 women aged 50-70, diagnosed with a primary, invasive, non-metastatic breast cancer between 1 April 1989 and 31 March 2011 in the West Midlands region of England, linked to Hospital Episode Statistics (HES) and the National Breast Screening Service (NBSS). Follow-up was complete on all women up to 31 July 2012. Analyses were stratified by screening status, age, tumour stage, deprivation and ethnicity. We estimated net survival for the whole cohort and each subgroup. Population 'cure' was evaluated by fitting flexible parametric log-cumulative excess hazard regression models in which the excess hazard of breast cancer death was assumed to be equal to zero after a given follow-up time. There was an overall lack of evidence for 'cure'. Across all subgroups examined, the general pattern was that of a continuous decrease in net survival over time, with no obvious asymptotic tendency within 12 years of follow-up. Model-based analyses confirmed this observation. Despite dramatic improvements in survival over past decades, diagnosis with breast cancer remains associated with a small but persistent increased risk of death for all groups of women, including those whose cancer is detected asymptomatically. These findings are unlikely to be due to methodological inadequacies. Communication of these long-term consequences of breast cancer among women recently diagnosed and to those considering undergoing screening should take due consideration of these patterns.