Semiparametric Nonlinear Mixed-Effects Models and Their Applications

Abstract

Nonlinear mixed effects models (NLMMs) and self-modeling nonlinear regression (SEMOR) models are often used to fit repeated measures data. They use a common function shared by all subjects to model variation within each subject and some fixed and/or random parameters to model variation between subjects. The parametric NLMM may be too restrictive, and the semiparametric SEMOR model ignores correlations within each subject. In this article we propose a class of semiparametric nonlinear mixed effects models (SNMMs) that extend NLMMs, SEMOR models, and many other existing models in a natural way. A SNMM assumes that the mean function depends on some parameters and nonparametric functions. The parameters provide an interpretable data summary. The nonparametric functions provide flexibility to allow the data to decide some unknown or uncertain components, such as the shape of the mean response over time. A second-stage model with fixed and random effects is used to model the parameters. Smoothing splines are used to model the nonparametric functions. Covariate effects on parameters can be built into the second-stage model, and covariate effects on nonparametric functions can be constructed using smoothing spline ANOVA decompositions. Laplace approximations to the marginal likelihood and penalized marginal likelihood are used to estimate all parameters and nonparametric functions. We propose and compare two estimation procedures, and also show how to construct approximate Bayesian confidence intervals for the nonparametric functions based on a Bayesian formulation of SNMMs. We evaluate the proposed estimation and inference procedures through a simulation study. Applications of SNMMs are illustrated with analyses of Canadian temperature data.