Nonlinear Model Selection Research Articles

PyDarwin machine learning algorithms application and comparison in nonlinear mixed-effect model selection and optimization

pyDarwin machine learning algorithms application and comparison in nonlinear mixed-effect model selection and optimization

PyDarwin: A Machine Learning Enhanced Automated Nonlinear Mixed-Effect Model Selection Toolbox.

pyDarwin is an open-source Python package for nonlinear mixed-effect model selection. pyDarwin combines machine-learning algorithms and NONMEM to perform a global search for the optimal model in a user-defined model search space. Compared with traditional stepwise search, pyDarwin provides an efficient platform for conducting an objective, robust, less labor-intensive model selection process without compromising model interpretability. In this tutorial, we will begin by introducing the essential components and concepts within the package. Subsequently, we will provide an overview of the pyDarwin modeling workflow and the necessary files needed for model selection. To illustrate the entire process, we will conclude with an example utilizing quetiapine clinical data.

Ranking-based selection of non-linear quantitative structure-property relationship models for prediction of bioconcentration factor of triazine derivatives as pesticide candidates

Ranking-based selection of non-linear quantitative structure-property relationship models for prediction of bioconcentration factor of triazine derivatives as pesticide candidates

Mathematical model to calculate the total number of decays in Peptide Receptor Radionuclide Therapy using nonlinear mixed effect modelling

The calculation of an individual total number of decays or time-integrated activities (TIAs) of a radiopharmaceutical for kidneys is desirable for dosimetry in molecular radiotherapy. The accuracy of the TIAs calculation relies heavily on the chosen fit model. Therefore, this study aimed to determine the best mathematical model of 177Lu-DOTATATE in peptide receptor radionuclide therapy (PRRT) using the nonlinear mixed effect (NLME) and model selection method. Pharmacokinetics data of 177Lu-DOTATATE in the kidneys of ten PRRT patients were obtained from the literature (PMID: 33443063). Eleven sums of exponential (SOE) functions were fitted to the pharmacokinetics data in the NLME framework. The model selection was performed based on the goodness of fit test and the corrected Akaike Information Criterion (AICc). The goodness of fit was evaluated based on the fitted graphs visualization, coefficient of variations (CV<50%), and the off-diagonal elements of the correlation matrix (-0,8≤CM≤0,8). In general, all SOE functions were successfully fitted to the pharmacokinetic data of 177Lu-DOTATATE in kidneys. The function was selected as the best mathematical model with an AICc weight of 77.58 %.

THE EFFECT OF DRIVER SAFETY PRIORITY ON SLEEPY DRIVING BEHAVIOR: LINEAR AND NON-LINEAR REGRESSION ANALYSIS

Driver safety priority is among the factors that affect the sleepy driving behavior of truck drivers. The aim of this paper is to determine the effect of truck drivers' driver safety priority on sleepy driving behavior with linear and non-linear models. Two hypotheses have been developed within the scope of the research. The sample area consists of 399 Turkish truck drivers. Surveys were collected face to face and online between January 2022 and April 2022. The sample area has been determined by simple random method. "Driver safety priority" and "Sleepy driving behavior" scales is used, which is tested for reliability and validity. Simple regression analysis has been performed for the linear model. In the selection of non-linear models, curve estimation models have been tested. The “Power” estimation model has been determined as the best model. According to the research findings, driver safety priority has a negative and significant effect on sleepy driving behavior in both linear and non-linear models. In addition, it has been determined that there is no significant difference between the percentages of explanation of linear and non-linear models. It has also been determined that the coefficient of the driver safety priority of the linear model on sleepy driving behavior is higher than the non-linear model. Based on the empirical findings, implications for both managers and drivers have been presented.

The analog computer: Beyond the museum artwork, a tool for studying linear and nonlinear systems

Analog computers have been a valuable resource for a time to solve linear as well as nonlinear differential equations before being superseded by digital computers. However, interest toward analog computing has not fully faded away. Though not competitive for accuracy or speed, analog computers have the important property of being true physical systems that can be made to behave according to a given equations set. This aspect makes analog computers interesting educational tools for hands-on work with real physical systems, where a number of effects, both linear and not linear, can easily be added at will (a feature not readily available in mechanical systems). This paper describes the implementation of a modular, very flexible, cheap, yet powerful analog computer, suitable for student laboratories. As examples of possible applications, a selection of linear and nonlinear oscillator models are described that can be used in the laboratory to introduce students to various nonlinear effects.

On the convergence rate of Fletcher‐Reeves nonlinear conjugate gradient methods satisfying strong Wolfe conditions: Application to parameter identification in problems governed by general dynamics

Over the last decades, many efforts have been made toward the understanding of the convergence rate of the gradient‐based method for both constrained and unconstrained optimization. The cases of the strongly convex and weakly convex payoff function have been extensively studied and are nowadays fully understood. Despite the impressive advances made in the convex optimization context, the nonlinear non‐convex optimization problems are still not fully exploited. In this paper, we are concerned with the nonlinear, non‐convex optimization problem under system dynamic constraints. We apply our analysis to parameter identification of systems governed by general nonlinear differential equations. The considered inverse problem is presented using optimal control tools. We tackle the optimization through the use of Fletcher‐Reeves nonlinear conjugate gradient method satisfying strong Wolfe conditions with inexact line search. We rigorously establish a convergence analysis of the method and report a new linear convergence rate which forms the main contribution of this work. The theoretical result reported in our analysis requires that the second derivative of the payoff functional be continuous and bounded. Numerical evidence on a selection of popular nonlinear models is presented as a direct application of parameter identification to support the theoretical findings.

Raphia-Microorganism Composite Biosorbent for Lead Ion Removal from Aqueous Solutions.

This study investigated the possibility of obtaining a raphia-microorganism composite for removing lead ions from aqueous solutions using immobilized yeast cells Saccharomyces cerevisiae on Raphia farinifera fibers. The obtained biocomposite was characterized using scanning electron microscopy and Fourier transform infrared spectroscopy. Studies were conducted to determine the influence of contact time, initial concentration of Pb(II), and pH allowed for the selection of nonlinear equilibrium and kinetic models. The results showed that the biocomposite had a better Pb(II) removal capacity in comparison to the raphia fibers alone, and its maximum Pb(II) adsorption capacity was 94.8 mg/g. The model that best describes Pb(II) sorption was the Temkin isotherm model, while kinetic studies confirmed the chemical nature of the sorption process following the Elovich model. The obtained research results provide new information on the full use of the adsorption function of biomass and the ubiquitous microbial resources and their use in the remediation of aqueous environments contaminated with heavy metals.

Fast selection of nonlinear mixed effect models using penalized likelihood

Nonlinear Mixed effects models are hidden variables models that are widely used in many fields such as pharmacometrics. In such models, the distribution characteristics of hidden variables can be specified by including several parameters such as covariates or correlations which must be selected. Recent development of pharmacogenomics has brought averaged/high dimensional problems to the field of nonlinear mixed effects modeling for which standard covariates selection techniques like stepwise methods are not well suited. The selection of covariates and correlation parameters using a penalized likelihood approach is proposed. The penalized likelihood problem is solved using a stochastic proximal gradient algorithm to avoid inner-outer iterations. Speed of convergence of the proximal gradient algorithm is improved using component-wise adaptive gradient step sizes. The practical implementation and tuning of the proximal gradient algorithm are explored using simulations. Calibration of regularization parameters is performed by minimizing the Bayesian Information Criterion using particle swarm optimization, a zero-order optimization procedure. The use of warm restart and parallelization allowed computing time to be reduced significantly. The performance of the proposed method compared to the traditional grid search strategy is explored using simulated data. Finally, an application to real data from two pharmacokinetics studies is provided, one studying an antifibrinolytic and the other studying an antibiotic.

Benchmarking Optimisation Methods for Model Selection and Parameter Estimation of Nonlinear Systems

Characterisation and quantification of nonlinearities in the engineering structures include selecting and fitting a good mathematical model to a set of experimental vibration data with significant nonlinear features. These tasks involve solving an optimisation problem where it is difficult to choose a priori the best optimisation technique. This paper presents a systematic comparison of ten optimisation methods used to select the best nonlinear model and estimate its parameters through nonlinear system identification. The model selection framework fits the structure’s equation of motions using time-domain dynamic response data and takes into account couplings due to the presence of the nonlinearities. Three benchmark problems are used to evaluate the performance of two families of optimisation methods: (i) deterministic local searches and (ii) global optimisation metaheuristics. Furthermore, hybrid local–global optimisation methods are examined. All benchmark problems include a free play nonlinearity commonly found in mechanical structures. Multiple performance criteria are considered based on computational efficiency and robustness, that is, finding the best nonlinear model. Results show that hybrid methods, that is, the multi-start strategy with local gradient-based Levenberg–Marquardt method and the particle swarm with Levenberg–Marquardt method, lead to a successful selection of nonlinear models and an accurate estimation of their parameters within acceptable computational times.

Nonlinear Model Selection for Fruit and Kernel Development as a Function of Heat in Pistachio

Knowing a tree crop’s seasonal growth and development as a function of heat accumulation can facilitate scheduling of irrigation, pesticide applications, and harvest. Our objective was to compare the goodness of fit of applied models and determine which provides the best description of pistachio nut growth as a function of thermal unit accumulation. Three fruit growth traits of pistachio—pericarp (hull) + endocarp (shell) size, endocarp thickening and hardening, and embryo (kernel) size—exhibited clear nonlinear dependence on heat accumulation. We tested three nonlinear models—Michaelis–Menten, three-parameter logistic, and Gompertz—fitted to fruit development data to create a tool to forecast pest susceptibility and harvest timing. Observation of development began at full bloom and ended at harvest. Data were collected from six pistachio cultivars in one experimental and eight commercial orchards over 3 years. Analyses of residual distribution, parameter standard errors, coefficient of determination (R2) and the Akaike information criterion (AIC) all demonstrated the Gompertz function was the best model. Cultivars differed significantly in all the three parameters (Asym, b, and c) for all three traits with the Gompertz model, demonstrating the Gompertz model can adjust to incorporate cultivar differences. The growth curve of the three traits together provided integrated information on nut biomass accumulation that facilitates predicting the critical timing for multiple orchard management practices.

A Deep Neural Network for Accurate and Robust Prediction of the Glass Transition Temperature of Polyhydroxyalkanoate Homo- and Copolymers.

The purpose of this study was to develop a data-driven machine learning model to predict the performance properties of polyhydroxyalkanoates (PHAs), a group of biosourced polyesters featuring excellent performance, to guide future design and synthesis experiments. A deep neural network (DNN) machine learning model was built for predicting the glass transition temperature, Tg, of PHA homo- and copolymers. Molecular fingerprints were used to capture the structural and atomic information of PHA monomers. The other input variables included the molecular weight, the polydispersity index, and the percentage of each monomer in the homo- and copolymers. The results indicate that the DNN model achieves high accuracy in estimation of the glass transition temperature of PHAs. In addition, the symmetry of the DNN model is ensured by incorporating symmetry data in the training process. The DNN model achieved better performance than the support vector machine (SVD), a nonlinear ML model and least absolute shrinkage and selection operator (LASSO), a sparse linear regression model. The relative importance of factors affecting the DNN model prediction were analyzed. Sensitivity of the DNN model, including strategies to deal with missing data, were also investigated. Compared with commonly used machine learning models incorporating quantitative structure–property (QSPR) relationships, it does not require an explicit descriptor selection step but shows a comparable performance. The machine learning model framework can be readily extended to predict other properties.

Selection of nonlinear mixed models for growth curves of dairy buffaloes (Bubalus bubalis)

AbstractThe determination of livestock growth patterns is important for meat or milk production systems, and nonlinear models are used to summarize and interpret the information. The aim of this study was to more accurately estimate growth curve parameters in buffalo cows by evaluating and selecting nonlinear mixed models that employ different types of residuals and include or not contemporary groups (CG) as a covariate. Weight records from 720 animals obtained over a period of 60 months were used. The growth curves were fit using nonlinear mixed-effects models. The Bertalanffy, Gompertz and Logistic models were evaluated. Modelling residuals using four structures (constant, combined, exponential and proportional) and the inclusion or not of CG in the models were also evaluated. The Akaike information criterion (AIC) and Bayesian information criterion (BIC) were used to select the model. In addition to estimating the parameters of the nonlinear growth models and their correlations, the instantaneous growth rate and inflection point were obtained. The Bertalanffy model with a combined residual structure and CG exhibited the lowest AIC and BIC values. Asymptotic weight (A) estimates ranged from 621.8 to 742.1 kg, and the maturity rate (k) ranged from 0.068 to 0.115 kg/month. The correlation between A and k ranged from −0.32 to −0.82 among the models evaluated. The selection criteria indicated that the Bertalanffy model was the most suitable for growth curve analysis in buffaloes.

An Optimization-Based Framework for Nonlinear Model Selection and Identification

This paper proposes an optimization-based framework to determine the type of nonlinear model present and identify its parameters. The objective in this optimization problem is to identify the parameters of a nonlinear model by minimizing the differences between the experimental and analytical responses at the measured coordinates of the nonlinear structure. The application of the method is demonstrated on a clamped beam subjected to a nonlinear electromagnetic force. In the proposed method, the assumption is that the form of nonlinear force is not known. For this reason, one may assume that any nonlinear force can be described using a Taylor series expansion. In this paper, four different possible nonlinear forms are assumed to model the electromagnetic force. The parameters of these four nonlinear models are identified from experimental data obtained from a series of stepped-sine vibration tests with constant acceleration base excitation. It is found that a nonlinear model consisting of linear damping and linear, quadratic, cubic, and fifth order stiffness provides excellent agreement between the predicted responses and the corresponding measured responses. It is also shown that adding a quadratic nonlinear damping does not lead to a significant improvement in the results.

Spatio-temporal additive regression model selection for urban water demand

Understanding the factors influencing urban water use is critical for meeting demand and conserving resources. To analyze the relationships between urban household-level water demand and potential drivers, we develop a method for Bayesian variable selection in partially linear additive regression models, particularly suited for high-dimensional spatio-temporally dependent data. Our approach combines a spike-and-slab prior distribution with a modified version of the Bayesian group lasso to simultaneously perform selection of null, linear, and nonlinear models and to penalize regression splines to prevent overfitting. We investigate the effectiveness of the proposed method through a simulation study and provide comparisons with existing methods. We illustrate the methodology on a case study to estimate and quantify uncertainty of the associations between several environmental and demographic predictors and spatio-temporally varying household-level urban water demand in Tampa, FL.

Nonlinear predictive model selection and model averaging using information criteria

ABSTRACTThis paper is concerned with the model selection and model averaging problems in system identification and data-driven modelling for nonlinear systems. Given a set of data, the objective of model selection is to evaluate a series of candidate models and determine which one best presents the data. Three commonly used criteria, namely, Akaike information criterion, Bayesian information criterion and an adjustable prediction error sum of squares (APRESS) are investigated and their performance in model selection and model averaging is evaluated via a number of case studies using both simulation and real data. The results show that APRESS produces better models in terms of generalization performance and model complexity.

Model selection and averaging of nonlinear mixed-effect models for robust phase III dose selection

Population model-based (pharmacometric) approaches are widely used for the analyses of phase IIb clinical trial data to increase the accuracy of the dose selection for phase III clinical trials. On the other hand, if the analysis is based on one selected model, model selection bias can potentially spoil the accuracy of the dose selection process. In this paper, four methods that assume a number of pre-defined model structure candidates, for example a set of dose–response shape functions, and then combine or select those candidate models are introduced. The key hypothesis is that by combining both model structure uncertainty and model parameter uncertainty using these methodologies, we can make a more robust model based dose selection decision at the end of a phase IIb clinical trial. These methods are investigated using realistic simulation studies based on the study protocol of an actual phase IIb trial for an oral asthma drug candidate (AZD1981). Based on the simulation study, it is demonstrated that a bootstrap model selection method properly avoids model selection bias and in most cases increases the accuracy of the end of phase IIb decision. Thus, we recommend using this bootstrap model selection method when conducting population model-based decision-making at the end of phase IIb clinical trials.

Tempered particle filters for non-linear model selection and uncertainty quantification of highly informative seabed data

Knowledge about seabed properties is important for many geoscientific and navy applications, such as sediment transport, sonar performance prediction, and detection of unexploded ordnance. Bayesian model selection and uncertainty estimation have been shown to provide detailed, quantitative seabed knowledge that is valuable for these applications. However, the extreme computational cost limits the utility of Bayesian methods for increasingly common big data sets. Here, we consider geoacoustic reflectivity surveys based on towed source and receiver arrays. Such systems produce thousands of data sets with high information content that require non-linear inversion along tracks many kilometers in length and cannot be analyzed by standard Bayesian sampling. A particle filter that includes reversible jump Markov chain Monte Carlo updates is applied here for efficient posterior probability estimation. Efficiency is improved by likelihood tempering of various particle subsets and including information exchange within the particle cloud. The tempering applies to reversible jump updates and leads to significantly improved exploration of the trans-dimensional seabed model which accounts for changes in the number of sediment layers and their properties along the track. For challenging track sections, where data change abruptly, the particle cloud is resampled to increase the number of tempered particles. [Work supported by the U.S. Dept. of Defense, through SERDP, and by ONR Ocean Acoustics.]

Bayesian nonlinear model selection for gene regulatory networks.

Gene regulatory networks represent the regulatory relationships between genes and their products and are important for exploring and defining the underlying biological processes of cellular systems. We develop a novel framework to recover the structure of nonlinear gene regulatory networks using semiparametric spline-based directed acyclic graphical models. Our use of splines allows the model to have both flexibility in capturing nonlinear dependencies as well as control of overfitting via shrinkage, using mixed model representations of penalized splines. We propose a novel discrete mixture prior on the smoothing parameter of the splines that allows for simultaneous selection of both linear and nonlinear functional relationships as well as inducing sparsity in the edge selection. Using simulation studies, we demonstrate the superior performance of our methods in comparison with several existing approaches in terms of network reconstruction and functional selection. We apply our methods to a gene expression dataset in glioblastoma multiforme, which reveals several interesting and biologically relevant nonlinear relationships.

Selection of nonlinear models for the description of the growth curves of coffee fruit

O cafe e um dos produtos mais importantes da agricultura brasileira, portanto estudos que visam uma melhor qualidade final do seu fruto tem sido, cada vez mais, objetivo de pesquisas. Conhecer as curvas de crescimento de frutos permite um melhor manejo da cultura, bem como identificar as epocas de maior demanda de nutrientes. Objetivou-se, neste trabalho, comparar o ajuste dos modelos de regressao nao lineares Logistico e Gompertz, na descricao das curvas de crescimento do fruto do cafeeiro, considerando a heterogeneidade das variâncias amostrais, nas diferentes faces de exposicao ao sol. Os dados analisados foram obtidos de Cunha e Volpe (2011) e correspondem aos resultados parciais de um experimento conduzido na Faculdade de Ciencias Agrarias e Veterinarias, UNESP, Campus Jaboticabal, SP, com a cultivar Obata IAC 1669-20, que pertence a especie Coffea arabica. Foram utilizados os modelos nao lineares Gompertz e Logistico, ponderados (variâncias amostrais heterogeneas) e nao ponderados. A estimacao dos parâmetros foi feita utilizando a funcao “gnls” da biblioteca “nlme” do software R, o metodo de minimos quadrados e o algoritmo de convergencia de Gauss-Newton. A qualidade dos ajustes foi comparada com base nos seguintes criterios: coeficiente de determinacao ajustado ( R a 2 ) ; criterio de informacao de Akaike e medidas de curvatura de Bates e Watts. Os modelos nao lineares estudados descreveram de forma satisfatoria a curva de crescimento do fruto do cafeeiro, fornecendo parâmetros com interpretacoes praticas. A incorporacao da heterocedasticidade foi eficiente, pois tornou as estimativas dos parâmetros mais confiaveis. O modelo que melhor descreveu o crescimento do fruto do cafeeiro foi o modelo Gompertz ponderado, em ambas as faces de exposicao ao sol.