Nonlinear Functional Relationships Research Articles

Elevating Model Predictive Control Using Feedforward Artificial Neural Networks: A Review

Process control in the field of chemical engineering has always been a challenging task for the chemical engineers. Hence, the majority of processes found in the chemical industries are non-linear and in these cases the performance of the linear models can be inadequate. Recently a promising alternative modelling technique, artificial neural networks (ANNs), has found numerous applications in representing non-linear functional relationships between variables. A feedforward multi-layered neural network is a highly connected set of elementary non-linear neurons. Model-based control techniques were developed to obtain tighter control. Many model-based control schemes have been proposed to incorporate a process model into a control system. Among them, model predictive control (MPC) is the most common scheme. MPC is a general and mathematically feasible scheme to integrate our knowledge about a target, process controller design and operation, which allows flexible and efficient exploitation of our understanding of a target, and thus produces optimal performance of a system under various constraints. The need to handle some difficult control problems has led us to use ANN in MPC and has recently attracted a great deal of attention. The efficacy of the neural predictive control with the ability to perform comparably to the non linear neural network strategy in both set point tracking and disturbance rejection proves to have less computation expense for the neural predictive control. The neural network model predictive control (NNMPC) method has less perturbations and oscillations when dealing with noise as compared to the PI controllers.

Cropland change in southern Romania: a comparison of logistic regressions and artificial neural networks

Changes in cropland have been the dominating land use changes in Central and Eastern Europe, with cropland abandonment frequently exceeding cropland expansion. However, surprisingly little is known about the rates, spatial patterns, and determinants of cropland change in Eastern Europe. We study cropland changes between 1995 and 2005 in Arges County in Southern Romania with two distinct modeling techniques. We apply and compare spatially explicit logistic regressions with artificial neural networks (ANN) using an integrated socioeconomic and environmental dataset. The logistic regressions allow identifying the determinants of cropland changes, but cannot deal with non-linear and complex functional relationships nor with collinearity between variables. ANNs relax some of these rigorous assumptions inherent in conventional statistical modeling, but likewise have drawbacks such as the unknown contribution of the parameters to the outcome of interest. We compare the outcomes of both modeling techniques quantitatively using several goodness-of-fit statistics. The resulting spatial predictions serve to delineate hotspots of change that indicate areas that are under more eminent threat of future abandonment. The two modeling techniques address two controversial issues of concern for land-change scientists: (1) to identify the spatial determinants that conditioned the observed changes and (2) to deal with complex functional relationships between influencing variables and land use processes. The spatially explicit insights into patterns of cropland change and in particular into hotspots of change derived from multiple methods provide useful information for decision-makers.

Flow-mediated feeding in the field: consequences for the performance and abundance of a sessile marine invertebrate

The ecology of sessile marine invertebrates that feed on waterborne particles is closely related to their surrounding fluid environment. However, few studies have quantified how water flow affects the feeding success of such organisms under field conditions and whether the results of shortterm feeding assays can predict longer-term patterns of growth and reproduction. I examined the influence of natural variation in current speed on the feeding success, colony growth rate and population abundance of an encrusting bryozoan Membranipora serrilamella using transplant experiments at nearshore reefs along the coast of Santa Barbara, California, USA. Feeding success was highest at intermediate ambient flow speeds (10 to 12 cm s–1) and lowest at low (0 to 5 cm s–1) and fast (> 20 cm s–1) speeds. Growth rates varied such that the greatest increases in bryozoan colony size occurred at reefs where average temperatures were the coolest and current speeds were intermediate. A test of the hypothesis that high feeding success translates into faster growth rates revealed that colonies with the highest predicted consumption grew the most. Further, spatial variation in M. serrilamella abundance mirrored that of colony growth, as percent cover of the bryozoan was highest at reefs where ambient current speeds were intermediate. These nonlinear functional relationships likely result from reductions in food delivery at low flows and deformation of feeding structures at high flows. The consequences of dynamic flow conditions for both short-term and integrated measures of performance suggest that flow-mediated food acquisition plays an important role in determining the distribution, abundance and morphology of encrusting bryozoans and other sessile, suspension-feeding invertebrates.

Identification and adaptation of linear look-up table parameters using an efficient recursive least-squares technique

Linear look-up tables are widely used to approximate and characterize complex nonlinear functional relationships between system input and output. However, both the initial calibration and subsequent real-time adaptation of these tables can be time consuming and prone to error as a result of the large number of table parameters typically required to map the system and the uncertainties and noise in the experimental data on which the calibration is based. In this paper, a new method is presented for identifying or adapting the look-up table parameters using a recursive least-squares based approach. The new method differs from standard recursive least squares algorithms because it exploits the structure of the look-up table equations in order to perform the identification process in a way that is highly computationally and memory efficient. The technique can therefore be implemented within the constraints of typical embedded applications. In the present study, the technique is applied to the identification of the volumetric efficiency look-up table commonly used in gasoline engine fueling strategies. The technique is demonstrated on a Ford 2.0L I4 Duratec engine using time-delayed feedback from a sensor in the exhaust manifold in order to adapt the table parameters online.

Prediction of wind loads on a large flat roof using fuzzy neural networks

Fuzzy neural networks (FNN) have capability to develop complex, nonlinear functional relationships between input–output patterns based on limited and sometimes inconsistent data, and are therefore suitable to predict wind loads on buildings on the basis of data obtained from model tests in wind tunnels. In this study, simultaneous pressure measurements are made on a large flat roof model in a boundary layer wind tunnel. An FNN approach is developed for prediction of mean pressure distributions on the roof model, and parts of the wind tunnel test results are used as the training sets for the FNN to recognize the pressure distribution patterns. The procedure is further extended to predict the power spectra and cross-power spectra of fluctuating wind pressures for some typical tap locations in the roof corners and leading edge areas under different wind directions. It is found that the developed FNN approach can generalize functional relationships of wind loads varying with incident wind directions and spatial locations on the roof, and can successfully predict the wind loads on the roof which are not fully covered by the wind tunnel measurements. It is demonstrated from this study that the adoption of the FNN approach can lead to a significant reduction of the pressure measurement programs (e.g., incident wind direction configurations and number of required pressure taps) in wind tunnel tests.

Multivariable regression model building by using fractional polynomials: Description of SAS, STATA and R programs

In fitting regression models data analysts are often faced with many predictor variables which may influence the outcome. Several strategies for selection of variables to identify a subset of ‘important’ predictors are available for many years. A further issue to model building is how to deal with non-linearity in the relationship between outcome and a continuous predictor. Traditionally, for such predictors either a linear functional relationship or a step function after grouping is assumed. However, the assumption of linearity may be incorrect, leading to a misspecified final model. For multivariable model building a systematic approach to investigate possible non-linear functional relationships based on fractional polynomials and the combination with backward elimination was proposed recently. So far a program was only available in Stata, certainly preventing a more general application of this useful procedure. The approach will be introduced, advantages will be shown in two examples, a new approach to present FP functions will be illustrated and a macro in SAS will be shortly introduced. Differences to Stata and R programs are noted.

Prediction of pressure coefficients on roofs of low buildings using artificial neural networks

This paper describes an artificial neural network (ANN) approach for the prediction of mean and root-mean-square (rms) pressure coefficients on the gable roofs of low buildings. The ANN models, which employ a backpropagation training algorithm, are capable of generalizing the complex, nonlinear functional relationships between the pressure coefficients and eave height, wind direction and spatial location on the roof. The performance of the ANN is demonstrated by the prediction of the pressure coefficients for roof tap locations in a corner bay. The mean bay uplift can be predicted accurately with an average error less than 2% for three cornering wind directions not seen by the ANN during training. The mean-square errors of all of the individual pressure taps in the corner bay were 12% and 9% for the mean and rms coefficients, respectively. This approach could be used to expand aerodynamic databases to a larger variety of geometries and increase its practical feasibility.

A kernel-based Adaline for function approximation

In this work the kernel Adaline algorithm is presented. The new algorithm is a generalisation of Widrow's and Hoff's linear Adaline, and allows to approximate non-linear functional relationships. Similar to the linear adaline, the proposed neural network algorithm minimises the least-mean-squared (LMS) cost function. It can be guaranteed that the kernel Adaline's cost function is always convex, therefore the method does not suffer from local optima as known in conventional neural networks. The algorithm uses potential function operators due to Aizerman and colleagues to map the training points in a first stage into a very high dimensional non-linear “feature” space. In the second stage the LMS solution in this space is determined by the algorithm. Weight decay regularisation allows to avoid overfitting effects, and can be performed efficiently. The kernel Adaline algorithm works in a sequential fashion, is conceptually simple, and numerically robust. The method shows a high performace in tasks like one dimensional curve fitting, system identification, and speech processing.

Persistent unstable equilibria and the grace period in dynamic models of environmental change

Models of the interactions between population, economy and environment often contain nonlinear functional relationships and variables that vary at different speeds. These properties foster apparent unpredictabilities in the system behavior. We identify a class of deterministic models on the interaction of demographic, economic and environmental interactions in which catastrophic changes in environmental quality can take place, but involve a delay between passing a critical threshold level of pollution and the final collapse of the environment. We denoted this delay as the 'environmental grace period'. We illustrate the usefulness of geometric singular perturbation theory and local bifurcation theory to analyse such models. In particular, we show how it is possible to obtain analytic expressions for: (1) the level of emissions above which environmental deterioration begins; (2) the time it takes from reaching the critical level of emissions to the beginning of rapid environmental deterioration and (3) the level of emissions at the time that rapid deterioration begins. Because our results are analytic, they make the outcomes of demo- graphic, economic and environmental interactions more predictable and, therefore, poten- tially more manageable.

Slow-fast dynamics in a model of population and resource growth.

"Models of the interaction of population, the economy, and the environment often contain nonlinear functional relationships and variables that move at different speeds. These properties foster apparent unpredictabilities in system behaviour. Using a simple deterministic model of demographic, economic and environmental interactions we illustrate the usefulness of geometric singular perturbation theory in environmental population economics. In contrast to local stability analysis, the theory of slow-fast dynamics helps to gain new insights into the global behaviour of the system. In particular, the knowledge of the basins of attraction of the stationary states enables one to determine the regions of sustainable future paths of resources and population." (SUMMARY IN FRE)

A Numerical Method for Finding Form-preserving Free Solutions of the Barotropic Vorticity Equation on a Sphere

Abstract A numerical method is presented that allows one to obtain steady or zonally moving (i.e., form-preserving) solutions of the barotropic vorticity equation on a sphere. The solutions are characterized by nonlinear functional relationships between the streakfunction (the streamfunction in a steady or zonally moving frame of reference) and the absolute vorticity. Three new examples of steady solutions are obtained that resemble atmospheric blocking. The examples have approximately the same global structure but differ in the degree of uniformity of the streamfunction and the absolute vorticity in the blocking region. From a numerical linear stability analysis it is concluded that the example in which the absolute vorticity in this region is the most uniform is to be preferred as a description of atmospheric blocking. This conclusion is consistent with recent observational evidence.

Bioprocess model building using artificial neural networks

Artificial neural networks are made upon of highly interconnected layers of simple ‘neuron-like’ nodes. The neurons act as non-linear processing elements within the network. An attractive property of artificial neural networks is that given the appropriate network topology, they are capable of learning and characterising non-linear functional relationships. Furthermore, the structure of the resulting neural network based process model may be considered generic, in the sense that little prior process knowledge is required in its determination. The methodology therefore provides a cost efficient and reliable process modelling technique. One area where such a technique could be useful is biotechnological systems. Here, for example, the use of a process model within an estimation scheme has long been considered an effective means of overcoming inherent on-line measurement problems. However, the development of an accurate process model is extremely time consuming and often results in a model of limited applicability. Artificial neural networks could therefore prove to be a useful model building tool when striving to improve bioprocess operability. Two large scale industrial fermentation systems have been considered as test cases; a fed-batch penicillin fermentation and a continuous mycelial fermentation. Both systems serve to demonstrate the utility, flexibility and potential of the artificial neural network approach to process modelling.

Artificial neural networks in process engineering

Artificial neural networks are made up of highly interconnected layers of simple ‘neuron-like’ nodes. The neurons act as nonlinear processing elements within the network. An attractive property of artificial neural networks is that, given the appropriate network topology, they are capable of characterising nonlinear functional relationships. Furthermore, the structure of the resulting neural network based process model may be considered generic, in the sense that little prior process knowledge is required in its determination. The methodology therefore provides a cost efficient and reliable process modelling technique. The concepts involved in the formulation of artificial neural networks are introduced. Their suitability for solving some process engineering problems is discussed and illustrated using results obtained from both simulation studies and recent applications to industrial process data. In the latter, neural network models were used to provide estimates of biomass concentration in industrial fermentation systems and of top product composition of an industrial distillation tower. Measurements from established instruments such as off-gas carbon dioxide in the fermenter and overheads temperature in the distillation column were used as the secondary variables for the respective processes. The advantage of using these estimates for feedback control is demonstrated. The possibility of using neural network models directly in model based control strategies is also considered. The range of applications is an indication of the utility of artificial neural network methodologies within a process engineering environment.

Estimation problems of non-linear functional relationships

Non-linear functional relationships between two variables (curve fitting with errors in both variables) may be defined in a similar way to linear functional relationships. The ratio of the error variances must be fixed to obtain a solution to the likelihood equations, but this solution may not be unique. The best fitting solution may change abruptly as the error variance ratio passes through a critical value. Some examples are given.

Approximate inference in multivariate nonlinear functional relationships

An estimation procedure based on estimating equations is presented for the parameters in a multivariate functional relationship model, where all observations are subject to error. The covariance matrix of the observational errors may be parametrized and is allowed to be different for different sets of observations. Estimators are defined for the unknown relation parameters and error parameters.For linear models (i.e. where the model function is linear in the incidental parameters) the estimators are consistent and asymptotically normal. A consistent expression for the covariance matrix of the estimators is derived. The results are valid for general error distributions.For nonlinear models the estimators are based on locally linear approximations to the model function. The afore mentioned properties of the estimators are now only approximately valid. The adequacy of the approximate inference, based on asymptotic theory for the linearized model, needs at least informal check. Some examples are given to illustrate the estimation procedure.

Tobin's q ratio, monopoly earnings, risk, and dividend policy

Inherently nonlinear functional relationships between Tobin's q ratio of market value to replacement cost and financial variables of the firm are developed and estimated. Statistical significance and material importance of returns on equity and dividend payout ratios are established, but systematic risk does not appear to provide significant explanation of differences in equilibrium q values for a sample of 97 firms.

Discussion of Professor Copas's Paper

Discussion of Professor Copas's Paper

International Intra-Company Transfer Pricing

This paper presents the important issues of the multinational corporation's transfer pricing determination problem. It discusses the behavioral consequences of the setting of transfer prices by top management. It presents a model of transfer pricing that is developed further to include all of the issues that affect the total profits of the multinational corporation through transfer prices. The model is then expanded to include economic externalities and interdependencies such as nonlinear cost functions and functional relationships among demand, supply and transfer prices. An example is given and solved via geometric programming.

Quadratic Limit States in Structural Reliability

Second-moment methods are widely applied in structural reliability. Recently, so-called first-order reliability methods have been developed that are capable of producing reliable estimates of the failure probability for arbitrary design situations and distributional assumptions for the uncertainity vector. In essence, nonlinear functional relationships or probability distribution transformations are approximated by linear Taylor expansions so that the simple second-moment calculus is retained. Failure probabilities are obtained by evaluating the standard normal integral, which is the probability content of a circular normal distribution in a domain bounded by a hyperplane. In this paper second-order expansions are studied to approximate the failure surface and some results of the statistical theory of quadratic forms in normal variates are used to calculate improved estimates of the failure probability.

Maximum likelihood estimation of multivariate non-linear functional relationships

The results of Dolby (1972) and Dolby and Feeeman (1975) on the estimation of an explicit non-linear functional relationship are examined for the multivariate case with Σknown up to a multiplicative constant. It is shown that in both univariate and multivariate situations the method of scoring for parameters is susceptible to major errors whenever the non-linearity implies asymmetric peaks or valleys and the data is relatively ”noisy”. Amendments to this method are proposed and demonstrated.