Parsimonious Model Structure Research Articles

Broad Learning Based Dynamic Fuzzy Inference System With Adaptive Structure and Interpretable Fuzzy Rules

This article investigates the feasibility of applying the broad learning system (BLS) to realize a novel Takagi–Sugeno–Kang (TSK) neuro-fuzzy model, namely a broad learning based dynamic fuzzy inference system (BL-DFIS). It not only improves the accuracy and interpretability of neuro-fuzzy models but also solves the challenging problem that models are incapable of determining the optimal architecture autonomously. BL-DFIS first accomplishes a TSK fuzzy system under the framework of BLS, in which an extreme learning machine auto-encoder is employed to obtain feature representation in a fast and analytical way, and an interpretable linguistic fuzzy rule is integrated into the enhancement node to ensure the high interpretability of the system. Meanwhile, the extended-enhancement unit is designed to achieve the first-order TSK fuzzy system. In addition, a dynamic incremental learning algorithm with internal pruning and updating mechanism is developed for the learning of BL-DFIS, which enables the system to automatically assemble the optimal structure to obtain a compact rule base and an excellent classification performance. Experiments on benchmark datasets demonstrate that the proposed BL-DFIS can achieve a better classification performance than some state-of-the-art nonfuzzy and neuro-fuzzy methods, simultaneously using the most parsimonious model structure.

Formulation of Parsimonious Urban Flash Flood Predictive Model with Inferential Statistics

The curve number (CN) rainfall–runoff model is widely adopted. However, it had been reported to repeatedly fail in consistently predicting runoff results worldwide. Unlike the existing antecedent moisture condition concept, this study preserved its parsimonious model structure for calibration according to different ground saturation conditions under guidance from inferential statistics. The existing CN model was not statistically significant without calibration. The calibrated model did not rely on the return period data and included rainfall depths less than 25.4 mm to formulate statistically significant urban runoff predictive models, and it derived CN directly. Contrarily, the linear regression runoff model and the asymptotic fitting method failed to model hydrological conditions when runoff coefficient was greater than 50%. Although the land-use and land cover remained the same throughout this study, the calculated CN value of this urban watershed increased from 93.35 to 96.50 as the watershed became more saturated. On average, a 3.4% increase in CN value would affect runoff by 44% (178,000 m3). This proves that the CN value cannot be selected according to the land-use and land cover of the watershed only. Urban flash flood modelling should be formulated with rainfall–runoff data pairs with a runoff coefficient > 50%.

The regionalization of a parameter of HYMOD, a conceptual hydrological model, using data from across the globe

The approximation of streamflow data in an un-gauged catchment is challenging and is generally addressed through parameter regionalization. Though identifying the relationships between catchment attributes and model parameters is straightforward, the uncertainties in both model parameters and functional relationships lead to poor regionalization. The key to successful regionalization is selecting a parsimonious model structure, proper catchment attributes, a better calibration strategy, and a proper regional model structure. The parameters of the HYMOD model were calibrated from 59 watersheds across the globe using a three-year calibration period and evolutionary algorithms, which are well-suited to account for the multiobjective nature of hydrological model calibration. The optimized parameter set satisfactorily reproduced the observed flow for all catchments used to calibrate the regional models. Over the calibration and validation period, the optimal regional relationship obtained from the multiple polynomial regression (MPR) and the multiobjective regional calibration (MORC) methods were developed and evaluated. For basins used for calibration of regional relationship, the performance loss of the model simulation with parameters estimated from the MPR and MORC methods was 14% and 10% respectively. The performance loss for basins presumed ungauged was 15 and 12.6% for MPR and MORC respectively. MORC was more effective in both basins considered for the derivation and evaluation of regional functions.

An uncommon guest is masked in fisheries landings: The case of the endemic sub-population of red spiny lobster Panulirus penicillatus (Decapoda: Palinuridae) in Costa Rica

Fishery landings composition in tropical developing countries is formed of high diversity of species and some of them are not well-reported in landings or national fisheries statistics. This is particularly the case of the endemic red spiny lobster Panulirus penicillatus (Olivier, 1791) on the Pacific coast of Costa Rica. The red spiny lobster is a circumtropical species which has the most extensive global distribution of all the species of spiny lobsters. In order to fill the gap of knowledge regarding the spatial distribution of this species in the eastern Pacific Ocean and highlight its importance to fishing communities, this study focused on data collected from a dive-based bentho-demersal fishery on the northern Pacific coast of Costa Rica. Environmental and fishing effort data were recorded as an input for fitting site-occupancy models to get estimates of occurrence and probability of detection. The most parsimonious model structure for detection suggested that the covariates of diving time and water column visibility had an important effect on detection probability. For the occupancy part of the model, the depth gradient had the strongest effect, with high probability of lobster encounters in shallow rocky reefs near the coastline. These findings point towards an enlargement of the area of occurrence of the endemic sub-population of P. penicillatus to which sampled animals belong. Further, it is discussed potential ecological and fishery factors that explain the low occurrence of the red spiny lobster at this restricted area of the eastern Pacific continental shelf. As a result, a set of research guidelines and management recommendations are suggested for promoting the conservation and sustainable use of this endemic species within the region.

Extracting Inherent Model Structures and Identifying Parameters of Time-Varying Systems Using Local Linear Neuro-Fuzzy Networks

The main challenges in identifying time-varying systems are to extract the inherent model structure and formulate a generic approach for various time-varying processes. In this article, a local linear neuro-fuzzy networks based approach is proposed for the first time to tackle this issue. The basic idea is to utilize the superior approximation capability of the fuzzy neural networks to expand the unknown time-varying parameters so that various nonlinear and discontinuous processes can be captured. In addition, the local linear model tree optimization algorithm is incorporated in our approach to optimize the network architecture. On one hand, the least number of neurons corresponding to the time-varying parameters can be automatically determined, thus avoiding the superfluous bases and the troublesome overfitting problem; on the other hand, the optimized distribution of the neurons could significantly increase the interpretability of the results, thus contributing to locating the domains where the parameters exhibit strong nonlinearity or change sharply. Moreover, we demonstrate that our approach could simultaneously extract the parsimonious model structure while identifying the time-varying parameters, without adding additional procedures or increasing computation burden. Considering the generality and the unique merits, our proposed approach could significantly advance the applications of artificial neural network techniques in system identification.

A proportionality-based multi-scale catchment water balance model and its global verification

Unifying the description of catchment hydrological partitioning across spatial and temporal scales has long been an important research topic within the hydrological community. As an effort to that, here we develop a conceptual water balance model based on the proportionality hypothesis (denoted PWBM) and for the first time, test the proportionality hypothesis using hydrologic observations across global catchments at varying time scales. The PWBM has a transparent and parsimonious model structure with only two model parameters, and requires precipitation, potential evapotranspiration and leaf area index as model inputs. To test PWBM, we apply the model to simulate streamflow (Q), evapotranspiration (E) and storage change (ΔS) in 22 large basins and 255 small basins globally and find an overall good model performance in reproducing all three water balance components at the daily (only in small basins), monthly, seasonal, annual and mean-annual time scales. Compared with three other conceptual hydrological models developed for specific time scales, the PWBM substantially outperforms the SCS model at the daily scale and the ABCD model at the monthly scale and performs similarly as the long-standing Budyko model at the mean-annual scale. In summary, our study verifies the robustness of PWBM and offers convincing evidence regarding the applicability of the proportionality hypothesis for enhanced hydrological system understanding at multiple spatial and temporal scales.

A Multiple Beta Wavelet-Based Locally Regularized Ultraorthogonal Forward Regression Algorithm for Time-Varying System Identification With Applications to EEG

Time-varying (TV) nonlinear systems widely exist in various fields of engineering and science. Effective identification and modeling of TV systems is a challenging problem due to the nonstationarity and nonlinearity of the associated processes. In this paper, a novel parametric modeling algorithm is proposed to deal with this problem based on a TV nonlinear autoregressive with exogenous input (TV-NARX) model. A new class of multiple beta wavelet (MBW) basis functions is introduced to represent the TV coefficients of the TV-NARX model to enable the tracking of both smooth trends and sharp changes of the system behavior. To produce a parsimonious model structure, a locally regularized ultraorthogonal forward regression (LRUOFR) algorithm aided by the adjustable prediction error sum of squares (APRESS) criterion is investigated for sparse model term selection and parameter estimation. Simulation studies and a real application to EEG data show that the proposed MBW-LRUOFR algorithm can effectively capture the global and local features of nonstationary systems and obtain an optimal model, even for signals contaminated with severe colored noise.

Exploring parsimonious daily rainfall-runoff model structure using the hyperbolic tangent function and Tank model

Mathematical models with small numbers of parameters are useful for hydrological analyses and water resources planning as they are easier to prepare, use, and interpret compared to more complex models. Many simple model structures and simulation strategies have been developed, but there is still substantial room for improvement in the efficiency of daily rainfall-runoff modeling. This study proposes daily rainfall-runoff models that require only two to four parameters yet are capable of predicting peak discharge and baseflow. Parsimonious model structures were obtained using the hyperbolic tangent function and combining it with one subsurface layer and two parameters of a Tank model. The daily rainfall-runoff models proposed based on the Two-Parameter Hyperbolic Model (TPHM) and Tank model were applied to 41 watersheds throughout South Korea. Model accuracy was evaluated by comparing observed and simulated watershed streamflow and compared with that of a nine-parameter Tank model. The original TPHM and its variants yielded slightly better efficiency with a smaller number of parameters than the Tank model when reproducing high flows on a daily basis. The accuracy of low flow and flow duration curve predictions provided by the TPHM-based models was higher in watersheds with relatively large baseflow temporal variability. The results suggested that the TPHM-based structures be alternative parsimonious daily rainfall-runoff simulation models. This study demonstrates that daily high streamflow can be efficiently simulated using a two-parameter hydrological model, and daily baseflow simulation can be improved by adding two additional parameters to the model.

Subspace-based spectrum estimation in innovation models by mixed norm minimization

In this paper, identification of discrete-time power spectra of multi-input/multi-output (MIMO) systems in innovation models from output-only time-domain measurements is considered.A hybrid identification algorithm unifying mixed norm minimization with subspace estimation method is proposed. The proposed algorithm first estimates a covariance matrix from measurements. A significant dimension reduction is achieved in this step. Next, a regularized nuclear norm optimization problem is solved to enforce sparsity on the selection of most parsimonious model structure. A modification of the covariance estimates in the proposed algorithm generates yet another algorithm capable of handling data records with sequentially and intermittently missing values. The new and the modified identification algorithms are tested on a numerical study and a real-life application example concerned with the estimation of joint power spectral density (PSD) of parallel road tracks.

Epileptic seizure detection in EEG signals using sparse multiscale radial basis function networks and the Fisher vector approach

Detecting epileptic seizures in electroencephalography (EEG) signals is a challenging task due to nonstationary processes of brain activities. Currently, the epilepsy is mainly detected by clinicians based on visual observation of EEG recordings, which is generally time consuming and sensitive to bias. This paper presents a novel automatic seizure detection method based on the multiscale radial basis function (MRBF) networks and the Fisher vector (FV) encoding. Specifically, the MRBF networks are first used to obtain high-resolution time-frequency (TF) images for feature extraction, where both a modified particle swarm optimization (MPSO) method and an orthogonal least squares (OLS) algorithm are implemented to determine optimal scales and detect a parsimonious model structure. Gray level co-occurrence matrix (GLCM) texture descriptors and the FV, which contribute to high-dimensional vectors, are then adopted to achieve discriminative features based on five frequency subbands of clinical interests from TF images. Furthermore, the dimensionality of the original feature space can be effectively reduced by the t-test statistical tool before feeding compact features into the SVM classifier for seizure detection. Finally, the classification performance of the proposed method is evaluated by using two widely used EEG database, and is observed to provide good classification accuracy on both datasets. Experimental results demonstrate that our proposed method is a powerful tool in detecting epileptic seizures.

Copula‐based downscaling of daily precipitation fields

AbstractA novel stochastic downscaling approach to simulate ensembles of daily precipitation fields using the Gaussian copula is presented. In contrast to many other statistical downscaling techniques, this approach uses spatial correlation (correlograms) to derive the transfer function between predictors and predictands for a parsimonious model structure. Daily regional climate model (RCM) simulations for a region in Central Europe in two different spatial resolutions (7 and 42 km) served as a training set to derive the statistics necessary to simulate fine scale precipitation values. The model was calibrated with RCM simulations for the year 1971, and the evaluation was performed for the period 1972–2000 to emulate the typical problem of limited availability of fine scale data. A comprehensive evaluation of the downscaling approach comprising the spatial correlations and statistical distributions of the simulated precipitation fields and several further performance measures was performed. The distribution of simulated precipitation is in close agreement with values simulated from a distribution function that was fitted to the complete evaluation period. Average Brier skill scores of 0.5 indicate a good performance of reproducing the daily dynamical simulations for most regions. A comparison with precipitation fields interpolated with inverse distance weighting revealed an average added skill of 42% for different precipitation thresholds; 87% of the dry days and 71% of the wet days were simulated correctly. An advantage of the proposed method over deterministic downscaling techniques is that ensembles of predictand fields are generated. Thus, the uncertainty that is inherent to downscaling can be estimated. The method has the potential to be used in other downscaling applications to generate ensembles of spatially correlated predictands based on other predictors. As copulas treat the dependence structure separately from the marginal distributions of the predictors and predictands, it is possible to simulate meteorological variables from any desired distribution function.

Finding the optimal time resolution for batch-end quality prediction: MRQP – A framework for multi-resolution quality prediction

Batch-end quality prediction is of paramount importance in industries producing high added-value products. However, all available methods hereto only use raw data at its native resolution. This paper proposes a new multi-resolution quality prediction (MRQP) methodology for batch-end quality, which exploits structured correlation in both the time and variables dimensions. The implementation of this methodology results in a more parsimonious and robust model structure, with a predictive performance bounded to be at least as good as the current standard approach. The implementation of MRQP is illustrated in three different case studies, where improvements over the standard single-resolution approach were found to be in the range of 10%–50%. From an interpretation standpoint, multi-resolution models are more robust with respect to the selection of too many predictors, facilitating the identification of key process variables, and providing information on the process time scales that influence final product quality, which can be further exploited for diagnosis, control, and optimization.

L0-constrained regression using mixed integer linear programming

In this work, sparse regression using a penalized least absolute deviations objective function is considered. Regression model sparsity is promoted using a L0 - pseudo norm penalty (the cardinality of the model parameter vector). Implemented using mixed integer linear programming (MILP) it is demonstrated that the use of the L0 - norm (without approximation) enables efficient and accurate solutions to sparse regression problems of practical size. For model development with a large number of potential model parameters (or features) methods to relax the MILP are also developed; using nonlinear function approximations to the L0- norm, penalty terms are linearized and solved using sequential linear programming. Experimental results (using both simulated and real data) demonstrate that these algorithms are also computationally efficient producing accurate and parsimonious model structures. Applications considered are the development of a calibration model for prediction with Near Infrared (NIR) data and the development of a model for the prediction of chemical toxicity - a quantitative structure activity relationship (QSAR).

The long-term population dynamics of common wasps in their native and invaded range.

Populations of introduced species are often thought to perform differently, or experience different population dynamics, in their introduced range compared to their native habitat. Differences between habitats in climate, competition or natural enemies may result in populations with varying density dependence and population dynamics. We examined the long-term population dynamics of the invasive common wasp, Vespula vulgaris, in its native range in England and its invaded range in New Zealand. We used 39years of wasp density data from four sites in England, and 23years of data from six sites in New Zealand. Wasp population time series was examined using partial rate correlation functions. Gompertz population models and multivariate autoregressive state-space (MARSS) models were fitted, incorporating climatic variation. Gompertz models successfully explained 59-66% of the variation in wasp abundance between years. Density dependence in wasp populations appeared to act similarly in both the native and invaded range, with wasp abundance in the previous year as the most important variable in predicting intrinsic rate of increase (r). No evidence of cyclic population dynamics was observed. Both the Gompertz and MARSS models highlighted the role of weather conditions in each country as significant predictors of annual wasp abundance. The temporal evolution of wasp populations at all sites was best modelled jointly using a single latent dynamic factor for local trends, with the inclusion of a latent spring weather covariate. That same parsimonious multivariate model structure was optimal in both the native and invaded range. Density dependence is overwhelmingly important in predicting wasp densities and 'wasp years' in both the native and invaded range. Spring weather conditions in both countries have a major influence, probably through their impact on wasp colony initiation and early development. The population dynamics in the native range and invaded range show no evidence of cyclic boom-and-bust dynamics. Invasive species may not exhibit different population dynamics despite considerable variation in abundances throughout their distribution.

Assessing the Impact of Retreat Mechanisms in a Simple Antarctic Ice Sheet Model Using Bayesian Calibration

The response of the Antarctic ice sheet (AIS) to changing climate forcings is an important driver of sea-level changes. Anthropogenic climate change may drive a sizeable AIS tipping point response with subsequent increases in coastal flooding risks. Many studies analyzing flood risks use simple models to project the future responses of AIS and its sea-level contributions. These analyses have provided important new insights, but they are often silent on the effects of potentially important processes such as Marine Ice Sheet Instability (MISI) or Marine Ice Cliff Instability (MICI). These approximations can be well justified and result in more parsimonious and transparent model structures. This raises the question of how this approximation impacts hindcasts and projections. Here, we calibrate a previously published and relatively simple AIS model, which neglects the effects of MICI and regional characteristics, using a combination of observational constraints and a Bayesian inversion method. Specifically, we approximate the effects of missing MICI by comparing our results to those from expert assessments with more realistic models and quantify the bias during the last interglacial when MICI may have been triggered. Our results suggest that the model can approximate the process of MISI and reproduce the projected median melt from some previous expert assessments in the year 2100. Yet, our mean hindcast is roughly 3/4 of the observed data during the last interglacial period and our mean projection is roughly 1/6 and 1/10 of the mean from a model accounting for MICI in the year 2100. These results suggest that missing MICI and/or regional characteristics can lead to a low-bias during warming period AIS melting and hence a potential low-bias in projected sea levels and flood risks.

Time-Varying System Identification Using an Ultra-Orthogonal Forward Regression and Multiwavelet Basis Functions With Applications to EEG.

A new parametric approach is proposed for nonlinear and nonstationary system identification based on a time-varying nonlinear autoregressive with exogenous input (TV-NARX) model. The TV coefficients of the TV-NARX model are expanded using multiwavelet basis functions, and the model is thus transformed into a time-invariant regression problem. An ultra-orthogonal forward regression (UOFR) algorithm aided by mutual information (MI) is designed to identify a parsimonious model structure and estimate the associated model parameters. The UOFR-MI algorithm, which uses not only the observed data themselves but also weak derivatives of the signals, is more powerful in model structure detection. The proposed approach combining the advantages of both the basis function expansion method and the UOFR-MI algorithm is proved to be capable of tracking the change of TV parameters effectively in both numerical simulations and the real EEG data.

A comparison of simple and complex versions of the Xinanjiang hydrological model in predicting runoff in ungauged basins

There are different views on the selection of hydrological model structural complexity for streamflow prediction in ungauged basins. Some studies suggest that complex models are better than simple models due to the former's prediction capability; whereas some studies favor parsimonious model structures to overcome a risk of over-parameterization. The Xinanjiang (XAJ) model, the most widely used hydrological model in China, has two different versions, as follows: (1) the simple version with seven parameters (XAJ7) and (2) the complex version with 14 parameters (XAJ14). In this study, the two versions of the XAJ model were comprehensively evaluated for streamflow prediction in ungauged basins based on their efficiency, parameter identifiability, and independence. The results showed that the complex XAJ14 model was more flexible than the simple XAJ7 in calibration mode; while the two models have similar performance in validation and regionalization modes. Lack of parameter identifiability and the presence of parameter interdependence most likely explain why the complex XAJ14 cannot consistently outperform the XAJ7 in different modes. Therefore, the simple XAJ7 is a better choice than XAJ14 for streamflow prediction in ungauged basins.

Selecting and estimating interest rate models with evolutionary methods

Selecting and estimating parsimonious models is often desired, but hard to achieve. This is particularly true when models can potentially contain a very large number of parameters but data are scarce—as is the case for many macro-economic models in general and interest-rate models in particular. These models need to cater for a large number of potential relationships and dependencies, but are fitted on low-frequency data to focus on the bigger picture and long-term effects. To identify the ideal model and estimating it is then particularly demanding from an optimization perspective. In this paper, we suggest an evolutionary approach that considers model selection and estimation simultaneously. Numerical experiments with artificial data suggest that the approach is well suited for this type of problem. In an empirical application for short-term and long-term interest rates denominated in US dollar, euro and the Japanese yen, respectively, parsimonious model structures are identified that highlight the dependencies as well as spill-overs across maturities and currencies.

Assessment of the adequacy of the regional relationships between catchment attributes and catchment response dynamics, calibrated by a multi‐objective approach

AbstractThis study aimed to evaluate the effectiveness of the regionalization method on the basis of a combination of a parsimonious model structure and a multi‐objective calibration technique. For this study, 12 gauged catchments in the Republic of Korea were used. The parsimonious model structure, requiring minimal input data, was used to avoid adverse effects arising from model complexity, over‐parameterization and data requirements. The IHACRES rainfall‐runoff model was applied to represent the dynamic response characteristics of catchments in Korea. A multi‐objective approach was adopted to reduce the predictive uncertainty arising from the calibration of a rainfall‐runoff model, by increasing the amount of information retrieved from the available data. The regional relationships (or models) between the model parameters and the catchment attributes were established via a multiple regression approach, incorporating correlation analysis and stepwise regression on linear and logarithmic scales. The impacts of the parameters, calibrated by the multi‐objective approach, on the adequacy of regional relationships were assessed by comparison with impacts obtained by the single‐objective approach. The regional relationships were well defined, despite limited available data. The drainage area, the effective soil depth, the mean catchment slope and the catchment gradient appeared to be the main factors for describing the hydrologic response characteristics in the areas studied. The overall model performance of the regional models based on the multi‐objective approach was good, producing reasonable results for high and low flows and for the overall water balance, simultaneously. The regional models based on the single‐objective approach yielded accurate predictions in high flows but showed limited predictive capability for low flows and the overall water balance. This was due to the optimal model parameter estimates when using a single‐objective measure. The parameters calibrated by the single‐objective approach decreased the predictability of the regional models. Copyright © 2013 John Wiley & Sons, Ltd.

Temperature and Precipitation Affect Steer Weight Gains Differentially by Stocking Rate in Northern Mixed-Grass Prairie

Cattle weight gain responses to seasonal weather variability are difficult to predict for rangelands because few long-term (>20 yr) studies have been conducted. However, an increased understanding of temperature and precipitation influences on cattle weight gains is needed to optimize stocking rates and reduce enterprise risk associated with climatic variability. Yearling steer weight gain data collected at the USDA-ARS High Plains Grasslands Research Station at light, moderate, and heavy stocking rates for 30 years (1982–2011) were used to examine the effects of spring (April–June) and summer (July–September) temperature and precipitation, as well as prior-growing-season (prior April–September) and fall/winter (October–March) precipitation, on beef production (kg · ha−1). At heavier stocking rates, steer production was more sensitive to seasonal weather variations. A novel finding was that temperature (relatively cool springs and warm summers) played a large predictive role on beef production. At heavier stocking rates, beef production was highest during years with cool, wet springs and warm, wet summers, corresponding to optimum growth conditions for this mixed C3–C4 plant community. The novelty and utility of these findings may increase the efficacy of stocking rate decision support tools. The parsimonious model structure presented here includes three-month seasonal clusters that are forecasted and freely available from the US National Oceanic and Atmospheric Administration up to a year in advance. These seasonal weather forecasts can provide ranchers with an increased predictive capacity to adjust stocking rates (in advance of the grazing season) according to predicted seasonal weather conditions, thereby reducing enterprise risk.