Input Subspaces Research Articles

Pattern‐orientedanalysis of system dynamics models via random forests

AbstractSystem dynamics (SD) modeling studies aim to reveal the causes of problematic dynamic behaviors and eliminate them through policy design and analysis. The analyst conducts sensitivity/scenario analyses and what‐if experiments to reveal the input–output relationships during modeling. However, during these analyses and investigations, the identification of input‐parameter spaces that cause the generation of different SD model behavior patterns is time consuming and susceptible to human bias. Therefore, we propose a metamodel‐based procedure for SD models that considers the necessity for unbiased and automated analysis and insight generation. The approach uses the random forest algorithm for metamodel generation and extracts interpretable IF–THEN rules from the metamodel, thereby identifying input subspaces that generate different qualitative or numerical SD model outputs. We illustrate the proposed approach using two well‐established SD models. These case studies reveal how the model analyst can utilize the proposed method to capture input–output relationships. © 2022 System Dynamics Society.

A novel concurrent approach for multiclass scenario discovery using Multivariate Regression Trees: Exploring spatial inequality patterns in the Vietnam Mekong Delta under uncertainty

To support equitable planning, model-based analyses can be used to explore inequality patterns arising from different scenarios. Scenario discovery is increasingly used to extract insights from ensembles of simulation. Here, we apply two scenario discovery approaches for unraveling inequality patterns and their drivers, with an application to spatial inequality of farms profitability in the Vietnam Mekong Delta. First, we follow an established sequential approach where we begin with clustering the inequality patterns from the simulation results and next identify model input subspaces that best explain each cluster. Second, we propose a novel concurrent approach using Multivariate Regression Trees to simultaneously classify inequality patterns and identify their corresponding input subspaces. Both approaches have comparable output space separability performance. The concurrent approach yields significantly better input space separability, but this comes at the expense of having a larger number of subspaces, requiring analysts to make extra effort to distill policy-relevant insights.

Analysis of a Fast Control Allocation approach for nonlinear over-actuated systems

Autonomous Robots with multiple directional thrusters are normally over-actuated systems that require nonlinear control allocation methods to map the forces that drive the robot’s dynamics and act as virtual control variables to the actuators. This process demands computational efforts that, sometimes, are not available in small robotic platforms. The present paper introduces a new control allocation approach with fast convergence, high accuracy, and dealing with complex nonlinear problems, especially in embedded systems. The adopted approach divides the desired nonlinear system into coupled linear problems. For that purpose, the Real Actions (RAs) and Virtual Control Variables (VCVs) are broke in two or more sets each. While the RA subsets are designed to linearize the system according to different input subspaces, the VCV is designed to be partially coupled to overlap the output subspaces. This approach generates smaller linear systems with fast and robust convergence used sequentially to solve nonlinear allocation problems. This methodology is assessed in mathematical tutorial cases and over-actuated UAV simulations.

Granular Fuzzy Modeling Guided Through the Synergy of Granulating Output Space and Clustering Input Subspaces

As an augmentation of classic fuzzy models, granular fuzzy models (GFMs) have been applied to many fields being in rapport with experimental data, models, and users. However, most of the existing methods used to construct GFMs are based on the principle of optimal allocation of information granularity, which requires that a numeric model be provided in advance. In this paper, a straightforward and convincing modeling method is proposed to directly construct GFM on a basis of experimental data. The method first granulates the output space to form some interval information granules with distinct semantics and then uses them to partition the entire input space into a series of input subspaces. Subsequently, an initial GFM is emerged by using "If-Then" rules to relate with those interval information granules positioned in the output space and structures expressed in prototypes that are produced by clustering individual input subspaces. Further, the initial GFM is also refined by continuously migrating prototypes in individual input subspaces. The experimental studies using the synthetic dataset and several real-world datasets are reported. They offer a useful insight into the feasibility and effectiveness of the proposed modeling method and reveal the impact of parameters on the performance of the ensuing GFMs. An application example is also presented to exhibit the advantages of the resulting GFM.

An Improved Just-in-Time Learning Scheme for Online Fault Detection of Nonlinear Systems

Just-in-time learning (JITL) scheme has been employed as an efficient tool of online soft sensor. It requires current measured data with high accuracy. However, in real industrial environments, it is difficult to ensure that no disturbance is added to the measurement. To solve this problem, this article proposes an improved JITL scheme, which employs leverage calculation to trim the weight of variables and abate the affect of disturbances. On this basis, an online modeling and output prediction algorithm is further presented. The experiment on a typical nonlinear system shows the better robustness and accuracy of the presented algorithm in comparison with conventional JITL-based approaches. Moreover, an online fault detection strategy for nonlinear systems is proposed based on JITL and partial least squares (PLS), for the purpose of simplifying the parameter setting and reducing the computational load of conventional fault detection approaches for nonlinear systems. Four statistic indexes are designed, including conventional T <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and SPE for fault detection, T <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">h</sub> and T <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</sub> of two orthogonal decomposition input subspaces for fault birth subspace observation. A numerical nonlinear system and an industrial benchmark of Tennessee Eastman process are employed for fault detection experiments, testifying the effectiveness of the proposed strategy.

FUZZY INFERENCE SYSTEMS BASE ON POLYNOMIALCONSEQUENTS OF FUZZY RULES

Various fuzzy inference systems that operate on the basis of polynomial consequents of fuzzy rules. As well as inference methods for such systems, in particular, Takagi-Sugeno fuzzy inference systems, their differences from other popular fuzzy systems, such as Mamdani systems, etc., are considered. The attention is focused on the features of the functioning of such systems both in the construction of elementary fuzzy systems. The Systems for which the calculation of the general logical conclusion involves intermediate levels of logical inference with many hierarchically interconnected blocks of fuzzy rules. Fuzzy sets of type 2 are considered, the membership index of which is a fuzzy term of the first type. This allows you to take into account the secondary fuzziness of linguistic concepts in the design of intelligent systems based on fuzzy inference. Fuzzy systems of the second type based on Takagi-Sugeno systems and the iterative Karnik-Mendel algorithm are considered to obtain a logical conclusion for fuzzy systems with the interval membership functions of the second type in the antecedents of fuzzy rules. The proposed procedure for lowering the order of fuzzy rules for higher-order Takagi-Sugeno fuzzy systems is described and justified. A fuzzy inference method for higher-order fuzzy systems based on the partition of a set of input variables is proposed. It is proposed to build a separate block of fuzzy rules for each of the input subspaces in the presence of a common polynomial. Which is a higher-order consequent, that reduces the total number of fuzzy rules in blocks.

The “Art of Trellis Decoding” Is Fixed-Parameter Tractable

Given n subspaces of a finite-dimensional vector space over a fixed finite field $\mathbb F$, we wish to find a linear layout $V_1,V_2,\ldots,V_n$ of the subspaces such that $\dim((V_1+V_2+\cdots+V_i) \cap (V_{i+1}+\cdots+V_n))\le k$ for all i, such a linear layout is said to have width at most k. When restricted to 1-dimensional subspaces, this problem is equivalent to computing the trellis-width (or minimum trellis state-complexity) of a linear code in coding theory and computing the path-width of an $\mathbb F$-represented matroid in matroid theory. We present a fixed-parameter tractable algorithm to construct a linear layout of width at most k, if it exists, for input subspaces of a finite-dimensional vector space over $\mathbb F$. As corollaries, we obtain a fixed-parameter tractable algorithm to produce a path-decomposition of width at most k for an input $\mathbb F$-represented matroid of path-width at most k, and a fixed-parameter tractable algorithm to find a linear rank-decomposition of width at most k for an input graph of linear rank-width at most k. In both corollaries, no such algorithms were known previously. It was previously known that a fixed-parameter tractable algorithm exists for the decision version of the problem for matroid path-width, a theorem by Geelen, Gerards, and Whittle~(2002) implies that for each fixed finite field $\mathbb F$, there are finitely many forbidden $\mathbb F$-representable minors for the class of matroids of path-width at most k. An algorithm by Hlin\v{e}n\'y (2006) can detect a minor in an input $\mathbb F$-represented matroid of bounded branch-width. However, this indirect approach would not produce an actual path-decomposition. Our algorithm is the first one to construct such a path-decomposition and does not depend on the finiteness of forbidden minors.

Genetic Design of Granular-oriented Radial Basis Function Neural Network Based on Information Proximity

In this study, we introduce and discuss a concept of a granular-oriented radial basis function neural networks (GRBF NNs). In contrast to the typical architectures encountered in radial basis function neural networks(RBF NNs), our main objective is to develop a design strategy of GRBF NNs as follows : (a) The architecture of the network is fully reflective of the structure encountered in the training data which are granulated with the aid of clustering techniques. More specifically, the output space is granulated with use of K-Means clustering while the information granules in the multidimensional input space are formed by using a so-called context-based Fuzzy C-Means which takes into account the structure being already formed in the output space, (b) The innovative development facet of the network involves a dynamic reduction of dimensionality of the input space in which the information granules are formed in the subspace of the overall input space which is formed by selecting a suitable subset of input variables so that the this subspace retains the structure of the entire space. As this search is of combinatorial character, we use the technique of genetic optimization to determine the optimal input subspaces. A series of numeric studies exploiting some nonlinear process data and a dataset coming from the machine learning repository provide a detailed insight into the nature of the algorithm and its parameters as well as offer some comparative analysis.

Granular Neural Networks and Their Development Through Context-Based Clustering and Adjustable Dimensionality of Receptive Fields

In this study, we present a new architecture of a granular neural network and provide a comprehensive design methodology as well as elaborate on an algorithmic setup supporting its development. The proposed neural network relates to a broad category of radial basis function neural networks (RBFNNs) in the sense that its topology involves a collection of receptive fields. In contrast to the standard architectures encountered in RBFNNs, here we form individual receptive fields in subspaces of the original input space rather than in the entire input space. These subspaces could be different for different receptive fields. The architecture of the network is fully reflective of the structure encountered in the training data which are granulated with the aid of clustering techniques. More specifically, the output space is granulated with use of K-means clustering while the information granules in the multidimensional input space are formed by using the so-called context-based fuzzy C-means, which takes into account the structure being already formed in the output space. The innovative development facet of the network involves a dynamic reduction of dimensionality of the input space in which the information granules are formed in the subspace of the overall input space which is formed by selecting a suitable subset of input variables so that this subspace retains the structure of the entire space. As this search is of combinatorial character, we use the technique of genetic optimization [genetic algorithms (GAs), to be more specific] to determine the optimal input subspaces. A series of numeric studies exploiting synthetic data and data coming from the Machine Learning Repository, University of California at Irvine, provide a detailed insight into the nature of the algorithm and its parameters as well as offer some comparative analysis.

Input space-adaptive optimization for embedded-software synthesis

This paper presents a technique for exploiting input statistics for energy and performance optimization of embedded software. The proposed technique is based on the fact that the computational complexities of programs or subprograms are often highly dependent on the values assumed by input and intermediate program variables during execution. This observation is exploited in the proposed software synthesis technique by augmenting the program with optimized versions of one or more subprograms that are specialized to, and executed under, specific input subspaces. We propose a methodology for input space-adaptive software synthesis that consists of the following steps: 1) control and value profiling of the input program; 2) application of compiler transformations in a preprocessing step; 3) identification of subprograms and corresponding input subspaces that hold the highest potential for optimization; and 4) an iterative application of known compiler transformations to realize performance and energy savings. We propose novel metrics based on the entropies of program variables to characterize subprograms and input subspaces that hold significant potential for optimization. The chosen subprograms are optimized by translating the input subspaces into value constraints on their variables, and iteratively applying known compiler transformations (that were not applicable in the context of the original program). We have evaluated input space-adaptive software synthesis by compiling the resulting optimized programs to two commercial embedded systems: an embedded system based on the Fujitsu SPARClite processor, and the Compaq iPAQ personal digital assistant (PDA) [64 MB memory, 206 MHz Intel StrongARM central processing unit (CPU)]. The energy and execution-time savings were calculated using energy-aware instruction-level simulators, as well as through direct-current measurement on the iPAQ. Our results demonstrate that the proposed technique can reduce energy by up to 54.5% (average of 30.6% and 25.6% for the SPARClite-based system and the iPAQ, respectively) while simultaneously improving performance by up to 59.6% (average of 31.3% and 31.5% for the SPARClite-based system and the iPAQ, respectively). In effect, improvements in the energy-delay product of up to 81.1% (average of 51.0% and 47.7% for the SPARClite-based system and the iPAQ, respectively) were observed. The energy savings resulting from our technique are fairly processor independent, and complementary to conventional compiler optimizations.

Input space adaptive design: a high-level methodology for optimizing energy and performance

This paper presents a high-level design methodology, called input space adaptive design, and new design automation algorithms for optimizing energy consumption and performance. Our techniques can be applied to behaviors described in hardware description languages, predesigned register-transfer level (RTL) circuits, or in the context of traditional high-level design methodologies. An input space adaptive design exploits the well-known fact that the quality of circuits can be significantly optimized by employing algorithms and implementation architectures that adapt to input statistics. This paper shows that harnessing the principles of input space adaptive design into a structured high-level design methodology can lead to large improvements in performance and energy consumption. We illustrate the tradeoffs involved in such designs, and demonstrate the need for a systematic design methodology in order to realize the full potential for performance and energy improvements. We propose a methodology for input space adaptive design that consists of the following steps: identification of parts of the behavior that hold the highest potential for optimization, selection of input subspaces whose occurrence can lead to significant reductions in implementation complexity, and transformation of the behavior to realize performance and/or energy savings. Evaluations of performance, energy, and area characteristics of input space adaptive designs in the context of a commercial high-level design flow indicate that such designs can reduce energy consumption by up to 58.9% (average of 40.0%), and simultaneously improve performance by up to 57.5% (average of 41.8%) compared to well-optimized designs that do not employ such techniques. The energy-delay product is reduced by up to 77.9% (average of 64.8%). When the performance improvements are translated into additional energy savings through supply voltage reduction, input space adaptive designs consume up to 74.2% (average of 68.8%) less energy at the same performance. The average area overhead is only 7.4%.

On the ill-conditioning of subspace identification with inputs

There is experimental evidence that the performance of standard subspace algorithms from the literature (e.g. the N4SID method) may be surprisingly poor in certain experimental conditions. This happens typically when the past signals (past inputs and outputs) and future input spaces are nearly parallel. In this paper we argue that the poor behavior may be attributed to a form of ill-conditioning of the underlying multiple regression problem, which may occur for nearly parallel regressors. An elementary error analysis of the subspace identification problem, shows that there are two main possible causes of ill-conditioning. The first has to do with near collinearity of the state and future input subspaces. The second has to do with the dynamical structure of the input signal and may roughly be attributed to “lack of excitation”. Stochastic realization theory constitutes a natural setting for analyzing subspace identification methods. In this setting, we undertake a comparative study of three widely used subspace methods (N4SID, Robust N4SID and PO-MOESP). The last two methods are proven to be essentially equivalent and the relative accuracy, regarding the estimation of the ( A, C) parameters, is shown to be the same.

Numerical conditioning and asymptotic variance of subspace estimates

New formulas for the asymptotic variance of the parameter estimates in subspace identification, show that the accuracy of the parameter estimates depends on certain indices of ‘near collinearity’ of the state and future input subspaces of the system to be identified. This complements the numerical conditioning analysis of subspace methods presented in the companion paper (On the ill-conditioning of subspace identification with inputs, Automatica, doi: 10.1016/j.automatica.2003.11.009).

Stacking Diverse Models to Achieve Reliable Error Response Distributions

Artificial Neural Networks (ANNs) can be useful for modeling real- world processes such as time series weather, financial, or chaotic data. The generalization and robustness of these models can be improved, and estimates of the modeling error distributions can be made, using a technique called Stacked Generalization (SG). SG uses a number of diverse models, each of which is trained and queried on independent cross validation subsets of the process data. The models are then combined in the stacking process to provide error estimates and improved accuracy. These improvements depend on the individual model response diversity between networks. Modified Series Association (MSA), an extension to SG, presents the various models with different input subspaces from the raw data as a catalyst for increased diversity. Model diversity is formulated, and an alternative model combination approach is derived from it, called Diversified Committee Machines (DCM). A framework for quantifying error estimation reliability is...