Order Selection Technique Research Articles

ATwo‐StageMethod for Order Selection inModel‐FreePredictive Control

One of the primary advantages of model‐free predictive control over conventional methods is that it does not use any mathematical models and relies only utilizes measured input/output data from the storage. The capability of model‐free predictive control has been already demonstrated in nonlinear systems using linear and polynomial regression for data storage. However, identifying the appropriate order that aligns with the actual system order remains a primary challenge, selecting an incorrect order may result in increasing redundant terms, ultimately leading to instability issues. In this study, we employed the Singular Value Decomposition (SVD) order selection technique, combined with the Bayesian Information Criterion (BIC), to identify the appropriate input and output orders of the system as well as the optimal horizon order in predictive control. This combined technique was subsequently applied to determine the appropriate order for model‐free predictive control. Our findings confirmed the effectiveness of the proposed method using numerical simulations in both linear and nonlinear systems. © 2024 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.

POLAR: Adaptive and Non-invasive Join Order Selection via Plans of Least Resistance

Join ordering and query optimization are crucial for query performance but remain challenging due to unknown or changing characteristics of query intermediates, especially for complex queries with many joins. Over the past two decades, a spectrum of techniques for adaptive query processing (AQP)---including inter-/intra-operator adaptivity and tuple routing---have been proposed to address these challenges. However, commercial database systems in practice do not implement holistic AQP techniques because they increase the system complexity (e.g., intertwined planning and execution) and thus, complicate debugging and testing. Additionally, existing approaches may incur large overheads, leading to problematic performance regressions. In this paper, we introduce POLAR, a simple yet very effective technique for a self-regulating selection of alternative join orderings with bounded overhead. We enhance left-deep join pipelines with alternative join orders, perform regret-bounded tuple routing to find and validate "plans of least resistance", and then process the majority of tuple batches through these plans. We study different join order selection techniques, different routing strategies, and a variety of workload characteristics. Our experiments with a POLAR prototype in DuckDB show runtime improvements of up to 9x and less than 7% overhead for all benchmark queries, while outperforming state-of-the-art AQP systems by up to 15x.

Model order selection for approximate Boolean matrix factorization problem

A key step in applying Boolean matrix factorization (BMF) is establishing the correct model order for the data, i.e., decide where the knowledge stops and the noise starts, or simply, decide the proper number of factors that describe the data well. There are two main approaches to BMF, namely, Discrete Basis Problem (DBP) and Approximation Factorization Problem (AFP). Although the model order selection technique for DBP exists, there is no technique tailored for AFP. We show that the number of factors for DBP cannot be used in AFP, and we present a novel way, reflecting the nature of AFP, how to establish the proper number of factors. Moreover, we show that the number of factors established for AFP is – from a knowledge-representation viewpoint – better than that for DBP.

A Data-Driven McMillan Degree Lower Bound

Given measurements of a linear time-invariant system, the McMillan degree is the dimension of the smallest such system that reproduces these observed dynamics. Using impulse response measurements where the system has been started in some (unknown) state and then allowed to evolve freely, a classical result by Ho and Kalman reveals the McMillan degree as the rank of a Hankel matrix built from these measurements. However, if measurements are contaminated by noise, this Hankel matrix will almost surely be full rank. Hence practitioners often estimate the rank of this matrix---and thus the McMillan degree---by manually setting a threshold between the large singular values that correspond to the non-zero singular values of the noise-free Hankel matrix and the small singular values that are pertubations of the zero singular values. Here we introduce a probabilistic upper bound on the perturbation of the singular values of this Hankel matrix when measurements are corrupted by additive Gaussian noise, and hence provide guidance on setting the threshold to obtain a lower bound on the McMillan degree. This result is powered by a new, probabilistic bound on the 2-norm of a random Hankel matrix with normally distributed entries. Unlike existing results for random Hankel matrices, this bound features no unknown constants and, moreover, is within a small factor of the empirically observed bound when entries are independent and identically distributed. This bound on the McMillan degree provides an inexpensive alternative to more general model order selection techniques such as the Akaike Information Criteria (AIC).

Interference Environment Model Recognition for Robust Adaptive Detection

The recognition of the interference environment for adaptive radar detection is addressed in this article. Typically, detectors are designed in one specific scenario which may not be appropriate for the varying interference environment, especially for the airborne and space-based radar system. In this article, the considered recognition task is cast in terms of multiple hypothesis tests and the theory of model order selection (MOS) techniques are exploited to devise suitable decision rules. The interference environments are divided into homogeneity, partial homogeneity, and spherically invariant random process. Three MOS techniques, namely, the Akaike information criterion (AIC), generalized information criterion, and corrected AIC, are adopted. At the analysis stage, illustrating examples for the influence of the environment model parameters on the recognition accuracy of the MOS rules are presented. Numerical experiments show the AIC rule has the most robust recognition performance.

High SNR Consistent Linear Model Order Selection and Subset Selection

High SNR consistency of model order selection criteria in linear regression models has attracted a lot of attention in the signal processing community recently. It is now known that Exponentially Embedded Family, versions of Minimum Description Length etc. are high SNR consistent. However, a general framework for high SNR consistency in linear regression is still missing. This paper fills this gap by deriving necessary and sufficient conditions (NSCs) for model order selection criteria in both known and unknown noise variance situations to be high SNR consistent. Many popular model order selection criteria are proved to be high SNR consistent using these NSCs. We also provide a convergence rate analysis to discriminate between various high SNR consistent model order selection criteria. A direct application of model order selection techniques to the very important subset selection problem is computationally infeasible. This paper establish the high SNR consistency of an existing t-statistics based index ordering scheme that allows the conversion of a subset selection problem into a model order selection problem. Combining this index ordering with high SNR consistent model order selection criteria leads to a novel subset selection procedure (SSP) that is numerically shown to outperform existing high SNR consistent SSPs.

New order selection technique using information criteria applied to SISO and MIMO systems predistortion

This paper presents a new order selection technique of matrix memory polynomial technique that models the nonlinearities of single-branch and multi-branch transmitters. The new criteria take into account the complexity of the model in addition to its mean-square error in the selection criteria. The quasi-convexity of the proposed criteria was proven in this work. By using this proposed Akaike information criterion (AIC) and Bayesian information criterion (BIC) criteria, the model order selection was cast as a cost minimization problem. To minimize the criteria, modified gradient descent and simulated annealing algorithms were utilized which resulted in a considerable reduction in the number of search iterations. The performances of the criteria were shown by comparing the normalized mean square error (NMSE) of a higher-order model and the optimum model. It has been shown that the NMSE difference is <0.5 dB, but the complexity is much smaller.

Multi-dimensional model order selection

Multi-dimensional model order selection (MOS) techniques achieve an improved accuracy, reliability, and robustness, since they consider all dimensions jointly during the estimation of parameters. Additionally, from fundamental identifiability results of multi-dimensional decompositions, it is known that the number of main components can be larger when compared to matrix-based decompositions. In this article, we show how to use tensor calculus to extend matrix-based MOS schemes and we also present our proposed multi-dimensional model order selection scheme based on the closed-form PARAFAC algorithm, which is only applicable to multi-dimensional data. In general, as shown by means of simulations, the Probability of correct Detection (PoD) of our proposed multi-dimensional MOS schemes is much better than the PoD of matrix-based schemes.

Estimating the Order of an Autoregressive Model Using Normalized Maximum Likelihood

This paper examines the estimation of the order of an autoregressive model using the minimum description length principle. A closed form for an approximation of the parametric complexity of the autoregressive model class is derived by exploiting a relationship between coefficients and partial autocorrelations. The parametric complexity over the complete parameter space is found to diverge. A model selection criterion is subsequently derived by bounding the parameter space, and simulations suggest that it compares well against standard autoregressive order selection techniques in terms of correct order identification and prediction error.

Recursive parametric tests for multichannel adaptive signal detection

The problem of detecting a multichannel signal in spatially and temporally coloured disturbances is considered. The parametric Rao and parametric generalised likelihood ratio test detectors, recently developed by modelling the disturbance as a multichannel autoregressive (AR) process, have been shown to perform well with limited or even no range training data. These parametric detectors, however, assume that the model order of the multichannel AR process is known a priori to the detector. In practice, the model order has to be estimated by some model order selection technique. Meanwhile, a standard non-recursive implementation of the parametric detectors is computationally intensive since the unknown parameters have to be estimated for all possible model orders before the best one is identified. To address these issues, herein the joint model order selection, parameter estimation and target detection are considered. We present recursive versions of the aforementioned parametric detectors by integrating the multichannel Levinson algorithm, which is employed for recursive and computationally efficient parameter estimation, with a generalised Akaike Information Criterion for model order selection. Numerical results show that the proposed recursive parametric detectors, assuming no knowledge of the model order, yield a detection performance nearly identical to that of their non-recursive counterparts at significantly reduced complexity.

Robust detection of gearbox deterioration using compromised autoregressive modeling and Kolmogorov–Smirnov test statistic—Part I: Compromised autoregressive modeling with the aid of hypothesis tests and simulation analysis

A novel technique for detection of gearbox deterioration is proposed in Part I of this study. The proposed technique makes use of a time-varying autoregressive (AR) model and establishes a compromised AR model based on healthy gear motion residual (GMR) signals under varying load conditions and employs the Kolmogorov–Smirnov (K–S) goodness-of-fit (GOF) test statistic as a measure of gear condition. The order of the time-varying AR model is selected using a novel model order selection technique with the aid of hypothesis tests. The principal criterion for the selection of AR model order requires that the normality of the AR model residuals of the non-stationary healthy GMR signals under varying load conditions can be guaranteed. In the case where such orders are available, the one that results in the statistically least variance of the gear condition indicator, i.e. the K–S test statistic, is selected with the aid of the Bartlett's test. In the case where, under all considered orders, the normality condition cannot be met for all non-stationary healthy GMR signals, the order that results in the least violation against the normality condition can be identified with the aid of the Satterthwaite's t′-test. The coefficients of the time-varying AR model are estimated by means of a noise-adaptive Kalman filter. Validation of the proposed technique is carried out by using two sets of simulated entire lifetime gear vibration signals, i.e. clean and contaminated signals, to simulate the cases of sufficient and insufficient removal of background noise, respectively. The simulated tests demonstrate that the proposed technique possesses appealing effectiveness in identifying the optimum AR model order for robust gear condition detection under varying load conditions. The optimum performance of this technique is further confirmed by examining alternative orders.

Distributed BOLD-response in association cortex vector state space predicts reaction time during selective attention

Human cortical information processing is thought to be dominated by distributed activity in vector state space (Churchland, P.S., Sejnowski, T.J., 1992. The Computational Brain. MIT Press, Cambridge.). In principle, it should be possible to quantify distributed brain activation with independent component analysis (ICA) through vector-based decomposition, i.e., through a separation of a mixture of sources. Using event-related functional magnetic resonance imaging (fMRI) during a selective attention-requiring task (visual oddball), we explored how the number of independent components within activated cortical areas is related to reaction time. Prior to ICA, the activated cortical areas were determined on the basis of a General linear model (GLM) voxel-by-voxel analysis of the target stimuli (checkerboard reversal). Two activated cortical areas (temporoparietal cortex, medial prefrontal cortex) were further investigated as these cortical regions are known to be the sites of simultaneously active electromagnetic generators which give rise to the compound event-related potential P300 during oddball task conditions. We found that the number of independent components more strongly predicted reaction time than the overall level of “activation” (GLM BOLD-response) in the left temporoparietal area whereas in the medial prefrontal cortex both ICA and GLM predicted reaction time equally well. Comparable correlations were not seen when principle components were used instead of independent components. These results indicate that the number of independently activated components, i.e., a high level of cortical activation complexity in cortical vector state space, may index particularly efficient information processing during selective attention-requiring tasks. To our best knowledge, this is the first report describing a potential relationship between neuronal generators of cognitive processes, the associated electrophysiological evidence for the existence of distributed networks and BOLD fMRI signals using information from model order selection techniques.

Enhancing the Performance of Gaussian Mixture Model-Based Text Independent Speaker Identification

In this paper, we seek to enhance the identification performance of Gaussian Mixture Model (GMM)-based speaker identification systems in the presence of a limited amount of training data and a relatively large number of speakers. The performance is characterized by the identification accuracy, the identification time, and the model complexity. A new model order selection technique based on the Goodness of Fit (GOF) statistical test is proposed in order to increase the identification accuracy. This technique has shown to outperform other well known model order selection techniques like the Minimum Description Length (MDL) and the Akaike Information Criterion (AIC) in terms of the identification accuracy and the robustness against telephone channel degradation effects. In addition, the identification time is decreased by adapting the Linear Discriminative Analysis (LDA) feature extraction technique to fit our basic assumption of asymmetric multimodal distribution of the training data of each speaker. This modification results in a large decrease in the identification time with a little effect on the identification accuracy.

Enhancing the Performance of Gaussian Mixture Model-Based Text Independent Speaker Identification

In this paper, we seek to enhance the identification performance of Gaussian Mixture Model (GMM)-based speaker identification systems in the presence of a limited amount of training data and a relatively large number of speakers. The performance is characterized by the identification accuracy, the identification time, and the model complexity. A new model order selection technique based on the Goodness of Fit (GOF) statistical test is proposed in order to increase the identification accuracy. This technique has shown to outperform other well known model order selection techniques like the Minimum Description Length (MDL) and the Akaike Information Criterion (AIC) in terms of the identification accuracy and the robustness against telephone channel degradation effects. In addition, the identification time is decreased by adapting the Linear Discriminative Analysis (LDA) feature extraction technique to fit our basic assumption of asymmetric multimodal distribution of the training data of each speaker. This modification results in a large decrease in the identification time with a little effect on the identification accuracy.

Optimal modeling of corneal surfaces with Zernike polynomials.

Zernike polynomials are often used as an expansion of corneal height data and for analysis of optical wavefronts. Accurate modeling of corneal surfaces with Zernike polynomials involves selecting the order of the polynomial expansion based on the measured data. We have compared the efficacy of various classical model order selection techniques that can be utilized for this purpose, and propose an approach based on the bootstrap. First, it is shown in simulations that the bootstrap method outperforms the classical model order selection techniques. Then, it is proved that the bootstrap technique is the most appropriate method in the context of fitting Zernike polynomials to corneal elevation data, allowing objective selection of the optimal number of Zernike terms. The process of optimal fitting of Zernike polynomials to corneal elevation data is discussed and examples are given for normal corneas and for abnormal corneas with significant distortion. The optimal model order varies as a function of the diameter of the cornea.

A Bootstrap Method for Narmax Model Order Selection

NARMAX (Nonlinear AutoRegressive, Moving Average exogenous) models describe nonlinear systems in terms of linear-in-the-parameters difference equations which represent the current output with present and past inputs and past outputs. Identification of NARMAX models requires determining both the model order and parameter values. Good parameter estimation methods exist if the model order is known, however, model order selection remains a problem. We propose a bootstrap- based model order selection algorithm for determining the order of NARMAX systems. The performance of this bootstrap model order selection technique was evaluated by applying it to the slow phase component of the vestibulo-ocular reflex system (Kukreja et ai 1999).

Time-resolved MR imaging by automatic data segmentation.

A method for time-resolved imaging that provides a flexible trade-off between imaging time and temporal resolution is presented. It is based on a view order selection technique that automatically segments the acquired raw data into appropriate temporal frames. When used with cardiac monitoring and phase-contrast imaging, data similar to that obtained with a conventional gated phase-contrast sequence are acquired rapidly. For many applications, the temporal resolution can be reduced enough to permit imaging within a breath-hold interval, while still allowing accurate time-averaged flow quantitation. This is a general technique that can be implemented within a variety of pulse sequences and can resolve other motion cycles, including the respiratory cycle.

The asymptotic average squared error for polynomial regression

An asymptotic upper bound is derived for the average expected squared error from polynomial regression. The bound is applied to determine guaranteed rates of convergence for estimation over certain function Classes, Two order selection techniques are shown to be optimal for selecting the number of terms to include in the estimator.