Optimal Weight Matrix Research Articles

The Partially-Matched-Sample Correction in Pseudo Panel Minimum Distance Estimation

Abstract Certain repeated cross-sectional data sets, such as the Current Population Survey (CPS), use special sampling designs by which samples from different times periods are partially-matched. This paper proposes a correction to the optimal weighting matrix in minimum distance (MD) estimation of pseudo panel models to account for such partially-matched samples. This partially-matched-sample correction may be needed if the sample matching rate is nontrivial and, at the same time, there is a fixed effect, a serially correlated idiosyncratic error, or both in the underlying linear panel data model data generating process, all of which lead to a block diagonal structure of the optimal weighting matrix. Using the correction can result in considerable efficiency gains both in finite sample and asymptotically. Furthermore, it is shown that this correction is needed not only for the optimal MD estimator but for any MD estimator to make the inference right. As an illustration, the correction is applied to the classical question of estimating the monetary return to education using the yearly Merged Outgoing Rotation Group (MORG) files from CPS.

Enhancing quantitative 1H NMR model generalizability on honey from different years through partial least squares subspace and optimal transport based unsupervised domain adaptation

Honey is a nourishing and natural food product that is widely favored by a diverse group of consumers. Proton Nuclear Magnetic Resonance (1H NMR) is a powerful tool for quantitative analysis of honey and plays a crucial role in ensuring its quality. The 1H NMR technique necessitates the utilization of multivariate calibration models to facilitate the quantitative analysis of key compounds present in honey. However, maintaining consistent measurement conditions across different years is scarcely possible, which can significantly impact the distribution of training and test spectra, ultimately leading to reduced performance of predictive models. Unsupervised domain adaptation (UDA) methods have gained considerable attention for their ability to match distribution differences between the labeled source spectra and the unlabeled target spectra without costly annotation. To enhance the quantitative model generalizability on honey from different years, we propose a UDA method known as partial least squares subspace and optimal transport-based UDA (PLSS-OT-UDA). This approach eliminates distribution differences between the source subspace and target subspace via partial least squares (PLS) dimensionality reduction and OT. Firstly, the optimal latent variable weight matrix from the source domain (i.e., labeled 1H NMR data in 2017) is extracted with PLS. Next, the dimension of both source and target domains (i.e., unlabeled 1H NMR data in 2018) is reduced and their corresponding subspaces are obtained with weight matrix of the source domain. Finally, OT is then employed to align the distribution of the source and target domains within the subspace. Experimental results on the honey dataset demonstrate that the PLSS-OT-UDA outperforms traditional methods, including transfer component analysis (TCA), optimal transport for domain adaptation (OTDA), domain adaptation based on principal component analysis and optimal transport (PCA-OTDA), and subspace alignment (SA), with respect to generalization performance on three components: baume degree, sugar content, and water content.

Comparison of GMM and MD estimators for a panel regression model with endogenous regressors and an AR(1) disturbance

ABSTRACT In this paper, we investigate the asymptotic properties of nonlinear GMM and linear MD estimators for a panel regression model with endogenous regressors and an AR(1) disturbance. We demonstrate that the linear MD estimator has a smaller asymptotic variance than the nonlinear GMM estimator when a non-optimal weighting matrix is used. However, when the optimal weighting matrix is used, both the nonlinear GMM and linear MD estimators have the same asymptotic distribution. The simulation results indicate that the linear two-step MD estimator is a practical choice due to its performance and simplicity.

Inference in Coarsened Time Series via Generalized Method of Moments

We study statistical inference procedures in coarsened time series through the generalized method of moments. A new model for the coarsened time series via multiple potential outcomes is proposed. It can be naturally extended for inferring multi‐variate coarsened time series. We show that this framework generates a general class of estimators. It neatly generalizes the classical Horvitz–Thompson estimator for handling coarsened time series data. Asymptotic properties, including consistency and limiting distribution, of the proposed estimators are investigated. Estimators of the optimal weight matrix and the long‐run covariance matrix are also derived. In particular, confidence intervals of the mean function of the potential outcome as a function of coarsening index can be constructed. A real‐data application on air quality in the USA is investigated.

A Weighted Least-Squares Method for Non-Asymptotic Identification of Markov Parameters from Multiple Trajectories

Markov parameters play a key role in system identification. There exists many algorithms where these parameters are estimated using least-squares in a first, pre-processing, step, including subspace identification and multi-step least-squares algorithms, such as Weighted Null-Space Fitting. Recently, there has been an increasing interest in non-asymptotic analysis of estimation algorithms. In this contribution we identify the Markov parameters using weighted least-squares and present non-asymptotic analysis for such estimator. To cover both stable and unstable systems, multiple trajectories are collected. We show that with the optimal weighting matrix, weighted least-squares gives a tighter error bound than ordinary least-squares for the case of non-uniformly distributed measurement errors. Moreover, as the optimal weighting matrix depends on the system’s true parameters, we introduce two methods to consistently estimate the optimal weighting matrix, where the convergence rate of these estimates is also provided. Numerical experiments demonstrate improvements of weighted least-squares over ordinary least-squares in finite sample settings.

Weighted Least-Squares PARSIM

Subspace identification methods (SIMs) have proven very powerful for estimating linear state-space models. To overcome the deficiencies of classical SIMs, a significant number of algorithms has appeared over the last two decades, where most of them involve a common intermediate step, that is to estimate the range space of the extended observability matrix. In this contribution, an optimized version of the parallel and parsimonious SIM (PARSIM), PARSIMopt, is proposed by using weighted least-squares. It not only inherits all the benefits of PARSIM but also attains the best linear unbiased estimator for the above intermediate step. Furthermore, inspired by SIMs based on the predictor form, consistent estimates of the optimal weighting matrix for weighted least-squares are derived. Essential similarities, differences and simulated comparisons of some key SIMs related to our method are also presented.

Machine-learning detection of the Berezinskii-Kosterlitz-Thouless transition and the second-order phase transition in XXZ models

We propose two machine-learning methods based on neural networks, which we respectively call the phase-classification method and the temperature-identification method, for detecting different types of phase transitions in the XXZ models without prior knowledge of their critical temperatures. The XXZ models have exchange couplings which are anisotropic in the spin space where the strength is represented by a parameter $\Delta(>0)$. The models exhibit the second-order phase transition when $\Delta>1$, whereas the Berezinskii-Kosterlitz-Thouless (BKT) phase transition when $\Delta<1$. In the phase-classification method, the neural network is trained using spin or vortex configurations of well-known classical spin models other than the XXZ models, e.g., the Ising models and the XY models, to classify those of the XXZ models to corresponding phases. We demonstrate that the trained neural network successfully detects the phase transitions for both $\Delta>1$ and $\Delta<1$, and the evaluated critical temperatures coincide well with those evaluated by conventional numerical calculations. In the temperature-identification method, on the other hand, the neural network is trained so as to identify temperatures at which the input spin or vortex configurations are generated by the Monte Carlo thermalization. The critical temperatures are evaluated by analyzing the optimized weight matrix, which coincide with the result of numerical calculation for the second-order phase transition in the Ising-like XXZ model with $\Delta=1.05$ but cannot be determined uniquely for the BKT transition in the XY-like XXZ model with $\Delta=0.95$.

Regularized fingerprinting in detection and attribution of climate change with weight matrix optimizing the efficiency in scaling factor estimation

Detection and attribution analyses play a central role in establishing the causal effect of human activities on global warming. The most commonly used method in such analyses, optimal fingerprinting, is a multiple regression where each covariate has a measurement error whose covariance matrix is the same as that of the regression error up to a known scale. Inferences about the regression coefficients are critical not only for making statements about detection and attribution but also for quantifying the uncertainty in important outcomes derived from detection and attribution analyses. When there are no errors-in-variables (EIV), the optimal weight matrix in estimating the regression coefficients is the precision matrix of the regression error. This matrix, however, is never known and has to be estimated from climate model simulations with regularization. The consequence is that the optimal fingerprinting method is not optimal, as believed in practice. We construct a weight matrix by inverting a nonlinear shrinkage estimate of the error covariance matrix that minimizes loss functions directly targeting the uncertainty of the resulting regression coefficient estimator. The resulting estimator of the regression coefficients is asymptotically optimal as the sample size of the climate model simulations and the matrix dimension go to infinity together with a limiting ratio. When EIVs are present, the estimator of the regression coefficients, based on the proposed weight matrix, is asymptotically more efficient than that based on the inverse of the existing linear shrinkage estimator of the error covariance matrix. The performance of the method is confirmed in finite sample simulation studies mimicking realistic situations in terms of the length of the confidence intervals and empirical coverage rates for the regression coefficients. In an application to detection and attribution analyses of the mean temperature at different spatial scales, the method yielded shorter confidence intervals which are important for such analyses in practice.

Tractable Calculation and Estimation of the Optimal Weighting Matrix for ALS Problems

We study autocovariance least squares (ALS) estimation methods for covariance estimation for linear time-invariant systems. Previous works have posited that calculation of the ALS weighting matrix is intractable unless the number of data points <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N_d$</tex-math></inline-formula> is small because it requires storage of a matrix whose number of elements scales as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N_d^4$</tex-math></inline-formula> . We derive a novel way to compute the weight that avoids this difficulty. In practice, the true optimal weight cannot be calculated because it is a function of the sought covariance matrices. However, our work enables implementation of two novel ALS algorithms that estimate the weight from data. For the purpose of comparison, we also discuss ALS with an arbitrary weight (such as an identity matrix) and present a previously published method for estimating the ALS weight. ALS with an identity weight guarantees unbiased and consistent covariance estimates, but algorithms that estimate the weight from data do not inherit these guarantees. Despite this drawback, we present a numerical example for which the best performing algorithm, iterative estimation of the covariances and the ALS weight, produces covariance estimates with a small amount of bias and a significantly reduced variance compared to all other algorithms.

Optimization of inelastic multistory structures under seismic vibrations using shape-memory-alloy material

This paper develops a novel optimization methodology for designing Shape-memory-alloy resisting devices (SMARDs) and optimally allocating them to inelastic multistory structures. The solution algorithm is a control gains optimization procedure that refers to a formal optimization problem with an objective function subject to the state-space equation and design limitations. The objective function integrates the squared state components in time, and the state-space equation consists of a newly introduced state vector form that reflects the system's inelasticity. The control gains are the number of total Shape-memory-alloy (SMA) wires attached to the devices in each story, and the design limitations dictate the minimum/maximum number of wires. The solution algorithm consists of five iterative steps that employ the defined Hamiltonian gradients in state and gains and cater to the necessary optimality conditions. The numerical example deals with upgrading an eight-story shear-type frame system. It studies the algorithm efficiency and elaborates on the effect of the optimal weighting matrix by investigating three different configurations. In all cases, the algorithm improves the system's inelastic seismic response—showcasing the reliability of the developed design methodology and the utilization of SMA material.

Modifying the linear two-step Windmeijer correction for the presence of spatial error dependence

The aim in the paper is to show how the presence of spatial dependence affects the often-adopted Windmeijer (J Econom 126:25–51, 2005) finite sample correction (For example it is an option facilitating robust estimation in the software package Stata, which is used by many applied econometricians and data analysts.), which corrects the downward bias in estimated parameter standard errors. Windmeijer (2005) explains why, with numerous instruments, the estimated asymptotic standard errors of the efficient, two-step, GMM estimator are downward biased in small samples. GMM estimation is based on an estimated optimal weight matrix, which is the inverse of the covariance of the sample moments, and the bias results from the weight matrix being evaluated at estimated, rather than the true values of parameters. Hwang et al. (J Econom 229(2):276–298, 2022) provide a correction to the Windmeijer (2005) finite sample correction to allow for over-identification bias. The novel contribution of the current paper is to show how the Windmeijer (2005) correction can be modified given spatial dependence in the error term of a model with moments conditions that are linear in parameters estimated by GMM, leading to corrected standard errors and therefore more accurate inference. Monte Carlo simulations are used to demonstrate the effect of the modification and two examples using real data shows how inference may be affected by ignoring the effect of spatial error dependence.

Robust Sparse Weighted Classification For Crowdsourcing

Data collected from nature is usually unlabeled, and it is difficult to be used directly. This issue is well addressed by crowdsourcing, which provides a reasonable way for effectively using these unlabeled data. Generally, workers in crowdsourcing tasks are not professionals, so it is hard to obtain high-quality labels. To address this issue, a robust sparse weighted classification algorithm is proposed, which try to adjust the samples that are not correctly classified in the original lables as much as possible. Specifically, we evalute the ability of different workers(indicator weight matrix) to accurately label different samples by fitting the real data matrix to its weighted reconstruction matrix. And then, <inline-formula><tex-math notation="LaTeX">$ l_{2,1}$</tex-math></inline-formula>-norm and worker labeling ability similarity matrix are added, and negative effects of some bad workers are eliminated through the row sparsity property of <inline-formula><tex-math notation="LaTeX">$ l_{2,1}$</tex-math></inline-formula>-norm. Finally, the optimal indicator weight matrix is obtained by optimizing the two matrices in the objective function simultaneously. Therefore, the obtained optimal indicator weight matrix takes the similarity of worker labeling ability into consideration, and infers all the predicted labels. The results on synthetic and real data sets demonstrate that our algorithm is superior to other state-of-the-art methods.

Multi-Mode Active Suspension Control Based on a Genetic K-Means Clustering Linear Quadratic Algorithm

The traditional Linear quadratic regulator (LQR) control algorithm depends too much on expert experience during the selection of weighting coefficients. To solve this problem, we proposed a Genetic K-means clustering Linear quadratic (GKL) algorithm. Firstly, a 2-DOF 1/4 vehicle model and road input model are established. The weights of an LQR controller are optimized using a genetic algorithm. Then, a possible weighting space is constructed based on this optimal solution. Random weighting coefficients of each performance index are generated in this space. Next, LQR control for the 1/4 vehicle model is performed, and the simulation data are recorded automatically, with these random weighting values, different road classes, and driving speed. A machine learning dataset is built from these simulations. Finally, a K-means clustering algorithm is used to classify the LQR control active suspension into three performance modes: safety mode, comprehensive mode, and comfort mode. The optimal weighting matrix of each performance mode is determined to satisfy requirements for different types of drivers. The results show that the new GKL algorithm not only improves the suspension control effect but also realizes different performance modes. It can better adapt to the changes in driving conditions and drivers.

Utilizing the Particle Swarm Optimization Algorithm for Determining Control Parameters for Civil Structures Subject to Seismic Excitation

Structural control of civil infrastructure in response to large external loads, such as earthquakes or wind, is not widely employed due to challenges regarding information exchange and the inherent latencies in the system due to complex computations related to the control algorithm. This study employs front-end signal processing at the sensing node to alleviate computations at the control node and results in a simplistic sum of weighted inputs to determine a control force. The control law simplifies to U = WP, where U is the control force, W is a pre-determined weight matrix, and P is a deconstructed representation of the response of the structure to the applied excitation. Determining the optimal weight matrix for this calculation is non-trivial and this study uses the particle swarm optimization (PSO) algorithm with a modified homing feature to converge on a possible solution. To further streamline the control algorithm, various pruning techniques are combined with the PSO algorithm in order to optimize the number of entries in the weight matrix. These optimization techniques are applied in simulation to a five-story structure and the success of the resulting control parameters are quantified based on their ability to minimize the information exchange while maintaining control effectiveness. It is found that a magnitude-based pruning method, when paired with the PSO algorithm, is able to offer the most effective control for a structure subject to seismic base excitation.

A Text Category Detection and Information Extraction Algorithm with Deep Learning

In order to solve the problem that the text classification model based on neural network is easy to over-fit and ignore the key words in sentences in the training process, a Bi-GRU Chinese text classification model based on hierarchical Attention mechanism is proposed. The model introduces the idea of layering, uses bi-directional gated cyclic neural network to learn the text representation at word level and sentence level, uses Self-Attention hierarchical model to obtain the information of the influence of words and sentences on text classification, shares the weight between embedded layer and softmax layer by binding, and uses AMSBound optimization method to obtain the optimal weight matrix quickly and effectively while reducing the parameters in the model. Two commonly used Chinese data sets, FudanSet and THUCNews, are tested on the long Chinese text classification data set FudanSet. The experimental results show that the accuracy, recall rate and F-score of this model are better than Text-CNN model, Attention-BiLSTM model and Bi-GRU_CNN model, and the accuracy, recall rate and F-score index are improved by 5.9%, 5.8% and 4.6%, respectively.

Robust estimation for Binomial conditionally nonlinear autoregressive time series based on multivariate conditional frequencies

A topical problem of robust statistical estimation of parameters for binomial conditionally nonlinear autoregressive (BiCNAR) time series under innovation outliers is considered. This problem is solved by means of s-order Markov properties for observed time series and probabilistic properties of multivariate conditional frequencies of the future state under its s-prehistory. The new robust statistical estimator ζˆ called frequencies-based estimator (FBE) is constructed for the BiCNAR parameters under innovation outliers with arbitrary discrete probability distribution having some fixed known expectation. Under mild regularity conditions the constructed FBE is shown to have the robustness properties: consistency and asymptotic normality with obtained asymptotic covariance matrix. FBE also has computational advantages: an explicit form and a fast recursive re-estimation algorithm for extension of the model. Asymptotic risk functional and its minimum are evaluated using Fisher information matrix for the considered model. Sensitivity analysis of the statistical estimator ζ̃ for the BiCNAR parameters, that is constructed for the hypothetical model without outliers, is carried out for the situation with innovation outliers: ζ̃ is shown to be inconsistent in this situation, its bias and the instability coefficient are evaluated and analyzed. The robust estimator ζˆ has a free parameter — weight matrix H. The optimal weight matrix H∗ is found by minimization of the asymptotic risk w.r.t. H. Statistical estimator for H∗ based on the observed time series is constructed. Results of multiple computer experiments on simulated and real data illustrate the theory.

Estimation of dynamic panel data models with a lot of heterogeneity

The commonly used 1-step and 2-step System GMM estimators for the panel AR(1) model are inconsistent under mean stationarity when the ratio of the variance of the individual effects to the variance of the idiosyncratic errors is unbounded when The reason for their inconsistency is that their weight matrices select moment conditions that do not identify the autoregressive parameter. This paper proposes a new 2-step System estimator that is still consistent in this case provided that Unlike the commonly used 2-step System estimator, the new estimator uses an estimator of the optimal weight matrix that remains consistent in this case. We also show that the commonly used 1-step and 2-step Arellano-Bond GMM estimators and the Random Effects Quasi MLE remain consistent under the same conditions. To illustrate the usefulness of our new System estimator we revisit the growth study of Levine et al. (2000).

Line loss prediction based on particle swarm optimization combined with extreme learning machine

As a key economic index to measure the planning and management level of power grid company, line loss is not only the main content of completing the construction and transformation task of power grid company, but also an important assessment standard for the operation level of power grid company. In order to improve the generalization ability and prediction accuracy of extreme learning machine (ELM), particle swarm optimization (PSO) algorithm is applied to extreme learning machine (ELM), and a line loss prediction method based on PSO combined with ELM is proposed. The particle swarm optimization algorithm is used to select the optimal hidden layer deviation and input weight matrix to calculate the output weight matrix, so as to improve the accuracy and stability of the extreme learning machine. Experimental results show that the proposed method can effectively improve the accuracy of line loss prediction.

Estimating and Testing for Smooth Structural Changes in Moment Condition Models

Numerous studies have been devoted to estimating and testing for moment condition models. Most of the existing studies assume that structural parameters are either fixed or changed abruptly over time. This study considers estimation of and testing for smooth structural changes in moment condition models where the data generating process is assumed to be locally stationary. A novel local generalized method of moment estimator and its boundary-corrected counterpart are proposed to estimate the smoothly changing structural parameters. Consistency and asymptotic normality are established, and an optimal weighting matrix and its consistent estimator are obtained. In particular, a consistent nonparametric test is proposed to check both smooth changes and abrupt breaks in structural parameters. The test is asymptotically pivotal and does not require prior information about the alternative. A Monte Carlo study is performed to illustrate the merits of the proposed test. In an empirical application, we document the time-varying features of the risk aversion parameter in an asset pricing model, which are consistent with business cycles and financial crisis.

Control of Cricket System Using LQR Controller Optimized by Particle Swarm Optimization

The cricket system, as a strongly coupled, nonlinear and multivariable two dimensional cue system, is a typical representative of unstable and underactuated system. In this paper, the system is analyzed by modeling based on the cricket experimental platform, and while designing the LQR controller, the particle swarm algorithm is introduced to optimize the controller parameters to determine the optimal weight matrix Q in order to solve the difficult problem of controller parameter rectification. The simulation results show that the PSO-LQR can restore the balance of the system in a shorter period of time with enhanced stability compared to a single LQR controller, which exhibits good control performance.