Latent Variable Selection Research Articles

The Improved EMS Algorithm for Latent Variable Selection in M3PL Model.

One of the main concerns in multidimensional item response theory (MIRT) is to detect the relationship between items and latent traits, which can be treated as a latent variable selection problem. An attractive method for latent variable selection in multidimensional 2-parameter logistic (M2PL) model is to minimize the observed Bayesian information criterion (BIC) by the expectation model selection (EMS) algorithm. The EMS algorithm extends the EM algorithm and allows the updates of the model (e.g., the loading structure in MIRT) in the iterations along with the parameters under the model. As an extension of the M2PL model, the multidimensional 3-parameter logistic (M3PL) model introduces an additional guessing parameter which makes the latent variable selection more challenging. In this paper, a well-designed EMS algorithm, named improved EMS (IEMS), is proposed to accurately and efficiently detect the underlying true loading structure in the M3PL model, which also works for the M2PL model. In simulation studies, we compare the IEMS algorithm with several state-of-art methods and the IEMS is of competitiveness in terms of model recovery, estimation precision, and computational efficiency. The IEMS algorithm is illustrated by its application to two real data sets.

Private parking space owners’ choice behavior of different sharing modes: hybrid choice model using justified latent variables

ABSTRACT Private parking spaces account for a large proportion of potential parking resources, whose utilization rate could be raised by sharing them. To investigate private parking space owners’ sharing choice behavior, this paper applies a framework of Combined Technology Acceptance Model and the Theory of Planned Behavior for a more comprehensive consideration of psychological attitudes on these suppliers in urban residential areas. A web-based survey was conducted to collect data. The justified factors are then incorporated in hybrid choice model (HCM) as latent variables. HCM results showed that incorporation of latent variables such as perceived risk provides better fitting effect than traditional multinominal models, and that revenues and community participance are influential factors for sharing choice. Our findings indicate behavioral intention framework could serve as argumentation of rational selection of latent variables. These findings could also support better implementation of shared parking from managerial and operational perspectives.

Comparing Sensor Fusion and Multimodal Chemometric Models for Monitoring U(VI) in Complex Environments Representative of Irradiated Nuclear Fuel.

Optical sensors and chemometric models were leveraged for the quantification of uranium(VI) (0-100 μg mL-1), europium (0-150 μg mL-1), samarium (0-250 μg mL-1), praseodymium (0-350 μg mL-1), neodymium (0-1000 μg mL-1), and HNO3 (2-4 M) with varying corrosion product (iron, nickel, and chromium) levels using laser fluorescence, Raman scattering, and ultraviolet-visible-near-infrared absorption spectra. In this paper, an efficient approach to developing and evaluating tens of thousands of partial least-squares regression (PLSR) models, built from fused optical spectra or multimodal acquisitions, is discussed. Each PLSR model was optimized with unique preprocessing combinations, and features were selected using genetic algorithm filters. The 7-factor D-optimal design training set contained just 55 samples to minimize the number of samples. The performance of PLSR models was evaluated by using an automated latent variable selection script. PLS1 regression models tailored to each species outperformed a global PLS2 model. PLS1 models built using fused spectra data and a multimodal (i.e., analyzed separately) approach yielded similar information, resulting in percent root-mean-square error of prediction values of 0.9-5.7% for the seven factors. The optical techniques and data processing strategies established in this study allow for the direct analysis of numerous species without measuring luminescence lifetimes or relying on a standard addition approach, making it optimal for near-real-time, in situ measurements. Nuclear reactor modeling helped bound training set conditions and identified elemental ratios of lanthanide fission products to characterize the burnup of irradiated nuclear fuel. Leveraging fluorescence, spectrophotometry, experimental design, and chemometrics can enable the remote quantification and characterization of complex systems with numerous species, monitor system performance, help identify the source of materials, and enable rapid high-throughput experiments in a variety of industrial processes and fundamental studies.

A generalized expectation model selection algorithm for latent variable selection in multidimensional item response theory models

A generalized expectation model selection algorithm for latent variable selection in multidimensional item response theory models

Accelerating L1-penalized expectation maximization algorithm for latent variable selection in multidimensional two-parameter logistic models.

One of the main concerns in multidimensional item response theory (MIRT) is to detect the relationship between observed items and latent traits, which is typically addressed by the exploratory analysis and factor rotation techniques. Recently, an EM-based L1-penalized log-likelihood method (EML1) is proposed as a vital alternative to factor rotation. Based on the observed test response data, EML1 can yield a sparse and interpretable estimate of the loading matrix. However, EML1 suffers from high computational burden. In this paper, we consider the coordinate descent algorithm to optimize a new weighted log-likelihood, and consequently propose an improved EML1 (IEML1) which is more than 30 times faster than EML1. The performance of IEML1 is evaluated through simulation studies and an application on a real data set related to the Eysenck Personality Questionnaire is used to demonstrate our methodologies.

Industrial process monitoring based on optimal active relative entropy components

The multivariate statistic method has been widely applied, but no clear mapping relationship exists between the latent variables and the fault information, which leads to various information in different latent variables. The traditional methods cause the loss of useful fault information and the decline of monitoring performance. In this paper, a process monitoring method based on the optimal active relative entropy component is proposed. Based on global and local preserving projection, relative entropy is selected as the monitoring statistic. The influence of latent variable selection on relative entropy calculation is analyzed, and a relative entropy activity index is proposed to select the relative entropy component beneficial to process monitoring. The parameters are optimized to generate the optimal active relative entropy component for process monitoring using the mayfly algorithm. Finally, a numerical case, Tennessee Eastman (TE) process, and three-phase flow facility are used to verify the superiority of the monitoring algorithm.

Latent variable sdelection in multidimensional item response theory models using the expectation model selection algorithm.

The aim of latent variable selection in multidimensional item response theory (MIRT) models is to identify latent traits probed by test items of a multidimensional test. In this paper the expectation model selection (EMS) algorithm proposed by Jiang etal. (2015) is applied to minimize the Bayesian information criterion (BIC) for latent variable selection in MIRT models with a known number of latent traits. Under mild assumptions, we prove the numerical convergence of the EMS algorithm for model selection by minimizing the BIC of observed data in the presence of missing data. For the identification of MIRT models, we assume that the variances of all latent traits are unity and each latent trait has an item that is only related to it. Under this identifiability assumption, the convergence of the EMS algorithm for latent variable selection in the multidimensional two-parameter logistic (M2PL) models can be verified. We give an efficient implementation of the EMS for the M2PL models. Simulation studies show that the EMS outperforms the EM-based L1 regularization in terms of correctly selected latent variables and computation time. The EMS algorithm is applied to a real data set related to the Eysenck Personality Questionnaire.

A Bayesian Latent Variable Selection Model for Nonignorable Missingness

Missing data are exceedingly common across a variety of disciplines, such as educational, social, and behavioral science areas. Missing not at random (MNAR) mechanism where missingness is related to unobserved data is widespread in real data and has detrimental consequence. However, the existing MNAR-based methods have potential problems such as leaving the data incomplete and failing to accommodate incomplete covariates with interactions, non-linear terms, and random slopes. We propose a Bayesian latent variable imputation approach to impute missing data due to MNAR (and other missingness mechanisms) and estimate the model of substantive interest simultaneously. In addition, even when the incomplete covariates involves interactions, non-linear terms, and random slopes, the proposed method can handle missingness appropriately. Computer simulation results suggested that the proposed Bayesian latent variable selection model (BLVSM) was quite effective when the outcome and/or covariates were MNAR. Except when the sample size was small, estimates from the proposed BLVSM tracked closely with those from the complete data analysis. With a small sample size, when the outcome was less predictable from the covariates, the missingness proportions of the covariates and the outcome were larger, and the missingness selection processes of the covariates and the outcome were more MNAR and MAR, the performance of BLVSM was less satisfactory. When the sample size was large, BLVSM always performed well. In contrast, the method with an MAR assumption provided biased estimates and undercoverage confidence intervals when the missingness was MNAR. The robustness and the implementation of BLVSM in real data were also illustrated. The proposed method is available in the Blimp software application, and the paper includes a data analysis example illustrating its use.

Achieving robustness across season, location and cultivar for a NIRS model for intact mango fruit dry matter content. II. Local PLS and nonlinear models

Abstract A range of modelling techniques were used in the estimation of dry matter content of intact mango fruit from short wave near infrared spectra, collected using an interactance geometry, with models developed on a data set collected across three seasons (n = 10,243) and tested on that of a fourth season (n = 1,448). Model types included Artificial Neural Network (ANN), Gaussian Process Regression (GPR), Local Optimized by Variance Regression (LOVR), Local Partial Least Squares Regression (LPLS), Local PLS Scores (LPLS-S) and Memory Based Learner (MBL), with manual tuning of parameters undertaken. Additionally, two commercially available cloud-based chemometric packages for automated model development were trialled. All of these models gave a better result than use of a global PLS model. The best result (lowest RMSEP) was achieved with an ensemble of ANN, GPR and LPLS-S, with the best individual model result achieved by LOVR, with RMSEP of 0.839 % and 0.881 %, respectively, compared to the global PLS result of 1.014 %. The best precision was achieved with the LPLS model, with a SEP of 0.846 %, compared to the global PLS result of 1.012 %. LOVR was twice as fast as a generalized latent variable selection method LPLS-S-cv in prediction of independent validation set (at 58.7 × 10−3 s compared to 163 x 10-3 s). The ANN model was satisfactory in all categories (prediction speed, model build speed, and prediction statistics) and insensitive to tuning, e.g., 33 of the 70 parameter combinations were within 0.05 units of RMSEP of the minimum combination. However, the ANN learning rate was low. For applications that require ‘real-time’ prediction, such as fruit packlines, use of ANN and GPR models is recommended. For non-cloud based handheld NIR devices lacking the computational power to perform local modelling, ANN is recommended, and LOVR or a model ensemble recommended in cloud based implementation. The automated cloud-based systems performed well (RMSEP of 0.850 % and 0.963 % for Hone Create and DataRobot, ensemble models, respectively), without human intervention for the choosing and tuning of models.

Learning Attribute Hierarchies From Data: Two Exploratory Approaches

In cognitive diagnostic assessment, multiple fine-grained attributes are measured simultaneously. Attribute hierarchies are considered important structural features of cognitive diagnostic models (CDMs) that provide useful information about the nature of attributes. Templin and Bradshaw first introduced a hierarchical diagnostic classification model (HDCM) that directly takes into account attribute hierarchies, and hence, HDCM is nested within more general CDMs. They also formulated an empirically driven hypothesis test to statistically test one hypothesized link (between two attributes) at a time. However, their likelihood ratio test statistic does not have a known reference distribution, so it is cumbersome to perform hypothesis testing at scale. In this article, we studied two exploratory approaches that could learn the attribute hierarchies directly from data, namely, the latent variable selection (LVS) approach and the regularized latent class modeling (RLCM) approach. An identification constraint was proposed for the LVS approach. Simulation results revealed that both approaches could successfully identify different types of attribute hierarchies, when the underlying CDM is either the deterministic input noisy and gate model or the saturated log-linear CDM. The LVS approach outperformed the RLCM approach, especially when the total number of attributes increases.

Adaptive Selection of Latent Variables for Process Monitoring

Latent-variable (LV) methods have been widely applied to multivariate statistical process monitoring. Using LV methods can simplify the process-monitoring problem through a projection of the high-dimensional process data into a low-dimensional LV space that retains most of the important information for fault detection. A key issue is the optimal selection of a subset of LVs to constitute such a low-dimensional LV space. Traditional LV-selection methods do not choose LVs from a fault-detection point of view, which may result in poor process-monitoring performance. To overcome this drawback, a cumulative-percent-contribution (CPC) criterion is proposed to select appropriate LVs for process monitoring. First, contributions of LVs to the T2 value of a sample are computed by the decomposition of the T2 statistic. The importance of LVs to fault detection is then evaluated by their contributions. The larger the contribution is, the more important the LV is. After sorting LVs in order of decreasing contributions,...

Empirical study on the evaluation model of public satisfaction with typhoon disaster information disclosure

PurposeThe purpose of this paper is to study major influencing factors on public satisfaction with regard to information disclosure amid typhoon disasters. Internal latent variables, including disaster information quality, information disclosure channels and disaster perception levels, significantly affected the degree of public satisfaction, which was key factors in determining account consumer satisfaction.Design/methodology/approachBy conducting the questionnaire survey in typhoon-prone areas, 326 copies of the questionnaire about public satisfaction towards government information disclosure about typhoon were collected. An exponential model of satisfaction about information disclosure amid typhoon disasters was constructed and analyzed through the selection of latent variables, the hypotheses of the structural relationship and the establishment of measurable variables.FindingsDisaster information quality, disaster information disclosure channels and disaster perception levels were all positively correlated with the dimensions of information performance perception dimensions. Public expectations also made positive contributions to public satisfaction. These findings strongly support the hypothesis that public satisfaction affects the government image and public trust.Originality/valueWith the Leizhou Peninsula, China attacked by the typhoon disasters as an example, the SEM was adopted to simulate the routes and its feasibility and scientificity were validated. It is valuable to build and analyze an evaluation model based on a structural equation model underlying the public satisfaction towards information disclosure amid typhoon disasters.

Latent Variable Selection for Multidimensional Item Response Theory Models via [Formula: see text] Regularization.

We develop a latent variable selection method for multidimensional item response theory models. The proposed method identifies latent traits probed by items of a multidimensional test. Its basic strategy is to impose an [Formula: see text] penalty term to the log-likelihood. The computation is carried out by the expectation-maximization algorithm combined with the coordinate descent algorithm. Simulation studies show that the resulting estimator provides an effective way in correctly identifying the latent structures. The method is applied to a real dataset involving the Eysenck Personality Questionnaire.

Latent variable selection in structural equation models

Structural equation models (SEMs) are often formulated using a prespecified parametric structural equation. In many applications, however, the formulation of the structural equation is unknown, and its misspecification may lead to unreliable statistical inference. This paper develops a general SEM in which latent variables are linearly regressed on themselves, thereby avoiding the need to specify outcome/explanatory latent variables. A penalized likelihood method with a proper penalty function is proposed to simultaneously select latent variables and estimate the coefficient matrix in formulating the structural equation. Under some regularity conditions, we show the consistency and the oracle property of the proposed estimators. We also develop an expectation/conditional maximization (ECM) algorithm involving a minorization–maximization algorithm that facilitates the second M-step. Simulation studies are performed and a real data set is analyzed to illustrate the proposed methods. • Propose a PL method to identify the structure of coefficient matrix. • Show the consistency and the oracle property of the proposed estimators. • Propose a minorization–maximization algorithm to facilitate the second M-step.

Multiple SNP Set Analysis for Genome-Wide Association Studies Through Bayesian Latent Variable Selection.

The power of genome-wide association studies (GWAS) for mapping complex traits with single-SNP analysis (where SNP is single-nucleotide polymorphism) may be undermined by modest SNP effect sizes, unobserved causal SNPs, correlation among adjacent SNPs, and SNP-SNP interactions. Alternative approaches for testing the association between a single SNP set and individual phenotypes have been shown to be promising for improving the power of GWAS. We propose a Bayesian latent variable selection (BLVS) method to simultaneously model the joint association mapping between a large number of SNP sets and complex traits. Compared with single SNP set analysis, such joint association mapping not only accounts for the correlation among SNP sets but also is capable of detecting causal SNP sets that are marginally uncorrelated with traits. The spike-and-slab prior assigned to the effects of SNP sets can greatly reduce the dimension of effective SNP sets, while speeding up computation. An efficient Markov chain Monte Carlo algorithm is developed. Simulations demonstrate that BLVS outperforms several competing variable selection methods in some important scenarios.

A Review of Genetic Algorithms in near Infrared Spectroscopy and Chemometrics: Past and Future

Global optimisation and search problems are abundant in science and engineering, including spectroscopy and its applications. Therefore, it is hardly surprising that general optimisation and search methods such as genetic algorithms (GAs) have also found applications in the area of near infrared (NIR) spectroscopy. A brief introduction to genetic algorithms, their objectives and applications in NIR spectroscopy, as well as in chemometrics, is given. The most popular application for GAs in NIR spectroscopy is wavelength, or more generally speaking, variable selection. GAs are both frequently used and convenient in multi-criteria optimisation; for example, selection of pre-processing methods, wavelength inclusion, and selection of latent variables can be optimised simultaneously. Wavelet transform has recently been applied to pre-processing of NIR data. In particular, hybrid methods of wavelets and genetic algorithms have in a number of research papers been applied to pre-processing, wavelength selection and regression with good success. In all calibrations and, in particular, when optimising, it is essential to validate the model and to avoid over-fitting. GAs have a large potential when addressing these two major problems and we believe that many future applications will emerge. To conclude, optimisation gives good opportunities to simultaneously develop an accurate calibration model and to regulate model complexity and prediction ability within a considered validation framework.

Statistical Significance Testing as a Guide to Partial Least-Squares (PLS) Modeling of Nonideal Data Sets for Fuel Property Predictions

Partial least-squares (PLS) was used to formulate property models for a set of 43 jet fuel samples using near-infrared (NIR), Raman, and gas chromatography (GC) data. A total of 28 different properties were evaluated for each technique. Given that the data set was small and several of the property distributions were nonideal, significance testing was used for model formulation and evaluation. A statistical F test was applied for selection of latent variables, determining the significance level of the model compared to random chance and evaluating the increase in prediction error when the expected sources of error were fully incorporated. A chart was used to categorize the confidence in the modeling ability as high, low, or indeterminate with the given data set.

The Causal Effects of Participation in the American Economic Association Summer Minority Program

1. IntroductionIn 1974, the American Economic Association sponsored a Summer Minority Program (hereafter AEASMP). An impetus for the AEASMP was the underrepresentation of racial minorities among the ranks of Ph.D. economists, in addition to the possible shortfalls in academic preparation that, if not addressed, would reduce the likelihood of minorities successfully completing doctoral programs in economics. Thus, from the start and continuing up to the present, the AEASMP provides participants with a curriculum that introduces, and perhaps remediates, core quantitative skills and, in recent years, research skills, essential for completing doctoral programs in economics.1Given the severe underrepresentation of black Ph.D. economists in the profession, as an intervention, it is conceivable that the AEASMP could have an effect of increasing the likelihood of black participants earning economics doctorates and securing employment as economics faculty.2 This seems especially plausible if the AEASMP enhances the capabilities of participants and/or cultivates a serious interest in becoming a Ph.D. economist. Notwithstanding such possibilities, nothing is known on the extent to which the AEASMP has causal effects.As an intervention, it is of interest to know if the AEASMP has causal effects on outcomes deemed favorable for minority economists. Many outcomes associated with the training of Ph.D. economists are those outcomes associated with placement in tenure-track faculty positions in particular types of academic institutions. Successfully obtaining tenure is also associated with factors such as scholarly productivity, success in securing research resources, and in the prestige associated with scholarly society affiliation. If indeed the AEASMP has causal effects that enhance participants' labor market prospects as academic economists, the AEASMP can be viewed as a successful labor-market intervention.This article examines the causal effects of AEASMP participation for minority Ph.D. economists. Methodologically, the treatment effect is identified with an extension of Rubin's (1974) potential outcomes approach. In this framework, there are two outcomes: a treatment state and a control state, and the causal effect of the treatment is simply the difference between the two potential outcomes. Given the nonrandomized nature of the data, differences in attributes between AEASMP participants and nonparticipants could lead to biased estimates of the causal effect of participation.Possible bias in estimated treatment effects attributed to differing attributes is, of course, the sample-selection problem. A conventional approach assumes that selection into the treatment is conditioned on both observable and unobservable characteristics, and the treatment effect is estimated with latent variable selection models (Heckman 1979). However, the influential analysis of Lalonde (1986) has cast doubts on the accuracy of such models in estimating treatment effects. In contrast, when selection into the treatment is assumed to be conditioned only on observable characteristics, propensity scoring methods have been demonstrated to replicate experimental benchmarks reasonably well (Dehejia and Wahba, 1998). Below, we use both propensity-score weighted and Heckit estimators to identify and estimate the treatment effects of AEASMP participation. Our estimates of the treatment effects reveal that AEASMP participation has causal and positive effects on research productivity and on the ability to obtain research resources. We find similar effects for both propensity-score weighted and Heckit estimates.The remainder of this article is organized as follows. A historical overview of the AEASMP is provided in section 2. Section 3 discusses the data. In section 4, we discuss our methodology--the potential outcomes approach to identifying treatment effects. Given selection on observable characteristics, unobservable characteristics, or both, we motivate the use of propensity-score weighted and Heckit estimators of the treatment effects attributed to AEASMP participation. …

Do dropouts suffer from dropping out? Estimation and prediction of outcome gains in generalized selection models

AbstractIn this paper we describe methods for predicting distributions of outcome gains in the framework of a latent variable selection model. We describe such procedures for Student‐t selection models and a finite mixture of Gaussian selection models. Importantly, our algorithms for fitting these models are simple to implement in practice, and also permit learning to take place about the non‐identified cross‐regime correlation parameter. Using data from High School and Beyond, we apply our methods to determine the impact of dropping out of high school on a math test score taken at the senior year of high school. Our results show that selection bias is an important feature of this data, that our beliefs about this non‐identified correlation are updated from the data, and that generalized models of selectivity offer an improvement over the ‘textbook’ Gaussian model. Further, our results indicate that on average dropping out of high school has a large negative impact on senior‐year test scores. However, for those individuals who actually drop out of high school, the act of dropping out of high school does not have a significantly negative impact on test scores. This suggests that policies aimed at keeping students in school may not be as beneficial as first thought, since those individuals who must be induced to stay in school are not the ones who benefit significantly (in terms of test scores) from staying in school. Copyright © 2004 John Wiley & Sons, Ltd.

On the Predictive Distributions of Outcome Gains in the Presence of an Unidentified Parameter

In this article we describe methods for obtaining the predictive distributions of outcome gains in the framework of a standard latent variable selection model. Although most previous work has focused on estimation of mean treatment parameters as the method for characterizing outcome gains from program participation, we show how the entire distributions associated with these gains can be obtained in certain situations. Although the out-of-sample outcome gain distributions depend on an unidentified parameter, we use the results of Koop and Poirier to show that learning can take place about this parameter through information contained in the identified parameters via a positive definiteness restriction on the covariance matrix. In cases where this type of learning is not highly informative, the spread of the predictive distributions depends more critically on the prior. We show both theoretically and in extensive generated data experiments how learning occurs, and delineate the sensitivity of our results to the prior specifications. We relate our analysis to three treatment parameters widely used in the evaluation literature—the average treatment effect, the effect of treatment on the treated, and the local average treatment effect—and show how one might approach estimation of the predictive distributions associated with these outcome gains rather than simply the estimation of mean effects. We apply these techniques to predict the effect of literacy on the weekly wages of a sample of New Jersey child laborers in 1903.