Mixed-type Variables Research Articles

Bayesian clustering of mixed-type data with relevant variable identification

This paper presents a Bayesian nonparametric model for clustering datasets with continuous, ordinal, and nominal variables. The ordinal and nominal variables are treated using the latent variables framework based on the multivariate probit and the multinomial probit models. Combining the continuous variables with the latent continuous variables allows us to jointly model a set of mixed-type variables via the Dirichlet process Gaussian mixture model. The use of hierarchical shrinkage prior on the component means leads to improved clustering performances and provides an intuitive way to identify relevant clustering variables. The numerical results on simulated and real data illustrate the applicability of the proposed model.

Connectivity-Based Clustering for Mixed Discrete and Continuous Data

This paper introduces a density-based clustering procedure for datasets with variables of mixed type. The proposed procedure, which is closely related to the concept of shared neighbourhoods, works particularly well in cases where the individual clusters differ greatly in terms of the average pairwise distance of the associated objects. Using a number of concrete examples, it is shown that the proposed clustering algorithm succeeds in allowing the identification of subgroups of objects with statistically significant distributional characteristics.

Relabeling & raking algorithm for imbalanced classification

Imbalanced data classification, where the class distribution exhibits significant skewness, presents a challenging problem in binary classification tasks. This issue is particularly pronounced for high-dimensional data, as the presence of unequal class proportions can significantly degrade the performance of classifiers. Existing approaches to address this problem involve undersampling the majority class or oversampling the minority class to create balanced samples, thereby improving classification performance. However, extending these sampling methods to high-dimensional data and mixed data, which includes categorical variables, is nontrivial due to the need for approximating attribute distributions. In this paper, we propose a novel sampling strategy that incorporates raking and relabeling procedures to construct balanced samples by imputing attribute values from the majority class to the minority class. Our proposed algorithms demonstrate comparable performance to popular existing methods, while offering greater flexibility in accommodating diverse data shapes and attribute sizes. The practical appeal of our sampling algorithm lies in its ability to generate synthetic data for oversampling without the reliance on density estimation and its capability to handle mixed-type variables seamlessly. Furthermore, our sampling strategy exhibits robustness across different classifiers, as the choice of classifier does not significantly impact classification performance.

Identifying green citizen typologies by mining household-level survey data

Some impactful but unfavorable results of rapid urbanization are human-nature disconnection, waste of energy/water resources, and increased greenhouse gas emissions. To save the future of our planet, a transition to a more sustainable urban life is a must. However, there is no single sustainable city model because cities differ in terms of their assets. Hence, locally customized sustainable actions linked to global sustainability should be developed, such as a change in individual behaviors leads to a sustainable society, city, and country. This research investigated green citizen profiles and variables affecting the profiles in the context of environmental behavior and sustainability. For this purpose, survey research was done at the household level in a metropolis in Turkey. Measurement scales about environmental concern, human-nature connections, and sustainable consumption behavior were used for collecting data. A data analysis approach was proposed as the survey dataset contains mixed-type variables. It amalgamates statistics with machine learning algorithms, namely two-stage clustering with multilayered self-organizing maps, k-medoid clustering algorithm, factor analysis, permutational multivariate analysis of variance, principal component analysis and classification and regression trees algorithm. The results reveal that (i) five distinct profiles, namely unconscious greens, risky greens, economic greens, potential greens, and wasters are identified, none of which is entirely green; (ii) district, family life-cycle, household size, number of rooms, altruistic and biocentric environmental concerns are the most critical variables in distinguishing profiles; (iii) the proposed approach enables processing socio-demographic, psychographic, behavioral and consumption variables together.

Bayesian causal inference for observational studies with missingness in covariates and outcomes.

Missing data are a pervasive issue in observational studies using electronic health records or patient registries. It presents unique challenges for statistical inference, especially causal inference. Inappropriately handling missing data in causal inference could potentially bias causal estimation. Besides missing data problems, observational health data structures typically have mixed-type variables - continuous and categorical covariates - whose joint distribution is often too complex to be modeled by simple parametric models. The existence of missing values in covariates and outcomes makes the causal inference even more challenging, while most standard causal inference approaches assume fully observed data or start their works after imputing missing values in a separate preprocessing stage. To address these problems, we introduce a Bayesian nonparametric causal model to estimate causal effects with missing data. The proposed approach can simultaneously impute missing values, account for multiple outcomes, and estimate causal effects under the potential outcomes framework. We provide three simulation studies to show the performance of our proposed method under complicated data settings whose features are similar to our case studies. For example, Simulation Study 3 assumes the case where missing values exist in both outcomes and covariates. Two case studies were conducted applying our method to evaluate the comparative effectiveness of treatments for chronic disease management in juvenile idiopathic arthritis and cysticfibrosis.

Multivariate Small-Area Estimation for Mixed-type Response Variables with Item Nonresponse

Abstract Many surveys collect information on discrete characteristics and continuous variables, that is, mixed-type variables. Small-area statistics of interest include means or proportions of the response variables as well as their domain means, which are the mean values at each level of a different categorical variable. However, item nonresponse in survey data increases the complexity of small-area estimation. To address this issue, we propose a multivariate mixed-effects model for mixed-type response variables subject to item nonresponse. We apply this method to two data structures where the data are missing completely at random by design. We use empirical data from two separate studies: a survey of pet owners and a dataset from the National Resources Inventory. In these applications, our proposed method leads to improvements relative to a direct estimator and a predictor based on a univariate model.

Clustering of football players based on performance data and aggregated clustering validity indexes

Abstract We analyse football (soccer) player performance data with mixed type variables from the 2014-15 season of eight European major leagues. We cluster these data based on a tailor-made dissimilarity measure. In order to decide between the many available clustering methods and to choose an appropriate number of clusters, we use the approach by Akhanli and Hennig (2020. “Comparing Clusterings and Numbers of Clusters by Aggregation of Calibrated Clustering Validity Indexes.” Statistics and Computing 30 (5): 1523–44). This is based on several validation criteria that refer to different desirable characteristics of a clustering. These characteristics are chosen based on the aim of clustering, and this allows to define a suitable validation index as weighted average of calibrated individual indexes measuring the desirable features. We derive two different clusterings. The first one is a partition of the data set into major groups of essentially different players, which can be used for the analysis of a team’s composition. The second one divides the data set into many small clusters (with 10 players on average), which can be used for finding players with a very similar profile to a given player. It is discussed in depth what characteristics are desirable for these clusterings. Weighting the criteria for the second clustering is informed by a survey of football experts.

A Bayesian approach to study design and analysis with type I error rate control for response variables of mixed types.

There has been increased interest in the design and analysis of studies consisting of multiple response variables of mixed types. For example, in clinical trials, it is desirable to establish efficacy for a treatment effect in primary and secondary outcomes. In this article, we develop Bayesian approaches for hypothesis testing and study planning for data consisting of multiple response variables of mixed types with covariates. We assume that the responses are correlated via a Gaussian copula, and that the model for each response is, marginally, a generalized linear model (GLM). Taking a fully Bayesian approach, the proposed method enables inference based on the joint posterior distribution of the parameters. Under some mild conditions, we show that the joint distribution of the posterior probabilities under any Bayesian analysis converges to a Gaussian copula distribution as the sample size tends to infinity. Using this result, we develop an approach to control the type I error rate under multiple testing. Simulation results indicate that the method is more powerful than conducting marginal regression models and correcting for multiplicity using the Bonferroni-Holm Method. We also develop a Bayesian approach to sample size determination in the presence of response variables of mixed types, extending the concept of probability of success (POS) to multiple response variables of mixed types.

Mixed-type Variables Clustering for Learners’ Behavior in Flipped Classroom Implementation

Numerous approaches have been developed to group learners’ behavior in an online/blended learning environment. However, most clustering analyses in this particular field only consider numeric features despite the existence of categoric features that are found important in other studies. In this study, we compare K-Means and K-Prototypes algorithms to cluster learners’ behavior in a flipped classroom implementation. From the model selection, we found that the model produced by the K-Prototypes algorithm — which included categoric features — is a better one. The statistical analysis of the clustering results of the selected K-Prototypes model shows significant differences in most of the inter-cluster comparisons, implying a good separation of the data. More importantly, we can identify the behavior in each cluster which then can be used to help learners in achieving better results in learning.

Vine copula statistical disclosure control for mixed-type data

In this paper, we develop a new statistical disclosure control (SDC) method for mixed-type data based on vine copulas. The use of Gaussian and skew-t copulas has been demonstrated to be capable of incorporating information from the marginal distributions of mixed-type variables, whether they are discrete or continuous. In particular, our proposed SDC method using vine copulas generalizes a data perturbation method using an extended skew-t copula. Our vine-SDC method improves the SDC method using the extended skew-t copula by allowing the bivariate copulas in the vine decomposition to take various forms, thus offering a better fit for the joint distribution of the data and more flexibility in data perturbation. An additional advantage of our vine-SDC method is the significant improvement in computational efficiency compared with that using the extended skew-t copula. We discuss some statistical properties of vine copulas and the methodology of vine-SDC. A simulation and a study of real healthcare survey data are provided to explore the performance and strength of vine-SDC and compare it with a common copula-based SDC method.

Generalized Factor Model for Ultra-High Dimensional Correlated Variables with Mixed Types

As high-dimensional data measured with mixed-type variables gradually become prevalent, it is particularly appealing to represent those mixed-type high-dimensional data using a much smaller set of so-called factors. Due to the limitation of the existing methods for factor analysis that deal with only continuous variables, in this article, we develop a generalized factor model, a corresponding algorithm and theory for ultra-high dimensional mixed types of variables where both the sample size n and variable dimension p could diverge to infinity. Specifically, to solve the computational problem arising from the non-linearity and mixed types, we develop a two-step algorithm so that each update can be carried out in parallel across variables and samples by using an existing package. Theoretically, we establish the rate of convergence for the estimators of factors and loadings in the presence of nonlinear structure accompanied with mixed-type variables when both n and p diverge to infinity. Moreover, since the correct specification of the number of factors is crucial to both the theoretical and the empirical validity of factor models, we also develop a criterion based on a penalized loss to consistently estimate the number of factors under the framework of a generalized factor model. To demonstrate the advantages of the proposed method over the existing ones, we conducted extensive simulation studies and also applied it to the analysis of the NFBC1966 dataset and a cardiac arrhythmia dataset, resulting in more predictive and interpretable estimators for loadings and factors than the existing factor model.

Bayesian semiparametric joint modeling of longitudinal explanatory variables of mixed types and a binary outcome.

Many prospective biomedical studies collect longitudinal clinical and lifestyle data that are both continuous and discrete. In some studies, there is interest in the association between a binary outcome and the values of these longitudinal measurements at a specific time point. A common problem in these studies is inconsistency in timing of measurements and missing follow-ups which can lead to few measurements at the time of interest. Some methods have been developed to address this problem, but are only applicable to continuous measurements. To address this limitation, we propose a new class of joint models for a binary outcome and longitudinal explanatory variables of mixed types. The longitudinal model uses a latent normal random variable construction with regression splines to model time-dependent trends in mean with a Dirichlet Process prior assigned to random effects to relax distribution assumptions. We also standardize timing of the explanatory variables by relating the binary outcome to imputed longitudinal values at a set time point. The proposed model is evaluated through simulation studies and applied to data from a cancer survivor study of participants in the Women's Health Initiative.

A comparison of two dissimilarity functions for mixed-type predictor variables in the delta -machine

The delta -machine is a statistical learning tool for classification based on dissimilarities or distances between profiles of the observations to profiles of a representation set, which was proposed by Yuan et al. (J Claasif 36(3): 442–470, 2019). So far, the delta -machine was restricted to continuous predictor variables only. In this article, we extend the delta -machine to handle continuous, ordinal, nominal, and binary predictor variables. We utilized a tailored dissimilarity function for mixed type variables which was defined by Gower. This measure has properties of a Manhattan distance. We develop, in a similar vein, a Euclidean dissimilarity function for mixed type variables. In simulation studies we compare the performance of the two dissimilarity functions and we compare the predictive performance of the delta -machine to logistic regression models. We generated data according to two population distributions where the type of predictor variables, the distribution of categorical variables, and the number of predictor variables was varied. The performance of the delta -machine using the two dissimilarity functions and different types of representation set was investigated. The simulation studies showed that the adjusted Euclidean dissimilarity function performed better than the adjusted Gower dissimilarity function; that the delta -machine outperformed logistic regression; and that for constructing the representation set, K-medoids clustering achieved fewer active exemplars than the one using K-means clustering while maintaining the accuracy. We also applied the delta -machine to an empirical example, discussed its interpretation in detail, and compared the classification performance with five other classification methods. The results showed that the delta -machine has a good balance between accuracy and interpretability.

Generative adversarial networks for imputing missing data for big data clinical research

BackgroundMissing data is a pervasive problem in clinical research. Generative adversarial imputation nets (GAIN), a novel machine learning data imputation approach, has the potential to substitute missing data accurately and efficiently but has not yet been evaluated in empirical big clinical datasets.ObjectivesThis study aimed to evaluate the accuracy of GAIN in imputing missing values in large real-world clinical datasets with mixed-type variables. The computation efficiency of GAIN was also evaluated. The performance of GAIN was compared with other commonly used methods, MICE and missForest.MethodsTwo real world clinical datasets were used. The first was that of a cohort study on the long-term outcomes of patients with diabetes (50,000 complete cases), and the second was of a cohort study on the effectiveness of a risk assessment and management programme for patients with hypertension (10,000 complete cases). Missing data (missing at random) to independent variables were simulated at different missingness rates (20, 50%). The normalized root mean square error (NRMSE) between imputed values and real values for continuous variables and the proportion of falsely classified (PFC) for categorical variables were used to measure imputation accuracy. Computation time per imputation for each method was recorded. The differences in accuracy of different imputation methods were compared using ANOVA or non-parametric test.ResultsBoth missForest and GAIN were more accurate than MICE. GAIN showed similar accuracy as missForest when the simulated missingness rate was 20%, but was more accurate when the simulated missingness rate was 50%. GAIN was the most accurate for the imputation of skewed continuous and imbalanced categorical variables at both missingness rates. GAIN had a much higher computation speed (32 min on PC) comparing to that of missForest (1300 min) when the sample size is 50,000.ConclusionGAIN showed better accuracy as an imputation method for missing data in large real-world clinical datasets compared to MICE and missForest, and was more resistant to high missingness rate (50%). The high computation speed is an added advantage of GAIN in big clinical data research. It holds potential as an accurate and efficient method for missing data imputation in future big data clinical research.Trial registrationClinicalTrials.gov ID: NCT03299010; Unique Protocol ID: HKUCTR-2232

Hybrid Density- and Partition-Based Clustering Algorithm for Data With Mixed-Type Variables

Clustering is an essential technique for discovering patterns in data. The steady increase in amount and complexity of data over the years led to improvements and development of new clustering algorithms. However, algorithms that can cluster data with mixed variable types (continuous and categorical) remain limited, despite the abundance of data with mixed types particularly in the medical field. Among existing methods for mixed data, some posit unverifiable distributional assumptions or that the contributions of different variable types are not well balanced. We propose a two-step hybrid density- and partition-based algorithm (HyDaP) that can detect clusters after variables selection. The first step involves both density-based and partition-based algorithms to identify the data structure formed by continuous variables and recognize the important variables for clustering; the second step involves partition-based algorithm together with a novel dissimilarity measure we designed for mixed data to obtain clustering results. Simulations across various scenarios and data structures were conducted to examine the performance of the HyDaP algorithm compared to commonly used methods. We also applied the HyDaP algorithm on electronic health records to identify sepsis phenotypes.

Visualizing Inequality in Health and Socioeconomic Wellbeing in the EU: Findings from the SHARE Survey.

The main objective of this paper is to visualize profiles of older Europeans to better understand differing levels of dependency across Europe. Data comes from wave 6 of the Survey of Health, Ageing and Retirement in Europe (SHARE), carried out in 18 countries and representing over 124 million aged individuals in Europe. Using the information of around 30 mixed-type variables, we design four composite indices of wellbeing for each respondent: self-perception of health, physical health and nutrition, mental agility, and level of dependency. Next, by implementing the k-prototypes clustering algorithm, profiles are created by combining those indices with a collection of socio-economic and demographic variables about the respondents. Five profiles are established that segment the dataset into the least to the most individuals at risk of health and socio-economic wellbeing. The methodology we propose is wide enough to be extended to other surveys or disciplines.

Determinants of establishment success: Comparing alien and native freshwater fishes in Taiwan.

Many parts of Asia, including Taiwan, have suffered severely from freshwater fish invasions. However, few studies using an assemblage approach have been conducted in the region so far that would help to prioritize suitable preventive actions. In this study, we focused on the invasion process from the import stage to the establishment stage, and defined establishment success as the success during this predefined process. We used datasets of freshwater fish assemblages in Taiwan to (1) compare established versus non-established alien species to distinguish the determinants of establishment success, and (2) to use these determinants to test a life history hypothesis which predicts that the magnitudes of the determinants should be significantly different between established alien species and native species. We collated a dataset for freshwater fish species which were imported into Taiwan (n = 118) of which some successfully established (n = 26), and another dataset for freshwater fish species native to Taiwan (n = 77). For each imported species, we collected data for 17 variables, including two phylogenetic, two human-use, two invasion history, and 11 life history variables. We then used decision tree methods, which have advantages in analyzing datasets with many variables of mixed types without the need to make assumptions about data distributions and input data for missing values. Our results showed that aquaculture use and maximum body length were the most important determinants for predicting establishment success of alien freshwater fish in Taiwan. Comparing five important determinants between established alien versus native species showed that the established alien species were significantly more often used in aquaculture, were associated with a higher number of established countries, and had a larger body length and greater highest water temperature tolerance than the native species. We thus conclude that our results provided evidence to support our stated hypothesis. We suggest that aquaculture use, measures of body size, and the number of previously invaded countries may alert researchers and conservation managers to species with a high establishment potential, especially for countries with similar conditions as those in Taiwan.

A new correlation coefficient between categorical, ordinal and interval variables with Pearson characteristics

A prescription is presented for a new and practical correlation coefficient, ϕK, based on several refinements to Pearson’s hypothesis test of independence of two variables. The combined features of ϕK form an advantage over existing coefficients. Primarily, it works consistently between categorical, ordinal and interval variables, in essence by treating each variable as categorical, and can therefore be used to calculate correlations between variables of mixed type. Second, it captures nonlinear dependency. The strength of ϕK is similar to Pearson’s correlation coefficient, and is equivalent in case of a bivariate normal input distribution. These are useful properties when studying the correlations between variables with mixed types, where some are categorical. Two more innovations are presented: to the proper evaluation of statistical significance of correlations, and to the interpretation of variable relationships in a contingency table, in particular in case of sparse or low statistics samples and significant dependencies. Two practical applications are discussed. The presented algorithms are easy to use and available through a public Python library.11https://github.com/KaveIO/PhiK.

Learning clinical networks from medical records based on information estimates in mixed-type data.

The precise diagnostics of complex diseases require to integrate a large amount of information from heterogeneous clinical and biomedical data, whose direct and indirect interdependences are notoriously difficult to assess. To this end, we propose an efficient computational approach to simultaneously compute and assess the significance of multivariate information between any combination of mixed-type (continuous/categorical) variables. The method is then used to uncover direct, indirect and possibly causal relationships between mixed-type data from medical records, by extending a recent machine learning method to reconstruct graphical models beyond simple categorical datasets. The method is shown to outperform existing tools on benchmark mixed-type datasets, before being applied to analyze the medical records of eldery patients with cognitive disorders from La Pitié-Salpêtrière Hospital, Paris. The resulting clinical network visually captures the global interdependences in these medical records and some facets of clinical diagnosis practice, without specific hypothesis nor prior knowledge on any clinically relevant information. In particular, it provides some physiological insights linking the consequence of cerebrovascular accidents to the atrophy of important brain structures associated to cognitive impairment.

Pairwise sparse + low-rank models for variables of mixed type

Factor models have been proposed for a broad range of observed variables such as binary, Gaussian, and variables of mixed types. They typically model a pairwise interaction parameter matrix with a low-rank and a diagonal component. The low-rank component can be interpreted as the effect of a few latent quantitative variables which are common in social science applications. Another line of research has investigated graphical models for the same types of observed variables, where the pairwise interactions are usually assumed to be sparse. Sparse network structures are often found in the natural sciences. Still overall, while factor and sparse models are suitable for many applications, they sometimes might not be expressive enough to fit certain data well. This has motivated the confluence of factor and graphical models, yielding models where the interaction parameters are decomposed into respectively sparse and low-rank components. Up to now, this has only been done separately for observed Gaussian and for observed binary variables, but never jointly. The present work accomplishes a unified treatment of pairwise sparse and low-rank models. It does so by simultaneously allowing observed binary and quantitative (conditional Gaussian) variables. The model parameters of the joint model can be estimated using a convex regularized likelihood optimization problem. We show that the resulting estimator has consistency properties in the high-dimensional setting.