Cluster-specific Model Research Articles

Active Client Selection for Clustered Federated Learning.

Federated learning (FL) is an emerging distributed machine learning (ML) framework that operates under privacy and communication constraints. To mitigate the data heterogeneity underlying FL, clustered FL (CFL) was proposed to learn customized models for different client groups. However, due to the lack of effective client selection strategies, the CFL process is relatively slow, and the model performance is also limited in the presence of nonindependent and identically distributed (non-IID) client data. In this work, for the first time, we propose selecting participating clients for each cluster with active learning (AL) and call our method active client selection for CFL (ACFL). More specifically, in each ACFL round, each cluster filters out a small set of clients, which are the most informative clients according to some AL metrics e.g., uncertainty sampling, query-by-committee (QBC), loss, and aggregates only its model updates to update the cluster-specific model. We empirically evaluate our ACFL approach on the public MNIST, CIFAR-10, and LEAF synthetic datasets with class-imbalanced settings. Compared with several FL and CFL baselines, the results reveal that ACFL can dramatically speed up the learning process while requiring less client participation and significantly improving model accuracy with a relatively low communication overhead.

Federated Meta-Learning with Attention for Diversity-Aware Human Activity Recognition.

The ubiquity of smartphones equipped with multiple sensors has provided the possibility of automatically recognizing of human activity, which can benefit intelligent applications such as smart homes, health monitoring, and aging care. However, there are two major barriers to deploying an activity recognition model in real-world scenarios. Firstly, deep learning models for activity recognition use a large amount of sensor data, which are privacy-sensitive and hence cannot be shared or uploaded to a centralized server. Secondly, divergence in the distribution of sensory data exists among multiple individuals due to their diverse behavioral patterns and lifestyles, which contributes to difficulty in recognizing activity for large-scale users or 'cold-starts' for new users. To address these problems, we propose DivAR, a diversity-aware activity recognition framework based on a federated Meta-Learning architecture, which can extract general sensory features shared among individuals by a centralized embedding network and individual-specific features by attention module in each decentralized network. Specifically, we first classify individuals into multiple clusters according to their behavioral patterns and social factors. We then apply meta-learning in the architecture of federated learning, where a centralized meta-model learns common feature representations that can be transferred across all clusters of individuals, and multiple decentralized cluster-specific models are utilized to learn cluster-specific features. For each cluster-specific model, a CNN-based attention module learns cluster-specific features from the global model. In this way, by training with sensory data locally, privacy-sensitive information existing in sensory data can be preserved. To evaluate the model, we conduct two data collection experiments by collecting sensor readings from naturally used smartphones annotated with activity information in the real-life environment and constructing two multi-individual heterogeneous datasets. In addition, social characteristics including personality, mental health state, and behavior patterns are surveyed using questionnaires. Finally, extensive empirical results demonstrate that the proposed diversity-aware activity recognition model has a relatively better generalization ability and achieves competitive performance on multi-individual activity recognition tasks.

Deep-Learning-Based Adaptive Model for Solar Forecasting Using Clustering

Accurate short-term solar forecasting is challenging due to weather uncertainties associated with cloud movements. Typically, a solar station comprises a single prediction model irrespective of time and cloud condition, which often results in suboptimal performance. In the proposed model, different categories of cloud movement are discovered using K-medoid clustering. To ensure broader variation in cloud movements, neighboring stations were also used that were selected using a dynamic time warping (DTW)-based similarity score. Next, cluster-specific models were constructed. At the prediction time, the current weather condition is first matched with the different weather groups found through clustering, and a cluster-specific model is subsequently chosen. As a result, multiple models are dynamically used for a particular day and solar station, which improves performance over a single site-specific model. The proposed model achieved 19.74% and 59% less normalized root mean square error (NRMSE) and mean rank compared to the benchmarks, respectively, and was validated for nine solar stations across two regions and three climatic zones of India.

Modeling Determinants of Low Birth Weight for Under Five-Children in Ethiopia

Low birth weight (LBW) is a major determinant of morbidity, mortality and disability in infancy and childhood and has a long-term impact on health outcomes in adult life. This study was aimed to model LBW using marginal and generalized linear mixed models as well as identify the potential risk factors of LBW in Ethiopia. Data was taken from the 2011 Ethiopian demographic and health survey, which is a nationally representative survey of children in the 0–59 month age groups. Two model families, generalized estimating equation and alternating logistic regression models from marginal model family, and generalized linear mixed model from cluster specific model family were used for the analysis. The result showed that 34.8% of children were born with LBW. Alternating logistic regression model was best fits the data for population-averaged effects of the given factors on birth weight than generalized estimating equation model. Generalized linear mixed model with two random intercepts was the best model to evaluate within and between regional heterogeneity of birth weight. Both the best-fitted models gave the same conclusion that sex, wealth status, age, antenatal care, marital status, vaccination, anemia and mother education level were the determinant factors of LBW. This study contributes to the understanding of the individual and collective effect of maternal, socio-economic and child related factors influencing infant birth weight in Ethiopia.

Alternating Relaxed Twin Bounded Support Vector Clustering

Maximum margin clustering (MMC) and its improved version are based on the spirit of support vector machine, which will inevitably lead to prohibitively computational complexity when these learning models are encountered with an enormous amount of patterns. To accelerate the clustering efficiency, we propose alternating twin bounded support vector clustering to decompose the original large problem in MMC and its variants into two smaller sized ones, in which solving expensive semi-definite programming is avoided by performing alternating optimization between cluster-specific model parameters and instance-specific labeling assignments. Also the structural risk minimization principle is implemented to obtain good generalization. Additionally, in order to avoid premature convergence, a relaxed version of our algorithm is proposed, in which the hinge loss in the original twin bounded support vector machine is replaced with the Laplacian loss. These two versions can be easily extended to the nonlinear context via kernel tricks. To investigate the efficacy of our clustering algorithm, several experiments are conducted on a number of synthetic and real-world datasets. Experimental results demonstrate that the proposed method has better performance than other existing clustering approaches in terms of clustering accuracy and time efficiency and also possesses the powerful ability to process larger-scaled datasets.

Simultaneous State Estimation of Cluster-Based Wireless Sensor Networks

There is rich literature on the state estimation of wireless sensor networks (WSNs). Most of these works are based on the dynamic state-space model via Kalman filter or similar Markov models. In any network, information exchange is inevitable, and in this paper, we propose a Bayesian non-parametric approach for addressing this issue in the context of state estimation of WSNs. We consider a cluster-based WSN and consider a discrete-time linear Markov model for estimating the state values of the sensor nodes over time. For measuring the amount of information shared by the model parameters across different clusters, we consider non-parametric matrix stick-breaking priors for the cluster-specific model parameters. We demonstrate the usefulness of our proposed model in locating an immobile anomalous node in the network. We compute the time to locate the anomalous object and the false positive rate of our proposed approach. Simulation studies are performed to assess the operating characteristics of the proposed model. The proposed approach will be useful in emergency monitoring, medical genetics, geosciences, and many other disciplines where WSNs are frequently used for decision making.

Normalization of medium-resolution NDVI by the use of coarser reference data: method and evaluation

Medium-resolution remote-sensing images with tens of metre spatial resolutions have spatial and spectral characteristics that are suited for mapping a range of structural and compositional properties of vegetation. However, many factors, such as the long revisit cycles and frequent cloud contamination, limit the availability of images for the monitoring and time-series analysis of vegetation. Thus, there is a strong incentive to combine data from more than one observation system in order to fill the gaps in observation and enhance the capability of remote sensing to monitor dynamics. In this paper, we introduce a framework for the normalization of the normalized difference vegetation index (NDVI) from different sensor systems by the use of synchronous coarse-resolution NDVI data. A new model called the Local Cluster-specific Linear Model (LCLM) is proposed. This model is designed to build the specific relationships for different clusters, block by block, considering the spatial heterogeneity of the influencing factors. To improve the stability of the parameter estimation, an M-estimation method is utilized to solve the coefficients. Based on an analysis of the previous evaluation methods, new schemes are designed for evaluating the accuracy of the parameter normalization. Different assessment experiments were undertaken with the new evaluation schemes, to validate the performance of the LCLM method. The results indicate that the LCLM method performs better than the existing methods. An application experiment was also undertaken, in which synchronous NDVI from Landsat ETM+ and Terra ASTER sensors were normalized by the use of a coarse-resolution MODIS product.

A Comparison of Models for Clustered Binary Outcomes: Analysis of a Designed Immunology Experiment

SUMMARY The lymphocyte proliferative assay (LPA) of immune competence was conducted on 52 subjects, with up to 36 processing conditions per subject, to evaluate whether samples could be shipped or stored overnight, rather than being processed on fresh blood as currently required. The LPA study resulted in clustered binary data, with both cluster level and cluster-varying covariates. Two modelling strategies for the analysis of such clustered binary data are through the cluster-specific and population-averaged approaches. Whereas most research in this area has focused on the analysis of matched pairs data, in many situations, such as the LPA study, cluster sizes are naturally larger. Through considerations of interpretation and efficiency of these models when applied to large clusters, the mixed effect cluster-specific model was selected as most appropriate for the analysis of the LPA data. The model confirmed that the LPA response is significantly impaired in individuals infected with the human immunodeficiency virus (HIV). The LPA response was found to be significantly lower for shipped and overnight samples than for fresh samples, and this effect was significantly stronger among HIV-infected individuals. Surprisingly, an anticoagulant effect was not detected.

A Comparison of Cluster-Specific and Population-Averaged Approaches for Analyzing Correlated Binary Data

Summary Clustered or correlated samples of binary responses arise frequently in practice due to repeated measurements or to subsampling the primary sampling units. Several recent approaches address intracluster correlation in binary regression problems including cluster-specific methods such as those based on mixed-effects logistic models and population-averaged methods such as those based on beta-binomial models. This paper considers the interpretations of the regression parameters in these two general approaches. We show that, unlike models for correlated Gaussian outcomes, the parameters of the cluster-specific and population-averaged models for correlated binary data describe different types of effects of the covariates on the response probabilities. In the case of random intercepts, we show that the covariate effects measured by the population-averaged approach are closer to zero than those of the cluster-specific approach when the cluster-specific model holds and that the difference in the magnitude of the covariate effects is increasing with intra-cluster correlation. The case of random slopes is also examined. These results are valid for arbitrary random effects distributions and are demonstrated using data on the ability to obtain samples of breast fluid from women.