Sample Inference Research Articles

Label-Only Membership Inference Attack Based on Model Explanation

It is well known that machine learning models (e.g., image recognition) can unintentionally leak information about the training set. Conventional membership inference relies on posterior vectors, and this task becomes extremely difficult when the posterior is masked. However, current label-only membership inference attacks require a large number of queries during the generation of adversarial samples, and thus incorrect inference generates a large number of invalid queries. Therefore, we introduce a label-only membership inference attack based on model explanations. It can transform a label-only attack into a traditional membership inference attack by observing neighborhood consistency and perform fine-grained membership inference for vulnerable samples. We use feature attribution to simplify the high-dimensional neighborhood sampling process, quickly identify decision boundaries and recover a posteriori vectors. It also compares different privacy risks faced by different samples through finding vulnerable samples. The method is validated on CIFAR-10, CIFAR-100 and MNIST datasets. The results show that membership attributes can be identified even using a simple sampling method. Furthermore, vulnerable samples expose the model to greater privacy risks.

Shapley Curves: A Smoothing Perspective

This article fills the limited statistical understanding of Shapley values as a variable importance measure from a nonparametric (or smoothing) perspective. We introduce population-level Shapley curves to measure the true variable importance, determined by the conditional expectation function and the distribution of covariates. Having defined the estimand, we derive minimax convergence rates and asymptotic normality under general conditions for the two leading estimation strategies. For finite sample inference, we propose a novel version of the wild bootstrap procedure tailored for capturing lower-order terms in the estimation of Shapley curves. Numerical studies confirm our theoretical findings, and an empirical application analyzes the determining factors of vehicle prices.

Superpopulation model inference for non-probability samples under informative sampling

Non-probability samples have been extensively applied in many various fields in recent years. However, it is difficult to infer the population from non-probability samples since selection probabilities of non-probability samples is unknown. Superpopulation modeling has been recently explored to make inference from non-probability samples. However, the existing superpopulation model approaches rely mostly on the noninformative sampling assumption. When sampling is informative, the sample distribution differs from the population distribution. Ignoring this point may result in biased estimation. In this paper, taking into account the informative sampling scheme, a superpopulation model approach for non-probability samples is proposed. Exponential model, logistic model and probit model are established to explain the informative sampling mechanism, respectively. The sample likelihood is derived to estimate the superpopulation model parameters for non-probability samples under various informative sampling models. The population mean estimator can be obtained based on the fitted superpopulation model from non-probability samples under different informative sampling approaches. The theoretical properties of the proposed estimator are established. Simulation results illustrate the performance of our proposed method for different informative sampling models and sample sizes. Also, the proposed method is applied to the data from the National Health and Nutrition Examination Survey.

Robust and privacy-preserving collaborative training: a comprehensive survey

Increasing numbers of artificial intelligence systems are employing collaborative machine learning techniques, such as federated learning, to build a shared powerful deep model among participants, while keeping their training data locally. However, concerns about integrity and privacy in such systems have significantly hindered the use of collaborative learning systems. Therefore, numerous efforts have been presented to preserve the model’s integrity and reduce the privacy leakage of training data throughout the training phase of various collaborative learning systems. This survey seeks to provide a systematic and comprehensive evaluation of security and privacy studies in collaborative training, in contrast to prior surveys that only focus on one single collaborative learning system. Our survey begins with an overview of collaborative learning systems from various perspectives. Then, we systematically summarize the integrity and privacy risks of collaborative learning systems. In particular, we describe state-of-the-art integrity attacks (e.g., Byzantine, backdoor, and adversarial attacks) and privacy attacks (e.g., membership, property, and sample inference attacks), as well as the associated countermeasures. We additionally provide an analysis of open problems to motivate possible future studies.

Small Sample Inference for Two‐Way Capture‐Recapture Experiments

SummaryThe properties of the generalised Waring distribution defined on the non‐negative integers are reviewed. Formulas for its moments and its mode are given. A construction as a mixture of negative binomial distributions is also presented. Then we turn to the Petersen model for estimating the population size in a two‐way capture‐recapture experiment. We construct a Bayesian model for by combining a Waring prior with the hypergeometric distribution for the number of units caught twice in the experiment. Credible intervals for are obtained using quantiles of the posterior, a generalised Waring distribution. The standard confidence interval for the population size constructed using the asymptotic variance of Petersen estimator and 0.5 logit transformed interval are shown to be special cases of the generalised Waring credible interval. The true coverage of this interval is shown to be bigger than or equal to its nominal converage in small populations, regardless of the capture probabilities. In addition, its length is substantially smaller than that of the 0.5 logit transformed interval. Thus, the proposed generalised Waring credible interval appears to be the best way to quantify the uncertainty of the Petersen estimator for populations size.

Large Language Model Inference Acceleration Based on Hybrid Model Branch Prediction

As the size of deep learning models continues to expand, the elongation of inference time has gradually evolved into a significant challenge to efficiency and practicality for autoregressive models. This work introduces a hybrid model acceleration strategy based on branch prediction, which accelerates autoregressive model inference without requiring retraining and ensures output consistency with the original model. Specifically, the algorithm employs two models with different parameter sizes aimed at the same task. The smaller model generates a series of potential tokens that are then parallelly validated by the larger model to determine their acceptability. By orchestrating the workflow of the large and small models through a branch-prediction strategy, the algorithm conceals the validation time of the larger model when predictions are successful, thereby accelerating inference. We propose a binomial distribution-based prediction function that blends theoretical principles with empirical evidence, specifically designed for the nuanced requirements of accelerating inference within a hybrid model framework. The entire algorithm was designed and implemented on the llama model for text generation and translation tasks. The experimental results indicate significant improvements. The proposed algorithm achieves a 1.2× to 3.4× increase in inference speed compared to the original model, consistently outperforming the speculative sampling inference acceleration algorithm.

Correction

This note provides correction to some numerical results in Krivitsky P. N., Coletti, P., and Hens, N. (2023), “A Tale of Two Datasets: Representativeness and Generalisability of Inference for Samples of Networks,” Journal of the American Statistical Association, 118, 2213–2224.

Superpopulation model inference for non probability samples under informative sampling with high-dimensional data

Non probability samples have been widely used in various fields. However, non probability samples suffer from selection biases due to the unknown selection probabilities. Superpopulation model inference methods have been discussed to solve this problem, but these approaches require the non informative sampling assumption. When the sampling mechanism is informative sampling, that is, selection probabilities are related to the outcome variable, the previous inference methods may be invalid. Moreover, we may encounter a large number of covariates in practice, which poses a new challenge for inference from non probability samples under informative sampling. In this article, the superpopulation model approaches under informative sampling with high-dimensional data are developed to perform valid inferences from non probability samples. Specifically, a semiparametric exponential tilting model is established to estimate selection probabilities, and the sample distribution is derived for estimating the superpopulation model parameters. Moreover, SCAD, adaptive LASSO, and Model-X knockoffs are employed to select variables, and estimate parameters in superpopulation modeling. Asymptotic properties of the proposed estimators are established. Results from simulation studies are presented to compare the performance of the proposed estimators with the naive estimator, which ignores informative sampling. The proposed methods are further applied to the National Health and Nutrition Examination Survey data.

Bayesian inference for nonprobability samples with nonignorable missingness

AbstractNonprobability samples, especially web survey data, have been available in many different fields. However, nonprobability samples suffer from selection bias, which will yield biased estimates. Moreover, missingness, especially nonignorable missingness, may also be encountered in nonprobability samples. Thus, it is a challenging task to make inference from nonprobability samples with nonignorable missingness. In this article, we propose a Bayesian approach to infer the population based on nonprobability samples with nonignorable missingness. In our method, different Logistic regression models are employed to estimate the selection probabilities and the response probabilities; the superpopulation model is used to explain the relationship between the study variable and covariates. Further, Bayesian and approximate Bayesian methods are proposed to estimate the response model parameters and the superpopulation model parameters, respectively. Specifically, the estimating functions for the response model parameters and superpopulation model parameters are utilized to derive the approximate posterior distribution in superpopulation model estimation. Simulation studies are conducted to investigate the finite sample performance of the proposed method. The data from the Pew Research Center and the Behavioral Risk Factor Surveillance System are used to show better performance of our proposed method over the other approaches.

Improving Species Level-taxonomic Assignment from 16S rRNA Sequencing Technologies.

Analysis of the bacterial community from a 16S rRNA gene sequencing technologies requires comparing the reads to a reference database. The challenging task involved in annotation relies on the currently available tools and 16S rRNA databases: SILVA, Greengenes and RDP. A successful annotation depends on the quality of the database. For instance, Greengenes and RDP have not been updated since 2013 and 2016, respectively. In addition, the nature of 16S sequencing technologies (short reads) focuses mainly on the V3-V4 hypervariable region sequencing and hinders the species assignment, in contrast to whole shotgun metagenome sequencing. Here, we combine the results of three standard protocols for 16S rRNA amplicon annotation that utilize homology-based methods, and we propose a new re-annotation strategy to enlarge the percentage of amplicon sequence variants (ASV) classified up to the species level. Following the pattern (reference) method: DADA2 pipeline and SILVA v.138.1 reference database classification (Basic Protocol 1), our method maps the ASV sequences to custom nucleotide BLAST with the SILVA v.138.1 (Basic Protocol 2), and to the 16S database of Bacteria and Archaea of NCBI RefSeq Targeted Loci Project databases (Basic Protocol 3). This new re-annotation workflow was tested in 16S rRNA amplicon data from 156 human fecal samples. The proposed new strategy achieved an increase of nearly eight times the proportion of ASV classified at the species level in contrast to the reference method for the database used in the present research. © 2023 The Authors. Current Protocols published by Wiley Periodicals LLC. Basic Protocol 1: Sample inference and taxonomic profiling through DADA2 algorithm. Basic Protocol 2: Custom BLASTN database creation and ASV taxonomical assignment. Basic Protocol 3: ASV taxonomical assignment using NCBI RefSeq Targeted Loci Project database. Basic Protocol 4: Definitive selection of lineages among the three methods.

Systematic Process for Determining Field-Sampling Effort Required to Know Vegetation Changes in Large, Disturbed Rangelands Where Management Treatments Have Been Applied

Adequate numbers of replicated, dispersed, and random samples are the basis for reliable sampling inference on resources of concern, particularly vegetation cover across large and heterogenous areas such as rangelands. Tools are needed to predict and assess data precision, specifically the sampling effort required to attain acceptable levels of precision, before and after sampling. We describe and evaluate a flexible and scalable process for assessing the sampling effort requirement for a common monitoring context (responses of rangeland vegetation cover to post-fire restoration treatments), using a custom R script called “SampleRange.” In SampleRange, vegetation cover is estimated from available digital-gridded or field data (e.g., using the satellite-derived cover from the Rangeland Assessment Platform). Next, the sampling effort required to estimate cover with 20% relative standard error (RSE) or to saturate sampling effort is determined using simulations across the environmental gradients in areas of interest to estimate the number of needed plots (“SampleRange quota”). Finally, the SampleRange quota are randomly identified for actual sampling. A 2022 full-cycle trial of SampleRange using the best available digital and prior field data for areas treated after a 2017 wildfire in sagebrush-steppe rangelands revealed that differences in the predicted compared with realized RSEs are inevitable. Future efforts to account for uncertainty in remotely sensed−based vegetative products will enhance tool utility.

Microbiome depiction through user-adapted bioinformatic pipelines and parameters.

Introduction. The role of the microbiome in health and disease continues to be increasingly recognized. However, there is significant variability in the bioinformatic protocols for analysing genomic data. This, in part, has impeded the potential incorporation of microbiomics into the clinical setting and has challenged interstudy reproducibility. In microbial compositional analysis, there is a growing recognition for the need to move away from a one-size-fits-all approach to data processing.Gap Statement. Few evidence-based recommendations exist for setting parameters of programs that infer microbiota community profiles despite these parameters significantly impacting the accuracy of taxonomic inference.Aim. To compare three commonly used programs (DADA2, QIIME2, and mothur) and optimize them into four user-adapted pipelines for processing paired-end amplicon reads. We aim to increase the accuracy of compositional inference and help standardize microbiomic protocol.Methods. Two key parameters were isolated across four pipelines: filtering sequence reads based on a whole-number error threshold (maxEE) and truncating read ends based on a quality score threshold (QTrim). Closeness of sample inference was then evaluated using a mock community of known composition.Results. We observed that raw genomic data lost were proportionate to how stringently parameters were set. Exactly how much data were lost varied by pipeline. Accuracy of sample inference correlated with increased sequence read retention. Falsely detected taxa and unaccounted for microbial constituents were unique to pipeline and parameter. Implementation of optimized parameter values led to better approximation of the known mock community.Conclusions. Microbial compositions generated based on the 16S rRNA marker gene should be interpreted with caution. To improve microbial community profiling, bioinformatic protocols must be user-adapted. Analysis should be performed with consideration for the select target amplicon, pipelines and parameters used, and taxa of interest.

Towards Inference Efficient Deep Ensemble Learning

Ensemble methods can deliver surprising performance gains but also bring significantly higher computational costs, e.g., can be up to 2048X in large-scale ensemble tasks. However, we found that the majority of computations in ensemble methods are redundant. For instance, over 77% of samples in CIFAR-100 dataset can be correctly classified with only a single ResNet-18 model, which indicates that only around 23% of the samples need an ensemble of extra models. To this end, we propose an inference efficient ensemble learning method, to simultaneously optimize for effectiveness and efficiency in ensemble learning. More specifically, we regard ensemble of models as a sequential inference process and learn the optimal halting event for inference on a specific sample. At each timestep of the inference process, a common selector judges if the current ensemble has reached ensemble effectiveness and halt further inference, otherwise filters this challenging sample for the subsequent models to conduct more powerful ensemble. Both the base models and common selector are jointly optimized to dynamically adjust ensemble inference for different samples with various hardness, through the novel optimization goals including sequential ensemble boosting and computation saving. The experiments with different backbones on real-world datasets illustrate our method can bring up to 56% inference cost reduction while maintaining comparable performance to full ensemble, achieving significantly better ensemble utility than other baselines. Code and supplemental materials are available at https://seqml.github.io/irene.

Diet-induced changes in fecal microbiota composition and diversity in dogs (Canis lupus familiaris): A comparative study of BARF-type and commercial diets

BackgroundDiet is known to strongly modulate the composition of the gut microbiota, thereby affecting health conditions and disease. Natural BARF-type and commercial diets have been used for feeding pets (e.g. dogs and cats) promoting changes in the canine microbiota in terms of abundance, richness, and diversity that may favor certain metabolic processes and resistance to certain infectious agents. Therefore, the present study sought to identify microbiota changes in dogs fed with a BARF-type diet versus dogs fed with a commercial diet by sequencing the V4 region of the 16S rRNA gene. MethodsThe microbiota of dogs fed with the BARF-diet (n = 20) and commercial-diet (n = 26) was studied using fecal samples. A metabarcoding strategy was employed by sequencing the V4 hypervariable region of the 16S rRNA gene using the Illumina HiSeq platform. DADA2 was used to assess the quality profile of the reads and to determine the core sample inference algorithm of the reads to infer amplicon sequence variants (ASVs). The taxonomic assignment was performed using sequences from the Silva v138 formatted reference database. The microbial diversity analysis was performed using the R package Phyloseq, which was used to calculate diversity and abundance indices and construct the respective graphs. Linear discriminant analysis (LDA) effect size analysis (LEfSe) was used to identify the differentially abundant taxa in the BARF group versus the commercial-diet group. ResultsThe diet causes changes in fecal microbiota composition and diversity, with richness and diversity being higher in BARF-fed dogs. Beta diversity analyses confirmed that diet is directly related to microbiota composition regardless of breed or sex. Differentially enriched taxa were identified in each of the diets as Fusobacterium, Bacteroides, and Clostridium perfringens in BARF-fed dogs and Prevotella, Turicibacter, Faecalibacterium, and Peptacetobacter (Clostridium) hiranonis, mostly relevant in carbohydrate metabolism, in commercial-fed dogs. ConclusionsThis study is the first one carried out in dogs from Colombia that seeks to identify changes in the intestinal microbiota concerning natural BARF type diet and commercial diet using a metabarcoding approach. Important differences were identified in terms of richness, diversity, and differentially enriched bacteria in each of the diets. The microbiota of dogs fed the BARF diet was characterized by higher richness and diversity compared to the commercial diet. However, it was identified that BARF-fed dogs can potentially acquire more opportunistic infections by pathogens of importance such as C. perfringens. Most of the taxa enriched in commercial diet-fed dogs are linked to carbohydrate metabolism, which may be directly related to diet composition.

Ambiguity aversion and the degree of ambiguity

We empirically show that sample information not only moderates prospects’ outcome ambiguity but also decision makers’ revealed aversion of them. Since most natural prospects permit at least some sample inference, accounting for their degree of ambiguity improves prediction of aversion. The special case of full ambiguity, as in Ellsberg-type designs, is typically averted—yet many decision makers systematically like low degrees of ambiguity while disliking higher degrees. Ambiguity attitudes might thus usefully be characterized by not only their sensitivity to degrees of ambiguity but also such ambiguity thresholds. Just as people like some risks but not others, they have ambiguity attitudes that depend on how much ambiguity there is. We thus show how attitudes towards a degree of ambiguity are systematic, enabling prediction across sources of ambiguity.

Finite sample inference for empirical Bayesian methods

AbstractIn recent years, empirical Bayesian (EB) inference has become an attractive approach for estimation in parametric models arising in a variety of real‐life problems, especially in complex and high‐dimensional scientific applications. However, compared to the relative abundance of available general methods for computing point estimators in the EB framework, the construction of confidence sets and hypothesis tests with good theoretical properties remains difficult and problem specific. Motivated by the Universal Inference framework, we propose a general and universal method, based on holdout likelihood ratios, and utilizing the hierarchical structure of the specified Bayesian model for constructing confidence sets and hypothesis tests that are finite sample valid. We illustrate our method through a range of numerical studies and real data applications, which demonstrate that the approach is able to generate useful and meaningful inferential statements in the relevant contexts.

Triple Sampling Inference Procedures for the Mean of the Normal Distribution When the Population Coefficient of Variation Is Known

This paper discusses the triple sampling inference procedures for the mean of a symmetric distribution—the normal distribution when the coefficient of variation is known. We use the Searls’ estimator as an initial estimate for the unknown population mean rather than the classical sample mean. In statistics literature, the normal distribution under investigation underlines almost all the natural phenomena with applications in many fields. First, we discuss the minimum risk point estimation problem under a squared error loss function with linear sampling cost. We obtained all asymptotic results that enhanced finding the second-order asymptotic risk and regret. Second, we construct a fixed-width confidence interval for the mean that satisfies at least a predetermined nominal value and find the second-order asymptotic coverage probability. Both estimation problems are performed under a unified optimal framework. The theoretical results reveal that the performance of the triple sampling procedure depends on the numerical value of the coefficient of variation—the smaller the coefficient of variation, the better the performance of the procedure.

MLE with datasets from populations having shared parameters

We consider maximum likelihood estimation with two or more datasets sampled from different populations with shared parameters. Although more datasets with shared parameters can increase statistical accuracy, this paper shows how to handle heterogeneity among different populations for correctness of estimation and inference. Asymptotic distributions of maximum likelihood estimators are derived under either regular cases where regularity conditions are satisfied or some non-regular situations. A bootstrap variance estimator for assessing performance of estimators and/or making large sample inference is also introduced and evaluated in a simulation study.

Cross-collection latent Beta-Liouville allocation model training with privacy protection and applications.

Cross-collection topic models extend previous single-collection topic models, such as Latent Dirichlet Allocation (LDA), to multiple collections. The purpose of cross-collection topic modeling is to model document-topic representations and reveal similarities between each topic and differences among groups. However, the restriction of Dirichlet prior and the significant privacy risk have hampered those models' performance and utility. Training those cross-collection topic models may, in particular, leak sensitive information from the training dataset. To address the two issues mentioned above, we propose a novel model, cross-collection latent Beta-Liouville allocation (ccLBLA), which operates a more powerful prior, Beta-Liouville distribution with a more general covariance structure that enhances topic correlation analysis. To provide privacy protection for the ccLBLA model, we leverage the inherent differential privacy guarantee of the Collapsed Gibbs Sampling (CGS) inference scheme and then propose a hybrid privacy protection algorithm for the ccLBLA model (HPP-ccLBLA) that prevents inferring data from intermediate statistics during the CGS training process without sacrificing its utility. More crucially, our technique is the first attempt to use the cross-collection topic model in image classification applications and investigate the cross-collection topic model's capabilities beyond text analysis. The experimental results for comparative text mining and image classification will show the merits of our proposed approach.

On small sample inference for common mean in heteroscedastic one-way model

On small sample inference for common mean in heteroscedastic one-way model