Hoeffding Inequality Research Articles

A Multitask Latent Feature Augmentation Method for Few-Shot Learning.

Few-shot learning (FSL) aims to learn novel concepts quickly from a few novel labeled samples with the transferable knowledge learned from base dataset. The existing FSL methods usually treat each sample as a single feature point in embedding space and classify through one single comparison task. However, the few-shot single feature points on the novel meta-testing episode are still vulnerable to noise easily although with the good transferable knowledge, because the novel categories are never seen on base dataset. Besides, the existing FSL models are trained by only one single comparison task and ignore that different semantic feature maps have different weights on different comparison objects and tasks, which cannot take full advantage of the valuable information from different multiple comparison tasks and objects to make the latent features (LFs) more robust based on only few-shot samples. In this article, we propose a novel multitask LF augmentation (MTLFA) framework to learn the meta-knowledge of generalizing key intraclass and distinguishable interclass sample features from only few-shot samples through an LF augmentation (LFA) module and a multitask (MT) framework. Our MTLFA treats the support features as sampling from the class-specific LF distribution, enhancing the diversity of support features and reducing the impact of noise based on few-shot support samples. Furthermore, an MT framework is introduced to obtain more valuable comparison-task-related and episode-related comparison information from multiple different comparison tasks in which different semantic feature maps have different weights, adjusting the prior LFs and generating the more robust and effective episode-related classifier. Besides, we analyze the feasibility and effectiveness of MTLFA from theoretical views based on the Hoeffding's inequality and the Chernoff's bounding method. Extensive experiments conducted on three benchmark datasets demonstrate that the MTLFA achieves the state-of-the-art performance in FSL. The experimental results verify our theoretical analysis and the effectiveness and robustness of MTLFA framework in FSL.

A Generalized Shuffle Framework for Privacy Amplification: Strengthening Privacy Guarantees and Enhancing Utility

The shuffle model of local differential privacy is an advanced method of privacy amplification designed to enhance privacy protection with high utility. It achieves this by randomly shuffling sensitive data, making linking individual data points to specific individuals more challenging. However, most existing studies have focused on the shuffle model based on (ε0,0)-Locally Differentially Private (LDP) randomizers, with limited consideration for complex scenarios such as (ε0,δ0)-LDP or personalized LDP (PLDP). This hinders a comprehensive understanding of the shuffle model's potential and limits its application in various settings. To bridge this research gap, we propose a generalized shuffle framework that can be applied to PLDP setting. This generalization allows for a broader exploration of the privacy-utility trade-off and facilitates the design of privacy-preserving analyses in diverse contexts. We prove that the shuffled PLDP process approximately preserves μ-Gaussian Differential Privacy with μ = O(1/√n). This approach allows us to avoid the limitations and potential inaccuracies associated with inequality estimations. To strengthen the privacy guarantee, we improve the lower bound by utilizing hypothesis testing instead of relying on rough estimations like the Chernoff bound or Hoeffding's inequality. Furthermore, extensive comparative evaluations clearly show that our approach outperforms existing methods in achieving strong central privacy guarantees while preserving the utility of the global model. We have also carefully designed corresponding algorithms for average function, frequency estimation, and stochastic gradient descent.

MamlFormer: Priori-experience guiding transformer network via manifold adversarial multi-modal learning for laryngeal histopathological grading

Pathologic grading of laryngeal squamous cell carcinoma (LSCC) plays a crucial role in diagnosis, prognosis, and migration. However, the grading performance and interpretability of the intelligent grading model based on LSCC low magnification images are poor. This is because it lacks the delicate nuclear information and information more relevant to grading contained in the high magnification images labeled by pathologists. Yet, low magnification images have information such as tissue texture and contours. Thus, we proposed an end-to-end transformer network with manifold adversarial multi-modal learning (MamlFormer). It effectively fuses and learns LSCC high and low magnification pathology image modalities. Firstly, we demonstrate the feasibility and sufficient conditions for modal fusion of LSCC high and low magnification images from Hoeffding's inequality and multimodal co-regularization. Secondly, we design a new manifold block. It constructs the manifold subspace by some principles. Those principles are divisibility, recoverability, and local distance closest of the feature matrix before and after the mapping of the LSCC each magnification image modalities. Meanwhile it can well solve the problems of redundant feature matrix information and weak modal semantic consistency after multimodal learning. Thirdly, we utilize the encoder and the adversarial loss function to implement adversarial block. It can adaptively learn the latent metrics of the modal distributions of LSCC high and low magnification images. Therefore, it also enhances the complementarity of LSCC high and low magnification image modalities. Then, numerous experiments show that MamlFormer outperforms other SOTA models in both grading performance and interpretability. Finally, we also performed generalization experiments on highly prevalent cervix squamous cell carcinoma. The MamlFormer over is superior to other SOTA models in terms of grading performance and interpretability. This indicates its excellent generalization performance and clinical practicability.

Hoeffding and Bernstein inequalities for weighted sums of exchangeable random variables

Hoeffding and Bernstein inequalities for weighted sums of exchangeable random variables

Elastic Data Binning: Time-Series Sketching for Time-Domain Astrophysics Analysis

Time-domain astrophysics analysis (TDAA) involves observational surveys of celestial phenomena that may contain irrelevant information because of several factors, one of which is the sensitivity of the optical telescopes. Data binning is a typical technique for removing inconsistencies and clarifying the main characteristics of the original data in astrophysics analysis. It splits the data sequence into smaller bins with a fixed size and subsequently sketches them into a new representation form. In this study, we introduce a novel approach, called elastic data binning (EBinning), to automatically adjust each bin size using two statistical metrics based on the Student's t-test for linear regression and Hoeffding inequality. EBinning outperforms well-known algorithms in TDAA for extracting relevant characteristics of time-series data, called lightcurve. We demonstrate the successful representation of various characteristics in the lightcurve gathered from the Kiso Schmidt telescope using EBinning and its applicability for transient detection in TDAA.

Elastic Data Binning: Time-Series Sketching for Time-Domain Astrophysics Analysis

Time-domain astrophysics analysis (TDAA) involves observational surveys of celestial phenomena that may contain irrelevant information because of several factors, one of which is the sensitivity of the optical telescopes. Data binning is a typical technique for removing inconsistencies and clarifying the main characteristics of the original data in astrophysics analysis. It splits the data sequence into smaller bins with a fixed size and subsequently sketches them into a new representation form. In this study, we introduce a novel approach, called elastic data binning (EBinning), to automatically adjust each bin size using two statistical metrics based on the Student's t-test for linear regression and Hoeffding inequality. EBinning outperforms well-known algorithms in TDAA for extracting relevant characteristics of time-series data, called lightcurve. We demonstrate the successful representation of various characteristics in the lightcurve gathered from the Kiso Schmidt telescope using EBinning and its applicability for transient detection in TDAA.

Estimating means of bounded random variables by betting

Abstract We derive confidence intervals (CIs) and confidence sequences (CSs) for the classical problem of estimating a bounded mean. Our approach generalizes and improves on the celebrated Chernoff method, yielding the best closed-form "empirical-Bernstein" CSs and CIs (converging exactly to the oracle Bernstein width) as well as non-closed-form "betting" CSs and CIs. Our method combines new composite nonnegative (super)martingales with Ville's maximal inequality, with strong connections to testing by betting and the method of mixtures. We also show how these ideas can be extended to sampling without replacement. In all cases, our bounds are adaptive to the unknown variance, and empirically vastly outperform prior approaches, establishing a new state-of-the-art for four fundamental problems: CSs and CIs for bounded means, when sampling with and without replacement.

FEAT: A Federated Approach for Privacy-Preserving Network Traffic Classification in Heterogeneous Environments

Network traffic classification is the foundation for many network security and network management applications. Recently, to preserve the privacy of the data which are generated in the mobile ends, federated learning (FL)-based classification methods are being proposed. Unfortunately, the performance of FL-based methods can seriously degrade when the client data have skewness. This is particularly true for mobile network traffic classification where the environments in the mobile ends are highly heterogeneous. In this article, we first conduct a measurement study on traffic classification accuracy through FL using real-world network traffic trace and we observe serious accuracy degradation due to heterogeneous environments. We propose a novel federated analytics (FA) approach, FEAT, to improve the accuracy. Note that FL emphasizes on model training, yet our FA performs local analytic tasks that can estimate traffic data skewness and select appropriate clients for FL model training. Our analytics tasks are performed locally and in a federated manner; thus, we preserve privacy as well. Our approach has strong theoretical properties where we exploit Hoeffding inequality to infer traffic data skewness and we leverage the Thompson Sampling for client selection. We evaluate our approach through extensive experiments using real-world traffic data sets QUIC and ISCX. The extensive experiments demonstrate that FEAT can improve traffic classification accuracy in heterogeneous environments.

Finite-key analysis of decoy model semi-quantum key distribution based on four-state protocol

Semi-quantum key distribution allows a full quantum user Alice and a classical user Bob to share a pair of security keys guaranteed by physical principles. Semi-quantum key distribution is proposed while verifying its robustness. Subsequently, its unconditional security of semi-quantum key distribution system is verified theoretically. In 2021, the feasibility of semi-quantum key distribution system based on mirror protocol was verified experimentally. However, the feasibility experimental system still uses the laser pulse with strong attenuation. It has been proved in the literature that the semi-quantum key distribution system still encounters the risk of secret key leakage under photon number splitting attack. Therefore, the actual security of key distribution can be further reasonably evaluated by introducing the temptation state and conducting the finite-key analysis in the key distribution process. In this work, for the model of adding one-decoy state only to Alice at the sending based on a four state semi-quantum key distribution system, the length of the security key in the case of finite-key is analyzed by using Hoeffding inequality, and then the formula of the security key rate is obtained. It is found in the numerical simulation that when the sample size is <inline-formula><tex-math id="M3">\begin{document}$ {10}^{5} $\end{document}</tex-math><alternatives><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="22-20230849_M3.jpg"/><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="22-20230849_M3.png"/></alternatives></inline-formula>, the security key rate of <inline-formula><tex-math id="M4">\begin{document}$ {10}^{-4} $\end{document}</tex-math><alternatives><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="22-20230849_M4.jpg"/><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="22-20230849_M4.png"/></alternatives></inline-formula>, which is close to the security key rate of the asymptotic limits, can be obtained in the case of close range. It is very important for the practical application of semi-quantum key distribution system.

An iterative stochastic procedure with a general step to a linear regular inverse problem

In this paper we consider a linear operator equation in a Hilbert space. Using Hoeffding inequalities, an exponential bound to the solution obtained by a stochastic procedure is established and the values of the step ak for which the procedure converges almost completely (a.co) are discussed. An illustrative application was treated to the solution of a Fredholm integral equation of the second kind.

Differential-quadrature-phase-shift quantum digital signature.

A novel quantum digital signature (QDS) scheme, called "differential quadrature phase-shift QDS," is presented. A message sender broadcasts a weak coherent pulse train with four phases of {0, π/2, π, 3π/2} and recipients create their own authentication keys from the broadcasted signal. Unlike conventional QDS protocols, there is no post-processing of information exchange between the sender and recipients and that between the recipients. Therefore, secured channels and/or authenticated channels for information exchange are not needed, and the key creation procedure is simpler than that of conventional QDS. Security issues are also discussed, using binominal distributions instead of Hoeffding's inequality utilized in conventional QDS studies, and calculation examples for system conditions achieving the QDS function are presented.

A multi-level weighted concept drift detection method

The concept drift detection method is an online learner. Its main task is to determine the position of drifts in the data stream, so as to reset the classifier after detecting the drift to improve the learning performance, which is very important in practical applications such as user interest prediction or financial transaction fraud detection. Aiming at the inability of existing drift detection methods to balance the detection delay, false positives, false negatives, and space–time efficiency, a new level transition threshold parameter is proposed, and a multi-level weighted mechanism including "Stable Level-Warning Level-Drift Level" is innovatively introduced in the concept drift detection. The instances in the window are weighted in levels, and the double sliding window is also applied. Based on this, a multi-level weighted drift detection method (MWDDM) is proposed. In particular, two variants which are MWDDM_H and MWDDM_M are proposed based on Hoeffding inequality and Mcdiarmid inequality, respectively. Experiments on artificial datasets show that MWDDM_H and MWDDM_M can detect abrupt and gradual concept drift faster than other comparison algorithms while maintaining a low false positive ratio and false negative ratio. Experiments on real-world datasets show that MWDDM has the highest classification accuracy in most cases while maintaining good space time efficiency.

Convenient Tail Bounds for Sums of Random Tensors

This work prepares new probability bounds for sums of random, independent, Hermitian tensors. These probability bounds characterize large-deviation behavior of the extreme eigenvalue of the sums of random tensors. We extend Laplace transform method and Lieb's concavity theorem from matrices to tensors, and apply these tools to generalize the classical bounds associated with the names Chernoff, Bennett, and Bernstein from the scalar to the tensor setting. Tail bounds for the norm of a sum of random rectangular tensors are also derived from corollaries of random Hermitian tensors cases. The proof mechanism can also be applied to tensor-valued martingales and tensor-based Azuma, Hoeffding and McDiarmid inequalities are established.

Hoeffding and Bernstein inequalities for U-statistics without replacement

Concentration inequalities quantify random fluctuations of functions of random variables, typically by bounding the probability that such a function differs from its expected value by more than a certain amount. In this paper, we extend Hoeffding’s inequality and Bernstein’s inequality for U-statistics to the setting of sampling without replacement from a finite population.

T-product tensors—part II: tail bounds for sums of random T-product tensors

This paper is the part II of a serious work about T-product tensors focusing at establishing new probability bounds for sums of random, independent, T-product tensors. These probability bounds characterize large-deviation behavior of the extreme eigenvalue of the sums of random T-product tensors. We apply Laplace transform method and Lieb’s concavity theorem for T-product tensors obtained from our part I paper, and apply these tools to generalize the classical bounds associated with the names Chernoff, and Bernstein from the scalar to the T-product tensor setting. Tail bounds for the norm of a sum of random rectangular T-product tensors are also derived from corollaries of random Hermitian T-product tensors cases. The proof mechanism is also applied to T-product tensor-valued martingales and T-product tensor-based Azuma, Hoeffding and McDiarmid inequalities are derived.

Generalizations of some concentration inequalities

For a real-valued measurable function f and a nonnegative, nondecreasing function ϕ, we first obtain a Chebyshev type inequality which provides an upper bound for ϕ(λ1)μ({x∈Ω:f(x)≥λ1})+∑k=2nϕ(λk)−ϕ(λk−1)μ({x∈Ω:f(x)≥λk}), where 0<λ1<λ2<⋯<λn<∞. Using this, generalizations of a few concentration inequalities such as Markov, reverse Markov, Bienaymé–Chebyshev, Cantelli and Hoeffding inequalities are obtained.

Bounds for probabilistic programming with application to a blend planning problem

In this paper, we derive deterministic inner approximations for single and joint independent or dependent probabilistic constraints based on classical inequalities from probability theory such as the one-sided Chebyshev inequality, Bernstein inequality, Chernoff inequality and Hoeffding inequality (see Pinter, 1989). The dependent case has been modelled via copulas. New assumptions under which the bounds based approximations are convex allowing to solve the problem efficiently are derived. When the convexity condition can not hold, an efficient sequential convex approximation approach is further proposed to solve the approximated problem. Piecewise linear and tangent approximations are also provided for Chernoff and Hoeffding inequalities allowing to reduce the computational complexity of the associated optimization problem. Extensive numerical results on a blend planning problem under uncertainty are finally provided allowing to compare the proposed bounds with the Second Order Cone (SOCP) formulation and Sample Average Approximation (SAA).

Factored LT and Factored Raptor Codes for Large-Scale Distributed Matrix Multiplication

We propose two coding schemes for distributed matrix multiplication in the presence of stragglers. These coding schemes are adaptations of Luby Transform (LT) codes and Raptor codes to distributed matrix multiplication and are termed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Factored LT (FLT) codes</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Factored Raptor (FRT) codes</i> . We show that all nodes in the Tanner graph of a randomly sampled code have a tree-like neighborhood with high probability. This ensures that the density evolution analysis gives a reasonable estimate of the average recovery threshold of FLT codes. The recovery threshold of the proposed FLT codes is asymptotically optimal when the output degree distribution is Soliton. Empirically, we show that FRT codes have an excellent recovery threshold while the number of worker nodes is moderately large. In addition, using Azuma–Hoeffding inequality, we derive concentration results to show that the recovery threshold of a randomly chosen FLT code is close to the ensemble average. FLT and FRT codes have better recovery thresholds when compared to Product codes and they are expected to have better numerical stability when compared to Polynomial codes, while they can also be decoded with a low-complexity decoding algorithm. Finally, the proposed codes are better matched to the practically important case of sparse matrix-matrix multiplication as compared to many previous schemes.

Generalization Error Bounds via Rényi-, f-Divergences and Maximal Leakage

In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two random variables. These results find applications in adaptive data analysis, where multiple dependencies are introduced and in learning theory, where they can be employed to bound the generalization error of a learning algorithm. Bounds are given in terms of Sibson's Mutual Information, α-Divergences, Hellinger Divergences, and f-Divergences. A case of particular interest is the Maximal Leakage (or Sibson's Mutual Information of order infinity), since this measure is robust to post-processing and composes adaptively. The corresponding bound can be seen as a generalization of classical bounds, such as Hoeffding's and McDiarmid's inequalities, to the case of dependent random variables.

New-type Hoeffding’s inequalities and application in tail bounds

It is well known that Hoeffding's inequality has a lot of applications in the signal and information processing fields. How to improve Hoeffding's inequality and find the refinements of its applications have always attracted much attentions. An improvement of Hoeffding inequality was recently given by Hertz [&lt;a href="#1"&gt;1&lt;/a&gt;]. Eventhough such an improvement is not so big, it still can be used to update many known results with original Hoeffding's inequality, especially for Hoeffding-Azuma inequality for martingales. However, the results in original Hoeffding's inequality and its refined version by Hertz only considered the first order moment of random variables. In this paper, we present a new type of Hoeffding's inequalities, where the high order moments of random variables are taken into account. It can get some considerable improvements in the tail bounds evaluation compared with the known results. It is expected that the developed new type Hoeffding's inequalities could get more interesting applications in some related fields that use Hoeffding's results.