Quantitative Information Flow Research Articles

Understanding Leakage in Searchable Encryption: a Quantitative Approach

Searchable encryption, or more generally, structured encryption, permits search over encrypted data. It is an important cryptographic tool for securing cloud storage. The standard security notion for structured encryption mandates that a protocol leaks nothing about the data or queries, except for some allowed leakage, defined by the leakage function. This is due to the fact that some leakage is unavoidable for efficient schemes.\\ Unfortunately, it was shown by numerous works that even innocuous-looking leakage can often be exploited by attackers to undermine users' privacy and recover their queries and/or data, despite the structured encryption schemes being provably secure. Nevertheless, the standard security remains the go-to notion used to show the 'security' of structured encryption schemes. While it is not likely that researchers will design practical structured encryption schemes with no leakage, it is not satisfactory that very few works study ways to assess leakage.This work proposes a novel framework to quantify leakage. Our methodology is inspired by the quantitative information flow, and we call our method q-leakage analysis. We show how $q$-leakage analysis is related to the standard security. We also demonstrate the usefulness of q-leakage analysis by analyzing the security of two existing schemes with complex leakage functions.

Defining and Controlling Information Leakage in US Equities Trading

We present a new framework for defining information leakage in the setting of US equities trading, and construct methods for deriving trading schedules that stay within specified information leakage bounds. Our approach treats the stock market as an interactive protocol performed in the presence of an adversary, and draws inspiration from the related disciplines of differential privacy as well as quantitative information flow. We apply a linear programming solver using examples from historical trade and quote (TAQ) data for US equities and describe how this framework can inform actual algorithmic trading strategies.

An Information-Flow Perspective on Algorithmic Fairness

This work presents insights gained by investigating the relationship between algorithmic fairness and the concept of secure information flow. The problem of enforcing secure information flow is well-studied in the context of information security: If secret information may "flow" through an algorithm or program in such a way that it can influence the program’s output, then that is considered insecure information flow as attackers could potentially observe (parts of) the secret. There is a strong correspondence between secure information flow and algorithmic fairness: if protected attributes such as race, gender, or age are treated as secret program inputs, then secure information flow means that these "secret" attributes cannot influence the result of a program. While most research in algorithmic fairness evaluation concentrates on studying the impact of algorithms (often treating the algorithm as a black-box), the concepts derived from information flow can be used both for the analysis of disparate treatment as well as disparate impact w.r.t. a structural causal model. In this paper, we examine the relationship between quantitative as well as qualitative information-flow properties and fairness. Moreover, based on this duality, we derive a new quantitative notion of fairness called fairness spread, which can be easily analyzed using quantitative information flow and which strongly relates to counterfactual fairness. We demonstrate that off-the-shelf tools for information-flow properties can be used in order to formally analyze a program's algorithmic fairness properties, including the new notion of fairness spread as well as established notions such as demographic parity.

Universal optimality and robust utility bounds for metric differential privacy1

We study the privacy-utility trade-off in the context of metric differential privacy. Ghosh et al. introduced the idea of universal optimality to characterise the “best” mechanism for a certain query that simultaneously satisfies (a fixed) ε-differential privacy constraint whilst at the same time providing better utility compared to any other ε-differentially private mechanism for the same query. They showed that the Geometric mechanism is universally optimal for the class of counting queries. On the other hand, Brenner and Nissim showed that outside the space of counting queries, and for the Bayes risk loss function, no such universally optimal mechanisms exist. Except for the universal optimality of the Laplace mechanism, there have been no generalisations of these universally optimal results to other classes of differentially-private mechanisms. In this paper, we use metric differential privacy and quantitative information flow as the fundamental principle for studying universal optimality. Metric differential privacy is a generalisation of both standard (i.e., central) differential privacy and local differential privacy, and it is increasingly being used in various application domains, for instance in location privacy and in privacy-preserving machine learning. Similar to the approaches adopted by Ghosh et al. and Brenner and Nissim, we measure utility in terms of loss functions, and we interpret the notion of a privacy mechanism as an information-theoretic channel satisfying constraints defined by ε-differential privacy and a metric meaningful to the underlying state space. Using this framework we are able to clarify Nissim and Brenner’s negative results by (a) that in fact all privacy types contain optimal mechanisms relative to certain kinds of non-trivial loss functions, and (b) extending and generalising their negative results beyond Bayes risk specifically to a wide class of non-trivial loss functions. Our exploration suggests that universally optimal mechanisms are indeed rare within privacy types. We therefore propose weaker universal benchmarks of utility called privacy type capacities. We show that such capacities always exist and can be computed using a convex optimisation algorithm. Further, we illustrate these ideas on a selection of examples with several different underlying metrics.

Obtaining Information Leakage Bounds via Approximate Model Counting

Information leaks are a significant problem in modern software systems. In recent years, information theoretic concepts, such as Shannon entropy, have been applied to quantifying information leaks in programs. One recent approach is to use symbolic execution together with model counting constraints solvers in order to quantify information leakage. There are at least two reasons for unsoundness in quantifying information leakage using this approach: 1) Symbolic execution may not be able to explore all execution paths, 2) Model counting constraints solvers may not be able to provide an exact count. We present a sound symbolic quantitative information flow analysis that bounds the information leakage both for the cases where the program behavior is not fully explored and the model counting constraint solver is unable to provide a precise model count but provides an upper and a lower bound. We implemented our approach as an extension to KLEE for computing sound bounds for information leakage in C programs.

Quantitative Information Flow Techniques for Studying Optimality in Differential Privacy

Universal optimality for differential privacy seeks to characterise the "best" mechanisms, which are the ones that provide the most utility to some class of consumers whilst simultaneously satisfying a fixed ε-differential privacy constraint. The study of universal optimality was initiated in 2009 in two important papers: the first by Ghosh et al., who discovered a universally optimal mechanism for a large class of consumers within the differential privacy constraint context of "counting queries" on statistical datasets; the second by Brenner and Nissim who found that outside of this context, no such universally optimal mechanisms exist. Since those remarkable results, there have been few advances in this field. In this article we detail some recent work which sheds new light on these results using Quantitative Information Flow (QIF), a mathematical framework for quantifying information leaks under a Bayesian adversarial model. Using standard QIF reasoning we show how the earlier results can be both generalised and extended, opening up new avenues for exploring optimality in a wide variety of differential privacy contexts.

Flexible and scalable privacy assessment for very large datasets, with an application to official governmental microdata

We present a systematic refactoring of the conventional treatment of privacy analyses, basing it on mathematical concepts from the framework of Quantitative Information Flow (QIF ). The approach we suggest brings three principal advantages: it is flexible, allowing for precise quantification and comparison of privacy risks for attacks both known and novel; it can be computationally tractable for very large, longitudinal datasets; and its results are explainable both to politicians and to the general public. We apply our approach to a very large case study: the Educational Censuses of Brazil, curated by the governmental agency inep, which comprise over 90 attributes of approximately 50 million individuals released longitudinally every year since 2007. These datasets have only very recently (2018–2021) attracted legislation to regulate their privacy — while at the same time continuing to maintain the openness that had been sought in Brazilian society. inep’s reaction to that legislation was the genesis of our project with them. In our conclusions here we share the scientific, technical, and communication lessons we learned in the process.

A novel reconstruction attack on foreign-trade official statistics, with a Brazilian case study

In this paper we describe, formalize, implement, and experimentally evaluate a novel transaction re-identification attack against official foreigntrade statistics releases in Brazil. The attack’s goal is to re-identify the importers of foreign-trade transactions (by revealing the identity of the company performing that transaction), which consequently violates those importers’ fiscal secrecy (by revealing sensitive information: the value and volume of traded goods). We provide a mathematical formalization of this fiscal secrecy problem using principles from the framework of quantitative information flow (QIF), then carefully identify the main sources of imprecision in the official data releases used as auxiliary information in the attack, and model transaction re-construction as a linear optimization problem solvable through integer linear programming (ILP). We show that this problem is NP-complete, and provide a methodology to identify tractable instances. We exemplify the feasibility of our attack by performing 2,003 transaction re-identifications that in total amount to more than $137M, and affect 348 Brazilian companies. Further, since similar statistics are produced by other statistical agencies, our attack is of broader concern.

Information Leakage Games: Exploring Information as a Utility Function

A common goal in the areas of secure information flow and privacy is to build effective defenses against unwanted leakage of information. To this end, one must be able to reason about potential attacks and their interplay with possible defenses. In this article, we propose a game-theoretic framework to formalize strategies of attacker and defender in the context of information leakage, and provide a basis for developing optimal defense methods. A novelty of our games is that their utility is given by information leakage, which in some cases may behave in a non-linear way. This causes a significant deviation from classic game theory, in which utility functions are linear with respect to players’ strategies. Hence, a key contribution of this work is the establishment of the foundations of information leakage games. We consider two kinds of games, depending on the notion of leakage considered. The first kind, the QIF -games , is tailored for the theory of quantitative information flow. The second one, the DP -games , corresponds to differential privacy.

Refinement Orders for Quantitative Information Flow and Differential Privacy

Quantitative Information Flow (QIF) and Differential Privacy (DP) are both concerned with the protection of sensitive information, but they are rather different approaches. In particular, QIF considers the expected probability of a successful attack, while DP (in both its standard and local versions) is a max-case measure, in the sense that it is compromised by the existence of a possible attack, regardless of its probability. Comparing systems is a fundamental task in these areas: one wishes to guarantee that replacing a system A by a system B is a safe operation that is the privacy of B is no worse than that of A. In QIF, a refinement order provides strong such guarantees, while, in DP, mechanisms are typically compared w.r.t. the privacy parameter ε in their definition. In this paper, we explore a variety of refinement orders, inspired by the one of QIF, providing precise guarantees for max-case leakage. We study simple structural ways of characterising them, the relation between them, efficient methods for verifying them and their lattice properties. Moreover, we apply these orders in the task of comparing DP mechanisms, raising the question of whether the order based on ε provides strong privacy guarantees. We show that, while it is often the case for mechanisms of the same “family” (geometric, randomised response, etc.), it rarely holds across different families.

A multi-flow information flow tracking approach for proving quantitative hardware security properties

Information Flow Tracking (IFT) is an established formal method for proving security properties related to confidentiality, integrity, and isolation. It has seen promise in identifying security vulnerabilities resulting from design flaws, timing channels, and hardware Trojans for secure hardware design. However, existing IFT methods tend to take a qualitative approach and only enforce binary security properties, requiring strict non-interference for the properties to hold while real systems usually allow a small amount of information flows to enable desirable interactions. Consequently, existing methods are inadequate for reasoning about quantitative security properties or measuring the security of a design in order to assess the severity of a security vulnerability. In this work, we propose two multi-flow solutions - multiple verifications for replicating existing IFT model and multi-flow IFT method. The proposed multi-flow IFT method provides more insight into simultaneous information flow behaviors and allows for proof of quantitative information flow security properties, such as diffusion, randomization, and boundaries on the amount of simultaneous information flows. Experimental results show that our method can be used to prove a new type of information flow security property with verification performance benefits.

ՏԵՂԵԿԱՏՎԱԿԱՆ ՔԱՂԱՔԱԿՐԹՈՒԹՅԱՆ ՄԱՐԴԱԲԱՆԱԿԱՆ ՀԵՏԵՎԱՆՔՆԵՐԸ/THE ANTHROPLOGICAL CONSEQUNECES OF INFORMATION CIVILIZATION/ АНТРОПОЛОГИЧЕСКИЕ ПОСЛЕДСТВИЯ ИНФОРМАЦИОННОЙ ЦИВИЛИЗАЦИИ

The article discusses the modern trends of information civilization and the deep reestablishment of quantitative information flow. The analysis is based on the theoretical and practical conclusions made by prominent researchers of the field. It is worth to mention that those conclusions are somewhat biased and as a matter of fact the article attempts to solve that issue as well.

On the Security-Privacy Tradeoff in Collaborative Security: A Quantitative Information Flow Game Perspective

To contest the rapidly developing cyber-attacks, numerous collaborative security schemes, in which multiple security entities can exchange their observations and other relevant data to achieve more effective security decisions, are proposed and developed in the literature. However, the security-related information shared among the security entities may contain some sensitive information and such information exchange can raise privacy concerns, especially when these entities belong to different organizations. With such consideration, the interplay between the attacker and the collaborative entities is formulated as Quantitative Information Flow (QIF) games, in which the QIF theory is adapted to measure the collaboration gain and the privacy loss of the entities in the information sharing process. In particular, three games are considered, each corresponding to one possible scenario of interest in practice. Based on the game-theoretic analysis, the expected behaviors of both the attacker and the security entities are obtained. In addition, the simulation results are presented to validate the analysis.

Information leakage of multi-threaded programs

Quantitative information flow is an important technique for measuring information leakage of a program. It is widely used in analyzing anonymity protocols and timing channels. One area of interest is to quantify the information leakage of multi-threaded programs, which has not been well-studied in prior work. In this paper, an automated trace-based approach is proposed to precisely quantify information leakage of shared-memory multi-threaded programs. The approach takes into account the effect of schedulers and leakage in intermediate states of executions. The programs are modeled by Markovian processes. Then, variants of information leakage, including expected, bounded time, maximum, and minimum leakages are measured. The validity of the approach is demonstrated by implementing it in a tool PRISM-Leak, which is built upon PRISM, a probabilistic model checker. Finally, two case studies are utilized to analyze and compare the approach against state-of-the-art leakage quantification tools.

Secure Service Composition with Quantitative Information Flow Evaluation in Mobile Computing Environments.

The advances in mobile technologies enable mobile devices to cooperate with each other to perform complex tasks to satisfy users’ composite service requirements. However, data with different sensitivities and heterogeneous systems with diverse security policies pose a great challenge on information flow security during the service composition across multiple mobile devices. The qualitative information flow control mechanism based on non-interference provides a solid security assurance on the propagation of customer’s private data across multiple service participants. However, strict discipline limits the service availability and may cause a high failure rate on service composition. Therefore, we propose a distributed quantitative information flow evaluation approach for service composition across multiple devices in mobile environments. The quantitative approach provides us a more precise way to evaluate the leakage and supports the customized disciplines on information flow security for the diverse requirements of different customers. Considering the limited energy feature on mobile devices, we use a distributed evaluation approach to provide a better balance on consumption on each service participant. Through the experiments and evaluations, the results indicate that our approach can improve the availability of composite service effectively while the security can be ensured.

Program algebra for quantitative information flow

Models for quantitative information flow traditionally assume that the secret, once set, never changes. More recently, however, Hidden Markov Models (HMM's) have been used to describe program features that include both state updates and information flow, thus supporting more realistic contexts where secrets can indeed be refreshed.In this paper we explore HMM's further, with the aim of bringing algebraic concepts to bear in the analysis of confidentiality properties of programs. Of particular importance is the idea that local reasoning about program fragments should remain sound even when those same fragments are executed within a larger system. We show how to extend the basic HMM model to incorporate this core idea within an algebraic setting and, in so doing, show how it is related to established notion about privacy and correlated data sets in statistical databases.Using our algebra for an HMM-style model we show how to describe and prove some foundational properties of quantitative information flow.

Quantifying Dynamic Leakage - Complexity Analysis and Model Counting-based Calculation -

A program is non-interferent if it leaks no secret information to an observable output. However, non-interference is too strict in many practical cases and quantitative information flow (QIF) has been proposed and studied in depth. Originally, QIF is defined as the average of leakage amount of secret information over all executions of a program. However, a vulnerable program that has executions leaking the whole secret but has the small average leakage could be considered as secure. This counter-intuition raises a need for a new definition of information leakage of a particular run, i.e., dynamic leakage. As discussed in [5], entropy-based definitions do not work well for quantifying information leakage dynamically; Belief-based definition on the other hand is appropriate for deterministic programs, however, it is not appropriate for probabilistic ones. In this paper, we propose new simple notions of dynamic leakage based on entropy which are compatible with existing QIF definitions for deterministic programs, and yet reasonable for probabilistic programs in the sense of [5]. We also investigated the complexity of computing the proposed dynamic leakage for three classes of Boolean programs. We also implemented a tool for QIF calculation using model counting tools for Boolean formulae. Experimental results on popular benchmarks of QIF research show the flexibility of our framework. Finally, we discuss the improvement of performance and scalability of the proposed method as well as an extension to more general cases.

An Information Analysis of Iterative Algorithms for Network Utility Maximization and Strategic Games

A variety of resource allocation problems on networked systems, for example, those in cyber-physical systems or Internet-of-things applications, require distributed solution methods. Modern distributed algorithms usually require bandwidth-limited digital communication between the system and its users, who are often modeled as independent decision makers with individual preferences. This paper presents a quantitative information flow and knowledge gain analysis of decentralized iterative algorithms with bounded trajectories in the context of convex network utility maximization problems and strategic games with a unique Nash equilibrium solution. First, a novel generic framework is introduced to quantify knowledge gain in network resource allocation problems using entropy by taking into account priors in the solution space. Second, a general result is presented on the interplay between quantization of information and distributed algorithm performance both for linear and sublinear convergence. Third, information flow in distributed algorithms is studied and a lower bound is derived on the total amount of information exchanged for convergence under uniform quantization. The well-known primal-dual decomposition algorithm is used as an example to illustrate the results. Finally, convergence guarantees for distributed algorithms with estimation are investigated. This paper establishes specific links between information concepts and iterative algorithms in addition to building a foundation for integrating learning schemes into distributed optimization.

An axiomatization of information flow measures

Quantitative information flow aims to assess and control the leakage of sensitive information by computer systems. A key insight in this area is that no single leakage measure is appropriate in all operational scenarios; as a result, many leakage measures have been proposed, with many different properties. To clarify this complex situation, this paper studies information leakage axiomatically, showing important dependencies among different axioms. It also establishes a completeness result about the g-leakage family, showing that any leakage measure satisfying certain intuitively-reasonable properties can be expressed as a g-leakage.

A Game-Theoretic Approach to Information-Flow Control via Protocol Composition.

In the inference attacks studied in Quantitative Information Flow (QIF), the attacker typically tries to interfere with the system in the attempt to increase its leakage of secret information. The defender, on the other hand, typically tries to decrease leakage by introducing some controlled noise. This noise introduction can be modeled as a type of protocol composition, i.e., a probabilistic choice among different protocols, and its effect on the amount of leakage depends heavily on whether or not this choice is visible to the attacker. In this work, we consider operators for modeling visible and hidden choice in protocol composition, and we study their algebraic properties. We then formalize the interplay between defender and attacker in a game-theoretic framework adapted to the specific issues of QIF, where the payoff is information leakage. We consider various kinds of leakage games, depending on whether players act simultaneously or sequentially, and on whether or not the choices of the defender are visible to the attacker. In the case of sequential games, the choice of the second player is generally a function of the choice of the first player, and his/her probabilistic choice can be either over the possible functions (mixed strategy) or it can be on the result of the function (behavioral strategy). We show that when the attacker moves first in a sequential game with a hidden choice, then behavioral strategies are more advantageous for the defender than mixed strategies. This contrasts with the standard game theory, where the two types of strategies are equivalent. Finally, we establish a hierarchy of these games in terms of their information leakage and provide methods for finding optimal strategies (at the points of equilibrium) for both attacker and defender in the various cases.