Average-case Analysis Research Articles

Average Case Analysis of Multichannel Sparse Recovery Using Convex Relaxation

This paper considers recovery of jointly sparse multichannel signals from incomplete measurements. Several approaches have been developed to recover the unknown sparse vectors from the given observations, including thresholding, simultaneous orthogonal matching pursuit (SOMP), and convex relaxation based on a mixed matrix norm. Typically, worst case analysis is carried out in order to analyze conditions under which the algorithms are able to recover any jointly sparse set of vectors. However, such an approach is not able to provide insights into why joint sparse recovery is superior to applying standard sparse reconstruction methods to each channel individually. Previous work considered an average case analysis of thresholding and SOMP by imposing a probability model on the measured signals. Here, the main focus is on analysis of convex relaxation techniques. In particular, the mixed <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> approach to multichannel recovery is investigated. Under a very mild condition on the sparsity and on the dictionary characteristics, measured for example by the coherence, it is shown that the probability of recovery failure decays exponentially in the number of channels. This demonstrates that most of the time, multichannel sparse recovery is indeed superior to single channel methods. The probability bounds are valid and meaningful even for a small number of signals. Using the tools developed to analyze the convex relaxation technique, also previous bounds for thresholding and SOMP recovery are tightened.

An Improved Version of Cuckoo Hashing: Average Case Analysis of Construction Cost and Search Operations

Cuckoo hashing is a hash table data structure introduced by Pagh and Rodler, that offers constant worst case search time. As a major contribution of this paper, we analyze modified versions of this algorithm with improved performance. Further, we provide an asymptotic analysis of the search costs of all these variants of cuckoo hashing and compare these results with the well known properties of double hashing and linear probing. The analysis is supported by numerical results. Finally, our analysis shows, that the expected number of steps of search operations can be reduced by using a modified version of cuckoo hashing instead of standard algorithms based on open addressing.

The relevant prefixes of coloured Motzkin walks: An average case analysis

In this paper we study some relevant prefixes of coloured Motzkin walks (otherwise called coloured Motzkin words). In these walks, the three kinds of step can have α , β and γ colours, respectively. In particular, when α = β = γ = 1 we have the classical Motzkin walks while for α = γ = 1 and β = 0 we find the well-known Dyck walks. By using the concept of Riordan arrays and probability generating functions we find the average length of the relevant prefix in a walk of length n and the corresponding variance in terms of α , β and γ . This result is interesting from a combinatorial point of view and also provides an average case analysis of algorithms related to the problem of ranking and generating uniformly at random the coloured Motzkin words.

Maximum likelihood analysis of algorithms and data structures

We present a new approach for an average-case analysis of algorithms and data structures that supports a non-uniform distribution of the inputs and is based on the maximum likelihood training of stochastic grammars. The approach is exemplified by an analysis of the expected size of binary tries as well as by three sorting algorithms and it is compared to the known results that were obtained by traditional techniques. Investigating traditional settings like the random permutation model, we rediscover well-known results formerly derived by pure analytic methods; changing to biased data yields original results. All but one step of our analysis can be automated on top of a computer-algebra system. Thus our new approach can reduce the effort required for an average-case analysis, allowing for the consideration of realistic input distributions with unknown distribution functions at the same time. As a by-product, our approach yields an easy way to generate random combinatorial objects according to various probability distributions.

Average-case analysis of incremental topological ordering

Many applications like pointer analysis and incremental compilation require maintaining a topological ordering of the nodes of a directed acyclic graph (DAG) under dynamic updates. All known algorithms for this problem are either only analyzed for worst-case insertion sequences or only evaluated experimentally on random DAGs. We present the first average-case analysis of incremental topological ordering algorithms. We prove an expected runtime of O ( n 2 polylog ( n ) ) under insertion of the edges of a complete DAG in a random order for the algorithms of Alpern et al. (1990) [4], Katriel and Bodlaender (2006) [18], and Pearce and Kelly (2006) [23].

Why Almost All k-Colorable Graphs Are Easy to Color

Coloring a k-colorable graph using k colors (k≥3) is a notoriously hard problem. Considering average case analysis allows for better results. In this work we consider the uniform distribution over k-colorable graphs with n vertices and exactly cn edges, c greater than some sufficiently large constant. We rigorously show that all proper k-colorings of most such graphs lie in a single “cluster”, and agree on all but a small, though constant, portion of the vertices. We also describe a polynomial time algorithm that whp finds a proper k-coloring of such a random k-colorable graph, thus asserting that most such graphs are easy to color. This should be contrasted with the setting of very sparse random graphs (which are k-colorable whp), where experimental results show some regime of edge density to be difficult for many coloring heuristics.

Average-case analysis of some plurality algorithms

Given a set of n elements, each of which is colored one of c colors, we must determine an element of the plurality (most frequently occurring) color by pairwise equal/unequal color comparisons of elements. We focus on the expected number of color comparisons when the c n colorings are equally probable. We analyze an obvious algorithm, showing that its expected performance is c 2 + c − 2/2 c n − O ( c 2 ), with variance Θ( c 2 n ). We present and analyze an algorithm for the case c = 3 colors whose average complexity on the 3 n equally probable inputs is 7083/5425 n + O (√ n ) = 1.3056… n + O (√ n ), substantially better than the expected complexity 5/3 n + O (1) = 1.6666… n + O (1) of the obvious algorithm. We describe a similar algorithm for c =4 colors whose average complexity on the 4 n equally probable inputs is 761311/402850 n + O (log n ) = 1.8898… n + O (log n ), substantially better than the expected complexity 9/4 n + O (1) = 2.25 n + O (1) of the obvious algorithm.

Formula omitted]; unlocking the potential of compositional static average-case analysis

Compositionality is the “golden key” to static analysis and plays a central role in static worst-case time analysis. We show that compositionality, combined with the capacity for tracking data distributions, unlocks a useful novel technique for average-case analysis.The applicability of the technique has been demonstrated via the static average-case analysis tool DISTRI. The tool automatically extracts average-case time from source code of programs implemented in the novel programming language MOQA1MOdular Quantitative Analysis.1. MOQA enables the prediction of the average number of basic steps performed in a computation, paving the way for static analysis of complexity measures such as average time or average power use. MOQA has as a unique feature a guaranteed average-case timing compositionality.The compositionality property brings a strong advantage for the programmer. The capacity to combine parts of code, where the average-time is simply the sum of the times of the parts, is a very helpful advantage in static analysis, something which is not available in current languages. Moreover, re-use is a key factor in the MOQA approach: once the average time is determined for a piece of code, then this time will hold in any context. Hence it can be re-used and the timing impact is always the same. Compositionality also improves precision of static average-case analysis, supporting the determination of accurate estimates on the average number of basic operations of MOQA programs.The MOQA “language” essentially consists of a suite of data-structuring operations together with conditionals, for-loops and recursion. As such MOQA can be incorporated in any traditional programming language, importing all of its benefits in a familiar context2MOQA is implemented at CEOL in Java 5.0 as MOQA-java.2.Compositionality for average-case is subtle and one may easily be tempted to conclude that compositionality “comes for free”. For genuine compositional reasoning however, one needs to be able to track data and their distribution throughout computations; a non-trivial problem. The lack of an efficient method to track distributions has plagued all prior static average-case analysis approaches. We show how MOQA enables the finitary representation and tracking of the distribution of data states throughout computations. This enables one to unlock the true potential of compositional reasoning. Links with reversible computing are discussed. The highly visual aspect of this novel and unified approach to the Analysis of Algorithms also has a pedagogical advantage, providing students with useful insights in the nature of algorithms and their analysis.

A Random Bag Preserving Product Operation

The author's current research programme is the development of a modular calculus for the average-cost of data structuring. This modular calculus provides a novel foundation for the analysis of algorithms. Its applicability to the analysis of algorithms has been demonstrated at the Center for Efficiency-Oriented Languages (CEOL) through the design of the novel programming language MOQA and the associated average-case analysis tool DISTRI-TRACK [M. Schellekens, D. Hickey and G. Bollella, ACETT, a Linearly-Compositional Programming Language for (semi-)automated Average-Case analysis, IEEE Real-Time Systems Symposium - Work In Progress Session, 2004; M. Boubekeur, D. Hickey, J. Mc Enery and M. Schellekens, A new Approach for Modular Average-Case Timing of Real-Time Java Programs, Accepted for publication in WSEAS Transactions on Computers, 2007; M. Boubekeur, D. Hickey, J. Mc Enery and M. Schellekens, Towards Modular Average-Case Timing in Real-Time Languages: An Application to Real-Time Java, Accepted for publication on the 6th WSEAS International Conference on APPLIED COMPUTER SCIENCE (ACS'06), Tenerife, December, 2006; M. Schellekens, A Modular Calculus for the Average Cost of Data Structuring, Springer book to appear, 250 pages, May 2008 (to appear), M. Schellekens, Compositional Average-Case Analysis, preprint, under review, 2006]. Modular computations of the average cost of data structuring are possible through the fundamental notion of random bag preservation. Random bag preserving operations enable the constructive tracking of the data and the distribution of the data states during a MOQA computation. This in turn enables the (semi-)automated derivation of the average cost of the operations. Two fundamental MOQA operations enable the creation and destruction of data structures: the MOQA product operation, which is the subject of this paper, and the MOQA delete operation, which forms the subject of [M. Schellekens, Compositional Average-Case Analysis, preprint, under review, 2006]. The introduction of the entire MOQA language is well beyond the scope of this paper and will be reported in a book [M. Schellekens, A Modular Calculus for the Average Cost of Data Structuring, Springer book to appear, 250 pages, May 2008 (to appear)]. The language has been implemented at CEOL and automated derivations of average-cost of data structuring are under way. Here we report on a (simplified) version of the fundamental notion of random bag preservation and demonstrate that the central MOQA product operation possesses this crucial property.

Functionally-Generalised MOQA Operations

The programming language MOQA was designed by Michel Schellekens in [Schellekens M.: Compositional Average-Case Analysis, Preprint, submitted to the Journal of ACM (2006), 78p.; M.P. Schellekens, A Modular Calculus for the Average Cost of Data Structuring, Springer book to appear (2008), 250 pp.] specifically to facilitate the average execution time analysis of its programs. MOQA is based on a special data structure and an associated suite of operations. This special data structure, said here as labeled partial orders (lpo), is a pair (P,ℓ) such that P=(X,⊑) is a poset and ℓ:X↦L a strictly increasing bijection between X and a totally ordered set L. MOQA basic operations are defined on lpo's under some restrictions. These restrictions guarantee the main property of MOQA's operations, said here as Automatic Random Preservation. Random Preservation (RP) is a fundamental property used to calculate the average time of programs, opening the way to a (semi-)automation of this calculation in MOQA.In the paper, we introduce functionally-generalised versions of some MOQA basic operations. Those versions are defined with more natural restrictions and in a more general way than the correspondent MOQA operations. As a consequence of their greater generality, they are not RP on all their domains in contrast to MOQA operations. Nevertheless, they have two good properties. The first one is that they have the intended semantics. For instance, the generalised split actually splits any lpo into two parts using a label as a pivot. The second one is that the subdomains on which they are RP are strictly bigger than the domains where MOQA basic operations are defined and RP. A tractable characterisation of those RP subdomains is an open problem and its solution would be an important step toward an implementation of the functionally-generalised operations as Automatic RP operations.

Atoms of All Channels, Unite! Average Case Analysis of Multi-Channel Sparse Recovery Using Greedy Algorithms

This paper provides new results on computing simultaneous sparse approximations of multichannel signals over redundant dictionaries using two greedy algorithms. The first one, p-thresholding, selects the S atoms that have the largest p-correlation while the second one, p-simultaneous matching pursuit (p-SOMP), is a generalisation of an algorithm studied by Tropp in (Signal Process. 86:572–588, 2006). We first provide exact recovery conditions as well as worst case analyses of all algorithms. The results, expressed using the standard cumulative coherence, are very reminiscent of the single channel case and, in particular, impose stringent restrictions on the dictionary. We unlock the situation by performing an average case analysis of both algorithms. First, we set up a general probabilistic signal model in which the coefficients of the atoms are drawn at random from the standard Gaussian distribution. Second, we show that under this model, and with mild conditions on the coherence, the probability that p-thresholding and p-SOMP fail to recover the correct components is overwhelmingly small and gets smaller as the number of channels increases. Furthermore, we analyse the influence of selecting the set of correct atoms at random. We show that, if the dictionary satisfies a uniform uncertainty principle (Candes and Tao, IEEE Trans. Inf. Theory, 52(12):5406–5425, 2006), the probability that simultaneous OMP fails to recover any sufficiently sparse set of atoms gets increasingly smaller as the number of channels increases.

TOWARDS POSSIBLE NON-EXTENSIVE THERMODYNAMICS OF ALGORITHMIC PROCESSING — STATISTICAL MECHANICS OF INSERTION SORT ALGORITHM

Tsallis entropy introduced in 1988 is considered to have obtained new possibilities to construct generalized thermodynamical basis for statistical physics expanding classical Boltzmann–Gibbs thermodynamics for nonequilibrium states. During the last two decades this q-generalized theory has been successfully applied to considerable amount of physically interesting complex phenomena. The authors would like to present a new view on the problem of algorithms computational complexity analysis by the example of the possible thermodynamical basis of the sorting process and its dynamical behavior. A classical approach to the analysis of the amount of resources needed for algorithmic computation is based on the assumption that the contact between the algorithm and the input data stream is a simple system, because only the worst-case time complexity is considered to minimize the dependency on specific instances. Meanwhile the article shows that this process can be governed by long-range dependencies with thermodynamical basis expressed by the specific shapes of probability distributions. The classical approach does not allow to describe all properties of processes (especially the dynamical behavior of algorithms) that can appear during the computer algorithmic processing even if one takes into account the average case analysis in computational complexity. The importance of this problem is still neglected especially if one realizes two important things. The first one: nowadays computer systems work also in an interactive mode and for better understanding of its possible behavior one needs a proper thermodynamical basis. The second one: computers from mathematical point of view are Turing machines but in reality they have physical implementations that need energy for processing and the problem of entropy production appears. That is why the thermodynamical analysis of the possible behavior of the simple insertion sort algorithm will be given here.

Average-Case Analysis of Cousins in m-ary Tries

We investigate the average similarity of random strings as captured by the average number of ‘cousins’ in the underlying tree structures. Analytical techniques including poissonization and the Mellin transform are used for accurate calculation of the mean. The string alphabets we consider arem-ary, and the corresponding trees arem-ary trees. Certain analytic issues arise in them-ary case that do not have an analog in the binary case.

Regularization independent of the noise level: an analysis of quasi-optimality

The quasi-optimality criterion chooses the regularization parameter in inverse problems without taking into account the noise level. This rule works remarkably well in practice, although Bakushinskii has shown that there are always counterexamples with very poor performance. We propose an average case analysis of quasi-optimality for spectral cut-off estimators (also known as truncated singular value decomposition, TSVD) and we prove that the quasi-optimality criterion determines estimators which are rate-optimal on average. Its practical performance is illustrated with a calibration problem from mathematical finance.

Optimization of Group Key Management Structure with a Client Join-Leave Mechanism

Many group key management schemes that reduce the total communication cost and/or the computational cost imposed on client devices have been proposed. However, optimizations of the key-management structure have not been studied. This paper proposes ways to optimize the key-management structure in a hybrid group key management scheme. The proposed method is able to minimize both the total communication cost and the computational cost imposed on client devices. First, we propose a probabilistic client join/leave model in order to evaluate the communication and computational costs of group key management schemes. This model idealizes client actions generally and considers the existence of the peaks of the joining/leaving frequency. Thus, we can analyze not only the average case scenario but also the worst case scenario using this model. Then, we formalize the total computation cost and the computational cost imposed on client devices in group key management schemes under the model. We present both an average case analysis and a worst case analysis. Finally, we show the parameters that minimize the total communication cost and the computational cost imposed on clients under the model. Our results should be useful in designing a secure group communication system for large and dynamic groups.

Random-order bin packing

The average-case analysis of algorithms usually assumes independent, identical distributions for the inputs. In [C. Kenyon, Best-fit bin-packing with random order, in: Proc. of the Seventh Annual ACM-SIAM Symposium on Discrete Algorithms, SIAM, 1996, pp. 359–364] Kenyon introduced the random-order ratio, a new average-case performance metric for bin packing heuristics, and gave upper and lower bounds for it for the Best Fit heuristics. We introduce an alternative definition of the random-order ratio and show that the two definitions give the same result for Next Fit. We also show that the random-order ratio of Next Fit equals to its asymptotic worst-case, i.e., it is 2.

Average Case Analysis of Bounded Space Bin Packing Algorithms

We consider the one-dimensional bin packing problem and analyze the average case performance of bounded space algorithms. The analysis covers a wide variety of bin packing algorithms including Next-K Fit, K-Bounded Best Fit and Harmonic algorithms, as well as of others. We assume discrete item sizes, an assumption which is true in most real-world applications of bin packing. We present a Markov chains method which is general enough to calculate results for any i.i.d. discrete item size distribution. The paper presents many results heretofore unknown or conjectured from simulation. Some of the results are surprising as they differ considerably from results for continuous distributions.

Average Case Analysis of Disk Scheduling, Increasing Subsequences and Spacetime Geometry

We consider the problem of estimating the average tour length of the asymmetric TSP arising from the disk scheduling problem with a linear seek function and a probability distribution on the location of I/O requests. We provide a law of large numbers expression, for the expected tour size and compute it explicitely in the most interesting cases. We also provide finer asymptotic estimates in the case of a uniform request distribution. Our methods relate disk scheduling to the problem of finding the longest increasing subsequence in a permutation. Our tour length estimates also answer questions on the asymptotics of the connect-the dots problem with Lipschitz functions which was raised in Adv. Appl. Probab. 37, 571–603 (2005). In the last section we show that all the above problems can be interpreted naturally, in terms of Lorentzian geometry. We generalize the main results to this setting.

Worst-case and average-case analysis of n -detection test sets and test generation strategies

Test sets that detect each target fault n times (n-detection test sets) are typically generated for restricted values of n because of the increase in test set size with n. Both a worst-case analysis and an average-case analysis is performed to investigate the effects of restricting n on the unmodelled fault coverage of an (arbitrary) n-detection test set of a full-scan circuit. The analysis is independent of any particular test set or any particular test generation procedure. It is based on a specific set of target faults and a specific set of untargeted faults. It shows that, depending on the circuit, very large values of n may be needed to guarantee the detection of all the untargeted faults. The implications of these results are discussed and it is also demonstrated that the proposed analysis methods can be used to evaluate the effects of incorporating into the n-detection test generation procedure specific strategies aimed at improving the n-detection test set quality.

Average case analysis for tree labelling schemes

We study how to label the vertices of a tree in such a way that we can decide the distance of two vertices in the tree given only their labels. Gavoille et al. proved that for any such distance labelling scheme, the maximum label length is at least 1 8 log 2 n − O ( log n ) bits, where n is the number of vertices in the input tree T . They also gave a separator-based labelling scheme that has the optimal label length Θ ( log n ⋅ log ( H n ( T ) ) ) , where H n ( T ) is the height of T . We present two distance labelling schemes, namely, the backbone-based scheme and rake-based scheme, which also achieve the optimal label length. The two schemes always perform at least as well as the separator scheme. Furthermore, the rake-based scheme has a much smaller expected label length under certain tree distributions. With these new schemes, we also can find the least common ancestor of any two vertices based on their labels only.