Johnson Lindenstrauss Research Articles

Restricted Isometry Property of Gaussian Random Projection for Finite Set of Subspaces

Dimension reduction plays an essential role when decreasing the complexity of solving large-scale problems. The well-known Johnson-Lindenstrauss (JL) Lemma and Restricted Isometry Property (RIP) admit the use of random projection to reduce the dimension while keeping the Euclidean distance, which leads to the boom of Compressed Sensing and the field of sparsity related signal processing. Recently, successful applications of sparse models in computer vision and machine learning have increasingly hinted that the underlying structure of high dimensional data looks more like a union of subspaces (UoS). In this paper, motivated by JL Lemma and an emerging field of Compressed Subspace Clustering (CSC), we study for the first time the RIP of Gaussian random matrices for the compression of two subspaces based on the generalized projection $F$-norm distance. We theoretically prove that with high probability the affinity or distance between two projected subspaces are concentrated around their estimates. When the ambient dimension after projection is sufficiently large, the affinity and distance between two subspaces almost remain unchanged after random projection. Numerical experiments verify the theoretical work.

Microstructure under the Microscope: Tools to Survive and Thrive in the Age of (Too Much) Information

Market Microstructure is the investigation of the process and protocols that govern the exchange of assets with the objective of reducing frictions that can impede the transfer. In financial markets, where there is an abundance of recorded information, this translates to the study of the dynamic relationships between observed variables, such as price, volume and spread, and hidden constituents, such as transaction costs and volatility, that hold sway over the efficient functioning of the system. "My dear, here we must process as much data as we can, just to stay in business. And if you wish to make a profit you must process at least twice as much data." - Red Queen to Alice in Hedge-Fund-Land. In this age of (Too Much) Information, it is imperative to uncover nuggets of knowledge (signal) from buckets of nonsense (noise). To aid in this effort to extract meaning from chaos and to gain a better understanding of the relationships between financial variables, we summarize the application of the theoretical results from (Kashyap 2016b) to microstructure studies. The central concept rests on a novel methodology based on the marriage between the Bhattacharyya distance, a measure of similarity across distributions, and the Johnson Lindenstrauss Lemma, a technique for dimension reduction, providing us with a simple yet powerful tool that allows comparisons between data-sets representing any two distributions. We provide an empirical illustration using prices, volumes and volatilities across seven countries and three different continents. The degree to which different markets or sub groups of securities have different measures of their corresponding distributions tells us the extent to which they are different. This can aid investors looking for diversification or looking for more of the same thing.

Microstructure Under the Microscope: Tools to Survive and Thrive in the Age of (Too Much) Information (Presentation Slides)

Market Microstructure is the investigation of the process and protocols that govern the exchange of assets with the objective of reducing frictions that can impede the transfer. In financial markets, where there is an abundance of recorded information, this translates to the study of the dynamic relationships between observed variables, such as price, volume and spread, and hidden constituents, such as transaction costs and volatility, that hold sway over the efficient functioning of the system. “My dear, here we must process as much data as we can, just to stay in business. And if you wish to make a profit you must process at least twice as much data.” - Red Queen to Alice in Hedge-Fund-Land. Necessity is the mother of all invention/creation/innovation, but the often forgotten father is frustration. In this age of (Too Much) Information, it is imperative to uncover nuggets of knowledge (signal) from buckets of nonsense (noise). To aid in this effort to extract meaning from chaos and to gain a better understanding of the relationships between financial variables, we summarize the application of the theoretical results from (Kashyap 2016b) to microstructure studies. The central concept rests on a novel methodology based on the marriage between the Bhattacharyya distance, a measure of similarity across distributions, and the Johnson Lindenstrauss Lemma, a technique for dimension reduction, providing us with a simple yet powerful tool that allows comparisons between data-sets representing any two distributions. We provide an empirical illustration using prices, volumes and volatilities across seven countries and three different continents. The degree to which different markets or sub groups of securities have different measures of their corresponding distributions tells us the extent to which they are different. This can aid investors looking for diversification or looking for more of the same thing. In Indian mythology, it is believed that in each era, God takes on an avatar or reincarnation to fight the main source of evil in that epoch and to restore the balance between good and bad. In this age of too much information and complexity, perhaps the supreme being needs to be born as a data scientist, conceivably with an apt nickname, the Infoman. Until higher powers intervene and provide the ultimate solution to completely eliminate information overload, we have to make do with marginal methods, such as this composition, to reduce information. As we wait for the perfect solution, it is worth meditating upon what superior beings would do when faced with a complex situation, such as the one we are in. It is said that the Universe is but the Brahma's (Creator's) dream. Research (Effort/Struggle) can help us understand this world; Sleep (Ease/Peace of Mind) can help us create our own world. A lesson from close by and down under: We need to “Do Some Yoga and Sleep Like A Koala”.

Combining Dimension Reduction, Distance Measures and Covariance (Presentation Slides)

We consider a measure of similarity, the Bhattacharyya distance, across distributions of random variables. We develop a novel methodology based on the marriage between the Bhattacharyya distance and the Johnson Lindenstrauss Lemma, a technique for dimension reduction, providing us with a simple yet powerful tool that allows comparisons between data-sets representing any two distributions. We demonstrate a relationship between covariance and distance measures based on a generic extension of Stein’s Lemma. The degree to which different markets or sub groups of securities have different measures of their corresponding distributions tells us the extent to which they are different. This can aid investors looking for diversification or looking for more of the same thing. We consider an asset pricing application and then briefly discuss how this methodology lends itself to numerous Marketstructure studies and even applications outside the realm of finance / social sciences by illustrating a biological application.

Microstructure under the Microscope

Market Microstructure is the investigation of the process and protocols that govern the exchange of assets with the objective of reducing frictions that can impede the transfer. In financial markets, where there is an abundance of recorded information, this translates to the study of the dynamic relationships between observed variables, such as price, volume and spread, and hidden constituents, such as transaction costs and volatility, that hold sway over the efficient functioning of the system.We consider a measure of similarity, the Bhattacharyya distance, across distributions of these variables. We illustrate a novel methodology based on the marriage between the Bhattacharyya distance and the Johnson Lindenstrauss Lemma, a technique for dimension reduction, providing us with a simple yet powerful tool that allows comparisons between data-sets representing any two distributions. We demonstrate a relationship between covariance and distance measures based on a generic extension of Stein's Lemma. The degree to which different markets or sub groups of securities have different measures of their corresponding distributions tells us the extent to which they are different. This can aid investors looking for diversification or looking for more of the same thing. We consider an asset pricing application and then briefly discuss how this methodology lends itself to numerous Marketstructure studies and even applications outside the realm of finance / social sciences.

Random Projection Estimation of Discrete-Choice Models with Large Choice Sets

We introduce random projection, an important dimension-reduction tool from machine learning, for the estimation of aggregate discrete-choice models with high-dimensional choice sets. Initially, high-dimensional data are projected into a lower-dimensional Euclidean space using random projections. Subsequently, estimation proceeds using cyclical monotonicity moment inequalities implied by the multinomial choice model; the estimation procedure is semiparametric and does not require explicit distributional assumptions to be made regarding the random utility errors. Our procedure is justified via the Johnson–Lindenstrauss lemma—the pairwise distances between data points are preserved through random projections. The estimator works well in simulations and in an application to a supermarket scanner data set. This paper was accepted by Juanjuan Zhang, marketing.

Notes on Uncertainty, Unintended Consequences and Everything Else

The following collection of essays are linked by one theme, uncertainty. The dynamic nature of financial markets and trading, as with the rest of social sciences, where changes can be observed and decisions can be taken by participants to influence the system, means that along with better models and predictive technologies, predictions need to be continuously revised; and yet unintended consequences set in. While a detailed axiomatic approach to uncertainty and unintended consequences is postponed for another time (or perhaps another lifetime), the present assortment has the following elements:1. Dynamic Multi-Factor Bid-Offer Adjustment ModelWe develop a simple model to incorporate a feedback loop into a financial system and review other situations where such an approach might be effective. This sets the stage for the next parts by illustrating how efforts at dealing with uncertainty can be improved by feedback loops or a continuous review of the system coupled with continued corrective action. The test results show how this can be immediately relevant to a real life setting.2. David vs Goliath (You against the Markets)We develop a set of stochastic dynamic programming models that seek to separate the effect of one's actions in a financial market from the actions of other players. Such a separation brings about a reduction in complexity since participants can better plan their course of action based on a greater focus on the set of controls and states that hold greater significance for their destiny while being cognizant of the implications of the actions of the rest of the participants.3. A Tale of Two Consequences We apply the methodology from part [enu:David-vs-Goliath] to a recent event in financial markets (TOPIX 100 tick size changes on the Tokyo Stock Exchange in 2014) and illustrate how the ability to understand the impact of one's actions in isolation to the actions of others can perhaps lead to more insightful results.4. Hong Kong - Shanghai Connect/Hong Kong - Beijing Disconnect (?) This case study applies the methodology from part [enu:David-vs-Goliath] to another recent event in financial markets (the opening of the Hong Kong Shanghai Connect, connecting the stock exchanges in the two cities in 2014) and illustrates a comparative approach at studying complex systems that perhaps can be one way to overcome a seemingly unsurmountable obstacle, that is, the lack of data.5. Microstructure under the MicroscopeWe consider a measure of similarity, the Bhattacharyya distance, across distributions of variables. We illustrate a novel methodology based on the marriage between the Bhattacharyya distance and the Johnson Lindenstrauss Lemma, a technique for dimension reduction, providing us with a simple yet powerful tool that allows comparisons between data-sets representing any two distributions. The degree to which different markets or sub groups of securities have different measures of their corresponding distributions tells us the extent to which they are different. This can aid investors looking for diversification or looking for more of the same thing. We briefly discuss how this methodology lends itself to numerous Marketstructure studies and even applications outside the realm of finance/social sciences.6. Securities Lending StrategiesWe develop a valuation methodology and corresponding bidding strategies for exclusive auctions as related to the securities lending industry, a less explored yet profit laden area of financial services. This part is orthogonal to the other portions in the sense that it illustrates how the application of even simple econometric techniques can reduce uncertainty and prove to be rewarding.Each part is meant to be self contained, though at times we refer the reader to different sections for a more detailed exposition on a particular sub topic. We regret any inconvenience that this loss of continuity might bring, but hope that this makes for a more concise presentation. There is a list of references and an appendix for each part where other relevant papers, graphical aids, results of statistical tests and proofs are given and linked into the narrative.

Microstructure under the Microscope: Tools to Survive and Thrive in The Age of (Too Much) Information

Market Microstructure is the investigation of the process and protocols that govern the exchange of assets with the objective of reducing frictions that can impede the transfer. In financial markets, where there is an abundance of recorded information, this translates to the study of the dynamic relationships between observed variables, such as price, volume and spread, and hidden constituents, such as transaction costs and volatility, that hold sway over the efficient functioning of the system. “My dear, here we must process as much data as we can, just to stay in business. And if you wish to make a profit you must process at least twice as much data.” - Red Queen to Alice in Hedge-Fund-Land. Necessity is the mother of all invention/creation/innovation, but the often forgotten father is frustration. In this age of (Too Much) Information, it is imperative to uncover nuggets of knowledge (signal) from buckets of nonsense (noise). To aid in this effort to extract meaning from chaos and to gain a better understanding of the relationships between financial variables, we summarize the application of the theoretical results from (Kashyap 2016b) to microstructure studies. The central concept rests on a novel methodology based on the marriage between the Bhattacharyya distance, a measure of similarity across distributions, and the Johnson Lindenstrauss Lemma, a technique for dimension reduction, providing us with a simple yet powerful tool that allows comparisons between data-sets representing any two distributions. We provide an empirical illustration using prices, volumes and volatilities across seven countries and three different continents. The degree to which different markets or sub groups of securities have different measures of their corresponding distributions tells us the extent to which they are different. This can aid investors looking for diversification or looking for more of the same thing. In Indian mythology, it is believed that in each era, God takes on an avatar or reincarnation to fight the main source of evil in that epoch and to restore the balance between good and bad. In this age of too much information and complexity, perhaps the supreme being needs to be born as a data scientist, conceivably with an apt nickname, the Infoman. Until higher powers intervene and provide the ultimate solution to completely eliminate information overload, we have to make do with marginal methods, such as this composition, to reduce information. As we wait for the perfect solution, it is worth meditating upon what superior beings would do when faced with a complex situation, such as the one we are in. It is said that the Universe is but the Brahma's (Creator's) dream. Research (Effort/Struggle) can help us understand this world; Sleep (Ease/Peace of Mind) can help us create our own world. A lesson from close by and down under: We need to “Do Some Yoga and Sleep Like A Koala”.

A Quantized Johnson–Lindenstrauss Lemma: The Finding of Buffon’s Needle

In 1733, Georges-Louis Leclerc, Comte de Buffon in France, set the ground of geometric probability theory by defining an enlightening problem: what is the probability that a needle thrown randomly on a ground made of equispaced parallel strips lies on two of them? In this paper, we show that the solution to this problem, and its generalization to N dimensions, allows us to discover a quantized form of the Johnson-Lindenstrauss (JL) lemma, i.e., one that combines a linear dimensionality reduction procedure with a uniform quantization of precision δ > 0. In particular, given a finite set S ⊂ ℝ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sup> of S points and a distortion level ϵ > 0, as soon as M > M <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> = O(ϵ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-2</sup> log S), we can (randomly) construct a mapping from (S, ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> ) to (δℤ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</sup> , ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ) that approximately preserves the pairwise distances between the points of S. Interestingly, compared with the common JL lemma, the mapping is quasi-isometric and we observe both an additive and a multiplicative distortions on the embedded distances. These two distortions, however, decay as O((log S/M) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/2</sup> ) when M increases. Moreover, for coarse quantization, i.e., for high δ compared with the set radius, the distortion is mainly additive, while for small δ we tend to a Lipschitz isometric embedding. Finally, we prove the existence of a nearly quasi-isometric embedding of (S, ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> ) into (δℤ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</sup> , ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> ). This one involves a non-linear distortion of the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -distance in S that vanishes for distant points in this set. Noticeably, the additive distortion in this case is slower, and decays as O((log S/M) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/4</sup> ).

Toward a unified theory of sparse dimensionality reduction in Euclidean space

Let $${\Phi\in\mathbb{R}^{m\times n}}$$ be a sparse Johnson–Lindenstrauss transform (Kane and Nelson in J ACM 61(1):4, 2014) with s non-zeroes per column. For a subset T of the unit sphere, $${\varepsilon\in(0,1/2)}$$ given, we study settings for m, s required to ensure $$\mathop{\mathbb{E}}_\Phi \sup_{x\in T}\left|\|\Phi x\|_2^2 - 1\right| < \varepsilon,$$ i.e. so that $${\Phi}$$ preserves the norm of every $${x\in T}$$ simultaneously and multiplicatively up to $${1+\varepsilon}$$ . We introduce a new complexity parameter, which depends on the geometry of T, and show that it suffices to choose s and m such that this parameter is small. Our result is a sparse analog of Gordon’s theorem, which was concerned with a dense $${\Phi}$$ having i.i.d. Gaussian entries. We qualitatively unify several results related to the Johnson–Lindenstrauss lemma, subspace embeddings, and Fourier-based restricted isometries. Our work also implies new results in using the sparse Johnson–Lindenstrauss transform in numerical linear algebra, classical and model-based compressed sensing, manifold learning, and constrained least squares problems such as the Lasso.

Limitations on quantum dimensionality reduction

The Johnson–Lindenstrauss Lemma is a classic result which implies that any set of n real vectors can be compressed to O( log n) dimensions while only distorting pairwise Euclidean distances by a constant factor. Here we consider potential extensions of this result to the compression of quantum states. We show that, by contrast with the classical case, there does not exist any distribution over quantum channels that significantly reduces the dimension of quantum states while preserving the 2-norm distance with high probability. We discuss two tasks for which the 2-norm distance is indeed the correct figure of merit. In the case of the trace norm, we show that the dimension of low-rank mixed states can be reduced by up to a square root, but that essentially no dimensionality reduction is possible for highly mixed states.

Sparse sign-consistent Johnson-Lindenstrauss matrices: compression with neuroscience-based constraints.

Johnson-Lindenstrauss (JL) matrices implemented by sparse random synaptic connections are thought to be a prime candidate for how convergent pathways in the brain compress information. However, to date, there is no complete mathematical support for such implementations given the constraints of real neural tissue. The fact that neurons are either excitatory or inhibitory implies that every so implementable JL matrix must be sign consistent (i.e., all entries in a single column must be either all nonnegative or all nonpositive), and the fact that any given neuron connects to a relatively small subset of other neurons implies that the JL matrix should be sparse. We construct sparse JL matrices that are sign consistent and prove that our construction is essentially optimal. Our work answers a mathematical question that was triggered by earlier work and is necessary to justify the existence of JL compression in the brain and emphasizes that inhibition is crucial if neurons are to perform efficient, correlation-preserving compression.

K-means algorithms for functional data

Cluster analysis of functional data considers that the objects on which you want to perform a taxonomy are functions f:X⊂Rp↦R and the available information about each object is a sample in a finite set of points fn={(xi,yi)∈X×R}i=1n. The aim is to infer the meaningful groups by working explicitly with its infinite-dimensional nature.In this paper the use of K-means algorithms to solve this problem is analysed. A comparative study of three K-means algorithms has been conducted. The K-means algorithm for raw data, a kernel K-means algorithm for raw data and a K-means algorithm using two distances for functional data are tested. These distances, called dVn and dϕ, are based on projections onto Reproducing Kernel Hilbert Spaces (RKHS) and Tikhonov regularization theory. Although it is shown that both distances are equivalent, they lead to two different strategies to reduce the dimensionality of the data. In the case of dVn distance the most suitable strategy is Johnson–Lindenstrauss random projections. The dimensionality reduction for dϕ is based on spectral methods.A key aspect that has been analysed is the effect of the sampling {xi}i=1n on the K-means algorithm performance. In the numerical study an ex professo example is given to show that if the sampling is not uniform in X, then a K-means algorithm that ignores the functional nature of the data can reduce its performance. It is numerically shown that the original K-means algorithm and that suggested here lead to similar performance in the examples when X is uniformly sampled, but the computational cost when working with the original set of observations is higher than the K-means algorithms based on dϕ or dVn, as they use strategies to reduce the dimensionality of the data.The numerical tests are completed with a case study to analyse what kind of problem the K-means algorithm for functional data must face.

On deterministic sketching and streaming for sparse recovery and norm estimation

We study classic streaming and sparse recovery problems using deterministic linear sketches, including ℓ1/ℓ1 and ℓ∞/ℓ1 sparse recovery problems (the latter also being known as ℓ1-heavy hitters), norm estimation, and approximate inner product. We focus on devising a fixed matrix A∈Rm×n and a deterministic recovery/estimation procedure which work for all possible input vectors simultaneously. Our results improve upon existing work, the following being our main contributions:• A proof that ℓ∞/ℓ1 sparse recovery and inner product estimation are equivalent, and that incoherent matrices can be used to solve both problems. Our upper bound for the number of measurements is m=O(ε-2min{logn,(logn/log(1/ε))2}), which holds for any 0<ε<1/2. We can also obtain fast sketching and recovery algorithms by making use of the Fast Johnson–Lindenstrauss transform. Both our running times and number of measurements improve upon previous work. We can also obtain better error guarantees than previous work in terms of a smaller tail of the input vector.• A new lower bound for the number of linear measurements required to solve ℓ1/ℓ1 sparse recovery. We show Ω(k/ε2+klog(n/k)/ε) measurements are required to recover an x′ with ‖x-x′‖1⩽(1+ε)‖xtail(k)‖1, where xtail(k) is x projected onto all but its largest k coordinates in magnitude.• A tight bound of m=Θ(ε-2log(ε2n)) on the number of measurements required to solve deterministic norm estimation, i.e., to recover ‖x‖2±ε‖x‖1.For all the problems we study, tight bounds are already known for the randomized complexity from previous work, except in the case of ℓ1/ℓ1 sparse recovery, where a nearly tight bound is known. Our work thus aims to study the deterministic complexities of these problems. We remark that some of the matrices used in our algorithms, although known to exist, currently are not yet explicit in the sense that deterministic polynomial time constructions are not yet known, although in all cases polynomial time Monte Carlo algorithms are known.

A Subspace Embedding Method inL2Norm via Fast Cauchy Transform

We propose a subspace embedding method via Fast Cauchy Transform (FCT) inL2norm. It is motivated by and complements the work of the subspace embedding method inLpnorm, for allp∈[1,∞]exceptp=2, by K. L. Clarkson (ACM-SIAM, 2013). Unlike the traditionally used orthogonal basis in Johnson-Lindenstrauss (JL) embedding, we employ the well-conditioned basis inL2norm to obtain concentration property of FCT inL2norm.

Particle Systems and Kinetic Equations Modeling Interacting Agents in High Dimension

In this paper we explore how concepts of high-dimensional data compression via random projections onto lower-dimensional spaces can be applied for tractable simulation of certain dynamical systems modeling complex interactions. In such systems, one has to deal with a large number of agents (typically millions) in spaces of parameters describing each agent of high dimension (thousands or more). Even with today’s powerful computers, numerical simulations of such systems are prohibitively expensive. We propose an approach for the simulation of dynamical systems governed by functions of adjacency matrices in high dimension, by random projections via Johnson–Lindenstrauss embeddings, and recovery by compressed sensing techniques. We show how these concepts can be generalized to work for associated kinetic equations by addressing the phenomenon of the delayed curse of dimension, known in information-based complexity for optimal numerical integration problems and measure quantization in high dimensions.

Accelerating Image Registration With the Johnson–Lindenstrauss Lemma: Application to Imaging 3-D Neural Ultrastructure With Electron Microscopy

We present a novel algorithm to accelerate feature based registration, and demonstrate the utility of the algorithm for the alignment of large transmission electron microscopy (TEM) images to create 3-D images of neural ultrastructure. In contrast to the most similar algorithms, which achieve small computation times by truncated search, our algorithm uses a novel randomized projection to accelerate feature comparison and to enable global search. Further, we demonstrate robust estimation of nonrigid transformations with a novel probabilistic correspondence framework, that enables large TEM images to be rapidly brought into alignment, removing characteristic distortions of the tissue fixation and imaging process. We analyze the impact of randomized projections upon correspondence detection, and upon transformation accuracy, and demonstrate that accuracy is maintained. We provide experimental results that demonstrate significant reduction in computation time and successful alignment of TEM images.

New and Improved Johnson–Lindenstrauss Embeddings via the Restricted Isometry Property

Consider an $m \times N$ matrix $\Phi$ with the restricted isometry property of order k and level $\delta$; that is, the norm of any k-sparse vector in $\mathbb{R}^N$ is preserved to within a multiplicative factor of $1 \pm \delta$ under application of $\Phi$. We show that by randomizing the column signs of such a matrix $\Phi$, the resulting map with high probability embeds any fixed set of $p = O(e^k)$ points in $\mathbb{R}^N$ into $\mathbb{R}^m$ without distorting the norm of any point in the set by more than a factor of $1 \pm 4 \delta$. Consequently, matrices with the restricted isometry property and with randomized column signs provide optimal Johnson–Lindenstrauss embeddings up to logarithmic factors in N. In particular, our results improve the best known bounds on the necessary embedding dimension m for a wide class of structured random matrices; for partial Fourier and partial Hadamard matrices, we improve the recent bound $m \gtrsim \delta^{-4} \log(p) \log^4(N)$ given by Ailon and Liberty to $m \gtrsim \delta^{-2} \log(p) \log^4(N)$, which is optimal up to the logarithmic factors in N. Our results also have a direct application in the area of compressed sensing for redundant dictionaries.

A variant of the Johnson–Lindenstrauss lemma for circulant matrices

We continue our study of the Johnson–Lindenstrauss lemma and its connection to circulant matrices started in Hinrichs and Vybíral (in press) [7]. We reduce the bound on k from k = Ω ( ε − 2 log 3 n ) proven there to k = Ω ( ε − 2 log 2 n ) . Our technique differs essentially from the one used in Hinrichs and Vybíral (in press) [7]. We employ the discrete Fourier transform and singular value decomposition to deal with the dependency caused by the circulant structure.

Stochastic iterative projection methods for large linear systems

We suggest a randomized version of the projection methods belonging to the class of a “row-action” methods which work well both for systems with quadratic nonsingular matrices and for overdetermined systems. These methods belong to a type known as Projection on Convex Sets methods. Here we present a method beyond the conventional Markov chain based Neumann–Ulam scheme. The main idea is in a random choice of blocks of rows in the projection method so that in average, the convergence is improved compared to the conventional periodic choice of the rows. We suggest an acceleration of the row projection method by using the Johnson–Lindenstrauss (J–L) theorem to find, among the randomly chosen rows, in a sense an optimal row. We extend this randomized method for solving linear systems coupled with systems of linear inequalities. Applied to finite-difference approximations of boundary value problems, the method appears to be an extremely efficient Random Walk algorithm whose convergence is exponential, and the cost does not depend on the dimension of the matrix. In addition, the algorithm calculates the solution in all grid points, and is easily parallelizable.