Stable Embeddings Research Articles

Invariant Representations Learning with Future Dynamics

High-dimension inputs limit the sample efficiency of deep reinforcement learning, increasing the difficulty of applying it to real-world continuous control tasks, especially in uncertain environments. A good state embedding is crucial for the performance of agents on downstream tasks. The bisimulation metric is an excellent representation learning method that can abstract task-relevant invariant latent embeddings of states based on behavior similarity. However, since only one time-step transition is considered, the features captured by this method we call short-term dynamics. We think long-term dynamics are also important for state representation learning. In this paper, we present Invariant Representations Learning with Future Dynamics (RLF), which uses graph neural networks to learn long-term dynamics and trains the representation network based on a new state metric inspired by bisimulation relation. We experimented with our method on continuous control tasks from DeepMind Control Suite and showed that the RLF can mine more stable embeddings than the state-of-the-art representation learning methods for both state and pixel inputs. The learned policy on top of the embeddings has higher sample efficiency and performance and generalizes well in different tasks.

Density-based structural embedding for anomaly detection in dynamic networks

Dynamic networks continuously change their structure and properties, and anomaly detection methods must identify these structural changes at both local and global levels. Network embedding is considered a powerful tool for low-dimension structural representation of network objects. However, most embedding techniques developed for dynamic networks are incompetent in capturing the changes in global structures. Besides, with incoming edge streams, these techniques change the embeddings of all network objects, including those vertices and edges that do not experience any change in subsequent timestamps. In this paper, we propose an embedding algorithm DSEDN to identify anomalous vertices and edges in dynamic networks utilizing structural changes in networks. The algorithm uses sparse autoencoder to generate network embeddings minimizing the pair-wise and neighborhood distance between vertex representations of every subgraph derived from random walks. The subgraphs have been weighted based on their respective clustering coefficient, and the clustering algorithm is employed to identify network anomalies. The advantages of proposed method include: (i) better accuracy in detecting network anomalies; (ii) structure-preserving embeddings, such that it maintains the local and global structure of every snapshot of growing graphs; (iii) density-based embeddings; (iv) stable embeddings over time such that embeddings of consecutive timestamps do not change much for those network objects that do not experience any change; (v) scalability. We evaluate the proposed algorithm on anomaly detection and graph visualization on five real-world datasets. Our algorithm achieves improvement in AUC, scalability, and stability across all baselines.

Stable Manifold Embeddings With Structured Random Matrices

The fields of compressed sensing (CS) and matrix completion have shown that high-dimensional signals with sparse or low-rank structure can be effectively projected into a low-dimensional space (for efficient acquisition or processing) when the projection operator achieves a stable embedding of the data by satisfying the Restricted Isometry Property (RIP). It has also been shown that such stable embeddings can be achieved for general Riemannian submanifolds when random orthoprojectors are used for dimensionality reduction. Due to computational costs and system constraints, the CS community has recently explored the RIP for structured random matrices (e.g., random convolutions, localized measurements, deterministic constructions). The main contribution of this paper is to show that any matrix satisfying the RIP (i.e., providing a stable embedding for sparse signals) can be used to construct a stable embedding for manifold-modeled signals by randomizing the column signs and paying reasonable additional factors in the number of measurements, thereby generalizing previous stable manifold embedding results beyond unstructured random matrices. We demonstrate this result with several new constructions for stable manifold embeddings using structured matrices. This result allows advances in efficient projection schemes for sparse signals to be immediately applied to manifold signal models.

Derived algebraic geometry, determinants of perfect complexes, and applications to obstruction theories for maps and complexes

Abstract A quasi-smooth derived enhancement of a Deligne–Mumford stack 𝒳 naturally endows 𝒳 with a functorial perfect obstruction theory in the sense of Behrend–Fantechi. We apply this result to moduli of maps and perfect complexes on a smooth complex projective variety. For moduli of maps, for X = S an algebraic K3-surface, g ∈ ℕ, and β ≠ 0 in H 2(S,ℤ) a curve class, we construct a derived stack ℝ 𝐌 ¯ g , n red ( S ; β ) ${\mathbb {R}\overline{\mathbf {M}}^{\text{red}}_{g,n}(S;\beta )}$ whose truncation is the usual stack 𝐌 ¯ g , n ( S ; β ) ${\overline{\mathbf {M}}_{g,n}(S;\beta )}$ of pointed stable maps from curves of genus g to S hitting the class β, and such that the inclusion 𝐌 ¯ g ( S ; β ) ↪ ℝ 𝐌 ¯ g red ( S ; β ) ${\overline{\mathbf {M}}_{g}(S;\beta )\hookrightarrow \mathbb {R}\overline{\mathbf {M}}^{\text{red}}_{g}(S;\beta )}$ induces on 𝐌 ¯ g ( S ; β ) ${\overline{\mathbf {M}}_{g}(S;\beta )}$ a perfect obstruction theory whose tangent and obstruction spaces coincide with the corresponding reduced spaces of Okounkov–Maulik–Pandharipande–Thomas. The approach we present here uses derived algebraic geometry and yields not only a full rigorous proof of the existence of a reduced obstruction theory – not relying on any result on semiregularity maps – but also a new global geometric interpretation. We give two further applications to moduli of complexes. For a K3-surface S we show that the stack of simple perfect complexes on S is smooth. This result was proved with different methods by Inaba for the corresponding coarse moduli space. Finally, we construct a map from the derived stack of stable embeddings of curves (into a smooth complex projective variety X) to the derived stack of simple perfect complexes on X with vanishing negative Ext's, and show how this map induces a morphism of the corresponding obstruction theories when X is a Calabi–Yau 3-fold. An important ingredient of our construction is a perfect determinant map from the derived stack of perfect complexes to the derived stack of line bundles whose tangent morphism is given by Illusie's trace map for perfect complexes.

A geometric approach to multi-view compressive imaging

In this paper, we consider multi-view imaging problems in which an ensemble of cameras collect images describing a common scene. To simplify the acquisition and encoding of these images, we study the effectiveness of non-collaborative compressive sensing encoding schemes wherein each sensor directly and independently compresses its image using randomized measurements. After these measurements and also perhaps the camera positions are transmitted to a central node, the key to an accurate reconstruction is to fully exploit the joint correlation among the signal ensemble. To capture such correlations, we propose a geometric modeling framework in which the image ensemble is treated as a sampling of points from a low-dimensional in the ambient signal space. Building on results that guarantee stable embeddings of manifolds under random measurements, we propose a manifold algorithm for recovering the ensemble that can operate even without knowledge of the camera positions. We divide our discussion into two scenarios, the near-field and far-field cases, and describe how the lifting algorithm could be applied to these scenarios. At the end of this paper, we present an in-depth case study of a far-field imaging scenario, where the aim is to reconstruct an ensemble of satellite images taken from different positions with limited but overlapping fields of view. In this case study, we demonstrate the impressive power of random measurements to capture single- and multi-image structure without explicitly searching for it, as the randomized measurement encoding in conjunction with the proposed lifting algorithm can even outperform image-by-image transform coding.

Concentration of Measure for Block Diagonal Matrices With Applications to Compressive Signal Processing

Theoretical analysis of randomized, compressive operators often depends on a concentration of measure inequality for the operator in question. Typically, such inequalities quantify the likelihood that a random matrix will preserve the norm of a signal after multiplication. Concentration of measure results are well established for unstructured compressive matrices, populated with independent and identically distributed (i.i.d.) random entries. Many real-world acquisition systems, however, are subject to architectural constraints that make such matrices impractical. In this paper we derive concentration of measure bounds for two types of block diagonal compressive matrices, one in which the blocks along the main diagonal are random and independent, and one in which the blocks are random but equal. For both types of matrices, we show that the likelihood of norm preservation depends on certain properties of the signal being measured, but that for the best case signals, both types of block diagonal matrices can offer concentration performance on par with their unstructured, i.i.d. counterparts. We support our theoretical results with illustrative simulations as well as analytical and empirical investigations of several signal classes that are highly amenable to measurement using block diagonal matrices. We also discuss applications of these results in ensuring stable embeddings for various signal families and in establishing performance guarantees for solving various signal processing tasks (such as detection and classification) directly in the compressed domain.

Stable Takens' Embeddings for Linear Dynamical Systems

Takens' embedding theorem remarkably established that concatenating <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</i> previous outputs of a dynamical system into a vector (called a delay coordinate map) can be a one-to-one mapping of a low-dimensional attractor from the system state space. However, Takens' theorem is fragile in the sense that even small imperfections can induce arbitrarily large errors in this attractor representation. We extend Takens' result to establish deterministic, explicit and nonasymptotic sufficient conditions for a delay coordinate map to form a stable embedding in the restricted case of linear dynamical systems and observation functions. Our work is inspired by the field of Compressive Sensing (CS), where results guarantee that low-dimensional signal families can be robustly reconstructed if they are stably embedded by a measurement operator. However, in contrast to typical CS results, i) our sufficient conditions are independent of the size of the ambient state space, and ii) some system and measurement pairs have fundamental limits on the conditioning of the embedding (i.e., how close it is to an isometry), meaning that further measurements beyond some point add no further significant value. We use several simple simulations to explore the conditions of the main results, including the tightness of the bounds and the convergence speed of the stable embedding. We also present an example task of estimating the attractor dimension from time-series data to highlight the value of stable embeddings over traditional Takens' embeddings.

Quantum Hall effect in a holographic model

We consider a holographic description of a system of strongly coupled fermions in 2+1 dimensions based on a D7-brane probe in the background of D3-branes, and construct stable embeddings by turning on worldvolume fluxes. We study the system at finite temperature and charge density, and in the presence of a background magnetic field. We show that Minkowski-like embeddings that terminate above the horizon describe a family of quantum Hall states with filling fractions that are parameterized by a single discrete parameter. The quantization of the Hall conductivity is a direct consequence of the topological quantization of the fluxes. When the magnetic field is varied relative to the charge density away from these discrete filling fractions, the embeddings deform continuously into black-hole-like embeddings that enter the horizon and that describe metallic states. We also study the thermodynamics of this system and show that there is a first order phase transition at a critical temperature from the quantum Hall state to the metallic state.

A short note on compressed sensing with partially known signal support

This short note studies a variation of the compressed sensing paradigm introduced recently by Vaswani et al., i.e., the recovery of sparse signals from a certain number of linear measurements when the signal support is partially known. In this framework, we propose a reconstruction method based on a convex minimization program coined innovative Basis Pursuit DeNoise (or iBPDN). Under the common ℓ 2 -fidelity constraint made on the available measurements, this optimization promotes the ( ℓ 1 ) sparsity of the candidate signal over the complement of this known part. In particular, this paper extends the results of Vaswani et al. to the cases of compressible signals and noisy measurements by showing that iBPDN is ℓ 2 − ℓ 1 instance optimal. The corresponding proof relies on a small adaption of the results of Candès in 2008 for characterizing the stability of the Basis Pursuit DeNoise (BPDN) program. We also emphasize an interesting link between our method and the recent work of Davenport et al. on the δ -stable embeddings and the cancel-then-recover strategy applied to our problem. For both approaches, reconstructions are indeed stabilized when the sensing matrix respects the Restricted Isometry Property for the same sparsity order.

Maximality Properties of Stable Partial Spreads and of Shear Planes

Shear planes are special stable planes which can be derived from stable partial spreads by a construction analogous to the construction of (punctured) dual translation planes from spreads. We give a sufficient condition for the maximality of shear planes in the category of stable planes and open embeddings. The condition is expressed in terms of the defining partial spread. Moreover, we give several examples of connected and disconnected shear planes in dimension 4, 8 and 16 which are maximal stable planes and have large automorphism groups. Most of these examples are direct limits of open subplanes of dual projective translation planes.

Resins for combined light and electron microscopy: a half century of development.

The last fifty years have seen enormous improvements in the way biological specimens are prepared for microscopy. The Fifties produced the essential groundwork upon which many of our current methodologies are based. Acrylic resin embedding was introduced in 1949, with subsequent publications seeking improvements to resin formulations, embedding protocols, and modes of polymerisation. Procedures for progressive lowering of temperature processing, cryosubstitution, freeze-drying and polymerisation by ultra-violet light at low temperatures, all had their genesis in this decade of great innovation. The Sixties marked the period when the acrylics were eclipsed by the more stable and reliable epoxy resins, and much of our present-day understanding of ultrastructure was elucidated. The Seventies carried on this work with advances in technical developments concerned mainly with freezing methodologies. The beginning of the Eighties saw a resurrection of the acrylic resins, with new formulations of these resins giving reliable and stable embeddings. The low temperature and freezing methodologies pioneered in the Fifties, backed up by recent improvements to low temperature technologies, were used to further our understanding of ultrastructure and breathe new life into the science of immunocytochemistry. The remainder of the Eighties and Nineties has seen the ever increasing application of these various microscopical techniques to a wide range of biological studies. The flexibility offered by the acrylic resins in choosing between different processing, embedding and polymerisation methods has provided the impetus for detailed studies to bring to the attention of microscopists the underlying trends governing specimen preparation. Therefore, looking forward to the new Millennium, this has allowed for a more reasoned choice in organising a strategy to deal with a variety of microscopical requirements and for planning an appropriate protocol.