Dense Codes Research Articles

Two-stage zero-shot sparse hashing with missing labels for cross-modal retrieval

Recently, zero-shot cross-modal hashing has gained significant popularity due to its ability to effectively realize the retrieval of emerging concepts within multimedia data. Although the existing approaches have shown impressive results, the following limitations still need to be solved: (1) Labels in large-scale multimodal datasets in real scenes are usually incomplete or partially missing. (2) The existing methods ignore the influence of features-wise low-level similarity and label distribution on retrieval performance. (3) The representation ability of dense hash codes limits its discriminative potential. To solve these issues, we introduce an effective cross-modal retrieval framework called two-stage zero-shot sparse hashing with missing labels (TZSHML). Specifically, we learn a classifier through the partially known labeled samples to predict the labels of unlabeled data. Then, we use the reliable information in the correctly marked labels to recover the missing labels. The predicted and recovered labels are combined to obtain more accurate labels for the samples with missing labels. In addition, we employ sample-wise fine-grained similarity and cluster-wise similarity to learn hash codes. Therefore, TZSHML ensures that more samples with similar semantics are clustered together. Besides, we apply high-dimensional sparse hash codes to explore richer semantic information. Finally, the drift and interaction terms are introduced into the learning of the hash function to further narrow the gap between different modalities. Extensive experimental results demonstrate the competitiveness of our approach over other state-of-the-art methods in zero-shot retrieval scenarios with missing labels. The source code of this paper can be obtained from https://github.com/szq0816/TZSHML.

Interplays between classical and quantum entanglement-assisted communication scenarios

Prepare-and-measure scenarios (pm), in their many forms, can be seen as the basic building blocks of communication tasks. As such, they can be used to analyze a diversity of classical and quantum protocols—of which dense coding and random access codes are key examples—in a unified manner. In particular, the use of entanglement as a resource in pm scenarios have only recently started to be systematically investigated, and many crucial questions remain open. In this work, we explore such scenarios and provide answers to some seminal questions. More specifically, we show that, in scenarios where entanglement is a free resource, quantum messages are equivalent to classical ones with twice the capacity. We also prove that, in such scenarios, it is always advantageous for the parties to share entangled states of dimension greater than the transmitted message. Finally, we show that unsteerable states cannot provide advantages in classical communication tasks, thus proving that not all entangled states are useful resources in these scenarios.

A High-Dimensional Sparse Hashing Framework for Cross-Modal Retrieval

In recent years, many achievements have been made in improving the performance of supervised cross-modal hashing. However, it remains an open issue on how to fully explore the data information to achieve fine-grained retrieval performance. Most methods employ logical labels or a binary similarity matrix to supervise the hash learning, losing a lot of useful information. From another point of view, the low expressiveness of dense hash code severely limits its preservation of fine-grained data information. With this motivation, in this paper, we propose a high-dimensional sparse hashing framework for cross-modal retrieval, i.e., High-dimensional Sparse Cross-modal Hashing, HSCH for short. It leverages not only high-level semantic labels but also low-level multi-modal features to construct a fine-grained similarity. In particular, based on two well-designed rules, i.e., multi-level and prioritized, it is able to avoid semantic conflicts. Additionally, it leverages the strong power of high-dimensional sparse hash codes to preserve the fine-grained similarity. Then, it efficiently solves the sparse and discrete constraints of sparse hash codes through an efficient discrete optimization algorithm. In light of this, it is much more efficient and scalable to large-scale datasets. More importantly, the computational complexity of HSCH in the retrieval phase is as efficient as those naive hashing methods that use dense hash codes. Moreover, to support online learning scenarios, this paper also extends HSCH into an online version, i.e., HSCH_on. Extensive experiments on three benchmark datasets demonstrate the superiority of our framework compared with some state-of-the-art cross-modal hashing approaches in terms of both accuracy and efficiency.

Streaming Database System for Deaf People

Most of the traditional SQL keeps database records that do not often alternate. Models are client connection control systems or web and website online systems that replace like clockwork/seconds. Recently, programmers had composed dense codes to make database alter with converting or streaming records. Creating vast micro services to cope with stream records is exorbitant and often instable. SQL streaming database cope with record streams with insignificant coding and convey non-stop research for an extensive variety of forms of cutting-edge packages. A streaming database works with dynamic records that modification constantly. This converting record triggers other inquiries and packages as opposed to traditional way. The main characteristics of streaming database are continuous record of new data input, update the database status in real time, and answer online queries with consistency of database state. Standard database requires human to supply a reaction for streaming events. For example, when a customer buys an object on a commercial web enterprise human contribution triggers the reaction. The records in a streaming database are sort of streaming events in which it examines, gathers, and cycles those records streams and catches the records to be used gradually alarms, perceptions, questions, or packages. This research follows streaming database of deaf people to simultaneously provide understanding of events. Experiments showed that the streaming database structures are more appropriate for deaf people communicating with each other and with normal people as well. Extracted from the fact that 70% per cent of the sample were satisfied with the suggested system , more than 80% percentage of normal people are satisfying with the system, while more than 85% percentage of deaf and normal people are happy with the system. This type of on line updating and its applications can be applied for people with learning special needs in various ways.

A sparse code increases the speed and efficiency of neuro-dynamic programming for optimal control tasks with correlated inputs

Sparse codes in neuroscience have been suggested to offer certain computational advantages over other neural representations of sensory data. To explore this viewpoint, a sparse code is used to represent natural images in an optimal control task solved with neuro-dynamic programming, and its computational properties are investigated. The central finding is that when feature inputs to a linear network are correlated, an over-complete sparse code increases the memory capacity of the network in an efficient manner beyond that possible for any complete code with the same-sized input, and also increases the speed of learning the network weights. A complete sparse code is found to maximise the memory capacity of a linear network by decorrelating its feature inputs to transform the design matrix of the least-squares problem to one of full rank. It also conditions the Hessian matrix of the least-squares problem, thereby increasing the rate of convergence to the optimal network weights. Other types of decorrelating codes would also achieve this. However, an over-complete sparse code is found to be approximately decorrelated, extracting a larger number of approximately decorrelated features from the same-sized input, allowing it to efficiently increase memory capacity beyond that possible for any complete code: a 2.25 times over-complete sparse code is shown to at least double memory capacity compared with a complete sparse code using the same input. This is used in sequential learning to store a potentially large number of optimal control tasks in the network, while catastrophic forgetting is avoided using a partitioned representation, yielding a cost-to-go function approximator that generalizes over the states in each partition. Sparse code advantages over dense codes and local codes are also discussed.

Sparse coding with a somato-dendritic rule

Cortical neurons are silent most of the time: sparse activity enables low-energy computation in the brain, and promises to do the same in neuromorphic hardware. Beyond power efficiency, sparse codes have favourable properties for associative learning, as they can store more information than local codes but are easier to read out than dense codes. Auto-encoders with a sparse constraint can learn sparse codes, and so can single-layer networks that combine recurrent inhibition with unsupervised Hebbian learning. But the latter usually require fast homeostatic plasticity, which could lead to catastrophic forgetting in embodied agents that learn continuously. Here we set out to explore whether plasticity at recurrent inhibitory synapses could take up that role instead, regulating both the population sparseness and the firing rates of individual neurons. We put the idea to the test in a network that employs compartmentalised inputs to solve the task: rate-based dendritic compartments integrate the feedforward input, while spiking integrate-and-fire somas compete through recurrent inhibition. A somato-dendritic learning rule allows somatic inhibition to modulate nonlinear Hebbian learning in the dendrites. Trained on MNIST digits and natural images, the network discovers independent components that form a sparse encoding of the input and support linear decoding. These findings confirm that intrinsic homeostatic plasticity is not strictly required for regulating sparseness: inhibitory synaptic plasticity can have the same effect. Our work illustrates the usefulness of compartmentalised inputs, and makes the case for moving beyond point neuron models in artificial spiking neural networks.

On f-Disjunctive Languages, f-Disjunctive Domains and Solid Codes

Every language is either the disjoint union or the intersection of two disjunctive languages. The family of f-disjunctive languages is a natural generalization of the family of disjunctive languages. Disjunctive domains (f-disjunctive domains) are defined as the languages which can be checked whether a given language is disjunctive (f-disjunctive) or not. The f-disjunctive domains were first studied by Guo et al. around 1986–1989. We continue their study on f-disjunctive domains. Some new results for f-disjunctive domains, containing a relation between disjunctive domains and f-disjunctive domains, are introduced. In this respect, we also make an appropriate opening for the completely dense languages and solid codes.

On Closest Pair in Euclidean Metric: Monochromatic is as Hard as Bichromatic

Given a set of n points in ℝd, the (monochromatic) Closest Pair problem asks to find a pair of distinct points in the set that are closest in the lp-metric. Closest Pair is a fundamental problem in Computational Geometry and understanding its fine-grained complexity in the Euclidean metric when d = ω(log n) was raised as an open question in recent works (Abboud-Rubinstein-Williams [6], Williams [48], David-Karthik-Laekhanukit [17]). In this paper, we show that for every p ∈ ℝ≥1 ∪ {0}, under the Strong Exponential Time Hypothesis (SETH), for every e > 0, the following holds: In particular, our first result is shown by establishing the computational equivalence of the bichromatic Closest Pair problem and the (monochromatic) Closest Pair problem (up to ne factor in the running time) for $$d = {\left({\log n} \right)^{{\Omega _\varepsilon}\left(1 \right)}}$$ dimensions. Additionally, under SETH, we rule out nearly-polynomial factor approximation algorithms running in subquadratic time for the (monochromatic) Maximum Inner Product problem where we are given a set of n points in no(1)-dimensional Euclidean space and are required to find a pair of distinct points in the set that maximize the inner product. At the heart of all our proofs is the construction of a dense bipartite graph with low contact dimension, i.e., we construct a balanced bipartite graph on n vertices with n2−e edges whose vertices can be realized as points in a $${\left({\log n} \right)^{{\Omega _\varepsilon}\left(1 \right)}}$$ -dimensional Euclidean space such that every pair of vertices which have an edge in the graph are at distance exactly 1 and every other pair of vertices are at distance greater than 1. This graph construction is inspired by the construction of locally dense codes introduced by Dumer-Micciancio-Sudan [18].

Urban Start-up Districts: Mapping Venture Capital and Start-up Activity Across ZIP Codes

Previous research has identified the clustering of high-tech industries, entrepreneurial start-ups, and venture capital across metropolitan areas. Using detailed ZIP code data on start-up activity and venture capital investment, this research tests three hypotheses informed by urban theory on the clustering of innovation, entrepreneurship, and high-technology industry: (1) that start-up activity and venture capital investment will concentrate in distinct microclusters within metro areas, (2) that a substantial level of start-up activity and venture capital investment will cluster in dense urban neighborhoods or ZIP codes, and (3) that the clustering of start-ups and venture capital investment will vary by industry or type of technology. The authors find evidence to support all three. Start-up activity and venture capital investment are concentrated in a relatively small number of ZIP codes in the United States, the majority of which are in dense urban neighborhoods, and this clustering varies by industry and type of technology.

Graph construction using adaptive Local Hybrid Coding scheme

It is well known that dense coding with local bases (via Least Square coding schemes) can lead to large quantization errors or poor performances of machine learning tasks. On the other hand, sparse coding focuses on accurate representation without taking into account data locality due to its tendency to ignore the intrinsic structure hidden among the data. Local Hybrid Coding (LHC) (Xiang et al., 2014) was recently proposed as an alternative to the sparse coding scheme that is used in Sparse Representation Classifier (SRC). The LHC blends sparsity and bases-locality criteria in a unified optimization problem. It can retain the strengths of both sparsity and locality. Thus, the hybrid codes would have some advantages over both dense and sparse codes. This paper introduces a data-driven graph construction method that exploits and extends the LHC scheme. In particular, we propose a new coding scheme coined Adaptive Local Hybrid Coding (ALHC). The main contributions are as follows. First, the proposed coding scheme adaptively selects the local and non-local bases of LHC using data similarities provided by Locality-constrained Linear code. Second, the proposed ALHC exploits local similarities in its solution. Third, we use the proposed coding scheme for graph construction. For the task of graph-based label propagation, we demonstrate high classification performance of the proposed graph method on four benchmark face datasets: Extended Yale, PF01, PIE, and FERET.

Parallel tiled Nussinov RNA folding loop nest generated using both dependence graph transitive closure and loop skewing

BackgroundRNA secondary structure prediction is a compute intensive task that lies at the core of several search algorithms in bioinformatics. Fortunately, the RNA folding approaches, such as the Nussinov base pair maximization, involve mathematical operations over affine control loops whose iteration space can be represented by the polyhedral model. Polyhedral compilation techniques have proven to be a powerful tool for optimization of dense array codes. However, classical affine loop nest transformations used with these techniques do not optimize effectively codes of dynamic programming of RNA structure predictions.ResultsThe purpose of this paper is to present a novel approach allowing for generation of a parallel tiled Nussinov RNA loop nest exposing significantly higher performance than that of known related code. This effect is achieved due to improving code locality and calculation parallelization. In order to improve code locality, we apply our previously published technique of automatic loop nest tiling to all the three loops of the Nussinov loop nest. This approach first forms original rectangular 3D tiles and then corrects them to establish their validity by means of applying the transitive closure of a dependence graph. To produce parallel code, we apply the loop skewing technique to a tiled Nussinov loop nest.ConclusionsThe technique is implemented as a part of the publicly available polyhedral source-to-source TRACO compiler. Generated code was run on modern Intel multi-core processors and coprocessors. We present the speed-up factor of generated Nussinov RNA parallel code and demonstrate that it is considerably faster than related codes in which only the two outer loops of the Nussinov loop nest are tiled.

Variable-length Prefix Codes with Multiple Delimiters

Let $m_{1},m_{2},\ldots, m_{t}$ be a fixed set of natural integers given in ascending order. A multi-delimiter code $D_{m_{1},\ldots, m_{t}}$ consists of $t$ words of the form $1^{m_{i}}0$ and all other binary words with suffixes having one of the forms $01^{m_{i}}0$ that cannot occur in other places of a word, $i=1,\ldots, t$ . Completeness, universality, and density characteristics of multi-delimiter codes are proved. Encoding of integers by multi-delimiter codes is considered in detail. For these codes, fast byte aligned decoding algorithms are also discussed. The comparison of compression rate and decoding speed of Fibonacci codes, $(s,c)$ -dense codes and different multi-delimiter codes is presented. By many useful properties, multi-delimiter codes are superior to Fibonacci codes.

Electric fishes: neural systems, behaviour and evolution

Weakly electric fish are a unique model system in neuroethology as musculoskeletal systems and mechanics are not involved in many of their behaviours. Rather, these behaviours are mediated solely by the activity of neurons and other excitable cells in the sensory and motor systems of these

ODC: Frame for definition of Dense codes

Natural language compression has made great progress in the last two decades. The main step in this evolution was the introduction of word-based compression by Moffat. Another improvement came with so-called Dense codes, which proved to be very fast in compression and decompression while keeping a good compression ratio and direct search capability. Many variants of the Dense codes have been described, each of them using its own definition. In this paper, we present a generalized concept of dense coding called Open Dense Code (ODC), which aims to be a frame for the definition of many other dense code schemas. ODC underlines common features of the dense code schemas but at the same time allows one to express the divergences of each of them. Using the frame of ODC, we present two new word-based statistical compression algorithms based on the dense coding idea: Two Byte Dense Code (TBDC) and Self-Tuning Dense Code (STDC). Our algorithms improve the compression ratio and are considerate to smaller files, which are very often omitted by other compressors.

Precise Feature Based Time Scales and Frequency Decorrelation Lead to a Sparse Auditory Code

Sparse redundancy reducing codes have been proposed as efficient strategies for representing sensory stimuli. A prevailing hypothesis suggests that sensory representations shift from dense redundant codes in the periphery to selective sparse codes in cortex. We propose an alternative framework where sparseness and redundancy depend on sensory integration time scales and demonstrate that the central nucleus of the inferior colliculus (ICC) of cats encodes sound features by precise sparse spike trains. Direct comparisons with auditory cortical neurons demonstrate that ICC responses were sparse and uncorrelated as long as the spike train time scales were matched to the sensory integration time scales relevant to ICC neurons. Intriguingly, correlated spiking in the ICC was substantially lower than predicted by linear or nonlinear models and strictly observed for neurons with best frequencies within a "critical band," the hallmark of perceptual frequency resolution in mammals. This is consistent with a sparse asynchronous code throughout much of the ICC and a complementary correlation code within a critical band that may allow grouping of perceptually relevant cues.

Improving semistatic compression via phrase-based modeling

In recent years, new semistatic word-based byte-oriented text compressors, such as Tagged Huffman and those based on Dense Codes, have shown that it is possible to perform fast direct search over compressed text and decompression of arbitrary text passages over collections reduced to around 30–35% of their original size. Much of their success is due to the use of words as source symbols and a byte-oriented target alphabet. This approach broke with traditional statistical compressors, which use characters as source symbols and a bit-oriented target alphabet. In this work we go one step beyond by using phrases as source symbols. We present two new semistatic modelers that we combined with a dense coding scheme to obtain two new compressors: Pair-Based End-Tagged Dense Code (PETDC), where source symbols can be either words or pairs of words, and Phrase-Based End-Tagged Dense Code (PhETDC), which considers words and sequences of words (phrases). PETDC compresses English texts to 28–29% and PhETDC to around 23%, outperforming the optimal byte-oriented zero-order prefix-free word-based semistatic compressor by up to 8 percentage points. Moreover, PETDC and PhETDC still permit random access and efficient direct searches using fast Boyer–Moore algorithms.

On the Usefulness of Fibonacci Compression Codes

Recent publications advocate the use of various variable length codes for which each codeword consists of an integral number of bytes in compression applications using large alphabets. This paper shows that another tradeoff with similar properties can be obtained by Fibonacci codes. These are fixed codeword sets, using binary representations of integers based on Fibonacci numbers of order m ≥ 2. Fibonacci codes have been used before, and this paper extends previous work presenting several novel features. In particular, the compression efficiency is analyzed and compared to that of dense codes, and various table-driven decoding routines are suggested.

Composite CDMA—a statistical mechanics analysis

Code division multiple access (CDMA) in which the spreading code assignment to userscontains a random element has recently become a cornerstone of CDMA research. Therandom element in the construction is particularly attractive as it provides robustnessand flexibility in utilizing multi-access channels, whilst not making significantsacrifices in terms of transmission power. Random codes are generated from someensemble; here we consider the possibility of combining two standard paradigms,sparsely and densely spread codes, in a single composite code ensemble. Thecomposite code analysis includes a replica symmetric calculation of performance inthe large system limit, and investigation of finite systems through a compositebelief propagation algorithm. A variety of codes are examined with a focus onthe high multi-access interference regime. We demonstrate scenarios both in thelarge size limit and for finite systems in which the composite code has typicalperformance exceeding those of sparse and dense codes at equivalent signal to noiseratio.

New adaptive compressors for natural language text

AbstractSemistatic byte‐oriented word‐based compression codes have been shown to be an attractive alternative to compress natural language text databases, because of the combination of speed, effectiveness, and direct searchability they offer. In particular, our recently proposed family of dense compression codes has been shown to be superior to the more traditional byte‐oriented word‐based Huffman codes in most aspects. In this paper, we focus on the problem of transmitting texts among peers that do not share the vocabulary. This is the typical scenario for adaptive compression methods. We design adaptive variants of our semistatic dense codes, showing that they are much simpler and faster than dynamic Huffman codes and reach almost the same compression effectiveness. We show that our variants have a very compelling trade‐off between compression/decompression speed, compression ratio, and search speed compared with most of the state‐of‐the‐art general compressors. Copyright © 2008 John Wiley & Sons, Ltd.

Lightweight natural language text compression

Variants of Huffman codes where words are taken as the source symbols are currently the most attractive choices to compress natural language text databases. In particular, Tagged Huffman Code by Moura et al. offers fast direct searching on the compressed text and random access capabilities, in exchange for producing around 11% larger compressed files. This work describes End-Tagged Dense Code and (s, c)-Dense Code, two new semistatic statistical methods for compressing natural language texts. These techniques permit simpler and faster encoding and obtain better compression ratios than Tagged Huffman Code, while maintaining its fast direct search and random access capabilities. We show that Dense Codes improve Tagged Huffman Code compression ratio by about 10%, reaching only 0.6% overhead over the optimal Huffman compression ratio. Being simpler, Dense Codes are generated 45% to 60% faster than Huffman codes. This makes Dense Codes a very attractive alternative to Huffman code variants for various reasons: they are simpler to program, faster to build, of almost optimal size, and as fast and easy to search as the best Huffman variants, which are not so close to the optimal size.