Dictionary Size Research Articles

Compressive sensing based high-resolution passive bistatic radar

Passive bistatic radar (PBR) systems using different communication signals can only offer low-resolution target detection due to their inherent low bandwidth. In this paper, compressive sensing (CS) is applied to multichannel FM and GSM PBR to achieve improved range-Doppler resolutions and avoid some limitations of classical multiband PBR processing. In CS context, block-structured time-domain dictionary which is formed from multichannel signals suffers from coherence when fine range resolution is employed. To overcome such a pitfall, this work first transforms the dictionary to spectral domain where only the most important spectral components are retained. Principle component analysis followed by a whitening method are then applied to this spectrally transformed data in order to reduce the dimensionality of problem, thereby reducing the dictionary size and most importantly fulfilling the required condition of dictionary incoherence for better CS-based recovery. Two different block-structured dictionary formations are tested. The performance of the recovery of spatially close targets, in both FM and GSM PBR setups, are reported.

Discriminative analysis-synthesis dictionary learning for image classification

Dictionary learning has played an important role in the success of sparse representation. Although discriminative synthesis dictionary learning for sparse representation with a high-computational-complexity l0 or l1 norm constraint has been well studied for image classification, jointly and discriminatively learning an analysis dictionary and a synthesis dictionary is still in its infant stage. As a dual of synthesis dictionary, the recently developed analysis dictionary can provide a complementary view of data representation, which can have a much lower time complexity than sparse synthesis representation. Although several class-specific analysis-synthesis dictionary, which may have a big correlation between different classes' dictionaries, have been developed, how to learn a more compact and discriminative universal analysis-synthesis dictionary is still open. In this paper, to provide a more complete view of discriminative data representation, we propose a novel model of discriminative analysis-synthesis dictionary learning (DASDL), in which a linear classifier based on the coding coefficient is jointly learned with the dictionary pair, thus the performance of the classifier and the representational power of the dictionary pair being considered at the same time by the same optimization procedure. The size of the learned dictionaries can be very small since the analysis-synthesis dictionary is shared by all class data. An iterative algorithm to efficiently solve the proposed DASDL is presented in this paper. The experiments on face recognition, gender classification, action recognition and image classification clearly show the superiority of the proposed DASDL.

Why Does a Hilbertian Metric Work Efficiently in Online Learning With Kernels?

The autocorrelation matrix of the kernelized input vector is well approximated by the squared Gram matrix (scaled down by the dictionary size). This holds true under the condition that the input covariance matrix in the feature space is approximated by its sample estimate based on the dictionary elements, leading to a couple of fundamental insights into online learning with kernels. First, the eigenvalue spread of the autocorrelation matrix relevant to the hyperplane projection along affine subspace algorithm is approximately a square root of that for the kernel normalized least mean square algorithm. This clarifies the mechanism behind fast convergence due to the use of a Hilbertian metric. Second, for efficient function estimation, the dictionary needs to be constructed in general by taking into account the distribution of the input vector, so as to satisfy the condition. The theoretical results are justified by computer experiments.

Dependent nonparametric bayesian group dictionary learning for online reconstruction of dynamic MR images

In this paper, we introduce a dictionary learning based approach applied to the problem of real-time reconstruction of MR image sequences that are highly undersampled in k-space. Unlike traditional dictionary learning, our method integrates both global and patch-wise (local) sparsity information and incorporates some priori information into the reconstruction process. Moreover, we use a Dependent Hierarchical Beta-process as the prior for the group-based dictionary learning, which adaptively infers the dictionary size and the sparsity of each patch; and also ensures that similar patches are manifested in terms of similar dictionary atoms. An efficient numerical algorithm based on the alternating direction method of multipliers (ADMM) is also presented. Through extensive experimental results we show that our proposed method achieves superior reconstruction quality, compared to the other state-of-the- art DL-based methods.

Accurate HEp-2 cell classification based on sparse bag of words coding

Autoimmune diseases (AD) are the abnormal response of the immune system of the body to healthy tissues. ADs have generally been on the increase. Efficient computer aided diagnosis of ADs through classification of the human epithelial type 2 (HEp-2) cells become beneficial. These methods make lower diagnosis costs, faster response and better diagnosis repeatability. In this paper, we present an automated HEp-2 cell image classification technique that exploits the sparse coding of the visual features together with the Bag of Words model (SBoW). In particular, SURF (Speeded Up Robust Features) and SIFT (Scale-invariant feature transform) features are specially integrated to work in a complementary fashion. This method helps greatly improve the cell classification accuracy. Additionally, a hierarchical max-pooling method is proposed to aggregate the local sparse codes in different layers to provide final feature vector. Furthermore, various parameters of the dictionary learning including the dictionary size, the learning iteration number, and the pooling strategy is also investigated. Experiments conducted on publicly available datasets show that the proposed technique clearly outperforms state-of-the-art techniques in cell and specimen levels.

Towards Dictionaries of Optimal Size: A Bayesian Non Parametric Approach

Solving inverse problems usually calls for adapted priors such as the definition of a well chosen representation of possible solutions. One family of approaches relies on learning redundant dictionaries for sparse representation. In image processing, dictionary learning is applied to sets of patches. Many methods work with a dictionary with a number of atoms that is fixed in advance. Moreover optimization methods often call for the prior knowledge of the noise level to tune regularization parameters. We propose a Bayesian non parametric approach that is able to learn a dictionary of adapted size. The use of an Indian Buffet Process prior permits to learn an adequate number of atoms. The noise level is also accurately estimated so that nearly no parameter tuning is needed. We illustrate the relevance of the resulting dictionaries on numerical experiments.

Combination of spatio-temporal and transform domain for sparse occlusion estimation by optical flow

Lack of information in occluded regions leads to ambiguity inherent, which is a big challenge for motion estimation. Recently, the sparse model has been widely used since the essential content of the motion field could be effectively preserved with sparse representation. The methods exploiting sparsity acquire representations either directly in the spatio-temporal domain or indirectly in the transform domain. Usually, the sparse model with sparsifying transform is based on patches and thus is more robust agints noise, while the sparse model without sparsifying transform can directly work for an overall image treatment. Aiming at tackling the motion ambiguity efficiently, this paper employs a distinct sparse representation model into a variational framework for estimating occlusion with optical flow. In order to deal with dictionary learning which is computationally expensive and requires a preprocess for extending the sparsifying transform model for arbitrary image sizes, we present a new unified framework to directly generate an overall dictionary via the sparse model without sparsifying transform, and then optimize for small size dictionaries over corresponding patches with the overall dictionary. Our framework is based on the Stein–Weiss analysis function acting as a novel regulariser and a sparsifying transform function respectively in variational and sparsity models. Experiments show that the proposed method outperforms the existing estimation methods of jointing occlusion and optical flow.

Robust Bayesian non-parametric dictionary learning with heterogeneous Gaussian noise

Bayesian non-parametric dictionary learning has become popular in computer vision applications due to its ability of dictionary size decision. A common assumption of this modelling approach is to place Gaussian priors on both dictionary matrix and weighting matrix. Although such simple treatment has a number of merits such as conjugate priors and easy inference, it may violate the reality since there may exist heterogeneous noise in a digital image. In this paper, we consider a general noise model for Bayesian non-parametric dictionary learning, which is able to adapt images with heterogeneous Gaussian noise. To this end, we adopt Student’s t distributions as priors of heterogeneous noise for both dictionary matrix and weighting matrix. As an infinite Gaussian scale mixture, Student’s t not only retains the similar properties as Gaussian but also tolerates different scales of noise. We propose an approximate inference algorithm, combining Gibbs sampling and empirical Bayesian, to estimate the posterior distribution of parameters. The experimental results show that the proposed model can clearly outperform the counterpart with Gaussian prior and the prevailing parametric methods in image de-noising with heterogeneous noise.

Improving of entropy adaptive on-line compression

Since energy efficiency, high bandwidth, and low transmission delay are challenging issues in mobile networks, due to resource constraints, there is a great importance in designing of new communication methods. In particular, lossless data compression may provide high performance under constrained resources. In this paper we present a novel on-line and entropy adaptive compression scheme for streaming unbounded length inputs. The scheme extends the window dictionary Lempel–Ziv compression and is adaptive and tailored to compress on-line non entropy stationary inputs. Specifically, the window dictionary size is changed in an adaptive manner to fit the current best compression rate for the input. On-line entropy adaptive compression scheme (EAC), introduced and analyzed in this paper, examines all possible sliding window sizes over the next input portion to choose the optimal window size for this portion; a size that implies the best compression ratio. The size found is then used in the actual compression of this portion. We suggest an adaptive encoding scheme, which optimizes the parameters block by block, and base the compression performance on the optimality proof of LZ77 when applied to blocks (Ziv in IEEE Trans Inf Theory 55(5):1941–1944, 2009). This adaptivity can be useful for many communication tasks. In particular, providing efficient utilization of energy consuming wireless devices by data compression. Due to the dynamic and non-uniform structure of multimedia data, adaptive approaches for data processing are of special interest. The EAC scheme was tested on different types of files (docx, ppt, jpeg, xls) and over synthesized files that were generated as segments of homogeneous Markov Chains. Our experiments demonstrate that the EAC scheme typically provides a higher compression ratio than LZ77 does, when examined in the scope of on-line per-block compression of transmitted (or compressed) files. We propose techniques intended to control the adaptive on-line compression process by estimating relative entropy between two sequential blocks of data. This approach may enhance performance of the mobile networks.

Sparse feature learning for instrument identification: Effects of sampling and pooling methods.

Feature learning for music applications has recently received considerable attention from many researchers. This paper reports on the sparse feature learning algorithm for musical instrument identification, and in particular, focuses on the effects of the frame sampling techniques for dictionary learning and the pooling methods for feature aggregation. To this end, two frame sampling techniques are examined that are fixed and proportional random sampling. Furthermore, the effect of using onset frame was analyzed for both of proposed sampling methods. Regarding summarization of the feature activation, a standard deviation pooling method is used and compared with the commonly used max- and average-pooling techniques. Using more than 47 000 recordings of 24 instruments from various performers, playing styles, and dynamics, a number of tuning parameters are experimented including the analysis frame size, the dictionary size, and the type of frequency scaling as well as the different sampling and pooling methods. The results show that the combination of proportional sampling and standard deviation pooling achieve the best overall performance of 95.62% while the optimal parameter set varies among the instrument classes.

Dynamic with Dictionary Technique for Arabic Text Compression

In this research paper we build a new, reliable, and sufficient algorithm for Arabic text language. The proposed algorithm should combine the features of the Huffman and Lempel Ziv algorithms, and is expected be able to reduce the general compression ratio . Our approach is different from Huffman algorithm in the sense that it assigns codes to n-gram symbols where n is a positive integer that is greater than or equal to one. Compared to Huffman algorithm, which assigns a code to each symbol individually, our approach is expected to assign codes to symbols in average. Our approach is different from Lempel Ziv algorithm in the sense that the size of dictionary that we build does not grow in an uncontrolled manner. The size of the dictionary is fixed and its size can be expected prior to process the text files that are to be compressed. This is because the size of each word in the dictionary we build is fixed and is equal to n. So for example, given that the number of different symbols in the text file at hand is m and that n is 2, the total number of entries in the dictionary that we propose to build will be m*m in the worst case.

Vision-based Classification of Pakistani Sign Language

Automated sign language recognition is one of the important areas of computer vision today, because of its applicability in vast fields of life. This paper presents automated recognition of signs taken from Pakistani Sign Language (PSL). The paper presents empirical analysis of two statistical and one transformation based shape descriptors for the recognition of PSL. A purely vision based, efficient, signer independent, multi-aspect invariant method is proposed for the recognition of 44 signs of PSL. The method has proved its worth by utilizing a very small shape descriptor and giving promising results for a reasonable size of sign dictionary. The proposed methodology achieved an accuracy of 92%.

Rank-Frequency Analysis of Characters in Garhwali Text: Emergence of Zipf's Law

Zipf's law is ubiquitous in a language system, which establishes a relation between rank and frequency of characters or words. In the present study, it is shown that the distribution of character frequencies for Garhwali language follows Zipf-Mandelbrot law. Garhwali language is an Indo-Aryan language, spoken in the Garhwal region of Uttarakhand, India (northwestern Himalayan belt of India). The present communication examines the rank-frequency distribution by generalization of Zipf-Mandelbrot law in Garhwali language having limited dictionary size. The study shows that the distribution of character frequencies of consonants (with matras), vowels (including vowels with consonants in shape of matras) and all characters (including vowels and consonants without matras) for continuous Garhwali corpus follows Zipf-Mandelbrot law.

Fast Traffic Sign Recognition via High-Contrast Region Extraction and Extended Sparse Representation

In this paper, we propose a high-performance traffic sign recognition (TSR) framework to rapidly detect and recognize multiclass traffic signs in high-resolution images. This framework includes three parts: a novel region-of-interest (ROI) extraction method called the high-contrast region extraction (HCRE), the split-flow cascade tree detector (SFC-tree detector), and a rapid occlusion-robust traffic sign classification method based on the extended sparse representation classification (ESRC). Unlike the color-thresholding or extreme region extraction methods used by previous ROI methods, the ROI extraction method of the HCRE is designed to extract ROI with high local contrast, which can keep a good balance of the detection rate and the extraction rate. The SFC-tree detector can detect a large number of different types of traffic signs in high-resolution images quickly. The traffic sign classification method based on the ESRC is designed to classify traffic signs with partial occlusion. Instead of solving the sparse representation problem using an overcomplete dictionary, the classification method based on the ESRC utilizes a content dictionary and an occlusion dictionary to sparsely represent traffic signs, which can largely reduce the dictionary size in the occlusion-robust dictionaries and achieve high accuracy. The experiments demonstrate the advantage of the proposed approach, and our TSR framework can rapidly detect and recognize multiclass traffic signs with high accuracy.

Bayesian Recurrent Neural Network for Language Modeling.

A language model (LM) is calculated as the probability of a word sequence that provides the solution to word prediction for a variety of information systems. A recurrent neural network (RNN) is powerful to learn the large-span dynamics of a word sequence in the continuous space. However, the training of the RNN-LM is an ill-posed problem because of too many parameters from a large dictionary size and a high-dimensional hidden layer. This paper presents a Bayesian approach to regularize the RNN-LM and apply it for continuous speech recognition. We aim to penalize the too complicated RNN-LM by compensating for the uncertainty of the estimated model parameters, which is represented by a Gaussian prior. The objective function in a Bayesian classification network is formed as the regularized cross-entropy error function. The regularized model is constructed not only by calculating the regularized parameters according to the maximum a posteriori criterion but also by estimating the Gaussian hyperparameter by maximizing the marginal likelihood. A rapid approximation to a Hessian matrix is developed to implement the Bayesian RNN-LM (BRNN-LM) by selecting a small set of salient outer-products. The proposed BRNN-LM achieves a sparser model than the RNN-LM. Experiments on different corpora show the robustness of system performance by applying the rapid BRNN-LM under different conditions.

Practical compressed string dictionaries

The need to store and query a set of strings – a string dictionary – arises in many kinds of applications. While classically these string dictionaries have accounted for a small share of the total space budget (e.g., in Natural Language Processing or when indexing text collections), recent applications in Web engines, Semantic Web (RDF) graphs, Bioinformatics, and many others handle very large string dictionaries, whose size is a significant fraction of the whole data. In these cases, string dictionary management is a scalability issue by itself. This paper focuses on the problem of managing large static string dictionaries in compressed main memory space. We revisit classical solutions for string dictionaries like hashing, tries, and front-coding, and improve them by using compression techniques. We also introduce some novel string dictionary representations built on top of recent advances in succinct data structures and full-text indexes. All these structures are empirically compared on a heterogeneous testbed formed by real-world string dictionaries. We show that the compressed representations may use as little as 5% of the original dictionary size, while supporting lookup operations within a few microseconds. These numbers outperform the state-of-the-art space/time tradeoffs in many cases. Furthermore, we enhance some representations to provide prefix- and substring-based searches, which also perform competitively. The results show that compressed string dictionaries are a useful building block for various data-intensive applications in different domains.

Novel Tactile-SIFT Descriptor for Object Shape Recognition

Using a tactile array sensor to recognize an object often requires multiple touches at different positions. This process is prone to move or rotate the object, which inevitably increases difficulty in object recognition. To cope with the unknown object movement, this paper proposes a new tactile-SIFT descriptor to extract features in view of gradients in the tactile image to represent objects, to allow the features being invariant to object translation and rotation. The tactile-SIFT segments a tactile image into overlapping subpatches, each of which is represented using a $dn$ -dimensional gradient vector, similar to the classic SIFT descriptor. Tactile-SIFT descriptors obtained from multiple touches form a dictionary of $k$ words, and the bag-of-words method is then used to identify objects. The proposed method has been validated by classifying 18 real objects with data from an off-the-shelf tactile sensor. The parameters of the tactile-SIFT descriptor, including the dimension size $dn$ and the number of subpatches $sp$ , are studied. It is found that the optimal performance is obtained using an 8-D descriptor with three subpatches, taking both the classification accuracy and time efficiency into consideration. By employing tactile-SIFT, a recognition rate of 91.33% has been achieved with a dictionary size of 50 clusters using only 15 touches.

Color Sparse Representations for Image Processing: Review, Models, and Prospects

Sparse representations have been extended to deal with color images composed of three channels. A review of dictionary-learning-based sparse representations for color images is made here, detailing the differences between the models, and comparing their results on the real and simulated data. These models are considered in a unifying framework that is based on the degrees of freedom of the linear filtering/transformation of the color channels. Moreover, this allows it to be shown that the scalar quaternionic linear model is equivalent to constrained matrix-based color filtering, which highlights the filtering implicitly applied through this model. Based on this reformulation, the new color filtering model is introduced, using unconstrained filters. In this model, spatial morphologies of color images are encoded by atoms, and colors are encoded by color filters. Color variability is no longer captured in increasing the dictionary size, but with color filters, this gives an efficient color representation.

Multiple Sparse Representations Classification.

Sparse representations classification (SRC) is a powerful technique for pixelwise classification of images and it is increasingly being used for a wide variety of image analysis tasks. The method uses sparse representation and learned redundant dictionaries to classify image pixels. In this empirical study we propose to further leverage the redundancy of the learned dictionaries to achieve a more accurate classifier. In conventional SRC, each image pixel is associated with a small patch surrounding it. Using these patches, a dictionary is trained for each class in a supervised fashion. Commonly, redundant/overcomplete dictionaries are trained and image patches are sparsely represented by a linear combination of only a few of the dictionary elements. Given a set of trained dictionaries, a new patch is sparse coded using each of them, and subsequently assigned to the class whose dictionary yields the minimum residual energy. We propose a generalization of this scheme. The method, which we call multiple sparse representations classification (mSRC), is based on the observation that an overcomplete, class specific dictionary is capable of generating multiple accurate and independent estimates of a patch belonging to the class. So instead of finding a single sparse representation of a patch for each dictionary, we find multiple, and the corresponding residual energies provides an enhanced statistic which is used to improve classification. We demonstrate the efficacy of mSRC for three example applications: pixelwise classification of texture images, lumen segmentation in carotid artery magnetic resonance imaging (MRI), and bifurcation point detection in carotid artery MRI. We compare our method with conventional SRC, K-nearest neighbor, and support vector machine classifiers. The results show that mSRC outperforms SRC and the other reference methods. In addition, we present an extensive evaluation of the effect of the main mSRC parameters: patch size, dictionary size, and sparsity level.

Referral system for hard exudates in eye fundus

Hard exudates are one of the most common anomalies/artifacts found in the eye fundus of patients suffering from diabetic retinopathy. These exudates are the major cause of loss of sight or blindness in people having diabetic retinopathy. Diagnosis of hard exudates requires considerable time and effort of an ophthalmologist. The ophthalmologists have become overloaded, so that there is a need for an automated diagnostic/referral system. In this paper a referral system for the hard exudates in the eye-fundus images has been presented. The proposed referral system works by combining different techniques like Scale Invariant Feature Transform (SIFT), K-means Clustering, Visual Dictionaries and Support Vector Machine (SVM). The system was also tested with Back Propagation Neural Network as a classifier. To test the performance of the system four fundus image databases were used. One publicly available image database was used to compare the performance of the system to the existing systems. To test the general performance of the system when the images are taken under different conditions and come from different sources, three other fundus image databases were mixed. The evaluation of the system was also performed on different sizes of the visual dictionaries. When using only one fundus image database the area under the curve (AUC) of maximum 0.9702 (97.02%) was achieved with accuracy of 95.02%. In case of mixed image databases an AUC of 0.9349 (93.49%) was recorded having accuracy of 87.23%. The results were compared to the existing systems and were found better/comparable.