Large Vector Size Research Articles

PartialHD: Toward Efficient Hyperdimensional Computing by Partial Processing

Hyperdimensional (HD) computing is a brain inspired learning method that is widely employed in resource-constrained applications such as the internet of things (IoT) regarding its lightweight computation. Although HD computing has high efficiency in the IoT applications, it suffers from high computation due to the large vector size. Thus, several studies are proposed to speedup and increase the efficiency of HD computing. In this work, we propose a method to process HD computing called PartialHD. Our method divides a long hypervector into multiple partial vectors, and process each vector separately. In the retraining phase, this method improves accuracy up to 1.93%. Moreover, by employing our proposed method in the retraining and inference, only a few partial vectors participate and this causes the computational overhead reduces in these phases. The evaluation shows that our method processes on average 22.2% and 36.23% of the entire hypervectors with negligible accuracy loss in inference and retraining, respectively. Furthermore, we propose two general architectures (light-weight and high-speed), which accelerate partial vector computations on different FPGA platforms. The result shows that the light-weight architecture can accelerate HD computing on resource-constrained FPGAs such as Artix while the high-speed architecture has 4.72× higher throughput than light-weight architecture.

Photonic Multiply-Accumulate Operations for Neural Networks

It has long been known that photonic communication can alleviate the data movement bottlenecks that plague conventional microelectronic processors. More recently, there has also been interest in its capabilities to implement low precision linear operations, such as matrix multiplications, fast and efficiently. We characterize the performance of photonic and electronic hardware underlying neural network models using multiply-accumulate operations. First, we investigate the limits of analog electronic crossbar arrays and on-chip photonic linear computing systems. Photonic processors are shown to have advantages in the limit of large processor sizes ( ${>}\text{100}\; \mu$ m), large vector sizes ( $N > 500)$ , and low noise precision ( ${\leq} 4$ bits). We discuss several proposed tunable photonic MAC systems, and provide a concrete comparison between deep learning and photonic hardware using several empirically-validated device and system models. We show significant potential improvements over digital electronics in energy ( ${>}10^2$ ), speed ( ${>}10^3$ ), and compute density ( ${>}10^2$ ).

On Estimating the Norm of a Gaussian Vector Under Additive White Gaussian Noise

This letter considers the task of estimating the norm of an $n$ -dimensional Gaussian random vector given a noisy/perturbed observation of it. In particular, the focus is on the case of additive Gaussian noise perturbation, which is assumed to be independent of the original vector. First, an expression for the optimal estimator is derived, and then the corresponding minimum mean square error (MMSE) is computed. The regime of large vector size is also analyzed, and it is shown that the MMSE normalized by $n$ equals zero when $n \rightarrow \infty$ .

A hybrid algorithm for parallel molecular dynamics simulations

This article describes algorithms for the hybrid parallelization and SIMD vectorization of molecular dynamics simulations with short-range forces. The parallelization method combines domain decomposition with a thread-based parallelization approach. The goal of the work is to enable efficient simulations of very large (tens of millions of atoms) and inhomogeneous systems on many-core processors with hundreds or thousands of cores and SIMD units with large vector sizes. In order to test the efficiency of the method, simulations of a variety of configurations with up to 74 million atoms have been performed. Results are shown that were obtained on multi-core systems with Sandy Bridge and Haswell processors as well as systems with Xeon Phi many-core processors.

Classification Performance of Bio-Marker and Disease Word using Word Representation Models

One of the most important processesin a machine learning-based natural language processing module is to represent words by inputting the module. This can be accomplished by representing words in one-hot form with a large vector size without applying the concept of semantic similarity between words, or by word representation (word embedding) with vectors to represent lexical similarity. This has attracted keen research interest by improving the performance of several natural language processing modelssuch as syntactic parsing and sentiment analysis (also known as opinion mining). In this study, classification performance of Word2Vec, canonical correlation analysis (CCA), and GloVeare tested on a corpus that established using the titles and abstractsof 204,674biomedical articles published in PubMed. Categories include disease name, disease symptom, and ovarian cancer marker.Ovarian cancer markers were used as biomarkers.The classification performance of each word representation model for each category is visualized by mapping the results in two-dimensional word representations using t-distributed stochastic neighbor embedding (t-SNE).

On the Likelihood and Fluctuations of Gaussian Realizations

The likelihood of Gaussian realizations, as generated by the Cholesky simulation method, is analyzed in terms of Mahalanobis distances and fluctuations in the variogram reproduction. For random sampling, the probability to observe a Gaussian realization vector can be expressed as a function of its Mahalanobis distance, and the maximum likelihood depends only on the vector size. The Mahalanobis distances are themselves distributed as a Chi-square distribution and they can be used to describe the likelihood of Gaussian realizations. Their expected value and variance are only determined by the size of the vector of independent random normal scores used to generate the realizations. When the vector size is small, the distribution of Mahalanobis distances is highly skewed and most realizations are close to the vector mean in agreement with the multi-Gaussian density model. As the vector size increases, the realizations sample a region increasingly far out on the tail of the multi-Gaussian distribution, due to the large increase in the size of the uncertainty space largely compensating for the low probability density. For a large vector size, realizations close to the vector mean are not observed anymore. Instead, Gaussian vectors with Mahalanobis distance in the neighborhood of the expected Mahalanobis distance have the maximum probability to be observed. The distribution of Mahalanobis distances becomes Gaussian shaped and the bulk of realizations appear more equiprobable. However, the ratio of their probabilities indicates that they still remain far from being equiprobable. On the other hand, it is observed that equiprobable realizations still display important fluctuations in their variogram reproduction. The variance level that is expected in the variogram reproduction, as well as the variance of the variogram fluctuations, is dependent on the Mahalanobis distance. Realizations with smaller Mahalanobis distances are, on average, smoother than realizations with larger Mahalanobis distances. Poor ergodic conditions tend to generate higher proportions of flatter variograms relative to the variogram model. Only equiprobable realizations with a Mahalanobis distance equal to the expected Mahalanobis distance have an expected variogram matching the variogram model. For large vector sizes, Cholesky simulated Gaussian vectors cannot be used to explore uncertainty in the neighborhood of the vector mean. Instead uncertainty is explored around the n-dimensional elliptical envelop corresponding to the expected Mahalanobis distance.

Image coding using entropy-constrained residual vector quantization

An entropy-constrained residual vector quantization design algorithm is used to design codebooks for image coding. Entropy-constrained residual vector quantization has several important advantages. It can outperform entropy-constrained vector quantization in terms of rate-distortion performance, memory, and computation requirements. It can also be used to design vector quantizers with relatively large vector sizes and high output rates. Experimental results indicate that good image reproduction quality can be achieved at relatively low bit rates. For example, a peak signal-to-noise ratio of 30.09 dB is obtained for the 512x512 LENA image at a bit rate of 0.145 b/p.