Vector Quantization Theory Research Articles

Mad Max: Affine Spline Insights Into Deep Learning

We build a rigorous bridge between deep networks (DNs) and approximation theory via spline functions and operators. Our key result is that a large class of DNs can be written as a composition of max-affine spline operators (MASOs) that provide a powerful portal through which we view and analyze their inner workings. For instance, conditioned on the spline partition region containing the input signal, the output of an MASO DN can be written as a simple affine transformation of the input. This implies that a DN constructs a set of signal-dependent, class-specific templates against which the signal is compared via a simple inner product; we explore the links to the classical theory of optimal classification via matched filters and the effects of data memorization. Going further, we propose a simple penalty term that can be added to the cost function of any DN learning algorithm to force the templates to be orthogonal with each other; this leads to significantly improved classification performance and reduced overfitting with no change to the DN architecture. The spline partition of the input signal space that is implicitly induced by an MASO directly links DNs to the theory of vector quantization (VQ) and K-means clustering, which opens up new geometric avenues to study how DNs organize signals in a hierarchical fashion. To validate the utility of the VQ interpretation, we develop and validate a new distance metric for signals and images that quantify the difference between their VQ encodings.

Multidimensional Vector Quantization-Based Signal Constellation Design Enabling Beyond 1 Pb/s Serial Optical Transport Networks

In this paper, we propose a hybrid multidimensional coded-modulation (CM) scheme based on a new multidimensional signal constellation design as a solution to limited bandwidth and high energy consumption of information infrastructure. This multidimensional signal constellation, herewith called vector-quantization-based signal constellation design (VQ-SCD), is based on the vector quantization theory. The proposed scheme employs both the electrical basis functions (in the form of the prolate spheroidal wave functions) and the optical basis functions (in the form of polarization and spatial mode states) as optical basis functions. The proposed coded VQ-SCD scheme is the enabling technology for optical serial transport with bit rates exceeding 1000 Tb/s (1 Pb/s). In addition, the CM scheme we proposed allows for the adaptive, elastic, and dynamic allocation of the bandwidth. This fine granularity bandwidth manipulation is envisioned as a part of future software-defined optical networking.

Scene Categorization with Classified Codebook Model

This paper presents an efficient yet powerful codebook model, named classified codebook model, to categorize natural scene category. The current codebook model typically resorts to large codebook to obtain higher performance for scene categorization, which severely limits the practical applicability of the model. Our model formulates the codebook model with the theory of vector quantization, and thus uses the famous technique of classified vector quantization for scene-category modeling. The significant feature in our model is that it is beneficial for scene categorization, especially at small codebook size, while saving much computation complexity for quantization. We evaluate the proposed model on a well-known challenging scene dataset: 15 Natural Scenes. The experiments have demonstrated that our model can decrease the computation time for codebook generation. What is more, our model can get better performance for scene categorization, and the gain of performance becomes more pronounced at small codebook size.

Real time digital signal processing implementation for an APD-based PET scanner with phoswich detectors

Recent progress in advanced digital signal processing provides an opportunity to expand the computation power required for real time extraction of event characteristics in avalanche photodiode (APD)-based Positron Emission Tomography (PET) scanners. These developments are made possible by a highly parallel data acquisition (DAQ) system based on an integrated analog front-end and a high-speed fully digital signal processing section that directly samples the output of each preamplifier with a free-running, off-the-shelf, 45-MHz analog-to-digital converter that feeds the sampled data into a field programmable gate array (FPGA) VirtexII PRO from Xilinx. This FPGA features /spl sim/ 31 000 logic cells and two PowerPC processors, which allows up to 64 channels to be processed simultaneously. Each channel has its own digital signal processing chain including a trigger, a baseline restorer and a timestamp algorithm. Various timestamp algorithms have been tested so far, achieving a coincidence timing resolution of 3.2-ns full-width at half-maximum (FWHM) for APD coupled to Lutetium Oxyorthosilicate (APD-LSO) and 11.4-ns FWHM for APD coupled to Bismuth Germanium Oxide (APD-BGO) detectors, respectively. Channels are then multiplexed into a DSP processor from Texas Instruments for crystal identification by an ARMAX recursive algorithm borrowed from identification and vector quantization theory. The system can sustain an event rate of 10 000 events/s/channel without electronic dead time.

High-density model for server allocation and placement

It is well known that optimal server placement is NP-hard. We present an approximate model for the case when both clients and servers are dense, and propose a simple server allocation and placement algorithm based on high-rate vector quantization theory. The key idea is to regard the location of a request as a random variable with probability density that is proportional to the demand at that location, and the problem of server placement as source coding, i.e., to optimally map a source value (request location) to a code-word (server location) to minimize distortion (network cost). This view has led to a joint server allocation and placement algorithm that has a time-complexity that is linear in the number of clients. Simulations are presented to illustrate its performance.

Matrix quantization for very low data rate voice coding

Matrix quantization extends the theory of vector quantization for voice coding to take advantage of the phonological and phonotactic properties of speech. The speech signal is first modeled by LPC spectra at 10‐ms time intervals. By adopting an acoustically defined (and demisyllable like) automatic segmentation procedure, consecutive LPC spectra are grouped into LPC coefficient matrices. A distortion measure that consists of nonlinear time warping and a standard LPC distance metric is also developed for comparing the “likeliness” of two matrices. The matrix codebook is generated from a speech database using a simple mini‐max procedure and the matrix distortion measure. A nearest‐neighbor full search procedure is used during speech encoding. Data rates under 100 bit/s can be achieved for the LPC spectra. Results for a single speaker experiment based on the matrix quantization method and subsequent research on very similar techniques will be discussed.

A Vector Quantizer Combining Energy and LPC Parameters and Its Application to Isolated Word Recognition

The theory of vector quantization (VQ) of linear predictive coding (LPC) coefficients has established a wide variety of techniques for quantizing LPC spectral shape to minimize overall spectral distortion. Such vector quantizers have been widely used in the areas of speech coding and speech recognition. The conventional vector quantizer utilizes only spectral shape information and essentially disregards the energy or gain term associated with the optimal LPC fit to the signal being modeled. In this paper we present a method of incorporating LPC spectral shape and energy into the code-book entries of the vector quantizer. To do this, we postulate a distortion measure for comparing two LPC vectors that uses the weighted sum of an LPC shape distortion and a log energy distortion. Based on this combined distortion measure, we have designed and studied vector quantizers of several sizes for use in isolated word speech recognition experiments. We found that a fairly significant correlation exists between LPC shape and signal energy. Hence, an LPC shape combined with energy vector quantizer with a given distortion requires far fewer code-book entries than one in which LPC shape and energy are quantized separately. Based on isolated word recognition tests on both a 10-digit and a 129-word airlines vocabulary, we found improvements in recognition accuracy by using the VQ with both LPC shape and energy over that obtained using a VQ with LPC shape alone.

An 800 bit/s vector quantization LPC vocoder

An 800 bit/s vector quantization linear predictive coding (LPC) vocoder has been developed. The recently developed LPC vector quantization theory is applied to reduce the bit rate for LPC coefficients coding by a factor of four. Branch search techniques and separation of voiced and unvoiced codebooks are applied for better algorithm efficiency. Differential coding is applied to reduce the bit rate for the pitch and gain parameters by one third. Formal subjective evaluation shows that the 800 bit/s vocoder preserves most of the intelligibility of an LPC system. It is also robust under different transmission error and acoustic conditions. Informal listening comparisons show the quality to be acceptable and sometimes very close to 2400 bit/s LPC speech. The computational cost of the 800 bit/s vocoder is equivalent to or even lower than the 2400 bit/s LPC-10. Compatibility with any LPC-10 vocoder is guaranteed because the 800 bit/s design only differs in the quantization and encoding algorithms. Further bit rate reduction can be achieved by removing frame to frame redundancy in the code.