Reduce Storage Requirements Research Articles

Reducing the Storage Requirements of a Test Sequence by Using One or Two Background Vectors

We describe a storage scheme for functional test sequences where a test sequence T is associated with a primary input vector B called a background vector. T is stored by storing only the differences between its test vectors and B . We describe a procedure for computing a background vector B for a given test sequence T. We also describe a procedure that modifies T so as to reduce its storage requirements with respect to B . We present experimental results demonstrating that the single background vector B, computed based on T, allows T to be modified such that a vast majority of its entries are equal to the corresponding entries of B. Consequently, storage of T reduces to storage of a small number of entries. We also extend the discussion to storage of T based on two background vectors. A second background vector provides more flexibility in storing T as a list of entries where it is different from its background vectors. This contributes to a further reduction in storage requirements for certain circuits.

Compression aware physical database design

Modern RDBMSs support the ability to compress data using methods such as null suppression and dictionary encoding. Data compression offers the promise of significantly reducing storage requirements and improving I/O performance for decision support queries. However, compression can also slow down update and query performance due to the CPU costs of compression and decompression. In this paper, we study how data compression affects choice of appropriate physical database design, such as indexes, for a given workload. We observe that approaches that decouple the decision of whether or not to choose an index from whether or not to compress the index can result in poor solutions. Thus, we focus on the novel problem of integrating compression into physical database design in a scalable manner. We have implemented our techniques by modifying Microsoft SQL Server and the Database Engine Tuning Advisor (DTA) physical design tool. Our techniques are general and are potentially applicable to DBMSs that support other compression methods. Our experimental results on real world as well as TPC-H benchmark workloads demonstrate the effectiveness of our techniques.

Using clustering techniques for intelligent camera-based user interfaces

The area of Human-Machine Interface is growing fast due to its high importance in all technological systems. The basic idea behind designing human-machine interfaces is to enrich the communication with the technology in a natural and easy way. Gesture interfaces are a good example of transparent interfaces. Such interfaces must identify properly the action the user wants to perform, so the proper gesture recognition is of the highest importance. However, most of the systems based on gesture recognition use complex methods requiring high-resource devices. In this work, we propose to model gestures capturing their temporal properties, which significantly reduce storage requirements, and use clustering techniques, namely self-organizing maps and unsupervised genetic algorithm, for their classification. We further propose to train a certain number of algorithms with different parameters and combine their decision using majority voting in order to decrease the false positive rate. The main advantage of the approach is its simplicity, which enables the implementation using devices with limited resources, and therefore low cost. The testing results demonstrate its high potential.

Should Pruning be a Pre-Processor of any Linear System?

There are many real-world problems whose mathematical models turn out to be linear systems Ax = b, where A is an m x n matrix. Each equation of the linear system is an information. An information, in a physical problem, such as 4 mangoes, 6 bananas, and 5 oranges cost $10, is mathematically modeled as an equation 4x(sub 1) + 6x(sub 2) + 5x(sub 3) = 10 , where x(sub 1), x(sub 2), x(sub 3) are each cost of one mango, that of one banana, and that of one orange, respectively. All the information put together in a specified context, constitutes the physical problem and need not be all distinct. Some of these could be redundant, which cannot be readily identified by inspection. The resulting mathematical model will thus have equations corresponding to this redundant information and hence are linearly dependent and thus superfluous. Consequently, these equations once identified should be better pruned in the process of solving the system. The benefits are (i) less computation and hence less error and consequently a better quality of solution and (ii) reduced storage requirements. In literature, the pruning concept is not in vogue so far although it is most desirable. It is assumed that at least one information, i.e. one equation is known to be correct and which will be our first equation. In a numerical linear system, the system could be slightly inconsistent or inconsistent of varying degree. If the system is too inconsistent, then we should fall back on to the physical problem (PP), check the correctness of the PP derived from the material universe, modify it, if necessary, and then check the corresponding mathematical model (MM) and correct it. In nature/material universe, inconsistency is completely nonexistent. If the MM becomes inconsistent, it could be due to error introduced by the concerned measuring device and/or due to assumptions made on the PP to obtain an MM which is relatively easily solvable or simply due to human error. No measuring device can usually measure a quantity with an accuracy greater that 0.005% or, equivalently with a relative error less than 0.005%. Hence measurement error is unavoidable in a numerical linear system when the quantities are continuous (or even discrete with extremely large number). Assumptions, though not desirable, are usually made when we find the problem sufficiently difficult to be solved within the available means/tools/resources and hence distort the PP and the corresponding MM. The . error thus introduced in the system could (not always necessarily though) make the system somewhat inconsistent. If the inconsistency (contradiction) is too much then one should definitely not proceed to solve the system in terms of getting a least-squares solution or the minimum-norm least-squares solution. All these solutions will be invariably of no real-world use. If, on the other hand, inconsistency is reasonably low, i.e. the system is near-consistent or, equivalently, has near-linearly-dependent rows, then the foregoing solutions are useful. Pruning in such a near-consistent system should be performed based on the desired accuracy and on the definition of near-linear dependence. In this article, we discuss pruning over various kinds of linear systems and strongly suggest its use as a pre-processor or as a part of an algorithm. Ideally pruning should (i) be a part of the solution process (algorithm) of the system, (ii) reduce both computational error and complexity of the process, and (iii) take into account the numerical zero defined in the context. These are precisely what we achieve through our proposed O(mn2) algorithm presented in Matlab, that uses a subprogram of solving a single linear equation and that has embedded in it the pruning.

Discrete-time simulation of a peak current controlled DC/DC buck converter using modal decomposition

This study presents an accurate and fast method for large-signal discrete-time simulation of current controlled DC/DC buck converter in continuous conduction mode. It employs modal decomposition of the state transition matrix for each topology, resulting in an exact and computationally efficient set of decoupled discrete-time state equations. This enables one to obtain an accurate solution for duty ratios iteratively, by equating the switching conditions of the state variables with state equations, which are non-linear in duty ratio. In the absence of a compensating ramp, an efficient way to compute duty ratios explicitly, without iteration, is also suggested. Subsequently, state variables are propagated through the ON and OFF periods, using the state equations exactly but without the need to compute a matrix exponential. This way numerical integration at multiple intermediate points between two switching instants of interest is avoided, which makes the simulation considerably faster, leading to significantly reduced storage requirement compared to common simulation methods, such as using SPICE. It is shown under different parametric conditions that the proposed method has superior accuracy over several approximate simulation methods proposed in the literature. The method can be generalised for other converter topologies, operational modes and control configurations with appropriate changes.

Multilevel-Structured Low-Density Parity-Check Codes for AWGN and Rayleigh Channels

We propose a novel class of protograph low-density parity-check (LDPC) codes having a combinatorial rather than a random structure, which are termed multilevel-structured (MLS) LDPC codes. It is demonstrated that they posses a strikingly simple structure and, thus, benefit from reduced storage requirements, hardware-friendly implementations, and low-complexity encoding. Our simulation results provided for both additive white Gaussian noise (AWGN) and uncorrelated Rayleigh (UR) channels demonstrate that these advantages accrue without compromising the attainable bit error ratio (BER) and block error ratio (BLER) performance, when compared with their previously proposed more complex random-construction-based counterparts, as well as with other structured codes of the same length.

Inertial force term approximations for the use of Global Modal Parameterization for planar mechanisms

Recently a new model reduction technique for flexible multibody systems has been introduced, namely the Global Modal Parameterization (GMP). This method uses a system-level model reduction, in which the reduction is parameterized by the rigid position of the system and a small flexible deformation superposed on this rigid position. Because of the non-linearity, the projection base for reduction is position dependent. This position dependence leads to strongly non-linear inertial terms, with associated increased storage requirements and computational load. This paper gives a short overview of the GMP-method and focuses predominantly on the derivation of the reduced inertial forces. Three different approximations for the inertial terms are presented and numerically validated. It is shown that a substantial reduction in storage requirements, while maintaining accurate results, is possible through these approximations. Copyright © 2010 John Wiley & Sons, Ltd.

Color-Aware Instructions for Embedded Superscalar Processors

This paper introduces a color-aware instruction set extension that enhances the performance and efficiency in the processing of color images and video. Traditional multimedia extensions (e.g., MMX, VIS, and MDMX) depend solely on generic subword parallelism whereas the proposed color-aware instruction set (CAX) supports parallel operations on two-packed 16-bit (6:5:5) YCbCr (luminance-chrominance) values. A 16-bit YCbCr representation reduces storage requirements by 33% over the baseline 24-bit YCbCr representation while maintaining satisfactory image quality. Experimental results on an identically configured, dynamically scheduled superscalar processor indicate that CAX outperforms MDMX (a representative MIPS multimedia extension) in terms of speedup (3.9× baseline ISA with CAX versus 2.1× with MDMX) and energy reduction (75.8% reduction over baseline with CAX versus 54.8% reduction with MDMX). CAX can improve the performance and efficiency of future embedded color imaging products.

Cholesky decomposition within local multireference singles and doubles configuration interaction

A local multireference singles and doubles configuration interaction method in which Cholesky vectors are used in place of conventional two-electron integrals has been developed (CD-LMRSDCI). To reduce the overall cost associated with our linear scaling LMRSDCI method presented earlier [T. S. Chwee et al., J. Chem. Phys. 128, 224106 (2008)], we adopt a two-pronged approach. First, localized orthogonal virtual orbitals, introduced by Subotnik et al. [J. Chem. Phys. 123, 114108 (2005)], are substituted for nonorthogonal projected atomic orbitals. This obviates the need for contraction with overlap matrices and simplifies our working formalism. In addition, we restructure the rate-limiting step of our LMRSDCI algorithm to be driven by the search for two-electron integrals instead of configuration state functions. The shift necessitates a flexible way of processing the four-indexed two-electron integrals, which is facilitated by use of two-indexed Cholesky vectors. Our restructured LMRSDCI method is an order of magnitude faster and has greatly reduced storage requirements so that we are able to apply it to molecules containing up to 50 heavy atoms. However, generation of the Cholesky vectors and their subsequent transformation to the molecular orbital (MO) basis is not linear scaling. Together with assembling the MO integrals from the Cholesky vectors, these now constitute the rate-limiting steps in our method.

An advanced fully 3D OSEM reconstruction for positron emission tomography

A fully 3D OSEM reconstruction method for positron emission tomography (PET) based on symmetries and sparse matrix technique is described. Great savings in both storage space and computation time were achieved by exploiting the symmetries of scanner and sparseness of the system matrix. More reduction of storage requirement was obtained by introducing the approximation of system matrix. Iteration-filter was performed to restrict image noise in reconstruction. Performances of simulation data and phantom data got from Micro-PET (Type: Epuls-166) demonstrated that similar image quality was achieved using the approximation of the system matrix.

Large Scale Simulation with Scaled Boundary Finite Element Method

AbstractNowadays scientific and engineering applications often require wave propagation in infinite or unbounded domains. In order to model such applications we separate our model into near‐field and far‐field. The near‐field is represented by the well‐known finite element method (FEM), whereas the far‐field is mapped by a scaled boundary finite element (SBFE) approach. This latter approach allows wave propagation in infinite domains and suppresses the reflection of waves at the boundary, thus being a suitable method to model wave propagation to infinity. It is non‐local in time and space. From a computational point of view, those characteristics are a drawback because they lead to storage consuming calculations with high computational time‐effort. The non‐locality in space causes fully populated unit‐impulse acceleration influence matrices for each time step, leading to immense storage consumption for problems with a large number of degrees of freedom. Additionally, a different influence matrix has to be assembled for each time step which yields unacceptable storage requirements for long simulation times. For long slender domains, where many nodes are rather far from each other and where the influence of the degrees of freedom of those distant nodes is neglectable, substructuring represents an efficient method to reduce storage requirements and computational effort. The presented simulation with substructuring still yields satisfactory results. (© 2009 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Integral approximations in ab initio, electron propagator calculations

Treatments of interelectronic repulsion that avoid four-center integrals have been incorporated in ab initio, electron-propagator calculations with diagonal self-energy matrices. Whereas the formal scaling of arithmetic operations in the propagator calculations is unaffected, the reduction of storage requirements is substantial. Moreover, the scaling of integral transformations to the molecular orbital base is lowered by one order. Four-index, electron-repulsion integrals are regenerated from three-index intermediates. Test calculations with widely applied self-energy approximations demonstrate the accuracy of this approach. Only small errors are introduced when this technique is used with quasiparticle virtual orbitals, provided that conventional techniques of integral evaluation are used in the construction of density-difference matrices.

The Efficient Storage of Text Documents in Digital Libraries

In this paper we investigate the possibility of improving the efficiency of data compression, and thus reducing storage requirements, for seven widely used text document formats. We propose an open-source text compression software library, featuring an advanced word-substitution scheme with static and semidynamic word dictionaries. The empirical results show an average storage space reduction as high as 78 percent compared to uncompressed documents, and as high as 30 percent compared to documents compressed with the free compression software gzip.

REDUCING DECISION TREE ENSEMBLE SIZE USING PARALLEL DECISION DAGS

This research presents a new learning model, the Parallel Decision DAG (PDDAG), and shows how to use it to represent an ensemble of decision trees while using significantly less storage. Ensembles such as Bagging and Boosting have a high probability of encoding redundant data structures, and PDDAGs provide a way to remove this redundancy in decision tree based ensembles. When trained by encoding an ensemble, the new model behaves similar to the original ensemble, and can be made to perform identically to it. The reduced storage requirements allow an ensemble approach to be used in cases where storage requirements would normally be exceeded, and the smaller model can potentially execute faster by reducing redundant computation.

Contraction of gaussian basis sets and the total energy of fcc copper

A brief survey of contraction schemes for atomic, molecular, and solid-state calculations is presented. General guidelines for a simple application of the LCAO contraction procedure to solids are described with emphasis on subtle differences with the cases of atoms and molecules. By means of Wang and Callaway's program package, test computations are performed on fcc copper to produce non-self-consistent and self-consistent results. The electronic energies, densities of states, and x-ray scattering form factors compare satisfactorily with experimental results as well as with data obtained by Bagayoko et al. using a basis of 75 functions. The advantages of the contraction procedure include a factor of 6.8 savings in CPU time and a significant reduction in storage requirements. An extended version of the program package is used to compute the total energy of fcc copper for various lattice constants by a method recently introduced by Callaway and co-workers.

Application of FFT-based Algorithms for Large-Scale Universal Kriging Problems

Looking at kriging problems with huge numbers of estimation points and measurements, computational power and storage capacities often pose heavy limitations to the maximum manageable problem size. In the past, a list of FFT-based algorithms for matrix operations have been developed. They allow extremely fast convolution, superposition and inversion of covariance matrices under certain conditions. If adequately used in kriging problems, these algorithms lead to drastic speedup and reductions in storage requirements without changing the kriging estimator. However, they require second-order stationary covariance functions, estimation on regular grids, and the measurements must also form a regular grid. In this study, we show how to alleviate these rather heavy and many times unrealistic restrictions. Stationarity can be generalized to intrinsicity and beyond, if decomposing kriging problems into the sum of a stationary problem and a formally decoupled regression task. We use universal kriging, because it covers arbitrary forms of unknown drift and all cases of generalized covariance functions. Even more general, we use an extension to uncertain rather than unknown drift coefficients. The sampling locations may now be irregular, but must form a subset of the estimation grid. Finally, we present asymptotically exact but fast approximations to the estimation variance and point out application to conditional simulation, cokriging and sequential kriging. The drastic gain in computational and storage efficiency is demonstrated in test cases. Especially high-resolution and data-rich fields such as rainfall interpolation from radar measurements or seismic or other geophysical inversion can benefit from these improvements.

The Acousonde: A miniature autonomous wideband recorder.

The Acousonde™ is a newly designed miniature acoustic/ultrasonic recording tag incorporating several improvements over its predecessor, the Bioacoustic Probe. Design trade-offs for its acoustic data paths targeted acquisition of both near and distant odontocete echolocation clicks with minimal distortion while preserving general-purpose utility and low-power operation. Two acoustic channels are available: a low-power channel for long-term recording of signals up to 9 kHz and a high-frequency channel for signals up to 100 kHz. Each channel has its own dedicated hydrophone. For antialiasing, the low-power channel uses an adjustable switched-capacitor elliptic filter, while the high-frequency channel uses a fixed-frequency linear-phase filter; both filters may be bypassed if raw acquisition is desired. “Ping-pong” alternating sampling may be used to acquire samples from both channels concurrently, possibly to assess time-of-arrival differences between the two hydrophones. The controlling microprocessor, an ARM 9 with vector floating point accelerator, can digitally filter and downsample acoustic data during acquisition to reduce storage requirements. Other sensors include a 3D accelerometer, a 3D compass, a depth transducer, and a temperature monitor. Eight gigabytes of data storage are available, with data offload via a MicroUSB connector. Initial tests with captive animals are planned shortly. [Work supported by ONR.]

Wang Tiles in Computer Graphics

Abstract Many complex signals in computer graphics, such as point distributions and textures, cannot be efficiently synthesized and stored. This book presents tile-based methods based on Wang tiles and corner tiles to solve both these problems. Instead of synthesizing a complex signal when needed, the signal is synthesized beforehand over a small set of Wang tiles or corner tiles. Arbitrary large amounts of that signal can then efficiently be generated when needed by generating a stochastic tiling, and storing only a small set of tiles reduces storage requirements. A tile-based method for generating a complex signal consists of a method for synthesizing the signal over a set of Wang tiles or corner tiles, and a method for generating a stochastic tiling using the set of tiles. The method for generating a stochastic tiling using the set of tiles is independent of the signal. This book covers scanline stochastic tiling algorithms and direct stochastic tiling algorithms for Wang tiles and corner tiles. The me...

Construction of Data-Sparse $\mathcal{H}^2$-Matrices by Hierarchical Compression

Hierarchical matrices ($\mathcal{H}$-matrices) provide an elegant approach to handling large densely populated matrices: The matrix is split into a hierarchy of blocks, and each block is approximated by a low-rank matrix in factorized form. It has been demonstrated that this representation can be used to treat integral and partial differential equations, solve matrix equations from the field of control theory, and evaluate matrix functions efficiently. $\mathcal{H}^2$-matrices use a refined representation that employs a multilevel structure in order to reduce the storage requirements of hierarchical matrices. It has been shown that $\mathcal{H}^2$-matrices can significantly reduce storage requirements for large problems, in particular when combined with modern error control schemes. Until now, all algorithms for constructing an efficient approximation of a general matrix by an $\mathcal{H}^2$-matrix required a representation of the entire original matrix to be kept in storage; therefore the storage requirements of $\mathcal{H}^2$-matrix algorithms could be far larger than those of the final approximation. This paper presents a new approach that allows us to construct an $\mathcal{H}^2$-matrix without storing the entire original matrix. The central idea is to approximate submatrices and combine them by an efficient new algorithm to form approximations of larger matrices until the entire matrix has been treated. Using this new approach, many $\mathcal{H}$-matrix algorithms can be “refitted” easily to compute results in the more efficient representation. Possible applications include efficient matrix arithmetics for the construction of preconditioners, the approximation of matrix functions or solutions of matrix equations, or efficient compression schemes based on the popular cross approximation algorithms.

Real-time identification using a canonical face depth map

A practical identification system based on 3D face scanning is presented. The speed of comparing a probe scan to the gallery is enabled by scan normalisation followed by extraction of higher level features. Our canonical face depth map (CFDM) is a standardised representation for three-dimensional (3D) face data in a face-based coordinate system. Our experiments demonstrate that the CFDM normalisation algorithm is (a) robust to noise and occlusion, (b) significantly reduces storage requirements and thus I/O time, and (c) improves the efficiency of face recognition algorithms. Producing the CFDM takes less than a second on a desktop for 320 times 240 rangel scans. Current 3D scanning and matching methods are too slow for person identification, even for a watch list of only a few hundred face models. Transforming scanned 3D faces into CFDM format enables a probe scan to be matched to hundreds or thousands of gallery scans in a fewtimesseconds on a commodity computer. The best results achieved so far are a rank-1 recognition rate of 98.2% and a speed of 1900 face matches per second. Extrapolating these results suggests that multistage systems could achieve even better performance on even larger galleries.