Index Compression Research Articles

Toward Energy-efficient STT-MRAM-based Near Memory Computing Architecture for Embedded Systems

Convolutional Neural Networks (CNNs) have significantly impacted embedded system applications across various domains. However, this exacerbates the real-time processing and hardware resource-constrained challenges of embedded systems. To tackle these issues, we propose spin-transfer torque magnetic random-access memory (STT-MRAM)-based near memory computing (NMC) design for embedded systems. We optimize this design from three aspects: Fast-pipelined STT-MRAM readout scheme provides higher memory bandwidth for NMC design, enhancing real-time processing capability with a non-trivial area overhead. Direct index compression format in conjunction with digital sparse matrix-vector multiplication (SpMV) accelerator supports various matrices of practical applications that alleviate computing resource requirements. Custom NMC instructions and stream converter for NMC systems dynamically adjust available hardware resources for better utilization. Experimental results demonstrate that the memory bandwidth of STT-MRAM achieves 26.7 GB/s. Energy consumption and latency improvement of digital SpMV accelerator are up to 64× and 1,120× across sparsity matrices spanning from 10% to 99.8%. Single-precision and double-precision elements transmission increased up to 8× and 9.6×, respectively. Furthermore, our design achieves a throughput of up to 15.9× over state-of-the-art designs.

Leveraging index compression techniques to optimize the use of co-processors


 
 
 The significant presence that many-core devices like GPUs have these days, and their enormous computational power, motivates the study of sparse matrix operations in this hardware. The essential sparse kernels in scientific computing, such as the sparse matrix-vector multiplication (SpMV), usually have many different high-performance GPU implementations. Sparse matrix problems typically imply memory-bound operations, and this characteristic is particularly limiting in massively parallel processors. This work revisits the main ideas about reducing the volume of data required by sparse storage formats and advances in understanding some compression techniques. In particular, we study the use of index compression combined with sparse matrix reordering techniques in CSR and explore other approaches using a blocked format. The systematic experimental evaluation on a large set of real-world matrices confirms that this approach achieves meaningful data storage reductions. Additionally, we find promising results of the impact of the storage reduction on the execution time when using accelerators to perform the mathematical kernels.
 
 

Soil Compaction Mechanism and Improvement in Farmland

To improve the quality of sloping soil in farmlands, exploring the mechanism of farmland soil consolidation is vital. This study investigated the improvement of soil pH, exchangeable acid, crop economic characteristics, and the soil compression index under different soil amendments using field experiments and laboratory-simulated cultures. The results show that (1) increasing the pre-consolidation pressure of the soil and reducing the soil compression index and bulk density significantly reduces the risk of soil compaction; (2) soil compression performance improved after the addition of organic and bio-carbon fertilizers and other modifiers to the compacted soil. The comprehensive physical index compression curve (S) value was higher than 0.05; (3) the addition (biomass carbon + chitosan 32 t/hm2) had the best improvement effect, which significantly increased soil pH, reduced soil exchange acid, and prevented the dissolution of soil H+ and Al3+. Dissolution also reduced the risk of soil compaction stress in rice and significantly increased root thickening and yield. Based on the stagnation mechanism of farmland soil and the improvement principle of physical properties, a comprehensive improvement technology for farmland soil compaction is proposed.

A scalable approach for index compression using wavelet tree and LZW

A scalable approach for index compression using wavelet tree and LZW

AN IMPROVED HAUSA WORD STEMMING ALGORITHM

The explosion of scientific publications in different domains coupled with the introduction and socialization of the internet experienced in the last few decades has made information more available than ever before. Consequently, digital storage capacity has been consistently doubling to reflect this geometric increase in information. In view of this, Information Retrieval (IR), nowadays considered the dominant form of information access has become even more critical. However, the problem of using free text in indexing and retrieval arising from spelling mistake, alternative in spelling, affixes and abbreviations has continued to bedevil the field of IR. To mitigate this problem, Stemming Algorithm was introduced in the 1960s. Stemming is an automated process of stripping all word derivatives of their inflectional affixes in order to obtain stem of the word. Because stemming is language specific, there are stemming algorithms designed specifically for most of the major languages in the world. With a speaker population of about 150 million Hausa language stands in need of a better stemming algorithm. This research is an attempt to improve upon the existing Hausa word stemming algorithm. Affix stripping method of conflation with reference lookup was used. Using Sirsat’s evaluation method, this research achieved 96.9% as Correctly Stemmed Word Factor (CSWF), Index Compression Factor – 74.76%, Words Stemmed Factor (WSF) – 70.44% and Average Word Conflation Factor – 59.47%.

MSTDB: A Hybrid Storage-Empowered Scalable Semantic Blockchain Database

Blockchain has been regarded as a trusted carrier for distributed data storage. With large volumes of valuable data stored on blockchain, data query has become a major requirement. However, the existing blockchains do not provide efficient query functionality because of their deep-rooted chain structure. Blockchain database is a new direction that constructs index on top of blockchain to provide rich query functionalities. The existing works are either insecure because the query process separates from the blockchain consensus, or inscalable because all the data needs to be stored in the block. In this paper, we propose a novel semantic blockchain database called MSTDB. We design a hybrid on/off chain blockchain storage architecture in which the majority of blockchain storage is offloaded to the off-chain storage and a novel index structure named Merkle Semantic Trie (MST) is designed to be a secure and semantic bridge between on- and off-chain. Based on MST, MSTDB provides a variety of semantic query functions including multi-keyword query, range query, Top-K query, and cross-chain query. To improve the performance further, we design some index compression and query preprocessing techniques for MSTDB. Extensive experiments demonstrate the effectiveness and efficiency of our blockchain database.

An Improved Framework for Bitmap Indexes and their Use in Data Warehouse Optimization

Compression technique is basically used to compress the size of table or reduce the storage area. Oracle already gives this feature for the table compression as well as for the index compression. when index is created on particular column of a table then it contain some space, which require some storage or disk space by this technique we can save our disk space because in industry the company have to purchase the disk space according to the size of the their data and pay according to their disk space. To utilize this disk space for useful records data rather than wasting it. In this paper used the data pump utility for the compression of Bitmap index and table. Data pump utility performed for the logical backups in database.in this paper implemented data pump for compression, to release the space and change the index pointing location. It will not release the space even after deletion of records. This is of special interest for the case to compress the bitmap index and table space along with the’S (Data Manipulation Language).

An Optimization of Bitmap Index Compression Technique in Bulk Data Movement Infrastructure

An index is an optional structure, associated with a table or clustered table that can sometimes speed up data access. By creating an index on one or more columns of a table, you gain the ability, in some cases, to retrieve a small set of randomly distributed rows from the table. This research aims to free the filled disk space after documents have been erased. We verify the current compression technologies and apply the data pump to the bitmap index compression. Bitmap is more effective for minimal unique value, according to the traditional wisdom index. However, it needs no Bitmap index at high cardinality or low cardinality by data pump. In this paper we recommend a pump function for space release in the database after documentation has been removed. This utility does not free disk space even though the elimination of records from the table is seen by the Bitmap Index. To free up the room and adjust the index point position, we have introduced data pump for compression. The data pump is commonly used in oracle database for logical backups. Finally, we study the bitmap index which is widely used for industrial purposes and address unanswered questions for the future.

Efficient Inverted Index Compression Algorithm Characterized by Faster Decompression Compared with the Golomb-Rice Algorithm.

This article deals with compression of binary sequences with a given number of ones, which can also be considered as a list of indexes of a given length. The first part of the article shows that the entropy H of random n-element binary sequences with exactly k elements equal one satisfies the inequalities . Based on this result, we propose a simple coding using fixed length words. Its main application is the compression of random binary sequences with a large disproportion between the number of zeros and the number of ones. Importantly, the proposed solution allows for a much faster decompression compared with the Golomb-Rice coding with a relatively small decrease in the efficiency of compression. The proposed algorithm can be particularly useful for database applications for which the speed of decompression is much more important than the degree of index list compression.

Techniques for Inverted Index Compression

The data structure at the core of large-scale search engines is the inverted index , which is essentially a collection of sorted integer sequences called inverted lists . Because of the many documents indexed by such engines and stringent performance requirements imposed by the heavy load of queries, the inverted index stores billions of integers that must be searched efficiently. In this scenario, index compression is essential because it leads to a better exploitation of the computer memory hierarchy for faster query processing and, at the same time, allows reducing the number of storage machines. The aim of this article is twofold: first, surveying the encoding algorithms suitable for inverted index compression and, second, characterizing the performance of the inverted index through experimentation.

CIC-PIM: Trading spare computing power for memory space in graph processing

Shared-memory graph processing is usually more efficient than in a cluster in terms of cost effectiveness, ease of programming and runtime. However, the limited memory capacity of a single machine and the huge sizes of graphs restrains its applicability. Hence, it is imperative to reduce memory footprint. We observe that index compression holds promise and propose CIC-PIM, a lightweight encoding with chunked index compression, to reduce the memory footprint and the runtime of graph algorithms. CIC-PIM aims for significant space saving, real random-access support and high cache efficiency by exploiting the ubiquitous power-law and sparseness features of large scale graphs. The basic idea is to divide index structures into chunks of appropriate size and compress the chunks with our lightweight fixed-length byte-aligned encoding. After CIC-PIM compression, two-fold larger graphs are processed with all data fit in memory, resulting in speedups or fast in-memory processing unattainable previously.

Weighted minimizer sampling improves long read mapping.

In this era of exponential data growth, minimizer sampling has become a standard algorithmic technique for rapid genome sequence comparison. This technique yields a sub-linear representation of sequences, enabling their comparison in reduced space and time. A key property of the minimizer technique is that if two sequences share a substring of a specified length, then they can be guaranteed to have a matching minimizer. However, because the k-mer distribution in eukaryotic genomes is highly uneven, minimizer-based tools (e.g. Minimap2, Mashmap) opt to discard the most frequently occurring minimizers from the genome to avoid excessive false positives. By doing so, the underlying guarantee is lost and accuracy is reduced in repetitive genomic regions. We introduce a novel weighted-minimizer sampling algorithm. A unique feature of the proposed algorithm is that it performs minimizer sampling while considering a weight for each k-mer; i.e. the higher the weight of a k-mer, the more likely it is to be selected. By down-weighting frequently occurring k-mers, we are able to meet both objectives: (i) avoid excessive false-positive matches and (ii) maintain the minimizer match guarantee. We tested our algorithm, Winnowmap, using both simulated and real long-read data and compared it to a state-of-the-art long read mapper, Minimap2. Our results demonstrate a reduction in the mapping error-rate from 0.14% to 0.06% in the recently finished human X chromosome (154.3 Mbp), and from 3.6% to 0% within the highly repetitive X centromere (3.1 Mbp). Winnowmap improves mapping accuracy within repeats and achieves these results with sparser sampling, leading to better index compression and competitive runtimes. Winnowmap is built on top of the Minimap2 codebase and is available at https://github.com/marbl/winnowmap.

Classification and consolidation test on peat soil of Asahan Regency Sumatera Utara Province

Defined as land with water-saturated soil, formed from deposits that originate from the accumulation of decaying past plant. Peat soil has poor physical properties for construction purposes, the soil has low shear strength and has a high shrinkage because of a lack of mineral content in it. The research purposes were to find out the index properties and the classification of peat soil and to analyze the compression index (Cc) and the coefficient of consolidation (Cv) of peat soil. The research was carried out by gradual consolidation and immediately consolidation. In gradual consolidation the load was given in step by step of 0.5 Kg, 1 Kg, 2 Kg, 4 Kg, 8 Kg, and immediately consolidation using a load of 1 Kg and 4 Kg for 7 days. Peat soil of Pertahanan Village had high water content (w) of 726.34 %, specific gravity (Gs) of 1.302, moist unit weight (γb) of 0.764 gr/cm3, dry unit weight (γd) of 0.144 gr/cm3, void ratio (e) of 8.06. Based on ASTM D442-84 (1989), the peat soil had 45.032 % ash peat and 54.968 % organic content, and it was classified as High Ash Peat. The acidity of peat soil was 4 and it was classified as Highly Acidic Peat with a pH less than 4.5. Index compression (Cc) of the peat soil was 0.659 and the coefficient of consolidation (Cv) was 0.317 cm2/seconds.

Outcome of Long Segment Transpedicular Screw Fixation in Unstable Thoracolumbar Spine Injury with Incomplete Neurological Deficit

Background: Long segment transpedicular screw fixation in thoracolumbar spine injury has gained popularity in the last decades as it provides immediate stability, prevents further risk of complications and allows early mobilization of patient. Objectives: Evaluation of clinical and functional outcome after long segment transpedicular screw fixation in unstable thoracolumbar spine injury with incomplete neurological deficit patients. Methods: This quasi experimental study was conducted at Dhaka Medical College and Hospital (DMCH), Dhaka, Bangladesh from January 2016 to December 2017. A total of twenty four (24) patients with traumatic unstable fracture of the thoracolumbar spine with incomplete neurological deficit underwent long segment transpedicular screw fixation were included in this study. Clinical and radiologic outcomes were assessed pre-operatively and post-operatively. Results: A total of 24 patients aged between 20 - 60 years of both sexes (Male-21, Female-03) were included in this study. Their mean (±SD) age was 33.12 (±8.57) years. Most of them were farmer [09 (37.5%)] and day labour [08 (33.33%)]. Motor vehicle accidents were the most common [16 (66.67%)] cause of injury and then fall from height [08 (33.33%)]. The most common level of injury involved was L1-10 (41.67%) patients, followed by L2-07 (29.17%), T12-05 (20.83%) and T11-02 (8.33%) of the study patients. Burst fracture was the commonest [22 (91.67%)] type of fracture followed by Chance fracture [02 (8.33%)]. Pre-operative mean (±SD) Cobb angle, Kyphotic deformation of vertebral body, Beck index and vertebral compression deformity were 21.83 ± 4.5 degrees, 22.09 ± 4.40 degrees, 0.78 ± 2.65 and 0.18 ± 0.93 mm respectively, after procedure which were significantly (p < 0.05) decreased to 12.5 ± 3 degrees, 11.50 ± 3.5 degrees, 0.86 ± 0.40 and 0.11 ± 0.00011 mm respectively at final follow up visit. Pain was significantly (p < 0.001) decreased, while neurologic function and working capability of the patients were improved from pre-operative to final follow up visit. Conclusions: Long segment transpedicular screw fixation is an effective method of treatment in patients having thoracolumbar spine injury with incomplete neurological deficit.

On Optimally Partitioning Variable-Byte Codes

The ubiquitous Variable-Byte encoding is one of the fastest compressed representation for integer sequences. However, its compression ratio is usually not competitive with other more sophisticated encoders, especially when the integers to be compressed are small that is the typical case for inverted indexes. This paper shows that the compression ratio of Variable-Byte can be improved by 2x by adopting a partitioned representation of the inverted lists. This makes Variable-Byte surprisingly competitive in space with the best bit-aligned encoders, hence disproving the folklore belief that Variable-Byte is space-inefficient for inverted index compression. Despite the significant space savings, we show that our optimization almost comes for free, given that: we introduce an optimal partitioning algorithm that does not affect indexing time because of its linear-time complexity; we show that the query processing speed of Variable-Byte is preserved, with an extensive experimental analysis and comparison with several other state-of-the-art encoders.

An optimization of bitmap index compression technique in bulk data movement infrastructure

An optimization of bitmap index compression technique in bulk data movement infrastructure

The Settlement Ratio Prediction of Compressible Soil In Area Under Embankment

This paper aims to predicting the ratio of settlement under the toe embankment, outer side of embankment and the centre of embankment. The methods used in this study are the empirical approach to estimate the index compression value (Cc) for compressible clay soil. The magnitude of soil subgrade settlement is carried out using a 1-dimensional compression formula developed by Terzaghi and considers the soil subgrade is an over consolidated clay. This study uses several variables such as the embankment dimension by considering the height and width of embankment and the depth of compressible layer of soil subgrade. Those variable uses to compare the ratio of settlement depend on those variables. The results obtained from this study are the settlement ratio of toe/centre is around 0.23 and the settlement ratio of outside/centre is around 0.84. The empirical formulation of settlement ratio for various condition are described in this study.

P3448Influence of severity of pectus excavatum as indicated by the Haller Index on actual heart compression and ECG findings

Abstract Background Pectus excavatum (PEX) is a depression of the sternum relative to the costal cartilages. The Haller index (HI) is an anatomical indicator of the severity of PEX that is calculated from CT. Purpose We evaluated the influence of anatomical severity of PEX as indicated by HI on the severity of actual heart compression and ECG findings Methods A total of 28 patients (23 males; 20±19 years) with PEX underwent unenhanced CT and ECG. On CT, HI and degree of compression of right (RA) and left atria (LA) were evaluated simultaneously. Results As indicated by CT, the mean HI was 5.80±2.34. We observed 7, 15, 5, and 1 patient with none, mild, moderate, or severe RA compression, respectively. In addition, we observed 8, 13, 4, and 3 patients with none, mild, moderate, or severe LA compressions, respectively. The mean HI was calculated as 4.28±0.92, 5.40±2.01, 6.85±1.81, and 9.61±0.00 in patients with none, mild, moderate, or severe RA, respectively, and the findings were significant (P=0.021). The mean HI was calculated as 4.17±0.91, 5.97±2.23, 5.95±1.51, and 6.69±2.93 in patients with none, mild, moderate, or severe LA compression, respectively, although the findings were not significant (P=0.145). On ECG, a P wave with a negative portion &gt;1 mm in depth and &gt;40 ms in duration in the V1 lead, complete or incomplete right bundle branch block (RBBB), sinus arrhythmia, or right-axis deviation of a QRS wave were diagnosed in 23, 10, 8, and 6 patients, respectively. The HI between patients with (n=23) and without (n=5) a negative portion of the P wave in the V1 lead showed a significant difference (6.45±2.43 vs. 4.62±1.68; P&lt;0.05). The correlation coefficient between absolute values of the maximum negative portion of the P wave in the V1 lead and HI was 0.36 (n=28). The HI between patients with and without a complete or incomplete RBBB showed a significant difference (6.98±2.43 vs. 5.14±2.07; P&lt;0.05). The HI between patients with and without a sinus arrhythmia did not show a significant difference (5.45±2.66 vs. 5.94±2.25). The HI between patients with and without a right axis deviation on a QRS wave also did not show a significant difference (4.76±1.53 vs. 6.08±2.64). The inter-observer agreement of HI between the two observers was 0.978. Haller index and heart compression on CT Conclusion In patients with PEX, compression of RA on CT, a negative portion of the P wave in V1, complete or incomplete RBBB, sinus arrhythmia, and a right axis deviation on a QRS wave are common. The degree of compression of RA on CT, negative portion of the P wave in the V1 lead and complete or incomplete RBBB on ECG are associated with anatomical severity as indicated by the HI. However, our results showed that HI severity was unrelated to either the degree of compression of LA on CT, or sinus arrhythmia and right-axis deviation of the QRS wave on ECG. These observations may be due to variation of location of maximum sternum depression, such as high- or low-positioned dominant-type compressions.

An Improved Group Similarity-Based Association Rule Mining Algorithm in Complex Scenes

Association rule (AR) mining in complex scene has attracted extensive attention of researchers in recent years. Typically, many researchers focused on an algorithm itself and ignored a generalization method to improve the performance of AR mining. Tuna et al., presented a general data structure Speeding-Up AR Structure with Inverted Index Compression (SAII) which could be utilized in most of the existing algorithms to improve their performance IEEE Trans. Cybern. 46(12) (2016) 3059–3072. However, we found that this algorithm consumes a lot of time in re-ordering data because a one-to-one comparison method is used in this process, which is the main reason that the speeding-up structure is difficult to establish when coping with much more large amount of data. To overcome these problems, this paper aims to propose an improved speeding-up AR algorithm based on group similarity and Apache Spark framework to further reduce the memory requirements and runtime. Our simulation results on the police business big dataset make clear that our improved approach performs well and is more suitable for a big data environment.

Optimizing partitioning strategies for faster inverted index compression

The inverted index is a key component for search engines to manage billions of documents and quickly respond to users’ queries.Whereas substantial effort has been devoted to reducing space occupancy and decoding speed, the encoding speed when constructing the index has been overlooked. Partitioning the index aligning to its clustered distribution can effectively minimize the compressed size while accelerating its construction procedure. In this study, we introduce compression speed as one criterion to evaluate compression techniques, and thoroughly analyze the performance of different partitioning strategies. Optimizations are also proposed to enhance state-of-the-art methods with faster compression speed and more flexibility to partition an index. Experiments show that our methods offer a much better compression speed, while retaining an excellent space occupancy and decompression speed. networks.