Main Memory Space Research Articles

TinyWolf — Efficient on-device TinyML training for IoT using enhanced Grey Wolf Optimization

Training a deep learning model generally requires a huge amount of memory and processing power. Once trained, the learned model can make predictions very fast with very little resource consumption. The learned weights can be fitted into a microcontroller to build affordable embedded intelligence systems which is also known as TinyML. Although few attempts have been made, the limits of the state-of-the-art training of a deep learning model within a microcontroller can be pushed further. Generally deep learning models are trained with gradient optimizers which predict with high accuracy but require a very high amount of resources. On the other hand, nature-inspired meta-heuristic optimizers can be used to build a fast approximation of the model’s optimal solution with low resources. After a rigorous test, we have found that Grey Wolf Optimizer can be modified for enhanced uses of main memory, paging and swap space among α,β,δ and ω wolves. This modification saved up to 71% memory requirements compared to gradient optimizers. We have used this modification to train the TinyML model within a microcontroller of 256KB RAM. The performances of the proposed framework have been meticulously benchmarked on 13 open-sourced datasets.

Planting Fast-Growing Forest by Leveraging the Asymmetric Read/Write Latency of NVRAM-Based Systems

Owing to the considerations of cell density and low static power consumption, nonvolatile random-access memory (NVRAM) has been a promising candidate for collaborating with a dynamic random-access memory (DRAM) as the main memory in modern computer systems. As NVRAM also brings technical challenges (e.g., limited endurance and high writing cost) to computer system developers, the concept of write reduction becomes the famous doctrine in NVRAM-based system design. Unfortunately, a well-known machine learning algorithm, random forest, will generate a massive amount of write traffic to the main memory space during its construction phase. In other words, a random forest hits the Achilles’ heel of NVRAM-based systems. For remedying this pain, our work proposes an NVRAM-friendly random forest algorithm, namely, Amine, for an NVRAM-based system. The design principle of Amine is to replace write operations with read accesses without raising the read complexity of the random forest algorithm. According to experimental results, Amine can effectively decrease the latency of random forest construction by 64%, compared with the original random forest algorithm.

Rethinking the Interactivity of OS and Device Layers in Memory Management

In the big data era, a huge number of services has placed a fast-growing demand on the capacity of DRAM-based main memory. However, due to the high hardware cost and serious leakage power/energy consumption, the growth rate of DRAM capacity cannot meet the increased rate of the required main memory space when the energy or hardware cost is a critical concern. To tackle this issue, hybrid main-memory devices/modules have been proposed to replace the pure DRAM main memory with a hybrid main memory module that provides a large main memory space by integrating a small-sized DRAM and a large-sized non-volatile memory (NVM) into the same memory module. Although NVMs have high-density and low-cost features, they suffer from the low read/write performance and low endurance issue, compared to DRAM. Thus, inside the hybrid main-memory module, it also includes a memory management design to use DRAM as the cache of NVMs to enhance its performance and lifetime. However, it also introduces new design challenges in both the OS and the memory module. In this work, we rethink the interactivity of OS and hybrid main-memory module, and propose a cross-layer cache design that (1) utilizes the information from the operating system to optimize the hit ratio of the DRAM cache inside the memory module, and (2) takes advantage of the bulk-size (or block-based) read/write feature of NVM to minimize the time overhead on the data movement between DRAM and NVM. At the same time, this cross-layer cache design is very lightweight and only introduces limited runtime management overheads. A series of experiments was conducted to evaluate the effectiveness of the proposed cross-layer cache design. The results show that the proposed design could improve access performance for up to 88%, compared to the investigated well-known page replacement algorithms.

Streaming Approach to In Situ Selection of Key Time Steps for Time‐Varying Volume Data

AbstractKey time steps selection, i.e., selecting a subset of most representative time steps, is essential for effective and efficient scientific visualization of large time‐varying volume data. In particular, as computer simulations continue to grow in size and complexity, they often generate output that exceeds both the available storage capacity and bandwidth for transferring results to storage, making it indispensable to save only a subset of time steps. At the same time, this subset must be chosen so that it is highly representative, to facilitate post‐processing and reconstruction with high fidelity. The key time steps selection problem is especially challenging in the in situ setting, where we can only process data in one pass in an online streaming fashion, using a small amount of main memory and fast computation. In this paper, we formulate the problem as that of optimal piece‐wise linear interpolation. We first apply a method from numerical linear algebra to compute linear interpolation solutions and their errors in an online streaming fashion. Using that method as a building block, we can obtain a global optimal solution for the piece‐wise linear interpolation problem via a standard dynamic programming (DP) algorithm. However, this approach needs to process the time steps in multiple passes and is too slow for the in situ setting. To address this issue, we introduce a novel approximation algorithm, which processes time steps in one pass in an online streaming fashion, with very efficient computing time and main memory space both in theory and in practice. The algorithm is suitable for the in situ setting. Moreover, we prove that our algorithm, which is based on a greedy update rule, has strong theoretical guarantees on the approximation quality and the number of time steps stored. To the best of our knowledge, this is the first algorithm suitable for in situ key time steps selection with such theoretical guarantees, and is the main contribution of this paper. Experiments demonstrate the efficacy of our new techniques.

Task‐aware swapping for efficient DNN inference on DRAM‐constrained edge systems

Object detection at the edge side is a common task in various environments. The deployment of convolutional neural networks in intelligent edge systems is very challenging because of the highly constrained main-memory space. This study aims at operating neural networks with a reduced memory requirement. The basic idea is that tasks of the same type would involve the same critical subnetwork. We propose identifying the critical network connections by considering the importance of channels. During runtime, the proposed method detects the task types and timely swaps the model parameters of the critical subnetworks from the external storage into dynamic random access memory (DRAM). Compared with conventional network pruning, the proposed approach further reduced the DRAM requirement by 34.6% while maintaining a high inference accuracy.

Efficient top/bottom-k fraction estimation in spatial databases using bounded main memory

Spatial databases store objects with their locations and certain types of attached items. A variety of modern applications have been developed by leveraging the utilization of locations and items in spatial objects, such as searching points of interest, hot topics, or users' attitude in specified spatial regions. In many scenarios, the high and low-frequency items in a spatial region are worth noticing, considering they represent the majority's interest or eccentric users' opinion. However, existing works have yet to identify such items in an interactive manner, despite the significance of the endeavor in decision-making systems. This study recognizes a novel type of analytical query, called top/bottom-k fraction query, to discover such items in spatial databases. To achieve fast query response, we propose a multilayered data summary that is spread out across the main memory and external memory. A memory-based estimation method for top/bottom-k fraction queries is proposed. To maximize the use of the main memory space, we design a data summary tuning method to dynamically allocate memory space among different spatial partitions. The proposed approach is evaluated with real-life datasets and synthetic datasets in terms of estimation accuracy. Evaluation results demonstrate the effectiveness of the proposed data summary and corresponding estimation and tuning algorithms.

Performance modeling for I/O‐intensive applications on virtual machines

AbstractModels for virtual machines running on cloud computing systems. Modeling system behaviors of clouds is a grand challenge because the resource utilization in VMs is heterogeneous due to variability in workload conditions. We address this challenging issue by uniquely (1) objectifying the usage prediction of virtualized resources and (2) predicting the performance trends of programs running on clouds. At the heart of the modeling system, we pay particular attention to CPU cores, disk size, main memory space, and input data volume, which serve as important factors for the developed prediction module. We devise two resource‐utilization prediction algorithms driven by two distinctive sets of I/O and CPU intensive benchmarks, where one algorithm deals with execution time and the other one revolves around input data size. We investigate the correlation between CPU/disk utilization and VM live migrations. Our system aims at not only providing performance optimization for virtualized resources but also ensuring service level agreement (SLA) and Quality of Service (QoS). The model fits the curve quite well, thereby advocating for the efficiency of the algorithm. The case studies conducted in this project draw the comparisons between the performance of striped and monolithic disks as well as bringing forth the problem of cache coherence that causes hindrance to the experiment. We also deal with the cache‐coherence problem to improve the accuracy of our prediction algorithms

Internal and external memory set containment join

A set containment join operates on two set-valued attributes with a subset ( $$\subseteq $$ ) relationship as the join condition. It has many real-world applications, such as in publish/subscribe services and inclusion dependency discovery. Existing solutions can be broadly classified into union-oriented and intersection-oriented methods. Based on several recent studies, union-oriented methods are not competitive as they involve an expensive subset enumeration step. Intersection-oriented methods build an inverted index on one attribute and perform inverted list intersection on another attribute. Existing intersection-oriented methods intersect inverted lists one-by-one. In contrast, in this paper, we propose to intersect all the inverted lists simultaneously while skipping many irrelevant entries in the lists. To share computation, we utilize the prefix tree structure and extend our novel list intersection method to operate on the prefix tree. To further improve the efficiency, we propose to partition the data and process each partition separately. Each partition will be associated with a much smaller inverted index, and the set containment join cost can be significantly reduced. Moreover, to support large-scale datasets that are beyond the available memory space, we develop a novel adaptive data partition method that is designed to fully leverage the available memory and achieve high I/O efficiency, and thereby exhibiting outstanding performance for external memory set containment join. We evaluate our methods using both real-world and synthetic datasets. Experimental results demonstrate that our method outperforms state-of-the-art methods by up to 10 $$\times $$ when the dataset is completely resided in memory. Furthermore, our approach achieves up to two orders of magnitude improvement on I/O efficiency compared with a baseline method when the dataset size exceeds the main memory space.

Large-scale terrain-adaptive LOD control based on GPU tessellation

The real-time rendering of large-scale terrain scenes has been a challenge and is a popular research topic in the fields of 3D games and virtual reality. To fully utilize GPU features, this paper proposes a real-time rendering algorithm for large-scale regular grid terrain based on GPU Tessellation. The CPU processes a coarse-grained quadtree to generate terrain patches and then passes the patches to the GPU after view culling. Then, the GPU refines the terrain by using GPU Tessellation and displacement mapping. The most important advantage of our method is that during the tessellation stage, a new rule that the distance, screen space projection error and variance of the height, as the standard of terrain roughness, is designed. During the tessellation stage, the distance, screen space projection error and variance of the height, as the standard of terrain roughness, are considered. The experimental results show that our algorithm can greatly reduce the CPU processing time and main memory space requirement, and fully utilize the new features of the GPU; In conclusion, the proposed approach can render large-scale terrain with high-precision in real-time.

A buffer based page replacement algorithm to reduce page fault

In the memory management system, the page replacement algorithm is an inevitable concept. When the Kernel creates a process, the pages are required to be in the main memory. If the pages are not found in the main memory [4], then it is called page fault. When a page fault occurs, the process needs to bring back the pages in disk. If there is a space in main memory it will be put in this area, otherwise a page replacement with the algorithm is implemented. A number of page replacing algorithms have been implemented. The main aim of this paper is to reduce the page fault, and this paper proposes a new idea for implementing a page replacement, so that the page fault reduces by a minimum of 5%.method: two more inputs read from the reference string, and are to be stored in a buffer for reference before replacing the pages in the frames. This novel algorithm increases the success rate.

Downsizing Without Downgrading: Approximated Dynamic Time Warping on Nonvolatile Memories

In recent years, time-series data have emerged in a variety of application domains, such as wireless sensor networks and surveillance systems. To identify the similarity between time-series data, the Euclidean distance and its variations are common metrics that quantify the differences between time-series data. However, the Euclidean distance is limited by its inability to elastically shift with the time axis, which motivates the development of dynamic time warping (DTW) algorithms. While DTW algorithms have been proven very useful in diversified applications like speech recognition, their efficacy might be seriously affected by the resolution of the time-series data. However, high-resolution time-series data might take up a gigantic amount of main memory and storage space, which will slow down the DTW analysis procedure. This makes the upscaling of DTW analysis more challenging, especially for in-memory data analytics platforms with limited nonvolatile memory space. In this paper, we propose a strategy to downsample time-series data to significantly reduce their size without seriously affecting the precision of the results obtained by DTW algorithms (downsizing without downgrading). In other words, this paper proposes a technique to remove the unimportant details that are largely ignored by DTW algorithms. The efficacy of the proposed technique is verified by a series of experimental studies, where the results are quite encouraging.

DHC: A Distributed Hierarchical Clustering Algorithm for Large Datasets

Hierarchical clustering is a classical method to provide a hierarchical representation for the purpose of data analysis. However, in practical applications, it is difficult to deal with massive datasets due to their high computation complexity. To overcome this challenge, this paper presents a novel distributed storage and computation hierarchical clustering algorithm, which has a lower time complexity than the standard hierarchical clustering algorithms. Our proposed approach is suitable for hierarchical clustering on massive datasets, which has the following advantages. First, the algorithm is able to store massive dataset exceeding the main memory space by using distributed storage nodes. Second, the algorithm is able to efficiently process nearest neighbor searching along parallel lines by using distributed computation at each node. Extensive experiments are carried out to validate the effectiveness of the DHC algorithm. Experimental results demonstrate that the algorithm is 10 times faster than the standard hierarchical clustering algorithm, which is an effective and flexible distributed algorithm of hierarchical clustering for massive datasets.

Hot-Spot Suppression for Resource-Constrained Image Recognition Devices With Nonvolatile Memory

Resource-constrained devices with convolutional neural networks (CNNs) for image recognition are becoming popular in various Internet of Things and surveillance applications. They usually have a low-power CPU and limited CPU cache space. In such circumstances, nonvolatile memory (NVM) has great potential to replace DRAM as main memory to improve overall energy efficiency and provide larger main-memory space. However, due to the iterative access pattern, performing CNN-based image recognition may introduce some write hot-spots on the NVM main memory. These write hot-spots may lead to reliability issues due to limited write endurance of NVM. In order to improve the endurance of NVM main memory, this paper leverages the CPU cache pinning technique and exploits the iterative access pattern of CNN to resolve the write hot-spot effect. In particular, we present a CNN-aware self-bouncing pinning strategy to minimize the maximal write cycles in NVM cells by proactively fastening CPU cache lines, so as to effectively suppress the write hot-spots to NVM main memory with limited performance degradation. The proposed strategy was evaluated by a series of intensive experiments and the results are encouraging.

Compact Data Structures to Represent and Query Data Warehouses into Main Memory

In this paper we propose the use of compact data structures to represent and process Data Warehouses (DWs) into main memory. Compact data structures are data structures that allow compacting the data without losing the capacity of querying the data in their compact form. A DW is a data repository to store historical data for decision support, and consists of dimensions and facts. The dimensions are abstract concepts that groups data with a similar meaning, usually, they are modelled as hierarchies of levels, which contain elements. The facts are quantitative data associated to dimensions. A data cube is a way to retrieve facts at different levels of granularity, which is achieved by navigation on dimensions hierarchies. Since a DW can store terabytes of data, the efficient processing of data cubes is key in OLAP (On-line Analytical Processing). We show that by using a compact representation of DWs we can improve the use of space in main memory, and achieve better performance for query processing. In this paper we extend a previous work to process aggregate queries with aggregate functions MAX, MIN, COUNT and AVG.

Mobile Unified Memory-Storage Structure Based on Hybrid Non-Volatile Memories

In mobile computing systems, the limited amount of main memory space leads to page swap operation overhead and data duplication in both main memory and secondary storage. Furthermore, SQLite write operations in mobile devices such as smartphones and tablet PCs tend to frequently overwrite data to storage, significantly degrading performance. Thus, this article presents a unified memory-storage structure that is optimized for mobile devices and blurs the boundary between the existing main memory layer and secondary storage layer. This structure can eliminate the conventional page-swap operations that cause significant performance degradation and support fast program execution time. The unified memory-storage structure consists of a dynamic RAM (DRAM) and phase change memory (PCM) -based dual buffering module, a hybrid unified memory-storage array consisting of DRAM and NAND Flash memory, and an associated unified storage translation layer devised for the memory address and file translation mechanism as a system software module. This hybrid array of non-volatile memories is formed as a single memory-disk integrated storage space that can be logically divided into static and dynamic spaces. Experimental results show that the overall performance of the hybrid unified memory-storage system with the buffering structure increases by around 13% and power consumption is also improved by 35%, compared to current mobile system.

Evaluation of triangular mesh layout techniques using large mesh simplification

Highly detailed polygonal meshes nowadays were created vastly from a great number of multimedia applications. Insufficient main memory space and inferior locality-of-reference nature of the input mesh sequence incurred frequent page swaps and extremely low runtime efficiency. To address this issue, techniques based on the models of graph layout, matrix bandwidth minimization, and space-filling-curves were reported. However, such models merely considered the optimization of vertex layout on the basis of the graph distance or matrix bandwidths while leave the optimization of face layout either unoptimized or resorted an additional sorting procedure. In this paper, we propose an improved theoretic model suggesting the optimization of index-faced triangular mesh layout on the basis of a jointly consideration on both the vertex and face layout. Unlike previous attempts, we do not need additional efforts in optimizing one layout after the other by sorting, the optimized layouts are generated on the fly with each other with respect to both the vertex or triangle bandwidths. According to the experimental results, our approach outperforms the CM and SFC methods not only at yielding better theoretical results but also at improving the runtime efficiency of large mesh simplification.

LibreKV: A Persistent In-Memory Key-Value Store

Emerging Non-Volatile Memory (NVM) possesses unique features including byte-addressability and high density which bring huge opportunities to combine DRAM and NVM in a unified main memory space. And key-value store (KVS) systems play an important role in many applications, such as large-scale websites. Several existing KVS have been proposed for NVMs. However, they have drawbacks such as extra write overhead leading, lower memory utilization and worse endurance. In this paper, we design and develop an NVM-based key-value store system named LibreKV. It specifically targets the hybrid DRAM and NVM memory architecture, leveraging the NVM as the eventual persistent storage medium. It uses both static hash table and dynamic hash tables to achieve a balance between system performance and memory utilization. It adopts a checksum based consistency mechanism to guarantee data consistency and persistent storage on NVM. LibreKV works independently without relying on an underlying file system and simplified the IO stack comparing to the traditional KVS system. Experimental results show that LibreKV outperforms the state-of-the-art KVS systems and achieves better scalability and consistency while with low overhead.

Multi-tenant Main Memory Index Tree with Shared Structure

Multi-tenant main memory index is an important tool to accelerate data access to software as a service. Establishing main memory indexes for each tenant occupies lots of memory space and results in performance bottleneck. The data schemas and access patterns of different tenants are similar, which provides the conditions for tenants storing their index entries with shared structure in main memory. In the paper, the designed structure of main memory index puts the indexes of different tenants on one tree to achieve the effects of saving memory space. Meanwhile, tenant placement algorithm in the cloud is proposed. Its places the tenants with similar indexes on the same node to further optimize the main memory space.

Practical compressed string dictionaries

The need to store and query a set of strings – a string dictionary – arises in many kinds of applications. While classically these string dictionaries have accounted for a small share of the total space budget (e.g., in Natural Language Processing or when indexing text collections), recent applications in Web engines, Semantic Web (RDF) graphs, Bioinformatics, and many others handle very large string dictionaries, whose size is a significant fraction of the whole data. In these cases, string dictionary management is a scalability issue by itself. This paper focuses on the problem of managing large static string dictionaries in compressed main memory space. We revisit classical solutions for string dictionaries like hashing, tries, and front-coding, and improve them by using compression techniques. We also introduce some novel string dictionary representations built on top of recent advances in succinct data structures and full-text indexes. All these structures are empirically compared on a heterogeneous testbed formed by real-world string dictionaries. We show that the compressed representations may use as little as 5% of the original dictionary size, while supporting lookup operations within a few microseconds. These numbers outperform the state-of-the-art space/time tradeoffs in many cases. Furthermore, we enhance some representations to provide prefix- and substring-based searches, which also perform competitively. The results show that compressed string dictionaries are a useful building block for various data-intensive applications in different domains.

응용프로그램의 기동시간 단축을 위한 파일 시스템 수준의 SSD 캐싱 기법

Application launch time is an important performance metric to user experience in desktop and laptop environment, which mostly depends on the performance of secondary storage. Application launch times can be reduced by utilizing solid-state drive (SSD) instead of hard disk drive (HDD). However, considering a cost-performance trade-off, utilizing SSDs as caches for slow HDDs is a practicable alternative in reducing the application launch times. We propose a new SSD caching scheme which migrates data blocks from HDDs to SSDs. Our scheme operates entirely in the file system level and does not require an extra layer for mapping SSD-cached data that is essential in most other schemes. In particular, our scheme does not incur mapping overheads that cause significant burdens on the main memory, CPU, and SSD space for mapping table. Experimental results conducted with 8 popular applications demonstrate our scheme yields 56% of performance gain in application launch, when data blocks along with metadata are migrated.