Main Memory Research Articles

Classic distance join queries using compact data structures

Distance-based Join Queries (DJQs) have multiple applications in spatial databases, Geographic Information Systems, and other areas. The K Closest Pairs Query (KCPQ) and the ε Distance Join Query (εDJQ) are well-known DJQs that have been widely studied and can be solved using plane-sweep techniques, which are efficient but must keep the whole datasets in main memory. In this work, we propose DJQ algorithms that work with data represented using a k2-tree, a compact data structure for binary grids. Our algorithms solve KCPQ and εDJQ queries, as well as several window-constrained variants, taking advantage of the indexing capabilities of k2-trees to efficiently answer queries without the need to decompress the data. Our experimental evaluation with large datasets shows that k2-tree algorithms are up to 5 times faster than plane-sweep algorithms in KCPQ, and 5–30 times faster in εDJQ. In variants that are window-constrained, our algorithms are competitive in most scenarios and faster for large windows. Additionally, our algorithms are not very affected by the distribution of the data and yield much more predictable query times, showing up to 30 times smaller variance in query times than plane sweep, depending on the location of the query window.

FSDedup: Feature-Aware and Selective Deduplication for Improving Performance of Encrypted Non-Volatile Main Memory

FSDedup: Feature-Aware and Selective Deduplication for Improving Performance of Encrypted Non-Volatile Main Memory

Noctua2 Supercomputer

Noctua 2 is a supercomputer operated at the Paderborn Center for Parallel Computing (PC2) at Paderborn University in Germany. Noctua 2 was inaugurated in 2022 and is an Atos BullSequana XH2000 system. It consists mainly of three node types: 1) CPU Compute nodes with AMD EPYC processors in different main memory configurations, 2) GPU nodes with NVIDIA A100 GPUs, and 3) FPGA nodes with Xilinx Alveo U280 and Intel Stratix 10 FPGA cards. While CPUs and GPUs are known off-the-shelf components in HPC systems, the operation of a large number of FPGA cards from different vendors and a dedicated FPGA-to-FPGA network are unique characteristics of Noctua 2. This paper describes in detail the overall setup of Noctua 2 and gives insights into the operation of the cluster from a hardware, software and facility perspective.

Enhancing Regular Expression Processing through Field-Programmable Gate Array-Based Multi-Character Non-Deterministic Finite Automata

This work investigates the advantages of FPGA-based Multi-Character Non-Deterministic Finite Automata (MC-NFA) for enhancing regular expression processing over traditional software-based methods. By integrating Field-Programmable Gate Arrays (FPGAs) within a data processing framework, our study showcases significant improvements in processing efficiency, accuracy, and resource utilization for complex pattern matching tasks. We present a novel approach that not only accelerates database and network security applications, but also contributes to the evolving landscape of computational efficiency and hardware acceleration. The findings illustrate that FPGA’s coherent access to main memory and the efficient use of resources lead to considerable gains in processing times and throughput for handling regular expressions, unaffected by expression complexity and driven primarily by dataset size and match location. Our research further introduces a phase shift compensation technique that elevates match accuracy to optimal levels, highlighting FPGA’s potential for real-time, accurate data processing. The study confirms that the benefits of using FPGA for these tasks do not linearly correlate with an increase in resource consumption, underscoring the technology’s efficiency. This paper not only solidifies the case for adopting FPGA technology in complex data processing tasks, but also lays the groundwork for future explorations into optimizing hardware accelerators for broader applications.

Optimizing code allocation for hybrid on-chip memory in IoT systems

With the increasing application of IoT devices, the memory subsystem, as the performance and energy bottleneck of IoT systems, has received a lot of attention. One of the keys is on-chip memory which can bridge the performance gap between the CPU and main memory. While many off-the-shelf embedded processors utilize the hybrid on-chip memory architecture containing scratchpad memories (SPMs) and caches, most existing literature ignores the collaboration between caches and SPMs. This paper proposes static SPM allocation strategies for the architecture mentioned above in IoT systems, which try to minimize the overall instruction memory subsystem latency and/or energy consumption. We capture the intra- and inter-task cache conflict misses via a fine-grained temporal cache behavior model. Based on this cache conflict information, we propose an integer linear programming (ILP) algorithm to generate an optimal static function level SPM allocation for system performance. Furthermore, to improve the scalability of the proposed allocation scheme for an enormous task set, we offer the interference factor to calculate the interference impact quantitatively. Then, based on the interference factor, we present two approximate knapsack based heuristic algorithms to provide near optimal static allocation schemes at both function- and basic block-level granularities, which favors fast design space exploration. The experiment results demonstrate that the proposed solution achieves a 30.85% improvement in memory performance, and up to 31.39% reduction in energy consumption, compared to the existing SPM allocation scheme at the function level. In addition, the proposed basic block level allocation algorithm shows better performance than our function level allocation algorithm and other basic block level allocation algorithm.

There Are Infinite Ways to Formulate Code: How to Mitigate the Resulting Problems for Better Software Vulnerability Detection

Given a set of software programs, each being labeled either as vulnerable or benign, deep learning technology can be used to automatically build a software vulnerability detector. A challenge in this context is that there are countless equivalent ways to implement a particular functionality in a program. For instance, the naming of variables is often a matter of the personal style of programmers, and thus, the detection of vulnerability patterns in programs is made difficult. Current deep learning approaches to software vulnerability detection rely on the raw text of a program and exploit general natural language processing capabilities to address the problem of dealing with different naming schemes in instances of vulnerability patterns. Relying on natural language processing, and learning how to reveal variable reference structures from the raw text, is often too high a burden, however. Thus, approaches based on deep learning still exhibit problems generating a detector with decent generalization properties due to the naming or, more generally formulated, the vocabulary explosion problem. In this work, we propose techniques to mitigate this problem by making the referential structure of variable references explicit in input representations for deep learning approaches. Evaluation results show that deep learning models based on techniques presented in this article outperform raw text approaches for vulnerability detection. In addition, the new techniques also induce a very small main memory footprint. The efficiency gain of memory usage can be up to four orders of magnitude compared to existing methods as our experiments indicate.

“My Brain Said That … ”: A Qualitative Study of Sources of Children’s Creative Ideas

ABSTRACT Despite decades of research, the creative process remains to be fully understood, and most theories and empirical evidence focus on adults’ creativity. Without understanding children’s creative processes, the generalizability of these theories is questionable, which is crucial for teaching, learning, and parenting. However, studying children’s creative processes is challenging because of their still-developing meta-cognition, which is an essential component of the creative process. In this study, we conducted interviews with middle childhood-aged students (n = 30) after they completed four different types of divergent thinking tasks, examining the creative process during and beyond idea generation. Using inductive thematic analysis, we identified four main themes with associated subthemes, with Memory (experiences, knowledge, references, and personal and social interests) and Cognitive Strategies (evaluative processes and hypothetical thinking) as the primary themes. These data offer insight into how the main themes Memory and Cognitive Strategies reflect associative and executive processes, respectively, and the interplay between these in children’s divergent thinking. Findings from this study revealed that children were able to explain their process of generating an idea in the vast majority of responses. Additionally, this work provides insight into improving creativity, methods of assessment, and our understanding of creativity development.

ISwap: A New Memory Page Swap Mechanism for Reducing Ineffective I/O Operations in Cloud Environments

This paper proposes iSwap, a new memory page swap mechanism that reduces the ineffective I/O swap operations and improves the QoS for applications with a high priority in the cloud environments. iSwap works in the OS kernel. iSwap accurately learns the reuse patterns for memory pages and makes the swap decisions accordingly to avoid ineffective operations. In the cases where memory pressure is high, iSwap compresses pages that belong to the latency-critical (LC) applications (or high-priority applications) and keeps them in main memory, avoiding I/O operations for these LC applications to ensure QoS; and iSwap evicts low-priority applications’ pages out of main memory. iSwap has a low overhead and works well for cloud applications with large memory footprints. We evaluate iSwap on Intel x86 and ARM platforms. The experimental results show that iSwap can significantly reduce ineffective swap operations (8.0% - 19.2%) and improve the QoS for LC applications (36.8% - 91.3%) in cases where memory pressure is high, compared with the latest LRU-based approach widely used in modern OSes.

Dynamic-MLCS: Fast searching for dynamic multiple longest common subsequences in sequence stream data

In recent years, data stream mining has become a popular research topic, wherein the dynamic multiple longest common subsequence problem for sequence streams (i.e., D-MLCS) is an important issue with a vast range of applications. However, the existing methods are only for static multiple longest common subsequence problem (i.e., S-MLCS) and there are no algorithms for the D-MLCS problem. Using the existing algorithms for the S-MLCS problem to solve the D-MLCS problem often leads to an overload of the main memory and is extremely time consuming as all the sequence segments should be stored into the main memory when the new segments flow in, and all updated problems should be resolved. To overcome this shortcoming, we designed a scheme for constructing dynamic Directed Acyclic Graphs (DAGs). It only needs to update the current DAG post each sequence segment flows in, without constructing the DAG from the beginning. We propose a novel algorithm for the D-MLCS problem called Dynamic-MLCS, which only needs to store a part of the key information of each streaming sequence segments, instead of storing all streaming sequence segments and scanning them repeatedly. Theoretical analyses demonstrate that the time and space complexities of the proposed algorithm are only relevant and linear to the number of the incomplete points and newly generated nodes, and the experiments conducted on the datasets of sequence streams indicate that the proposed Dynamic-MLCS is efficient and performs better than the compared state-of-the-art algorithms.

Iterative Reconstruction of Micro Computed Tomography Scans Using Multiple Heterogeneous GPUs.

Graphics processing units (GPUs) facilitate massive parallelism and high-capacity storage, and thus are suitable for the iterative reconstruction of ultrahigh-resolution micro computed tomography (CT) scans by on-the-fly system matrix (OTFSM) calculation using ordered subsets expectation maximization (OSEM). We propose a finite state automaton (FSA) method that facilitates iterative reconstruction using a heterogeneous multi-GPU platform through parallelizing the matrix calculations derived from a ray tracing system of ordered subsets. The FSAs perform flow control for parallel threading of the heterogeneous GPUs, which minimizes the latency of launching ordered-subsets tasks, reduces the data transfer between the main system memory and local GPU memory, and solves the memory-bound of a single GPU. In the experiments, we compared the operation efficiency of OS-MLTR for three reconstruction environments. The heterogeneous multiple GPUs with job queues for high throughput calculation speed is up to five times faster than the single GPU environment, and that speed up is nine times faster than the heterogeneous multiple GPUs with the FIFO queues of the device scheduling control. Eventually, we proposed an event-triggered FSA method for iterative reconstruction using multiple heterogeneous GPUs that solves the memory-bound issue of a single GPU at ultrahigh resolutions, and the routines of the proposed method were successfully executed on each GPU simultaneously.

Detection, exploitation and mitigation of memory errors

Abstract Software vulnerabilities are the root cause for a multitude of security problems in computer systems. Owing to their efficiency and tight control over low-level system resources, the C and C++ programming languages are extensively used for a myriad of purposes, from implementing operating system kernels to user-space applications. However, insufficient or improper memory management frequently leads to invalid memory accesses, eventually resulting in memory corruption vulnerabilities. These vulnerabilities are used as a foothold for elaborated attacks that bypass existing defense methods. In this paper, we summarise the main memory safety violation types (i.e. memory errors), and analyse how they are exploited by attackers and the main mitigation methods proposed in the research community. We further systematise the most relevant techniques with regards to memory corruption identification in current programs.

Towards Buffer Management with Tiered Main Memory

The scaling of per-GB DRAM cost has slowed down in recent years. Recent research has suggested that adding remote memory to a system can further reduce the overall memory cost while maintaining good performance. Remote memory (i.e., tiered memory), connected to host servers via high-speed interconnect protocols such as RDMA and CXL, is expected to deliver 100x (less than 1µs) lower latency than SSD and be more cost-effective than local DRAM through pooling or adopting cheaper memory technologies. Tiered memory opens up a large number of potential use cases within database systems. But previous work has only explored limited ways of using tiered memory. Our study provides a systematic study for DBMS to build tiered memory buffer management with respect to a wide range of hardware performance characteristics. Specifically, we study five different indexing designs that leverage remote memory in different ways and evaluate them through a wide range of metrics including performance, tiered-memory latency sensitivity, and cost-effectiveness. In addition, we propose a new memory provisioning strategy that allocates an optimal amount of local and remote memory for a given workload. Our evaluations show that while some designs achieve higher performance than others, no design can win in all measured dimensions.

Starling: An I/O-Efficient Disk-Resident Graph Index Framework for High-Dimensional Vector Similarity Search on Data Segment

High-dimensional vector similarity search (HVSS) is gaining prominence as a powerful tool for various data science and AI applications. As vector data scales up, in-memory indexes pose a significant challenge due to the substantial increase in main memory requirements. A potential solution involves leveraging disk-based implementation, which stores and searches vector data on high-performance devices like NVMe SSDs. However, implementing HVSS for data segments proves to be intricate in vector databases where a single machine comprises multiple segments for system scalability. In this context, each segment operates with limited memory and disk space, necessitating a delicate balance between accuracy, efficiency, and space cost. Existing disk-based methods fall short as they do not holistically address all these requirements simultaneously. In this paper, we present Starling, an I/O-efficient disk-resident graph index framework that optimizes data layout and search strategy within the segment. It has two primary components: (1) a data layout incorporating an in-memory navigation graph and a reordered disk-based graph with enhanced locality, reducing the search path length and minimizing disk bandwidth wastage; and (2) a block search strategy designed to minimize costly disk I/O operations during vector query execution. Through extensive experiments, we validate the effectiveness, efficiency, and scalability of Starling. On a data segment with 2GB memory and 10GB disk capacity, Starling can accommodate up to 33 million vectors in 128 dimensions, offering HVSS with over 0.9 average precision and top-10 recall rate, and latency under 1 millisecond. The results showcase Starling's superior performance, exhibiting 43.9x higher throughput with 98% lower query latency compared to state-of-the-art methods while maintaining the same level of accuracy.

Cache Memory Organization and Virtual Memory

Cache memory is mainly inculcated in systems to overcome the gap created in-between the main memory and CPUs due to their performance issues. Since, the speed of the processors is ever-increasing, so a need arises for a faster speed cache memory that can definitely assist in bridging the gap between the speed of processor and memory. Therefore, this paper proposes architecture circumscribed with three improvement techniques namely victim cache, sub-blocks, and memory bank. These three techniques will be implemented one after other to improve and make the speed and performance of cache comparative to main memory.

Extension VM: Interleaved Data Layout in Vector Memory

While vector architecture is widely employed in processors for neural networks, signal processing, and high-performance computing; however, its performance is limited by inefficient column-major memory access. The column-major access limitation originates from the unsuitable mapping of multidimensional data structures to two-dimensional vector memory spaces. In addition, the traditional data layout mapping method creates an irreconcilable conflict between row- and column-major accesses. Ideally, both row- and column-major accesses can take advantage of the bank parallelism of vector memory. To this end, we propose the Interleaved Data Layout (IDL) method in vector memory, which can distribute vector elements into different banks regardless of whether they are in the row- or column-major category, so that any vector memory access can benefit from bank parallelism. Additionally, we propose an Extension Vector Memory (EVM) architecture to achieve IDL in vector memory. EVM can support two data layout methods and vector memory access modes simultaneously. The key idea is to continuously distribute the data that needs to be accessed from the main memory to different banks during the loading period. Thus, EVM can provide a larger spatial locality level through careful programming and the extension ISA support. The experimental results showed a 1.43-fold improvement of state-of-the-art vector processors by the proposed architecture, with an area cost of only 1.73%. Furthermore, the energy consumption was reduced by 50.1%.

Using fog computing (FC) and optimization techniques for tasks migration and resource allocation in the internet of things (IoT)

The present paper proposes a novel algorithm with the highest accuracy and efficiency for resource allocation in the IoT, considering the fog computing technique. To meet the need for resource allocation, including central processing unit, bandwidth, and main memory, the Genetic Algorithm, Particle Swarm Optimization, Adaptive Weighted Particle Swarm Optimization, Crystal Structure Algorithm, and Simulated Annealing were examined. The obtained results emphasize the better performance of Simulated Annealing compared to the rest. The findings of this paper can benefit optimizing the tasks migration in internet of things. The major innovation of this research lies in the heart of the comparison made between the selected algorithms to see which one conducted the migration tasks with the highest accuracy and remarkable convergence rate.

The effects of document's format, size, and storage media on memory forensics

Main memory or RAM contains volatile but critical data about the system's state and its recent activities. Often, RAM based artifacts are hard to be found elsewhere. Digital investigators can find in this volatile data an essential information about the recent usage of a system including the used documents. Nowadays, documents are often fetched from a variety of storage media, most of which are internet based. This can complicate the digital investigation process due to the remote nature of these storage media; most of these remote files cannot be traced on the local hard disk drive (HDD) of the captured machine. However, whenever the document's contents are successfully recovered from RAM images, it can ensure the actual usage of the document. This paper studies the effects of various storage media (local and remote) on the amount of volatile artifacts of different types of documents. Experiments are designed to evaluate the effects of local hard drives, removable media, and a set of cloud based platforms such as Dropbox, Google Drive, and OneDrive on the RAM based artifacts of a used document. Results show that the recovered contents are significantly affected by the used storage media. Moreover, the document's type has an effect too. Frequently, a good ratio of the document's contents are recovered from RAM even when the document is living on the cloud, the document is closed, and the connection is terminated.

Improving the Performance and Endurance of Persistent Memory with Loose-Ordering Consistency

Persistent memory provides high-performance data persistence at main memory. Memory writes need to be performed in strict order to satisfy storage consistency requirements and enable correct recovery from system crashes. Unfortunately, adhering to such a strict order significantly degrades system performance and persistent memory endurance. This paper introduces a new mechanism, Loose-Ordering Consistency (LOC), that satisfies the ordering requirements at significantly lower performance and endurance loss. LOC consists of two key techniques. First, Eager Commit eliminates the need to perform a persistent commit record write within a transaction. We do so by ensuring that we can determine the status of all committed transactions during recovery by storing necessary metadata information statically with blocks of data written to memory. Second, Speculative Persistence relaxes the write ordering between transactions by allowing writes to be speculatively written to persistent memory. A speculative write is made visible to software only after its associated transaction commits. To enable this, our mechanism supports the tracking of committed transaction ID and multi-versioning in the CPU cache. Our evaluations show that LOC reduces the average performance overhead of memory persistence from 66.9% to 34.9% and the memory write traffic overhead from 17.1% to 3.4% on a variety of workloads.

Notice of Violation of IEEE Publication Principles: An Energy-Efficient Heterogeneous Memory Architecture for Future Dark Silicon Embedded Chip-Multiprocessors

Main memories play an important role in overall energy consumption of embedded systems. Using conventional memory technologies in future designs in nanoscale era causes a drastic increase in leakage power consumption and temperature-related problems. Emerging non-volatile memory (NVM) technologies offer many desirable characteristics such as near-zero leakage power, high density and non-volatility. They can significantly mitigate the issue of memory leakage power in future embedded chip-multiprocessor (eCMP) systems. However, they suffer from challenges such as limited write endurance and high write energy consumption which restrict them for adoption in modern memory systems. In this article, we present a convex optimization model to design a 3D stacked hybrid memory architecture in order to minimize the future embedded systems energy consumption in the dark silicon era. This proposed approach satisfies endurance constraint in order to design a reliable memory system. Our convex model optimizes numbers and placement of eDRAM and STT-RAM memory banks on the memory layer to exploit the advantages of both technologies in future eCMPs. Energy consumption, the main challenge in the dark silicon era, is represented as a major target in this work and it is minimized by the detailed optimization model in order to design a dark silicon aware 3D Chip-Multiprocessor. Experimental results show that in comparison with the Baseline memory design, the proposed architecture improves the energy consumption and performance of the 3D CMP on average about 61.33 and 9 percent respectively.

Single‐ and multi‐GPU computing on NVIDIA‐ and AMD‐based server platforms for solidification modeling application

SummaryThis work explores the performance of single‐ and multi‐GPU computing on state‐of‐the‐art NVIDIA‐ and AMD‐based server‐class hardware using various programming interfaces to accelerate a real‐world scientific application for solidification modeling based on the phase‐field method. The main computations of this memory‐bound application correspond to 20 stencils computed across grid nodes. We investigate the application's scalability for two basic schemes of organizing computation: without and with hiding data transfers behind computation, combined with using either peer‐to‐peer inter‐GPU data transfers through NVIDIA NVLink and AMD Infinity interconnects or communication over the PCIe and main memory. Among the studied programming interfaces is CUDA, HIP, and OpenMP Accelerator Model. While the first two are designed to write the codes for a specific hardware platform, OpenMP enables code portability between NVIDIA and AMD GPUs. The resulting performance is experimentally assessed on computing platforms containing NVIDIA V100 (up to 8 GPUs) and A100 (one GPU), as well as AMD MI210 (one device) and MI250 (up to 8 logical GPUs).