Dynamic Memory Allocation Research Articles

Scalable Codes for Precision Calculations of Properties of Complex Atomic Systems

High precision atomic data are indispensable for studies of fundamental symmetries, tests of fundamental physics postulates, developments of atomic clocks, ultracold atom experiments, astrophysics, plasma science, and many other fields of research. We have developed a new parallel atomic structure code package that enables computations that were not previously possible due to system complexity. This code package also allows much quicker computations to be run with higher accuracy for simple systems. We explored different methods of load-balancing matrix element calculations for many-electron systems, which are very difficult due to the intrinsic nature of the computational methods used to calculate them. Furthermore, dynamic memory allocation and MPI parallelization have been implemented to optimize and accelerate the computations. We have achieved near-perfect linear scalability and efficiency with the number of processors used for calculation, paving the way towards the future where most open-shell systems will finally be able to be treated with good accuracy. We present several examples illustrating new capabilities of the newly developed codes, specifically correlating up to all 60 electrons in the highly charged Ir17+ ion and predicting certain properties of Fe16+.

A fast Monte-Carlo method to predict failure probability of offshore wind turbine caused by stochastic variations in soil

As spatial seabed soil variations are stochastic and considerable, it is important to predict the changes in the dynamic behavior of offshore structures for safety. Particularly, if the natural frequency of a structure is significantly changed by the soil variations, resonance can cause structural failure; hereafter, this probability is referred to as failure probability. Although several models have been proposed, they require a long computational time to predict failure probabilities. To overcome this limitation, this work proposes a fast method for estimating the failure probability. First, since a very large computation memory is required to construct a finite element model for stochastic soil, block-wise matrix calculations and a dynamic memory allocation technique were adopted. Second, conventional models require a large number of soil samples to calculate the failure probability, which results in heavy computations. A newly developed fast Monte-Carlo (FMC) method is 5.8 times faster than the conventional method with high accuracy (99.41%). This method was applied to offshore wind turbines and successfully predicted various structural characteristics. A noteworthy prediction is that slender and long foundation is more robust to stochastic soil variations than thick and short foundation. The FMC method can be used for preliminary design of offshore structures.

Polymorphic Memory: A Hybrid Approach for Utilizing On-Chip Memory in Manycore Systems

The key challenges of manycore systems are the large amount of memory and high bandwidth required to run many applications. Three-dimesnional integrated on-chip memory is a promising candidate for addressing these challenges. The advent of on-chip memory has provided new opportunities to rethink traditional memory hierarchies and their management. In this study, we propose a polymorphic memory as a hybrid approach when using on-chip memory. In contrast to previous studies, we use the on-chip memory as both a main memory (called M1 memory) and a Dynamic Random Access Memory (DRAM) cache (called M2 cache). The main memory consists of M1 memory and a conventional DRAM memory called M2 memory. To achieve high performance when running many applications on this memory architecture, we propose management techniques for the main memory with M1 and M2 memories and for polymorphic memory with dynamic memory allocations for many applications in a manycore system. The first technique is to move frequently accessed pages to M1 memory via hardware monitoring in a memory controller. The second is M1 memory partitioning to mitigate contention problems among many processes. Finally, we propose a method to use M2 cache between a conventional last-level cache and M2 memory, and we determine the best cache size for improving the performance with polymorphic memory. The proposed schemes are evaluated with the SPEC CPU2006 benchmark, and the experimental results show that the proposed approaches can improve the performance under various workloads of the benchmark. The performance evaluation confirms that the average performance improvement of polymorphic memory is 21.7%, with 0.026 standard deviation for the normalized results, compared to the previous method of using on-chip memory as a last-level cache.

RETRACTED ARTICLE: Cloud management architecture to improve the resource allocation in cloud IAAS platform

Cloud computing (CC) is an attractive emerging technology due to offering services based on-demand by the process of virtualization. Since CC platform offers services based on-demand it has been widely used in the field of various emerging IT infrastructure. In cloud platform each application is run in individual virtual machine (VM) for execution of services within the host. Since cloud platform operates on on-demand service it need to cope with multiple application in single time hence it is necessary to adopt an effective approach for balancing memory utilization in cloud network. For effective utilization of available memory existing approaches uses probability distribution method for allocating resources in cloud platform but still there exists a lack of utilization of available memory in cloud platform. This paper aims to develop an effective approach for dynamic memory allocation in VM in cloud platform. For memory allocation among VM in cloud platform proposed approach uses cloud vertical elasticity manager (CVEM), memory reporter (MR), memory over subscription granter (MOG). The MOG uses a scheduler to allocate the memory in a dynamic way inside a host. Finally, we adopt host elasticity rule to balance the available memory to allocate dynamically the memory inside an available host in cloud.

Reducing the Impact of Intensive Dynamic Memory Allocations in Parallel Multi-Threaded Programs

Frequent dynamic memory allocations (DyMAs) can significantly hinder the scalability of parallel multi-threaded programs. As the number of threads grows, DyMAs can even become the main performance bottleneck. We introduce modern tools and methods for evaluating the impact of DyMAs and present techniques for its reduction, which include scalable heap implementations, small buffer optimization, and memory pooling. Additionally, we provide a survey of state-of-the-art implementations of these techniques and study them experimentally by using a benchmark program, server simulator software, and a real-world high-performance computing application. As a result, we show that relatively small modifications in parallel program's source code or a way of its execution may substantially reduce the runtime overhead associated with the use of dynamic data structures.

PVC.js: visualizing C programs on web browsers for novices

Many researchers have proposed program visualization tools for memory management. Examples include state-of-the-art tools for C languages such as SeeC and Python Tutor (PT). However, three problems hinder the use of these and other tools: capability (P1), installability (P2), and usability (P3). (P1) Tools do not fully support dynamic memory allocation or File Input / Output (I/O) and Standard Input. (P2) Novice programmers often have difficulty installing SeeC due to its dependence on Clang and setting up an offline environment that uses PT. (P3) Revisualization of the modified source code in SeeC requires several steps. To alleviate these issues, we propose a new visualization tool called PlayVisualizerC.js (PVC.js). PVC.js, which is designed for novice C language programmers to provide solutions (S1–3) for P1–3. S1 offers complete support for dynamic memory allocation, standard I/O, and file I/O. S2 involves installation in a user web browser. This system is composed of JavaScript programs, including C language execution functions. S3 reduces the steps required for revisualization. To evaluate PVC.js, we conducted two experiments. The first experiment found that students using PVC solved a set of four programming tasks on average 1.7—times faster and with 19% more correct answers than those using SeeC. The second experiment found that PVC.js has a visualization performance equivalent to PT, and that PVC.js is more effective than existing general debugging tools for novices to understand programs in cases where the values of important variables change and the control flow is complicated.

Pangolin

There is growing interest in graph pattern mining (GPM) problems such as motif counting. GPM systems have been developed to provide unified interfaces for programming algorithms for these problems and for running them on parallel systems. However, existing systems may take hours to mine even simple patterns in moderate-sized graphs, which significantly limits their real-world usability. We present Pangolin , an efficient and flexible in-memory GPM framework targeting shared-memory CPUs and GPUs. Pangolin is the first GPM system that provides high-level abstractions for GPU processing. It provides a simple programming interface based on the extend-reduce-filter model, which allows users to specify application specific knowledge for search space pruning and isomorphism test elimination. We describe novel optimizations that exploit locality, reduce memory consumption, and mitigate the overheads of dynamic memory allocation and synchronization. Evaluation on a 28-core CPU demonstrates that Pangolin outperforms existing GPM frameworks Arabesque, RStream, and Fractal by 49×, 88×, and 80× on average, respectively. Acceleration on a V100 GPU further improves performance of Pangolin by 15× on average. Compared to state-of-the-art hand-optimized GPM applications, Pangolin provides competitive performance with less programming effort.

Verification of dynamic memory allocators based on symbolic program execution

Subject of Research. The paper presents the study of vulnerability exploitation techniques in the implementation of dynamic memory allocation algorithms (glibc library allocator): Poisoned Null-byte, Overlapped Chunks, Fastbin Attack,Unsafe Unlink, House of Einherjar, House of Force, House of Spirit, House of Lore, Unsorted Bin Attack. Examples of vulnerability exploitation code and classification of the presented techniques are given in accordance with the Common Weakness Enumeration list. The modern methods and means of vulnerabilities detection are studied; their advantages and disadvantages are shown using the Heap Hopper framework as an example. Modern methods of appropriate software verification are considered. Method. The proposed software verification method combines the approaches of static analysis and symbolic execution using an accurate model of algorithms for dynamic memory allocation. In the compilation process of program being tested, the Kripke structure is created. Dynamic memory vulnerabilities are described by temporal logic formulas. The resulting structure and formulas are passed at the input of the model checking algorithm. Concrete-symbolic execution of the assembled binary file is performed. Vulnerability conditions expressed in the form of propositional logic formulas are checked for symbolic execution paths. Main Results. The practical use of the proposed approach to detection of dynamic memory vulnerabilities in software applications is shown. Symbolic execution is implemented in the form of a low-level debugger, which reduces the operating time of algorithms due to the execution of the application being tested on a real processor. Practical Relevance. The paper presents an integrated approach for solving the problem of automatic vulnerabilities detecting at different stages of the software development life cycle. This approach is applicable for verification of the similar implementations of dynamic memory allocators, such as ptmalloc, dlmalloc, tcmalloc, jemalloc and musl.

МОДИФІКАЦІЯ МЕТОДУ А-БУФЕРА ДЛЯ ЗГЛАДЖУВАННЯ ТРИВИМІРНИХ ЗОБРАЖЕНЬ

According to the A-buffer method, a dynamic list of polygon fragments that cover a pixel is used for antialiasing of three-dimensional images. The A-buffer method uses less memory than the additional sampling method, but the need for dynamic memory allocation complicates its hardware implementation in computer graphics systems. Using a fixed number of fragments per pixel of the image simplifies the hardware implementation of the A-buffer method, but memory is used inefficiently because the number of fragments to calculate the pixel color intensity can differ significantly for different pixels. It was proposed to use a combined static-dynamic principle of memory organization, which is to introduce a common fragment buffer. New modifications of the A-buffer method have been developed, which use: - a combined static-dynamic approach to memory organization, which consists in the introduction of a common buffer for storing redundant fragments. The proposed modification has a simpler hardware implementation compared to dynamic memory allocation and requires less memory to store fragments of polygons than a fixed allocation; - a new approach to reducing the error of color intensity calculations, which occurs due to the rejection of fragments of polygons that did not have enough space in the fragment buffer. The essence of the approach is that the two most distant fragments from the observer are combined into one. The block diagram of the rasterization subsystem of three-dimensional graphic objects is developed, in which the modified A-buffer method is used. A computer program for modeling and testing smoothing methods based on the Abufer method has been developed. The simulation showed that the use of the proposed principle of memory allocation allows to reduce by 1.3-3.1 times the amount of memory required for smoothing three-dimensional images using the A-buffer method, compared with fixed memory allocation. The simulation results showed that the proposed solution, based on the combination of redundant fragments, provides a smaller value of the normalized standard error compared to the approach in which excess fragments are discarded. When using the new method to process polygon fragments, smoothing even complex scenes with the A-buffer method can be performed using only three fragments per pixel of the image, which reduces the amount of memory required to store fragments of polygons.

Visual Analysis of Actions Performed with Big Graphs

The basic concepts of using application development environments are presented in this paper. The way of using the GraphAnalyser application and its basic functions is also presented. All results of the experiments conducted are generated with this application. According to the experimental methodology, they fall into two groups: the first one includes actions related to the vertices of a graph, and the second one includes actions related to the edges and the dynamic allocation of memory to store the structure of a graph. The results show that when the number of vertices in a graph increases linearly, the time to add and remove these vertices also increases linearly. When the number of graph vertices increases linearly, the number of added vertices per millisecond remains relatively constant. However, the number of vertices removed for one millisecond for graphs containing between 10 and 70 million vertices varies. Similarly, when the number of graph edges increases linearly, the number of added edges per millisecond remains relatively constant. The summarized results of the two experiments show that the actions associated with adding, removing, and calculating the edge lengths are performed much more slowly than adding and removing the vertices.

OTA Firmware Update for Texas Instruments C2000 Controllers

Firmware updates are necessary to enhance the embedded systems performance and to remove the bugs. But it’s not an efficient manner to be installed it by the technician in the field. For c2000 controllers of Texas instruments dynamic memory allocation is not possible. So, using the proposed method we can maintain both updated firmware and current running firmware for the continuity of the system operation without dynamic memory allocation requirement. In this over the air (OTA) firmware update method, both application and the secondary boot loader (SBL) handles the update process without intervention of the technician. When there is any update the current running application copies the incoming updated firmware using its linker file. After the update process when there is a system reset the SBL calculates the checksum of the updated application, if it matches with the received checksum it executes the updated firmware else it executes the current running firmware without interrupting the application operation.

Precise reasoning with structured time, structured heaps, and collective operations

Despite decades of progress, static analysis tools still have great difficulty dealing with programs that combine arithmetic, loops, dynamic memory allocation, and linked data structures. In this paper we draw attention to two fundamental reasons for this difficulty: First, typical underlying program abstractions are low-level and inherently scalar, characterizing compound entities like data structures or results computed through iteration only indirectly. Second, to ensure termination, analyses typically project away the dimension of time, and merge information per program point, which incurs a loss in precision. As a remedy, we propose to make collective operations first-class in program analysis – inspired by Σ-notation in mathematics, and also by the success of high-level intermediate languages based on @map/reduce@ operations in program generators and aggressive optimizing compilers for domain-specific languages (DSLs). We further propose a novel structured heap abstraction that preserves a symbolic dimension of time, reflecting the program’s loop structure and thus unambiguously correlating multiple temporal points in the dynamic execution with a single point in the program text. This paper presents a formal model, based on a high-level intermediate analysis language, a practical realization in a prototype tool that analyzes C code, and an experimental evaluation that demonstrates competitive results on a series of benchmarks. Remarkably, our implementation achieves these results in a fully semantics-preserving strongest-postcondition model, which is a worst-case for analysis/verification. The underlying ideas, however, are not tied to this model and would equally apply in other settings, e.g., demand-driven invariant inference in a weakest-precondition model. Given its semantics-preserving nature, our implementation is not limited to analysis for verification, but can also check program equivalence, and translate legacy C code to high-performance DSLs.

FuzzFactory: domain-specific fuzzing with waypoints

Coverage-guided fuzz testing has gained prominence as a highly effective method of finding security vulnerabilities such as buffer overflows in programs that parse binary data. Recently, researchers have introduced various specializations to the coverage-guided fuzzing algorithm for different domain-specific testing goals, such as finding performance bottlenecks, generating valid inputs, handling magic-byte comparisons, etc. Each such solution can require non-trivial implementation effort and produces a distinct variant of a fuzzing tool. We observe that many of these domain-specific solutions follow a common solution pattern. In this paper, we present FuzzFactory, a framework for developing domain-specific fuzzing applications without requiring changes to mutation and search heuristics. FuzzFactory allows users to specify the collection of dynamic domain-specific feedback during test execution, as well as how such feedback should be aggregated. FuzzFactory uses this information to selectively save intermediate inputs, called waypoints, to augment coverage-guided fuzzing. Such waypoints always make progress towards domain-specific multi-dimensional objectives. We instantiate six domain-specific fuzzing applications using FuzzFactory: three re-implementations of prior work and three novel solutions, and evaluate their effectiveness on benchmarks from Google's fuzzer test suite. We also show how multiple domains can be composed to perform better than the sum of their parts. For example, we combine domain-specific feedback about strict equality comparisons and dynamic memory allocations, to enable the automatic generation of LZ4 bombs and PNG bombs.

Computational Smart Grid and DAM Queue

The layout of the Smart Computational Grid is an integral part of our information services infrastructure. In this paper, an anticipation of the architectural model of Smart Computational Grid is derived, which provides petabytes storage, tera-byte processing speed, and Gigabyte data transmission, where many supercomputers forms node with parallel data transmission. Impulsive demands posed by unexpected flows of data from users for complex computations, however, is exceedingly challenging. The paper specifies the design and implementation of a Dynamic Allocation of Memory (DAM) algorithm. DAM algorithm identifies the design of memory at supercomputing node, for ergodic functioning and for ensuring that no data overflow, using the methodology of queuing theory. The overall performance of the DAM algorithm is shown using MM1 and GG1 queue models and results are compared accordingly.

LiwePMS

Next-generation Storage Class Memory (SCM) offers low-latency, high-density, byte-addressable access and persistency. The potent combination of these attractive characteristics makes it possible for SCM to unify the main memory and storage to reduce the storage hierarchy. Aiming for this, several persistent memory systems were designed. However, the heavy metadata and transaction cost degrade the system performance. Moreover, neither of them pays attention to wear-leveling strategy. In this article, we present a lightweight persistent memory system, LiwePMS, which allows a fast access to persistent data stored in SCM with wear-aware memory management. LiwePMS makes performance improvement by simplifying the metadata management and the consistency method. LiwePMS abstracts SCM as heap space with container-based dynamic address mapping. Also, LiwePMS implements efficient wear-aware dynamic memory allocator and lightweight transaction mechanism for data consistency in user-space library. The experiments showed that LiwePMS persists key-value records 1.5× faster than Redis RDB mechanism. LiwePMS improves the performance of persistent region operation by more than 45%, 63%, and 1.1× comparing with HEAPO, Mnemosyne, and NVML, respectively. Also, the wear-leveling policy of memory allocator outperforms that of NVMalloc from 35% to 30%, and the transaction method promotes the transaction performance to 1.8× compared to NVML.

Formal Approaches to Secure Compilation

Secure compilation is a discipline aimed at developing compilers that preserve the security properties of the source programs they take as input in the target programs they produce as output. This discipline is broad in scope, targeting languages with a variety of features (including objects, higher-order functions, dynamic memory allocation, call/cc, concurrency) and employing a range of different techniques to ensure that source-level security is preserved at the target level. This article provides a survey of the existing literature on formal approaches to secure compilation with a focus on those that prove fully abstract compilation, which has been the criterion adopted by much of the literature thus far. This article then describes the formal techniques employed to prove secure compilation in existing work, introducing relevant terminology, and discussing the merits and limitations of each work. Finally, this article discusses open challenges and possible directions for future work in secure compilation.

A Dynamic Memory Allocation Approach Based on Balloon Technology on Virtualization Platform

In virtualization platforms, Virtual Machines(VMs) have different memory demand when running different applications. Applications with high memory demand often result in guest swapping and performance degradation of the VM. Because of the semantic gap between the Virtual Machine Monitor(VMM) and the guest Operating System(OS), it is impossible to achieve reasonable allocation and use of memory resources between the guests. Our goal is to monitor the memory information in real time and achieve memory recovery and replenish of each VM in case of memory fluctuation. This method builds up a reasonable swapping mechanism, which can achieve automatic balance of memory allocation among multiple VMs, optimize guest swapping and improve memory utilization. We display it by visualization module. Based on ballooning technology, this paper designs a dynamic memory allocation method in KVM virtualization platform, which consists of memory monitoring, memory dynamic allocation, memory resource visualization and other modules. The results of many tests and analysis demonstrate that our method has the ability to basically achieve our expected goals and requirements.

A Dynamic Memory Allocation Optimization Mechanism Based on Spark

Spark is a distributed data processing framework based on memory. Memory allocation is a focus question of Spark research. A good memory allocation scheme can effectively improve the efficiency of task execution and memory re... | Find, read and cite all the research you need on Tech Science Press

Design and analysis of a parallel hybrid memory architecture for per-flow buffering in high-speed switches and routers

The network switches and routers require both high-speed and large-capacity packet buffers. However, existing packet buffer architectures have the problems of speed scaling and flow number scaling limitations. To address the two problems simultaneously, this paper proposes a parallel hybrid SRAM/DRAM architecture for per-flow buffering in high-speed switches and routers. Tail SRAM and head SRAM are used to apply per-flow buffering for packet aggregation, so that the middle DRAM is accessed in a larger granularity and the DRAM's bandwidth utilization is improved. To mitigate the flow number scaling limitation, a dynamic memory allocation with hard timeout (DMA-HT) memory management algorithm is designed. The key idea of DMA-HT is that the memory space is dynamically allocated for the newly arrived flows, and a hard timeout is assigned for each queue. After a specific period of time, the memory space is freed, so that the SRAM space is efficiently utilized by the most recently active flows. A queuing system is used to model the proposed method, and theoretical analysis is performed to optimize the timeout value. With the derived formulas, multiple performance parameters are quantitatively analyzed, and the optimal timeout can be obtained. Both numerical results and simulations show that the proposed architecture can and reduce packet loss rate and average delay significantly compared with previous solutions with the same SRAM capacity.

Formal modelling of list based dynamic memory allocators

Existing implementations of dynamic memory allocators (DMA) employ a large spectrum of policies and techniques. The formal specifications of these techniques are quite complicated in isolation and very complex when combined. Therefore, the formal reasoning on a specific DMA implementation is difficult for automatic tools and mostly single-use.This paper proposes a solution to this problem by providing formal models for a full class of DMA, the class using various kinds of lists to manage the memory blocks controlled by the DMA.To obtain reusable formal models and tractable formal reasoning, we organise these models in a hierarchy ranked by refinement relations. We prove the soundness of models and the refinement relations using the modeling framework Event-B and the theorem prover Rodin.We demonstrate that our hierarchy is a basis for an algorithm theory for list based DMA:it abstracts various existing implementations of DMA and leads to new DMA implementations.The applications of this formalisation include model-based code generation, testing, and static analysis.