Amount Of Physical Memory Research Articles

Tests with FALKSOL A massively parallel multi-level domain decomposing direct solver

An implementation of a massively parallel domain decomposing direct finite element equation solver named FALKSOL is tested using the parallel version of the state of the art open-source solver MUMPS (MUltifrontal Massively Parallel Solver) as a reference. FALKSOL includes its own very advanced multi-level domain decomposition and load balancing procedure. In the elemental mode, MUMPS uses a single level domain decomposition, and then solves the interface problem. FALKSOL offers a full out-of-core functionality, while MUMPS offers only a partial one. The problem size that can be solved by MUMPS is therefore hard limited by the amount of physical memory plus swap space1 present. This is an actual bottleneck. FALKSOL can solve much larger problems. It is essentially only limited by the size of the secondary storage available. With modern so-called Non-Volatile Memory express (NVMe) on the PCI bus, enough I/O bandwidth is obtained with a couple of raided units. The results show that MUMPS is about twice as fast as FALKSOL in the limited range of problem size it copes with. The solvers are complementary. The out-of-core multi-level domain decomposition algorithm in FALKSOL makes it scale. The test indicates that there is a considerable potential gain in scalability/elasticity choosing the FALKSOL multi-level domain-decomposition out-of-core approach. Especially considering more powerful and larger compute systems. The price to be paid in speed compared with MUMPS is reasonable. The current stand-alone implementation of FALKSOL is interfaced with the 3Dhp code Demkowicz (2007) [32], Demkowicz et al. (2008) [33]. The described MUMPS-like interface can be used to select the best solver for the task.

Implementing smart swapping algorithm to boost the performance of Linux PC’s.

Abstract Swap space in Linux is used when the amount of physical memory (RAM) is full. If the system needs more memory resources and the RAM is full, inactive pages in memory are moved to the swap space. While swap space can help machines with a small amount of RAM, it should not be considered a replacement for more RAM. Swap space is located on hard drives, which have a slower access time than physical memory. If any memory-intensive application is deployed on a PC with limited amount of Random-Access Memory, the inactive (not recently used) application which is generally the system UI is shifted to the Swap memory (much slower than RAM). This makes the entire UI slow but the higher priority processes (the most recent applications) remain unaffected because they remain in the RAM. When we are finished with our work with the memory intensive application and close it, the RAM gets cleared and has ample of space to accommodate the contents of the SWAP memory. But the swap memory stays as it is, and does not get transferred back to the RAM. This results in slower UI even though the system has enough RAM and no memory intensive process is running. This then results in lag in UI transition, delayed application launches, slower processing of programs and a large overhead.In years past, the recommended amount of swap space increased linearly with the amount of RAM in the system. But because the amount of memory in modern systems has increased into the hundreds of gigabytes, it is now recognized that the amount of swap space that a system needs is a function of the memory workload running on that system.In the world of enhanced hardware and superior machines, the software enhancements and optimization for low end PC’s is sometimes neglected. Hence, we propose a technique to overcome the lag created by the swap space y moving the content of the swap memory back to the RAM and a total boost is provided to the system. The technique is evaluated under various configurations and proves to solve the problem.

Cluster Optimisation using Cgroups at a Tier-2

The Linux kernel feature Control Groups (cgroups) has been used to gather metrics on the resource usage of single and eight-core ATLAS workloads. It has been used to study the effects on performance of a reduction in the amount of physical memory. The results were used to optimise cluster performance, and consequently increase cluster throughput by up to 10%.

Memory reservation and shared page management for real-time systems

Memory reservations are used to provide real-time tasks with guaranteed memory access to a specified amount of physical memory. However, previous work on memory reservation primarily focused on private pages, and did not pay attention to shared pages, which are widely used in current operating systems. With previous schemes, a real-time task may experience unexpected timing delays from other tasks through shared pages that are shared by another process, even though the task has enough free pages in its own reservation. In this paper, we first describe the problems that arise when real-time tasks share pages. We then propose a shared-page management framework which enhances the temporal isolation provided by memory reservations in resource kernels that use the resource reservation approach. Our proposed solution consists of two schemes, Shared-Page Conservation (SPC) and Shared-Page Eviction Lock (SPEL), each of which prevents timing penalties caused by the seemingly arbitrary eviction of shared pages. The framework can manage shared data for inter-process communication and shared libraries, as well as pages shared by the kernel's copy-on-write technique and file caches. We have implemented and evaluated our schemes on the Linux/RK platform, but it can also be applied to other operating systems with paged virtual memory.

A de novo next generation genomic sequence assembler based on string graph and MapReduce cloud computing framework

BackgroundState-of-the-art high-throughput sequencers, e.g., the Illumina HiSeq series, generate sequencing reads that are longer than 150 bp up to a total of 600 Gbp of data per run. The high-throughput sequencers generate lengthier reads with greater sequencing depth than those generated by previous technologies. Two major challenges exist in using the high-throughput technology for de novo assembly of genomes. First, the amount of physical memory may be insufficient to store the data structure of the assembly algorithm, even for high-end multicore processors. Moreover, the graph-theoretical model used to capture intersection relationships of the reads may contain structural defects that are not well managed by existing assembly algorithms.ResultsWe developed a distributed genome assembler based on string graphs and MapReduce framework, known as the CloudBrush. The assembler includes a novel edge-adjustment algorithm to detect structural defects by examining the neighboring reads of a specific read for sequencing errors and adjusting the edges of the string graph, if necessary. CloudBrush is evaluated against GAGE benchmarks to compare its assembly quality with the other assemblers. The results show that our assemblies have a moderate N50, a low misassembly rate of misjoins, and indels of > 5 bp. In addition, we have introduced two measures, known as precision and recall, to address the issues of faithfully aligned contigs to target genomes. Compared with the assembly tools used in the GAGE benchmarks, CloudBrush is shown to produce contigs with high precision and recall. We also verified the effectiveness of the edge-adjustment algorithm using simulated datasets and ran CloudBrush on a nematode dataset using a commercial cloud. CloudBrush assembler is available at https://github.com/ice91/CloudBrush.