CPU Efficiency Research Articles

Leveraging the checkpoint-restart technique for optimizing CPU efficiency of ATLAS production applications on opportunistic platforms

Data processing applications of the ATLAS experiment, such as event simulation and reconstruction, spend considerable amount of time in the initialization phase. This phase includes loading a large number of shared libraries, reading detector geometry and condition data from external databases, building a transient representation of the detector geometry and initializing various algorithms and services. In some cases the initialization step can take as long as 10-15 minutes. Such slow initialization has a significant negative impact on overall CPU efficiency of the production job, especially when the job is executed on opportunistic, often short-lived, resources such as commercial clouds or volunteer computing. In order to improve this situation, we can take advantage of the fact that ATLAS runs large numbers of production jobs with similar configuration parameters (e.g. jobs within the same production task). This allows us to checkpoint one job at the end of its configuration step and then use the generated checkpoint image for rapid startup of thousands of production jobs. By applying this technique we can bring the initialization time of a job from tens of minutes down to just a few seconds. In addition to that we can leverage container technology for restarting checkpointed applications on the variety of computing platforms, in particular of platforms different from the one on which the checkpoint image was created.We will describe the mechanism of creating checkpoint images of Geant4 simulation jobs with AthenaMP (the multi-process version of the ATLAS data simulation, reconstruction and analysis framework Athena) and the usage of these images for running ATLAS Simulation production jobs on volunteer computing resources (ATLAS@Home) and on Supercomputers.

VBMq: pursuit baremetal performance by embracing block I/O parallelism in virtualization

Barely acceptable block I/O performance prevents virtualization from being widely used in the High-Performance Computing field. Although the virtio paravirtual framework brings great I/O performance improvement, there is a sharp performance degradation when accessing high-performance NAND-flash-based devices in the virtual machine due to their data parallel design. The primary cause of this fact is the deficiency of block I/O parallelism in hypervisor, such as KVM and Xen. In this paper, we propose a novel design of block I/O layer for virtualization, named VBMq. VBMq is based on virtio paravirtual I/O model, aiming to solve the block I/O parallelism issue in virtualization. It uses multiple dedicated I/O threads to handle I/O requests in parallel. In the meanwhile, we use polling mechanism to alleviate overheads caused by the frequent context switches of the VM’s notification to and from its hypervisor. Each dedicated I/O thread is assigned to a non-overlapping core to improve performance by avoiding unnecessary scheduling. In addition, we configure CPU affinity to optimize I/O completion for each request. The CPU affinity setting is very helpful to reduce CPU cache miss rate and increase CPU efficiency. The prototype system is based on Linux 4.1 kernel and QEMU 2.3.1. Our measurements show that the proposed method scales graciously in the multi-core environment, and provides performance which is 39.6x better than the baseline at most, and approaches bare-metal performance.

Assessment of gas radiative property models in the presence of nongray particles

ABSTRACTIn this study, a radiation code based on the method of lines solution of the discrete ordinates method for the prediction of radiative heat transfer in nongray gaseous media is developed by incorporation of two different spectral gas radiative property models, banded spectral line-based weighted sum of gray gases (banded SLW) and gray wide band (GWB) approximation in the presence of nongray absorbing–emitting–scattering particles. The aim is to introduce an accurate and CPU efficient spectral gas radiation model, which is compatible with spectral fuel/ash particle property models. Input data required for the radiation code and its validation are provided from two combustion tests previously performed in a 300 kWt atmospheric bubbling fluidized bed combustor test rig burning low calorific value Turkish lignite with high volatile matter/fixed carbon (VM/FC) ratio in its own ash. The agreement between wall heat fluxes and source term predictions obtained by global and banded SLW models reveal that global SLW model can be converted to an accurate wide band gas model (banded SLW) which can directly be coupled with spectral particle radiation. Furthermore, assessment of GWB approximation by benchmarking its predictions against banded SLW model shows that GWB gives reasonable agreement with a higher CPU efficiency when the particle absorption coefficient is at least in the same order of magnitude with the gas absorption coefficient.

Predictions of droplet heating and evaporation: An application to biodiesel, diesel, gasoline and blended fuels

This work is focused on the analyses of automotive fuel droplets’ heating and evaporation with application to biodiesel, diesel, gasoline, and blended diesel–biodiesel fuels. The analyses are made considering conditions representative of real internal combustion engines. The evolutions of droplet radii and temperatures for gasoline, diesel, and a broad range of biodiesel fuels and their selective diesel fuel blends have been predicted using the Discrete Component model (DCM). These mixtures are combined with up to 116 components of 98 hydrocarbons and 4–18 methyl esters. The results are compared with the predictions of the case when blended diesel-biodiesel fuels are represented by pure fossil and biodiesel fuels. In contrast to previous studies, it is shown that droplet evaporation time and surface temperature predicted for 100% biodiesel are not always close to those predicted for pure diesel fuel, but they are entirely dependent on the biodiesel fuel composition. Also, the previously introduced multi-dimensional quasi-discrete model and its application to the aforementioned fuels are discussed. The application of the latter model contributes to improving the CPU efficiency with up to 96% in computational time compared to the original approach (the DCM), with a minimal loss in the accuracy of the model predictions.

Automatic benchmark profiling through advanced workflow‐based trace analysis

SummaryBenchmarking has proven to be crucial for the investigation of the behavior and performances of a system. However, the choice of relevant benchmarks still remains a challenge. To help the process of comparing and choosing among benchmarks, we propose a solution for automatic benchmark profiling. It computes unified benchmark profiles reflecting benchmarks' duration, function repartition, stability, CPU efficiency, parallelization, and memory usage. Our approach identifies the needed system information for profile computation and collects it from execution traces captured without benchmark code modifications. It structures profile computation as a reproducible workflow for automatic trace analysis, which efficiently manages important trace volumes. In this paper, we report on the design and the implementation of our approach, which involves the collection and analysis of about 500 GB of trace data coming from 2 different platforms (an x86 desktop machine and the Juno SoC board). The computed benchmark profiles provide valuable insights about the benchmarks' behavior and help compare different benchmarks on the same platform as well as the behavior of the same benchmark on different platforms.

A 3D parallel Particle-In-Cell solver for wave interaction with vertical cylinders

In this paper, the Particle-In-Cell (PIC) based PICIN solver is extended to three spatial dimensions and parallelised using the Message Passing Interface (MPI) approach. The PICIN solver employs both Eulerian grid and Lagrangian particles to solve the incompressible Navier-Stokes equations for free-surface flows. The particles are employed to carry all the fluid properties, solve the non-linear advection term and track the free-surface, while the grid is used solely for computational efficiency in solving the non-advection terms. Validation of the new 3D model concentrates on test cases involving multiple wave types (including regular waves, focused waves and solitary waves) interacting with vertical cylinders in several spatial configurations. The results are compared with laboratory data and numerical results from state-of-the-art Volume of Fluid (VOF) based Eulerian solvers such as those from the OpenFOAM® suite. It is shown that the 3D parallel PICIN model is able to well simulate highly non-linear water waves, and the interaction of such waves with vertical cylinders, with a CPU efficiency similar to Eulerian solvers. Moreover, the innovative use of particles in PICIN, akin to meshless Lagrangian solvers, gives the model a particular flexibility in handling complex, full 3D, water-wave scenarios involving large free-surface deformations.

Parallel Monte Carlo Search for Hough Transform

We investigate the problem of line detection in digital image processing and in special how state of the art algorithms behave in the presence of noise and whether CPU efficiency can be improved by the combination of a Monte Carlo Tree Search, hierarchical space decomposition, and parallel computing.The starting point of the investigation is the method introduced in 1962 by Paul Hough for detecting lines in binary images. Extended in the 1970s to the detection of space forms, what came to be known as Hough Transform (HT) has been proposed, for example, in the context of track fitting in the LHC ATLAS and CMS projects. The Hough Transform transfers the problem of line detection, for example, into one of optimization of the peak in a vote counting process for cells which contain the possible points of candidate lines. The detection algorithm can be computationally expensive both in the demands made upon the processor and on memory. Additionally, it can have a reduced effectiveness in detection in the presence of noise.Our first contribution consists in an evaluation of the use of a variation of the Radon Transform as a form of improving theeffectiveness of line detection in the presence of noise. Then, parallel algorithms for variations of the Hough Transform and the Radon Transform for line detection are introduced. An algorithm for Parallel Monte Carlo Search applied to line detection is also introduced. Their algorithmic complexities are discussed. Finally, implementations on multi-GPU and multicore architectures are discussed.

Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric.

Data imbalance is frequently encountered in biomedical applications. Resampling techniques can be used in binary classification to tackle this issue. However such solutions are not desired when the number of samples in the small class is limited. Moreover the use of inadequate performance metrics, such as accuracy, lead to poor generalization results because the classifiers tend to predict the largest size class. One of the good approaches to deal with this issue is to optimize performance metrics that are designed to handle data imbalance. Matthews Correlation Coefficient (MCC) is widely used in Bioinformatics as a performance metric. We are interested in developing a new classifier based on the MCC metric to handle imbalanced data. We derive an optimal Bayes classifier for the MCC metric using an approach based on Frechet derivative. We show that the proposed algorithm has the nice theoretical property of consistency. Using simulated data, we verify the correctness of our optimality result by searching in the space of all possible binary classifiers. The proposed classifier is evaluated on 64 datasets from a wide range data imbalance. We compare both classification performance and CPU efficiency for three classifiers: 1) the proposed algorithm (MCC-classifier), the Bayes classifier with a default threshold (MCC-base) and imbalanced SVM (SVM-imba). The experimental evaluation shows that MCC-classifier has a close performance to SVM-imba while being simpler and more efficient.

The operational eEMEP model version 10.4 for volcanic SO<sub>2</sub> and ash forecasting

Abstract. This paper presents a new version of the EMEP MSC-W model called eEMEP developed for transportation and dispersion of volcanic emissions, both gases and ash. EMEP MSC-W is usually applied to study problems with air pollution and aerosol transport and requires some adaptation to treat volcanic eruption sources and effluent dispersion. The operational set-up of model simulations in case of a volcanic eruption is described. Important choices have to be made to achieve CPU efficiency so that emergency situations can be tackled in time, answering relevant questions of ash advisory authorities. An efficient model needs to balance the complexity of the model and resolution. We have investigated here a meteorological uncertainty component of the volcanic cloud forecast by using a consistent ensemble meteorological dataset (GLAMEPS forecast) at three resolutions for the case of SO2 emissions from the 2014 Barðarbunga eruption. The low resolution (40 × 40 km) ensemble members show larger agreement in plume position and intensity, suggesting that the ensemble here does not give much added value. To compare the dispersion at different resolutions, we compute the area where the column load of the volcanic tracer, here SO2, is above a certain threshold, varied for testing purposes between 0.25 and 50 Dobson units. The increased numerical diffusion causes a larger area (+34 %) to be covered by the volcanic tracer in the low resolution simulations than in the high resolution ones. The higher resolution (10 × 10 km) ensemble members show higher column loads farther away from the volcanic eruption site in narrower clouds. Cloud positions are more varied between the high resolution members, and the cloud forms resemble the observed clouds more than the low resolution ones. For a volcanic emergency case this means that to obtain quickly results of the transport of volcanic emissions, an individual simulation with our low resolution is sufficient; however, to forecast peak concentrations with more certainty for forecast or scientific analysis purposes, a finer resolution is needed. The model is further developed to simulate ash from highly explosive eruptions. A possibility of increasing the number of vertical layers, achieving finer vertical resolution, as well as a higher model top, is included in the eEMEP version. Ash size distributions may be altered for different volcanic eruptions and assumptions. Since ash particles are larger than typical particles in the standard model, gravitational settling across all vertical layers is included. We attempt finally a specific validation of the simulation of ash and its vertical distribution. Model simulations with and without gravitational settling for the 2010 Eyjafjallajökull eruption are compared to lidar observations over central Europe. The results show that with gravitation the centre of the ash mass can be 1 km lower over central Europe than without gravitation. However, the height variations in the ash layer caused by real weather situations are not captured perfectly well by either of the two simulations, playing down the role of gravitation parameterization imperfections. Both model simulations have on average an ash centre of mass below the observed values. Correlations between the observed and corresponding model centres of mass are higher for the model simulation with gravitational settling for four of the six stations studied here. The inclusion of gravitational settling is suggested to be required for a volcanic ash model.

CPU Efficiency Enhancement through Offload

CPU Efficiency Enhancement through Offload

Runtime Vectorization Transformations of Binary Code

In many cases, applications are not optimized for the hardware on which they run. Several reasons contribute to this unsatisfying situation, such as legacy code, commercial code distributed in binary form, or deployment on compute farms. In fact, backward compatibility of ISA guarantees only the functionality, not the best exploitation of the hardware. In this work, we focus on maximizing the CPU efficiency for the SIMD extensions. The first contribution was originally published in the International Conference on Embedded Computer Systems: Architectures, Modeling and Simulation, SAMOS XV, July 2015, Agios Konstantinos, Greece. It is a binary-to-binary optimization framework where loops vectorized for an older version of the processor SIMD extension are automatically converted to a newer one. It is a lightweight mechanism that does not include a vectorizer, but instead leverages what a static vectorizer previously did. We show that many loops compiled for x86 SSE can be dynamically converted to the more recent and more powerful AVX; as well as, how correctness is maintained with regards to challenges such as data dependencies and reductions. We obtain speedups in line with those of a native compiler targeting AVX. The second contribution is the runtime vectorization of loops in binary codes that were not originally vectorized. For this purpose, we use open source frameworks that we have tuned and integrated to (1) dynamically lift the x86 binary into the Intermediate Representation form of the LLVM compiler, (2) abstract hot loops in the polyhedral model, (3) use the power of this mathematical framework to vectorize them, and (4) finally compile them back into executable form using the LLVM Just-In-Time compiler. In most cases, the obtained speedups are close to the number of elements that can be simultaneously processed by the SIMD unit. The re-vectorizer and auto-vectorizer are implemented inside a dynamic optimization platform; it is completely transparent to the user, does not require any rewriting of the binaries, and operates during program execution.

Study of ATLAS TRT performance with GRID and supercomputers

One of the most important studies dedicated to be solved for ATLAS physical analysis is a reconstruction of proton-proton events with large number of interactions in Transition Radiation Tracker. Paper includes Transition Radiation Tracker performance results obtained with the usage of the ATLAS GRID and Kurchatov Institute’s Data Processing Center including Tier-1 grid site and supercomputer as well as analysis of CPU efficiency during these studies.

A dynamic lattice method for elastic seismic modeling in anisotropic media

Basing on a particle network model, we have developed a dynamic lattice approach to simulate seismic wave propagation in transversely isotropic (TI) media with tilted symmetry axis (TTI). Different from other wave-equation-based numerical methods, the dynamic lattice approach calculates the micromechanical interactions between particles in the lattice instead of solving the wave equation. We implement Lagrange’s equations to transform these interactions into elastic forces acting upon each particle. By solving the equations of motion, we obtain the disturbances of particles. Therefore, seismic wave motions in continua are approximated by displacements of these particles. Elastic features of the continuum are represented by properties of the particle lattice, including the physical properties of particles and the micromechanical interactions between particles. In the case of TI media, it is a challenge to find the correct particle lattice model that can reflect the anisotropic nature of TI media. We have determined the theoretical connection between the TI medium and the particle lattice model, allowing us to model elastic seismic waves in heterogeneous TI media. Furthermore, we have linearized the propagator to improve the CPU efficiency of our method for seismic wave simulation. We have applied the method to reverse time migration on a TI model to test its usefulness in complex media imaging.

Modelling spray and combustion processes in diesel engine by using the coupled Eulerian–Eulerian and Eulerian–Lagrangian method

The focus of this paper is to demonstrate that the existing and developed models, within the coupled approach, are capable for describing spray, combustion and pollutant formation processes. The Eulerian–Eulerian (EE) method, multi-continuum model, is used for spray modelling including the fuel jet disintegration and droplet evaporation processes. The combustion process is modelled by employing the Eulerian–Lagrangian (EL) discrete droplet model (DDM). The EE–EL coupling has been performed by running simultaneous simulations on a separate computational domains with constantly exchanging mass, enthalpy and momentum sources. The calculated results were compared to the available experimental data and to the previously published results. It is concluded that the pressure and temperature traces, energy conversion and the emission trends are well described and a good agreement to the experimental data is observed. The advantage of the presented approach is retaining a high CPU efficiency, characteristic for the EL method, with an increased solution accuracy in the near-nozzle region characteristic for the EE method.

The Geant4 Bertini Cascade

One of the medium energy hadron–nucleus interaction models in the Geant4 simulation toolkit is based partly on the Bertini intranuclear cascade model. Since its initial appearance in the toolkit, this model has been largely re-written in order to extend its physics capabilities and to reduce its memory footprint. Physics improvements include extensions in applicable energy range and incident particle types, and improved hadron–nucleon cross-sections and angular distributions. Interfaces have also been developed which allow the model to be coupled with other Geant4 models at lower and higher energies. The inevitable speed reductions due to enhanced physics have been mitigated by memory and CPU efficiency improvements. Details of these improvements, along with selected comparisons of the model to data, are discussed.

An experimental evaluation and analysis of database cracking

Database cracking has been an area of active research in recent years. core idea of database cracking is to create indexes adaptively and incrementally as a side product of query processing. Several works have proposed different cracking techniques for different aspects including updates, tuple reconstruction, convergence, concurrency control, and robustness. Our 2014 VLDB paper The Uncracked Pieces in Database Cracking (PVLDB 7:97---108, 2013/VLDB 2014) was the first comparative study of these different methods by an independent group. In this article, we extend our published experimental study on database cracking and bring it to an up-to-date state. Our goal is to critically review several aspects, identify the potential, and propose promising directions in database cracking. With this study, we hope to expand the scope of database cracking and possibly leverage cracking in database engines other than MonetDB. We repeat several prior database cracking works including the core cracking algorithms as well as three other works on convergence (hybrid cracking), tuple reconstruction (sideways cracking), and robustness (stochastic cracking), respectively. Additionally to our conference paper, we now also look at a recently published study about CPU efficiency (predication cracking). We evaluate these works and show possible directions to do even better. As a further extension, we evaluate the whole class of parallel cracking algorithms that were proposed in three recent works. Altogether, in this work we revisit 8 papers on database cracking and evaluate in total 18 cracking methods, 6 sorting algorithms, and 3 full index structures. Additionally, we test cracking under a variety of experimental settings, including high selectivity (Low selectivity means that many entries qualify. Consequently, a high selectivity means, that only few entries qualify) queries, low selectivity queries, varying selectivity, and multiple query access patterns. Finally, we compare cracking against different sorting algorithms as well as against different main memory optimized indexes, including the recently proposed adaptive radix tree (ART). Our results show that: (1) the previously proposed cracking algorithms are repeatable, (2) there is still enough room to significantly improve the previously proposed cracking algorithms, (3) parallelizing cracking algorithms efficiently is a hard task, (4) cracking depends heavily on query selectivity, (5) cracking needs to catch up with modern indexing trends, and (6) different indexing algorithms have different indexing signatures.

...like Commanding an Anthill

Computer system architecture has changed: an assembly of autonomous components has replaced the omnipotent CPU and its legion of dumb devices. Database Management System (DBMS) architecture, however, does not yet reflect this change: it is still dominated by a centralized kernel that limits the autonomy of the devices and, thus, their ability to exploit their increased 'smartness'. Distributed data management research can serve as an inspiration for an architecture that addresses this problem. However, the respective algorithms were never designed with CPU efficiency in mind implementing principles like dynamic programming and recursion. More than two decades ago, the transition to memory resident databases spawned a plethora of research on CPU-efficient query processors. We predict that hardware heterogeneity will trigger a similar line of research on CPU-efficient distributed algorithms and architectures. In this paper, we examine benefits and challenges that come with such a micro-distributed database management system. We also discuss a number of approaches that we consider steps towards a micro-distributed system

Efficient hyper reduced-order model (HROM) for parametric studies of the 3D thermo-elasto-plastic calculation

This paper focuses on a 3D thermo-elasto-plastic localized thermal source simulation and its parametric analysis with high CPU efficiency in the reduced-order model (ROM) framework. The hyper reduced-order model (HROM) is introduced and improved with two choices. Firstly, three reduced bases are constructed: one for the displacement increments, one for the plastic strain increments and one for the stress state. Equilibrium equation in plasticity relies on the knowledge of plastic strain rate, hence the plastic strain has to be included into the variable to be reduced, and the incremental form is adopted in the paper. It is shown that the introduction of an extra stress basis greatly improves the quality and the efficiency of the ROM. Secondly, the reduced state variables of plastic strain increments are determined in a reduced integration domain. Concerning the parametric analysis, the interpolation of the reduced bases is based on the Grassmann manifold, which permits to generate the new proper orthogonal decomposition bases for the modified parameters. In order to increase the convergence rate, the plastic strain interpolated from snapshots (the reference cases with full FEM calculations) is considered as the initial value of each time step for the modified problem of parametric studies. As a result, the plastic calculation is always done on the confined domain and only a few iterations are then required to reach static and plastic admissibility for each time step. The parametric studies on varying thermal load and yield stress show high versatility and efficiency of the HROM coupled with Grassmann manifold interpolation. A gain of CPU time of 25 is obtained for both cases with a level of accuracy smaller than 10%.

The ALICE analysis train system

In the ALICE experiment hundreds of users are analyzing big datasets on a Grid system. High throughput and short turn-around times are achieved by a centralized system called the LEGO trains. This system combines analysis from different users in so-called analysis trains which are then executed within the same Grid jobs thereby reducing the number of times the data needs to be read from the storage systems. The centralized trains improve the performance, the usability for users and the bookkeeping in comparison to single user analysis. The train system builds upon the already existing ALICE tools, i.e. the analysis framework as well as the Grid submission and monitoring infrastructure. The entry point to the train system is a web interface which is used to configure the analysis and the desired datasets as well as to test and submit the train. Several measures have been implemented to reduce the time a train needs to finish and to increase the CPU efficiency.

An Optimized Approach to CPU Scheduling Algorithm: Min-max

CPU scheduling is the basis of multi-programmed operating systems. The scheduler is accountable for multiplexing processes on the CPU. By switching the CPU among processes, the operating system results in the computer more prolific. Various CPU scheduling algorithms exist for a multi-programmed operating system like First Come First Served (FCFS), Shortest Job First (SJF), Shortest Remaining Time Next (SRTN) and Round Robin (RR) scheduling algorithm. This paper has threefold objective. Firstly, it throws light on a new-fangled CPU scheduling algorithm called MIN-MAX whichbehaves as both preemptive and non-preemptive algorithm basis on the burst time. Secondly, it would helps to understand how MIN-MAXalgorithm improves the CPU efficiency in multi programming operating system and also trims down the starvation problem amongprocesses. Thirdly, it focuses on the comparative study of the existing algorithms on basis of various scheduling parameters with the proposed algorithm MIN-MAX.