Benchmark Kernels Research Articles

Projective Multiple Kernel Subspace Clustering

Multiple kernel subspace clustering (MKSC), as an important extension for handling multi-view non-linear subspace data, has shown notable success in a wide variety of machine learning tasks. The key objective of MKSC is to build a flexible and appropriate graph for clustering from the kernel space. However, existing MKSC methods apply a mechanism utilizing the kernel trick to the traditional self-expressive principle, where the similarity graphs are built on the respective high-dimensional (or even infinite) reproducing kernel Hilbert space (RKHS). Regarding this strategy, we argue that the original high-dimensional spaces usually include noise and unreliable similarity measures and, therefore, output a low-quality graph matrix, which degrades clustering performance. In this paper, inspired by projective clustering, we propose the utilization of a complementary similarity graph by fusing the multiple kernel graphs constructed in the low-dimensional partition space, termed projective multiple kernel subspace clustering (PMKSC). By incorporating intrinsic structures with multi-view data, PMKSC alleviates the noise and redundancy in the original kernel space and obtains high-quality similarity to uncover the underlying clustering structures. Furthermore, we design a three-step alternate algorithm with proven convergence to solve the proposed optimization problem. The experimental results on ten multiple kernel benchmark datasets validate the effectiveness of our proposed PMKSC, compared to the state-of-the-art multiple kernel and kernel subspace clustering methods, by a large margin. Our code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/MengjingSun/PMKSC-code</uri> .

Spare simple MKKM with semi‐infinite linear program optimization

Multiple kernel clustering (MKC) optimally combines a group of predefined kernel matrices to improve clustering performance. Although demonstrating promising performance in various applications, most of existing approaches adopt the min–min formulation, which could be sensitive to perturbation with adversarial samples. Moreover, existing MKC algorithms often involve several hypermeters preventing them into further real applications. To address these issues, we propose a parameter-free effective sparse simple multiple kernel k-means algorithm with max–min optimization formulation in this paper. To be specific, we propose to optimize the widely used unsupervised kernel alignment criterion by minimizing the kernel coefficient and maximizing the clustering partition matrix. Unlike traditional min–min formulation, the max–min kernel alignment is robust to adversarial sample perturbation and free of hyper-parameters. An optimization method based on semi-infinite linear program is designed to solve the complicated optimization problem. Extensive experiments on six multiple kernel benchmark data sets demonstrate the effectiveness of the proposed method.

Multiple kernel clustering with late fusion consensus local graph preserving

Multiple kernel clustering (MKC) methods aim at integrating an optimal partition from a set of precalculated kernel matrices. Though achieving success in various applications, we observe that existing MKC methods: (i) lack of representation flexibility; and (ii) do not considerably preserve the locality structure in partition space. These issues may adversely affect the learning procedure of MKC, leading to unsatisfying clustering performance. In this paper, we propose a late fusion MKC method with local graph refinement to address the aforementioned issues. Different from existing MKC mechanisms, our method unifies the traditional weighted multiple kernel k-means, kernel partition, and graph construction into a single optimization procedure. The local graph is utilized to preserve the locality information in partition space and therefore all of the counterparts can be boosted for mutual clustering improvements. By this way, our approach enhances the local graph structure in partition space and enjoys more flexible kernel representations, leading to significant clustering improvements. Moreover, a three-step alternate algorithm is developed to solve the resultant optimization problem with proved convergence. Extensive experiments are conducted on several multiple kernel benchmark datasets to compare the proposed algorithm with the state-of-the-art ones, and the results well demonstrate its effectiveness and superiority.

M-ark-Support Vector Machine for Early Detection of Parkinson’s Disease from Speech Signals

Recent advances in the state-of-the-art open-source kernel functions for support vector machines (SVMs) have widened the choices of benchmark kernels for Machine Learning (ML)-based classification. However, it is still challenging to achieve margin maximisation in SVM, and further evidence is required to ensure such novel kernel functions can have translational applications with tangible impact. Noteworthily, m-arcsinh, freely available in scikit-learn, was preliminarily proven as a benchmark kernel function on 15 datasets in its seminal paper. Quantifying the benefit from leveraging this kernel in a specific application is essential to provide further evidence of its accuracy and reliability on real-life supervised ML-aided tasks. Thus, the predictive capability of SVM, including that with Lagrange multipliers for the first time coupled with m-arcsinh (m-ark-SVM with soft margin; m-arK-SVM with hard margin), is hereby assessed in aiding early detection of Parkinson’s Disease (PD) from speech data. This is important to leverage the m-arcsinh kernel ‘trick’ to maximise the margin width and, therefore, the linear separability of input speech features via automated pattern recognition. In this study, we demonstrate the accuracy and reliability of m-ark-SVM to aid early diagnosis of PD, evaluated against other gold standard kernel functions. Two benchmark datasets from the University of California-Irvine (UCI) database, pre-processed solely via min-max normalisation, were used to discriminate between speech patterns of 72 healthy subjects and 211 patients with PD. Overtraining was avoided via cross validation and the models were developed and tested in Python 3.7. The supervised model (m-ark-SVM) could detect early Parkinson’s Disease with 87.18% and 86.9% classification accuracy from the two datasets respectively (F1- scores: 85 and 86.2% correspondingly). Furthermore, the model achieved high precision (89.2% and 86.8%) and specificity (87% and 86.8%). Thus, this study validates the application of m-arcsinh to aid real-life supervised ML-based classification, in particular early diagnosis of Parkinson’s Disease from speech data.

A mechanism for balancing accuracy and scope in cross-machine black-box GPU performance modeling

The ability to model, analyze, and predict execution time of computations is an important building block that supports numerous efforts, such as load balancing, benchmarking, job scheduling, developer-guided performance optimization, and the automation of performance tuning for high performance, parallel applications. In today’s increasingly heterogeneous computing environment, this task must be accomplished efficiently across multiple architectures, including massively parallel coprocessors like GPUs, which are increasingly prevalent in the world’s fastest supercomputers. To address this challenge, we present an approach for constructing customizable, cross-machine performance models for GPU kernels, including a mechanism to automatically and symbolically gather performance-relevant kernel operation counts, a tool for formulating mathematical models using these counts, and a customizable parameterized collection of benchmark kernels used to calibrate models to GPUs in a black-box fashion. With this approach, we empower the user to manage trade-offs between model accuracy, evaluation speed, and generalizability. A user can define their own model and customize the calibration process, making it as simple or complex as desired, and as application-targeted or general as desired. As application examples of our approach, we demonstrate both linear and nonlinear models; these examples are designed to predict execution times for multiple variants of a particular computation: two matrix-matrix multiplication variants, four discontinuous Galerkin differentiation operation variants, and two 2D five-point finite difference stencil variants. For each variant, we present accuracy results on GPUs from multiple vendors and hardware generations. We view this highly user-customizable approach as a response to a central question arising in GPU performance modeling: how can we model GPU performance in a cost-explanatory fashion while maintaining accuracy, evaluation speed, portability, and ease of use, an attribute we believe precludes approaches requiring manual collection of kernel or hardware statistics.

Mass-based finite volume scheme for aggregation, growth and nucleation population balance equation.

In this paper, a new mass-based numerical method is developed using the notion of Forestier-Coste & Mancini (Forestier-Coste & Mancini 2012, SIAM J. Sci. Comput. 34, B840-B860. (doi:10.1137/110847998)) for solving a one-dimensional aggregation population balance equation. The existing scheme requires a large number of grids to predict both moments and number density function accurately, making it computationally very expensive. Therefore, a mass-based finite volume is developed which leads to the accurate prediction of different integral properties of number distribution functions using fewer grids. The new mass-based and existing finite volume schemes are extended to solve simultaneous aggregation-growth and aggregation-nucleation problems. To check the accuracy and efficiency, the mass-based formulation is compared with the existing method for two kinds of benchmark kernels, namely analytically solvable and practical oriented kernels. The comparison reveals that the mass-based method computes both number distribution functions and moments more accurately and efficiently than the existing method.

Optimizing Remote Communication in X10

X10 is a partitioned global address space programming language that supports the notion of places ; a place consists of some data and some lightweight tasks called activities. Each activity runs at a place and may invoke a place-change operation (using the at-construct) to synchronously perform some computation at another place. These place-change operations can be very expensive, as they need to copy all the required data from the current place to the remote place. However, identifying the necessary number of place-change operations and the required data during each place-change operation are non-trivial tasks, especially in the context of irregular applications (like graph applications) that contain complex code with large amounts of cross-referencing objects—not all of those objects may be actually required, at the remote place. In this article, we present AT-Com, a scheme to optimize X10 code with place-change operations. AT-Com consists of two inter-related new optimizations: (i) AT-Opt, which minimizes the amount of data serialized and communicated during place-change operations, and (ii) AT-Pruning, which identifies/elides redundant place-change operations and does parallel execution of place-change operations. AT-Opt uses a novel abstraction, called abstract-place-tree , to capture place-change operations in the program. For each place-change operation, AT-Opt uses a novel inter-procedural analysis to precisely identify the data required at the remote place in terms of the variables in the current scope. AT-Opt then emits the appropriate code to copy the identified data-items to the remote place. AT-Pruning introduces a set of program transformation techniques to emit optimized code such that it avoids the redundant place-change operations. We have implemented AT-Com in the x10v2.6.0 compiler and tested it over the IMSuite benchmark kernels. Compared to the current X10 compiler, the AT-Com optimized code achieved a geometric mean speedup of 18.72× and 17.83× on a four-node (32 cores per node) Intel and two-node (16 cores per node) AMD system, respectively.

Lattice Boltzmann benchmark kernels as a testbed for performance analysis

Lattice Boltzmann methods (LBM) are an important part of current computational fluid dynamics (CFD). They allow easy implementations and boundary handling. However, competitive time to solution not only depends on the choice of a reasonable method, but also on an efficient implementation on modern hardware. Hence, performance optimization has a long history in the lattice Boltzmann community. A variety of options exists regarding the implementation with direct impact on the solver performance. Experimenting and evaluating each option often is hard as the kernel itself is typically embedded in a larger code base. With our suite of lattice Boltzmann kernels we provide the infrastructure for such endeavors. Already included are several kernels ranging from simple to fully optimized implementations. Although these kernels are not fully functional CFD solvers, they are equipped with a solid verification method. The kernels may act as an reference for performance comparisons and as a blue print for optimization strategies. In this paper we give an overview of already available kernels, establish a performance model for each kernel, and show a comparison of implementations and recent architectures.

Improving Loop Dependence Analysis

Programmers can no longer depend on new processors to have significantly improved single-thread performance. Instead, gains have to come from other sources such as the compiler and its optimization passes. Advanced passes make use of information on the dependencies related to loops. We improve the quality of that information by reusing the information given by the programmer for parallelization. We have implemented a prototype based on GCC into which we also add a new optimization pass. Our approach improves the amount of correctly classified dependencies resulting in 46% average improvement in single-thread performance for kernel benchmarks compared to GCC 6.1.

HyBar: high efficient barrier synchronization based on a hybrid packet-circuit switching Network-on-Chip

Realizing barrier synchronization in multi-/many-core processors with high efficiency becomes more and more challenging as the number of cores integrated in a single chip keeps growing. Quite a few barrier solutions have been proposed, while they provide limited improvements for synchronizing large amounts of cores or incur unfavorable restrictions on performing concurrent barriers. This paper presents HyBar, a hardware barrier based on a hybrid switching NoC which adopts packet switching and circuit switching methods in two sub-networks respectively. Dedicated channels in the circuit-switching sub-network are dynamically built and removed when barrier requests traverse the packet-switching sub-network according to a modified dimension-order routing algorithm. The efficiency of inter-core communication for concurrent barriers is improved by merging barrier arrival requests and broadcasting release requests along the circuit channels. The execution time of synthetic cases, benchmark kernels and parallel applications using various barrier solutions are evaluated in an RTL-based simulation platform. Experimental results show that our proposal provides about 15\%--50\% performance improvement compared to previous solutions, while the hardware overhead is marginal under SMIC 40 nm technology. Moreover, HyBar introduces a minor efficiency loss for concurrent barriers with no limitation on their layouts of participating cores in the on-chip network.

Enabling PGAS Productivity with Hardware Support for Shared Address Mapping

Due to its rich memory model, the partitioned global address space (PGAS) parallel programming model strikes a balance between locality-awareness and the ease of use of the global address space model. Although locality-awareness can lead to high performance, supporting the PGAS memory model is associated with penalties that can hinder PGAS’s potential for scalability and speed of execution. This is because mapping the PGAS memory model to the underlying system requires a mapping process that is done in software, thereby introducing substantial overhead for shared accesses even when they are local. Compiler optimizations have not been sufficient to offset this overhead. On the other hand, manual code optimizations can help, but this eliminates the productivity edge of PGAS. This article proposes a processor microarchitecture extension that can perform such address mapping in hardware with nearly no performance overhead. These extensions are then availed to compilers through extensions to the processor instructions. Thus, the need for manual optimizations is eliminated and the productivity of PGAS languages is unleashed. Using Unified Parallel C (UPC), a PGAS language, we present a case study of a prototype compiler and architecture support. Two different implementations of the system were realized. The first uses a full-system simulator, gem5, which evaluates the overall performance gain of the new hardware support. The second uses an FPGA Leon3 soft-core processor to verify implementation feasibility and to parameterize the cost of the new hardware. The new instructions show promising results on all tested codes, including the NAS Parallel Benchmark kernels in UPC. Performance improvements of up to 5.5× for unmodified codes, sometimes surpassing hand-optimized performance, were demonstrated. We also show that our four-core FPGA prototype requires less than 2.4% of the overall chip’s area.

IMSuite: A benchmark suite for simulating distributed algorithms

Considering the diverse nature of real-world distributed applications that makes it hard to identify a representative subset of distributed benchmarks, we focus on their underlying distributed algorithms. We present and characterize a new kernel benchmark suite (named IMSuite) that simulates some of the classical distributed algorithms in task parallel languages. We present multiple variations of our kernels, broadly categorized under two heads: (a) varying synchronization primitives (with and without fine grain synchronization primitives); and (b) varying forms of parallelization (data parallel and recursive task parallel). Our characterization covers interesting aspects of distributed applications such as distribution of remote communication requests, number of synchronization, task creation, task termination and atomic operations. We study the behavior (execution time) of our kernels by varying the problem size, the number of compute threads, and the input configurations. We also present an involved set of input generators and output validators.

Chisel

The accuracy of an approximate computation is the distance between the result that the computation produces and the corresponding fully accurate result. The reliability of the computation is the probability that it will produce an acceptably accurate result. Emerging approximate hardware platforms provide approximate operations that, in return for reduced energy consumption and/or increased performance, exhibit reduced reliability and/or accuracy. We present Chisel, a system for reliability- and accuracy-aware optimization of approximate computational kernels that run on approximate hardware platforms. Given a combined reliability and/or accuracy specification, Chisel automatically selects approximate kernel operations to synthesize an approximate computation that minimizes energy consumption while satisfying its reliability and accuracy specification. We evaluate Chisel on five applications from the image processing, scientific computing, and financial analysis domains. The experimental results show that our implemented optimization algorithm enables Chisel to optimize our set of benchmark kernels to obtain energy savings from 8.7% to 19.8% compared to the fully reliable kernel implementations while preserving important reliability guarantees.

Algorithmic GPGPU Memory Optimization

The performance of General-Purpose computation on Graphics Processing Units (GPGPU) is heavily dependent on the memory access behavior. This sensitivity is due to a combination of the underlying Massively Parallel Processing (MPP) execution model present on GPUs and the lack of architectural support to handle irregular memory access patterns. Application performance can be significantly improved by applying memory-access-pattern-aware optimizations that can exploit knowledge of the characteristics of each access pattern. In this paper, we present an algorithmic methodology to semi-automatically find the best mapping of memory accesses present in serial loop nest to underlying data-parallel architectures based on a comprehensive static memory access pattern analysis. To that end we present a simple, yet powerful, mathematical model that captures all memory access pattern information present in serial data-parallel loop nests. We then show how this model is used in practice to select the most appropriate memory space for data and to search for an appropriate thread mapping and work group size from a large design space. To evaluate the effectiveness of our methodology, we report on execution speedup using selected benchmark kernels that cover a wide range of memory access patterns commonly found in GPGPU workloads. Our experimental results are reported using the industry standard heterogeneous programming language, OpenCL, targeting the NVIDIA GT200 architecture.

Profiling e-Science infrastructures with kernel and application benchmarks

Resources benchmarking is a promising though challenging opportunity for research communities to better exploit e-Science infrastructures. The paper investigates the suitability of an integrated two-level benchmarking approach in supporting resources ranking on a performance basis. The coupling of different benchmarking techniques i.e. kernel and application benchmarks is intended to grasp the behaviour of computational environments also with respect to real workloads so as to address a more suitable and efficient assignment of resources to the applications submitted by users. To confirm the appropriateness of our approach, an experimentation on a test bed was carried out which highlighted significant differences amongst the performance figures of the two levels of benchmarks. Some useful rules of thumb emerged from our experimentation that could be taken into account to fruitfully adopt this double-space approach to similar scenarios of use.

Tile size selection revisited

Loop tiling is a widely used loop transformation to enhance data locality and allow data reuse. In the tiled code, however, tiles of different sizes can lead to significant variation in performance. Thus, selection of an optimal tile size is critical to performance of tiled codes. In the past, tile size selection has been attempted using both static analytical and dynamic empirical (auto-tuning) models. Past work using static models assumed a direct-mapped cache for the purpose of analysis and thus proved to be less robust. On the other hand, the auto-tuning models involve an exhaustive search in a large space of tiled codes. In this article, we propose a new analytical model for tile size selection that leverages the high set associativity in modern caches to minimize conflict misses. Our tile size selection model targets data reuse in multiple levels of cache. In addition, it considers the interaction of tiling with the SIMD unit in modern processors in estimating the optimal tile size. We find that these factors, not considered in previous models, are critical in developing a robust model for tile size selection. We implement our tile size selection model in a polyhedral compiler and test it on 12 benchmark kernels using two different problem sizes. Our model outperforms the previous analytical models that are based on reusing data in a single level of cache and achieves an average performance improvement of 9.7% and 20.4%, respectively, over the best square (cubic) tiles for the two problem sizes. In addition, the tile size chosen by our tile size selection algorithm is similar to the best performing size obtained through an extensive search, validating the analytical model underlying the algorithm.

Transparent Trace-Based Binary Acceleration for Reconfigurable HW/SW Systems

This paper presents a novel approach to accelerate program execution by mapping repetitive traces of executed instructions, called Megablocks, to a runtime reconfigurable array of functional units. An offline tool suite extracts Megablocks from microprocessor instruction traces and generates a Reconfigurable Processing Unit (RPU) tailored for the execution of those Megablocks. The system is able to transparently movebcomputations from the microprocessor to the RPU at runtime. A prototype implementation of the system using a cacheless MicroBlaze microprocessor running code located in external memory reaches speedups from <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$2.2\times$</tex> </formula> to <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$18.2 \times$</tex> </formula> for a set of 14 benchmark kernels. For a system setup which maximizes microprocessor performance by having the application code located in internal block RAMs, speedups from <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$1.4 \times$</tex></formula> to <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$2.8 \times$</tex></formula> were estimated.

Polyhedral Bubble Insertion: A Method to Improve Nested Loop Pipelining for High-Level Synthesis

High-level synthesis (HLS) allows hardware to be directly produced from behavioral description in C/C++, thus accelerating the design process. Loop pipelining is a key transformation of HLS, as it improves the throughput of the design at the price of a small hardware overhead. However, for small loops, its use often results in a poor hardware utilization due to the pipeline latency overhead. Overlapping the iterations of the whole loop nest instead of only overlapping the innermost loop is a way to overcome this difficulty, but currently available techniques are restricted to perfectly nested loops with constant bounds, involving uniform dependences only. Using the polyhedral model, we extend the applicability of the nested loop pipelining transformation by proposing a new legality check and a new loop correction technique, called polyhedral bubble insertion. This method was implemented in a source-to-source compiler targeting HLS, and results on benchmark kernels show that polyhedral bubble insertion is effective in practice on a much larger class of loop nests.

High‐Level Synthesis: Productivity, Performance, and Software Constraints

FPGAs are an attractive platform for applications with high computation demand and low energy consumption requirements. However, design effort for FPGA implementations remains high—often an order of magnitude larger than design effort using high‐level languages. Instead of this time‐consuming process, high‐level synthesis (HLS) tools generate hardware implementations from algorithm descriptions in languages such as C/C++ and SystemC. Such tools reduce design effort: high‐level descriptions are more compact and less error prone. HLS tools promise hardware development abstracted from software designer knowledge of the implementation platform. In this paper, we present an unbiased study of the performance, usability and productivity of HLS using AutoPilot (a state‐of‐the‐art HLS tool). In particular, we first evaluate AutoPilot using the popular embedded benchmark kernels. Then, to evaluate the suitability of HLS on real‐world applications, we perform a case study of stereo matching, an active area of computer vision research that uses techniques also common for image denoising, image retrieval, feature matching, and face recognition. Based on our study, we provide insights on current limitations of mapping general‐purpose software to hardware using HLS and some future directions for HLS tool development. We also offer several guidelines for hardware‐friendly software design. For popular embedded benchmark kernels, the designs produced by HLS achieve 4X to 126X speedup over the software version. The stereo matching algorithms achieve between 3.5X and 67.9X speedup over software (but still less than manual RTL design) with a fivefold reduction in design effort versus manual RTL design.

Definition of an ESM Benchmark for Evaluating Parallel Architectures

Different approaches exist for evaluating the computational performance of a parallel system. These approaches are based on the appliance of some benchmarking tools for evaluating either the whole system or some of its sub-components (i.e. I/O system, memory bandwidth, node interconnection, etc). Different kind of benchmarks can be considered: real program benchmarks are based on real applications; kernel benchmarks include some key codes normally abstracted from actual programs (i.e. linear algebra operations); component benchmarks are focused on the evaluation of computer’s basic components; synthetic benchmarks are built taking statistics of all types of operation from many application programs and writing a program based on a proportional invocation of such operations.This report describes the development of an ESM (Earth System Model) benchmark, based on real applications, for evaluating the performance of a parallel system and its suitability for running climate models. The development of the ESM benchmark started from the composition of an evaluation suite that includes some of the most significant ESM models adopted in the climate community. The selection of the ESM models has been made within the ENES community involving all the main climate centers in Europe. Finally, we have defined a metric as index for measuring the system’s performance. The benchmark will be used for both comparing different parallel architectures and highlighting the hotspots of the target one. The benchmark’s results can provide useful hints for tuning and better configuring the analyzed system.