Number Of Memory Accesses Research Articles

Fourier-Based Fast 3-D Ultrasound Imaging Using Row-Column-Addressed 2-D Arrays.

A Fourier-based fast 3-D ultrasound imaging method using row-column-addressed (RCA) 2-D arrays is presented. The row elements in an RCA array are activated sequentially, and all the column elements are used to receive. The obtained dataset is adapted to approximate to that obtained using a fully sampled array after a plane wave at a given incident angle is transmitted. In this way, the fast algorithm in plane-wave Fourier imaging (PWFI) can be applied to the adapted dataset. In addition, synthesizing multiple datasets based on multiple incident angles enables angular compounding, which improves the image quality. The proposed method was validated using computer simulations and physical-phantom experiments. The results show that the spatial resolution and contrast of the proposed method are comparable with those of its PWFI counterpart without requiring a fully sampled (FS) array. Compared with the delay-and-sum (DAS) method using the RCA array, the proposed method provides comparable spatial resolution but lower contrast; however, the computational complexity is significantly reduced from O(N4Nz) to O(WN2Nz log2(N2Nz)) , where N is the number of elements on each side of the RCA array, Nz is the number of voxels in the axial direction in the output image, and W is the number of compounding angles. For example, in the simulated results when the maximum compounding angle M is 5°, at a given point the lateral - 6-dB width provided by the proposed method is 0.241 mm (0.267 mm for DAS), the contrast ratio of a hyperechoic cyst is 8.87 dB (9.10 dB for DAS), the number of real number operations is reduced by a factor of 20.62, and the number of memory accesses is reduced by a factor of 47.21, both compared with DAS. This novel fast algorithm could facilitate the development of compact real-time 3-D imaging systems, especially when the channel count is high and a large field of view (FOV) is required.

QuickFPS: Architecture and Algorithm Co-Design for Farthest Point Sampling in Large-Scale Point Clouds

Point clouds have been employed extensively in machine perception applications. Farthest point sampling (FPS) is a critical kernel for point cloud processing. With the rapid growth of point cloud scale, FPS introduces a large number of memory accesses, which become the bottleneck of the large-scale point cloud processing. In this paper, we present QuickFPS, an architecture and algorithm co-design of farthest point sampling in large-scale point clouds. First, We systemically analyze the characteristics of FPS and put forward a bucket-based farthest point sampling algorithm. The algorithm introduces a two-level tree data structure to organize the large-scale point cloud into multiple buckets. By using two mechanisms named merged computation and implicit computation for the buckets, the external memory accesses and compute cost are significantly reduced. Then, we design an efficient domain-specific accelerator for farthest point sampling in large-scale point clouds. The accelerator takes advantage of different forms of parallelism and further improves the accelerator’s efficiency. Finally, we evaluate QuickFPS with several widely used point cloud datasets, which include small-scale and large-scale point clouds (up to 120,000 points). Overall, QuickFPS achieves performance speedups of 43.4x and 12.2x compared to GTX 1080Ti GPU and state-of-the-art point cloud accelerator PointAcc, respectively.

Multi-directional Sobel operator kernel on GPUs

Sobel is one of the most popular edge detection operators used in image processing. To date, most users utilize the two-directional 3×3 Sobel operator as detectors because of its low computational cost and reasonable performance. Simultaneously, many studies have been conducted on using large multi-directional Sobel operators to satisfy their needs considering the high stability, but at an expense of speed. This paper proposes a fast graphics processing unit (GPU) kernel for the four-directional 5×5 Sobel operator. To improve kernel performance, we implement the kernel based on warp-level primitives, which can significantly reduce the number of memory accesses. In addition, we introduce the prefetching mechanism and operator transformation into the kernel to significantly reduce the computational complexity and data transmission latency. Compared with the OpenCV-GPU library, our kernel shows high performances of 6.7x speedup on a Jetson AGX Xavier GPU and 13x on a GTX 1650Ti GPU.

A Learned Bloom Filter-Assisted Scheme for Packet Classification in Software-Defined Networking

Traditional routing technologies based on a single IP address domain faces the challenge to meet the increasing demand for network services and security. Packet classification is a technique to differentiate multi-domain network traffic in a fine-grained manner using packet header fields. Packet classification requires to operate efficiently to avoid it becoming a bottleneck in the packet routing process. Tuple space search (TSS) used in SDN supports fast rule updates but low-speed packet classification. In this paper, we propose a learned Bloom filter (LBF)-based packet classification algorithm that combines LBF and TSS to promote classification speed by avoiding invalid hash table accesses. Specifically, LBF consists of multiple RNN models and one support Bloom filter (SBF), in which the learned models are trained with the positive and negative sets, and used as a pre-filter to identify the two sets. For the filter outcomes with a negative result from learned models, SBF is constructed to perform the second filtration. To ensure efficiency of RNN and SBF, we carefully select key features to be used in RNN and SBF, which can also maintain efficient search in the final stage of hash checking. Our experimental results show that the proposed algorithm saves more memory space than Tuple space pruning (TSP) given the same false positive rate. The proposed algorithm is competitive in terms of the number of memory accesses, while achieving almost one order of magnitude improvement on pre-processing time over NeuroCuts which is an advanced decision tree classifier.

CARMI

Learned indexes, which use machine learning models to replace traditional index structures, have shown promising results in recent studies. However, existing learned indexes exhibit a performance gap between synthetic and real-world datasets, making them far from practical indexes. In this paper, we identify that ignoring the importance of data partitioning during model training is the main reason for this problem. Thus, we explicitly apply data partitioning to index construction and propose a new efficient and updatable cache-aware RMI framework, called CARMI. Specifically, we introduce entropy as a metric to quantify and characterize the effectiveness of data partitioning of tree nodes in learned indexes and propose a novel cost model, laying a new theoretical foundation for future research. Then, based on our novel cost model, CARMI can automatically determine tree structures and model types under various datasets and workloads by a hybrid construction algorithm without any manual tuning. Furthermore, since memory accesses limit the performance of RMIs, a new cache-aware design is also applied in CARMI, which makes full use of the characteristics of the CPU cache to effectively reduce the number of memory accesses. Our experimental study shows that CARMI performs better than baselines, achieving an average of 2.2X/1.9X speedup compared to B+ Tree/ALEX, while using only about 0.77X memory space of B+ Tree. On the SOSD platform, CARMI outperforms all baselines, with an average speedup of 1.2X over the nearest competitor RMI, which has been carefully tuned for each dataset in advance.

Scalable name identifier lookup for Industrial Internet

Name identifiers are considered more effective than IP addresses in routing and forwarding billions of heterogeneous resource-restrained Industrial Internet objects. Although the design of name-based longest prefix matching lookup has been developed for nearly a decade, it is still attractive to design an efficient name lookup scheme for the Industrial Internet due to hierarchical variable-length names, large-scale forwarding tables, and frequent updates. In this paper, we present a memory-optimized binary search (MOBS) algorithm in the counting Bloom filter (CBF), which decreases memory consumption by reducing the storage of prefix marker entries while maintaining the fast lookup efficiency of binary search. Furthermore, we propose CBF-HT, an efficient and scalable name lookup method that combines counting Bloom filter and hash table to facilitate content forwarding. CBF-HT consists of two stages. Firstly, the longest prefix matching results are obtained through binary search in the counting Bloom filter with MOBS, and then the final lookup results are acquired by linear backtracking in the hash table. CBF-HT can achieve high-speed lookups and fast updates, and significantly reduces memory consumption caused by marker entries through MOBS. We have implemented and evaluated CBF-HT with 10 billion name identifiers. Experimental results show that, compared with the existing binary search on hash table algorithm (BS-HT), CBF-HT increases the forwarding throughput by 45%, eliminates 73% and 90% of memory consumption, and improves the update performance by 48.1%, at the cost of a 7.8% increase in the average number of memory accesses.

A 4.4–75-TOPS/W 14-nm Programmable, Performance- and Precision-Tunable All-Digital Stochastic Computing Neural Network Inference Accelerator

We present the first programmable and precision-tunable Stochastic Computing (SC) neural network (NN) inference accelerator. The use of SC makes it possible to achieve multiply-accumulate (MAC) density of 38.4k MAC/mm2, enabling a level of spatial data reuse unachievable to conventional, fixed-point architectures. This extensive reuse amortizes the cost of SC conversion and reduces the number of memory accesses, which can otherwise consume significant energy and latency. Our accelerator is a stand-alone architecture, with a custom instruction set architecture (ISA), and support for end-to-end model inference with convolutional and fully-connected layers of variable input and filter sizes. Further, it demonstrates extensive accuracy-latency trade-offs by varying the stream length. The 14nm demonstration chip achieves 2.4 TOPS and 75 TOPS/W peak throughput and energy efficiency, outperforming comparable fixed-point accelerators.

ROMANet: Fine-Grained Reuse-Driven Off-Chip Memory Access Management and Data Organization for Deep Neural Network Accelerators

Enabling high energy efficiency is crucial for embedded implementations of deep learning. Several studies have shown that the DRAM-based off-chip memory accesses are one of the most energy-consuming operations in deep neural network (DNN) accelerators and, thereby, limit the designs from achieving efficiency gains at the full potential. DRAM access energy varies depending upon the number of accesses required and the energy consumed per-access. Therefore, searching for a solution toward the minimum DRAM access energy is an important optimization problem. Toward this, we propose the ROMANet methodology that aims at reducing the number of memory accesses, by searching for the appropriate data partitioning and scheduling for each layer of a network using a design space exploration, based on the knowledge of the available on-chip memory and the data reuse factors. Moreover, ROMANet also targets decreasing the number of DRAM row buffer conflicts and misses by exploiting the DRAM multibank burst feature to improve the energy-per-access. Besides providing the energy benefits, our proposed DRAM data mapping also results in an increased effective DRAM throughput, which is useful for latency-constraint scenarios. Our experimental results show that the ROMANet saves DRAM access energy by 12% for the AlexNet, 36% for the VGG-16, 46% for the MobileNet, and 45% for the SqueezeNet while improving the DRAM throughput by 10% on average across different networks compared to the state of the art, i.e., bus-width aware (BWA) technique.

IDRA: An In-Storage Data Reorganization Accelerator for Multidimensional Databases

In the multidimensional databases, various storage methods for the main memory have been proposed for efficient data processing. On the other hand, storage devices such as SSD still leverage the conventional row-oriented storage method to avoid complicated operating system modification. Since there is a difference in storage methods between the main memory and storage device, it is necessary to reorganize the data between them. However, data reorganization in the conventional CPU-based systems causes a long latency due to a large number of memory accesses. In addition, it also causes high dynamic power consumption of a CPU. In this letter, we propose an in-storage data reorganization accelerator for multidimensional databases (IDRA). We place the IDRA in an SSD to reorganize data on the fly while loading it into the main memory without intervention of the CPU. In our evaluation on the off-the-shelf system (not simulation), the IDRA-based system improves the performance by 78.6% and reduces the system-wide energy by 30.3%, on average, compared to the conventional CPU-based system.

Статически-детерминированный метод прогнозирования динамических характеристик параллельных программ

In this paper, we consider the problem of prediction of parallel program dynamic characteristics, like execution time, count of floating-point operations, energy consumption, count of memory accesses and others. Prediction of dynamic characteristics allows solving many problems, related to design of new architectures, selection of the most suitable configurations of multiprocessor systems for solving specific problems, porting applications to new systems, task flow planning and more. The task of predicting characteristics is being actively investigated. Increasing complexity of the architectures of modern high-performance systems requires the development of new methods for solving the prediction problem. The article provides an overview of the existing approaches and software for predicting dynamic characteristics and proposes an approach based on a static analysis of the source code of a parallel application. Based on the text of the parallel program, the formal description of the target computing platform and the launch parameters, a method is implemented that allows predicting the operating time, the number of floating-point operations, number of memory accesses, and other characteristics of the parallel application. The applicability of the proposed approach is demonstrated by solving the test 3-dimensional numerical simulation problem on a multiprocessor cluster based on IBM Power8 processors.

An efficient pipeline processing scheme for programming Protocol-independent Packet Processors

OpenFlow is unable to provide customized flow tables, resulting in memory explosions and high switch retirement rates. This is the bottleneck for the development of SDN. Recently, P4 (Programming Protocol-independent Packet Processors) attracts much attentions from both academia and industry. It provides customized networking services by offering flow-level control. P4 can “produce” various forwarding tables according to packets. P4 increases the speed of custom ASICs. However, with the prevalence of P4, the multiple forwarding tables could explode when used in large scale networks. The explosion problem can slow down the lookup speed, which causes congestions and packet losses. In addition, the pipelined structure of forwarding tables brings additional processing delay. In this study, we will improve the lookup performance by optimizing the forwarding tables of P4. Intuitively, we will install the rules according to their popularity, i.e., the popular rules will appear earlier than others. Thus, the packets can hit the matched rule sooner. In this paper, we formalize the optimization problem, and prove that the problem is NP-hard. To solve the problem, we propose a heuristic algorithm called EPSP (Efficient Pipeline Processing Scheme for P4), which can largely reduce the lookup time while keeping the forwarding actions the same. Because running the optimization algorithm frequently brings additional processing burdens, wedesign an incremental update algorithm to alleviate this problem. To evaluate the proposed algorithms, we set up the simulation environments based on ns-3. The simulation results show that the algorithm greatly reduces both the lookup time and the number of memory accesses. The incremental algorithm largely reduces the processing burdens while the lookup time remains almost the same with the non-incremental algorithm. We also implemented a prototype using floodlight and mininet. The results show that our algorithm brings acceptable burder, and performs better than traditional algorithm.

Fast Number Theoretic Transform for Ring-LWE on 8-bit AVR Embedded Processor.

In this paper, we optimized Number Theoretic Transform (NTT) and random sampling operations on low-end 8-bit AVR microcontrollers. We focused on the optimized modular multiplication with secure countermeasure (i.e., constant timing), which ensures high performance and prevents timing attack and simple power analysis. In particular, we presented combined Look-Up Table (LUT)-based fast reduction techniques in a regular fashion. This novel approach only requires two times of LUT access to perform the whole modular reduction routine. The implementation is carefully written in assembly language, which reduces the number of memory access and function call routines. With LUT-based optimization techniques, proposed NTT implementations outperform the previous best results by 9.0% and 14.6% for 128-bit security level and 256-bit security level, respectively. Furthermore, we adopted the most optimized AES software implementation to improve the performance of pseudo random number generation for random sampling operation. The encryption of AES-256 counter (CTR) mode used for random number generator requires only 3184 clock cycles for 128-bit data input, which is 9.5% faster than previous state-of-art results. Finally, proposed methods are applied to the whole process of Ring-LWE key scheduling and encryption operations, which require only 524,211 and 659,603 clock cycles for 128-bit security level, respectively. For the key generation of 256-bit security level, 1,325,171 and 1,775,475 clock cycles are required for H/W and S/W AES-based implementations, respectively. For the encryption of 256-bit security level, 1,430,601 and 2,042,474 clock cycles are required for H/W and S/W AES-based implementations, respectively.

Optimized Block-Based Algorithms to Label Connected Components on GPUs

Connected Components Labeling (CCL) is a crucial step of several image processing and computer vision pipelines. Many efficient sequential strategies exist, among which one of the most effective is the use of a block-based mask to drastically cut the number of memory accesses. In the last decade, aided by the fast development of Graphics Processing Units (GPUs), a lot of data parallel CCL algorithms have been proposed along with sequential ones. Applications that entirely run in GPU can benefit from parallel implementations of CCL that allow to avoid expensive memory transfers between host and device. In this paper, two new eight-connectivity CCL algorithms are proposed, namely Block-based Union Find (BUF) and Block-based Komura Equivalence (BKE). These algorithms optimize existing GPU solutions introducing a block-based approach. Extensions for three-dimensional datasets are also discussed. In order to produce a fair comparison with previously proposed alternatives, YACCLAB, a public CCL benchmarking framework, has been extended and made suitable for evaluating also GPU algorithms. Moreover, three-dimensional datasets have been added to its collection. Experimental results on real cases and synthetically generated datasets demonstrate the superiority of the new proposals with respect to state-of-the-art, both on 2D and 3D scenarios.

Matched Filtering Accelerated by Tensor Cores on Volta GPUs With Improved Accuracy Using Half-Precision Variables

Matched Filtering can be applied to various fields owing to its ability to compute a correlation coefficient of two vectors and detect many template events. With an improvement in observation techniques, massive observation data and templates have been accumulated, in which a reduction of computation cost of Matched Filtering has become an important issue. This computation is mainly matrix-matrix product and Tensor Core on NVIDIA Volta GPU is expected to compute it rapidly. However, actual performance of Tensor Core is usually limited by the bandwidth of shared memory or global memory. In addition, only lower-precision data types are supported in the current API for Tensor Core. Therefore, we have to prevent a decline in accuracy in the computation. In this letter, we designed a Matched Filtering algorithm to solve these problems mentioned above and utilized high arithmetic capacity on Tensor Core. Specifically, we reduced the number of memory access to global memory and shared memory by using low-level description. In addition, we introduced local normalization to reduce the numerical error. We applied our developed kernel to template matching of seismic observation data and compared the performance and the accuracy with cuBLAS, a common library in GPU computation. When we compared the performance with the function in cuBLAS that offered almost the same accuracy as our kernel, we reduced the elapsed time by a factor of 4.74.

A labeling algorithm based on a forest of decision trees

Connected component labeling (CCL) is one of the most fundamental operations in image processing. CCL is a procedure for assigning a unique label to each connected component. It is a mandatory step between low-level and high-level image processing. In this work, a general method is given to improve the neighbourhood exploration in a two-scan labeling. The neighbourhood values are considered as commands of a decision table. This decision table can be represented as a decision tree. A block-based approach is proposed so that values of several pixels are given by one decision tree. This block-based approach can be extended to multiple connectivities, 2D and 3D. In a raster scan, already seen pixels can be exploited to generate smaller decision trees. New decision trees are automatically generated from every possible command. This process creates a decision forest that minimises the number of memory accesses. Experimental results show that this method is faster than the state-of-the-art labelling algorithms and require fewer memory accesses. The whole process can be generalised to any given connectivity.

A First Step Toward Using Quantum Computing for Low-level WCETs Estimations

Low-Level analysis of Worst Case Execution Time (WCET) is an important field for real-time system validation. It stands between computer architecture and mathematics, as it relies strongly on variants of abstract interpretation. One of the features that causes the largest uncertainty regarding WCET evaluation for low-level analysis of sequential execution on a single processor is taking Cache Memory-related Delays (CMRD) and Cache-related Preemption Delays (CRPD) correctly into account. Research work from the 1990s provides a good basic framework for this problem as long as a task runs without preemption. But when preemption of tasks is allowed, although several formalisms exist, their predictive power is lower and the usual approach relies on analyses of NP-hard problems. In this article, we want to show some potential advantages of using a formalism inspired by Quantum Computing (QC) to evaluate CMRDs with preemptions while avoiding the NP-hard problem underneath. The experimental results, with a classic (non-quantum) numerical approach, on a selection of Malardalen benchmark programs display very good accuracy, while the complexity of the evaluation is a low-order polynomial of the number of memory accesses. While it is not yet a fully parallel quantum algorithm, we provide a first roadmap on how to reach such an objective.

Comparison of the performance of skip lists and splay trees in classification of internet packets.

Due to the increasing number of Internet users and the volume of information exchanged by software applications, Internet packet traffic has increased significantly, which has highlighted the need to accelerate the processing required in network systems. Packet classification is one of the solutions implemented in network systems. The most important issue is to use an approach that can classify packets at the speed of the network and show optimum performance in terms of memory usage. In this study, we evaluated the performance in packet classification of two of the most important data structures used in decision trees, i.e. the skip list and splay tree. Our criteria for performance were the time of packet classification, the number of memory accesses, and memory usage of each event. These criteria were tested by the ACL and IPC rules with different numbers of rules as well as by different packet numbers. The results of the evaluation showed that the performance of skip lists is higher than that of splay trees. By increasing the number of classifying rules, both the difference in the speed of packet classification and the superiority of the performance of the skip list over that of the splay tree become more significant. The skip list also maintains its superiority over the splay tree in lower memory usage. The results of the experiments confirm the scalability of this method in comparison to the splay tree method.

Comparative Study and Proof of Single-Pass Connected Components Algorithms

Union-find algorithms form the basis of managing sets of equivalent labels within most connected components labelling algorithms. The new class of single-pass connected components analysis (CCA) algorithms (where a feature vector of each component is extracted during processing) are analysed and compared within this context. Such algorithms have been developed for stream processing, using customised hardware architectures. Many of these use an improved union-find algorithm requiring only a single lookup for its find operation. This paper analyses this optimisation and formally proves that the resulting single lookup connected components algorithm associates each pixel with its correct component when extracting the components’ feature vectors. Analysis of the algorithm led to a new double lookup algorithm that reduces the total number of memory accesses and is a step towards unifying pixel-based methods and run-based methods. State-of-the-art CCA algorithms are compared in terms of the number of memory accesses, which is a limiting factor for hardware-based acceleration, with key implementation trade-offs identified between hardware resources and worst-case processing speed.

IMT: A Memory-Efficient and Fast Updatable IP Lookup Architecture Using an Indexed Multibit Trie

IP address lookup is a function to determine nexthop for a given destination IP address. It takes an important role in modern routers because of its computation time and increasing Internet traffic. TCAM-based IP lookup approaches can exploit the capability of parallel searching but have a limitation of its size due to latency, power consumption, updatability, and cost. On the other hand, multibit trie-based approaches use SRAM which has relatively low power consumption and cost. They reduce the number of memory accesses required for each lookup, but it still needs several accesses. Moreover, the memory efficiency and updatability are proportional to the number of memory accesses. In this paper, we propose a novel architecture using an Indexed Multibit Trie (IMT) which is based on combined TCAM and SRAM. In the proposed architecture, each lookup takes at most two memory accesses. We present how the IMT is constructed so as to be memory-efficient and fast updatable. Experiment results with real-world forwarding tables show that our scheme achieves good memory efficiency as well as fast updatability.

A Robust Digital RRAM-Based Convolutional Block for Low-Power Image Processing and Learning Applications

Currently, there is a growing attention toward developing efficient hardware convolutional blocks for several applications such as computer vision or image processing. Recent works have shown that using binary values in convolutional blocks can considerably reduce the overall power consumption while achieving a high degree of accuracy. In parallel, some works employed resistive random-access memory (RRAM) as an in-memory accelerator to directly store the convolution kernels and perform analog dot product operations in the array, reducing the overall power consumption by limiting the number of memory accesses. However, such architecture is hampered by the limited resistance precision and large intrinsic variability of RRAMs. In this paper, we present a purely digital robust RRAM-based convolutional block using single-ended XNOR sensing capable of performing dot product operations in a single cycle. By carefully considering physical design and RRAM limitations at the 28-nm technology node, we show that at the circuit level, our architecture can tolerate a resistance window as low as 1.09, ensuring reliable operations even under a high RRAM variability ($\sigma /\mu = 25\%$ for a resistance window between both states around 50). When integrated in ISAAC, a state-of-the-art learning accelerator, our block can reduce the power by $2.7\times $ while guaranteeing robust operations.