Memory Access Time Research Articles

A High Scalability Memory NoC with Shared-Inside Hierarchical-Groupings for Triplet-Based Many-Core Architecture

Innovative processor architecture designs are shifting towards Many-Core Architectures (MCAs) to meet the future demands of high-performance computing as the limits of Moore’s Law have almost been reached. Many-core processors utilize shared memory hierarchies to achieve high-speed memory systems, improving memory access efficiency. However, as the number of cores multiplies, the scalability of this system is significantly constrained by the increased proportion of long-distance and Non-Uniform Memory Access (NUMA). Improving the scalability of MCAs is crucial for achieving large/super-scale general-purpose many-core processors. This work proposes a high scalability memory Network-on-Chip (NoC) for Triplet-Based Many-Core Architecture (TriBA), named TriBA-mNoC. TriBA-mNoC maintains a consistent core-to-core spacing as the network scale increases, effectively preventing increased long-distance memory access latency. Moreover, it leverages an inherent advantage of shared-inside hierarchical-groupings, alleviating common NUMA issues in the NoC design. Evaluations of static network characteristics show that TriBA-mNoC outperforms most classical NoCs in network diameter, average distance, and cost. TriBA-mNoC can be integrated with TriBA in the same silicon die with a tile-like floorplan, forming a novel NoC called TriBA-NoC, which can combine the strengths of both networks to maximize the architecture performance. We evaluated the memory access performance and scalability of TriBA-NoC using the mathematical evaluation models and actual simulations with real traffic (PARSEC 3.0 and SPLASH-2) at different network scales. The mathematical evaluation results indicate that TriBA-NoC achieves an aggregate speedup of approximately 3x compared with 2D-Mesh for a similar number of cores. Furthermore, TriBA-NoC’s single-core speedup efficiency remains stable as the number of cores increases under the same cache hit ratio, while 2D-Mesh experiences a rapid decline, highlighting TriBA-NoC’s exceptional scalability. Finally, the actual traffic simulation results show that TriBA-NoC achieves an average memory access latency and time reduction of 25.90% − 40.50% and 5.61% − 31.69% respectively, compared with 2D-Mesh.

Exploring memory synchronization and performance considerations for FPGA platform using the high-abstracted OpenCL framework: Benchmarks development and analysis.

A key benefit of the Open Computing Language (OpenCL) software framework is its capability to operate across diverse architectures. Field programmable gate arrays (FPGAs) are a high-speed computing architecture used for computation acceleration. This study investigates the impact of memory access time on overall performance in general FPGA computing environments through the creation of eight benchmarks within the OpenCL framework. The developed benchmarks capture a range of memory access behaviors, and they play a crucial role in assessing the performance of spinning and sleeping on FPGA-based architectures. The results obtained guide the formulation of new implementations and contribute to defining an abstraction of FPGAs. This abstraction is then utilized to create tailored implementations of primitives that are well-suited for this platform. While other research endeavors concentrate on creating benchmarks with the Compute Unified Device Architecture (CUDA) to scrutinize the memory systems across diverse GPU architectures and propose recommendations for future generations of GPU computation platforms, this study delves into the memory system analysis for the broader FPGA computing platform. It achieves this by employing the highly abstracted OpenCL framework, exploring various data workload characteristics, and experimentally delineating the appropriate implementation of primitives that can seamlessly integrate into a design tailored for the FPGA computing platform. Additionally, the results underscore the efficacy of employing a task-parallel model to mitigate the need for high-cost synchronization mechanisms in designs constructed on general FPGA computing platforms.

Enhancing Lifetime and Performance of MLC NVM Caches Using Embedded Trace Buffers

Large volumes of on-chip and off-chip memory are required by contemporary applications. Emerging non-volatile memory technologies including STT-RAM, PCM, and ReRAM are becoming popular for on-chip and off-chip memories as a result of their desirable properties. Compared to traditional memory technologies such as SRAM and DRAM, they have minimal leakage current and high packing density. Non Volatile Memories (NVM), however, have a low write endurance, a high write latency, and high write energy. Non-volatile Single Level Cell (SLC) memories can store a single bit of data in each memory cell, whereas Multi Level Cells (MLC) can store two or more bits in each memory cell. Although MLC NVMs have substantially higher packing density than SLCs, their lifetime and access speed are key concerns. For a given cache size, MLC caches consume 1.84× less space and 2.62× less leakage power than SLC caches. We propose Trace buffer Assisted Non-volatile Memory Cache (TANC), an approach that increases the lifespan and performance of MLC-based last-level caches using the underutilized Embedded Trace Buffers (ETB). TANC improves the lifetime of MLC LLCs up to 4.36× and decreases average memory access time by 4% compared to SLC NVM LLCs and by 6.41× and 11%, respectively, compared to baseline MLC LLCs.

A novel SpaSA based hyper-parameter optimized FCEDN with adaptive CNN classification for skin cancer detection

Skin cancer is the most prevalent kind of cancer in people. It is estimated that more than 1 million people get skin cancer every year in the world. The effectiveness of the disease’s therapy is significantly impacted by early identification of this illness. Preprocessing is the initial detecting stage in enhancing the quality of skin images by removing undesired background noise and objects. This study aims is to compile preprocessing techniques for skin cancer imaging that are currently accessible. Researchers looking into automated skin cancer diagnosis might use this article as an excellent place to start. The fully convolutional encoder–decoder network and Sparrow search algorithm (FCEDN-SpaSA) are proposed in this study for the segmentation of dermoscopic images. The individual wolf method and the ensemble ghosting technique are integrated to generate a neighbour-based search strategy in SpaSA for stressing the correct balance between navigation and exploitation. The classification procedure is accomplished by using an adaptive CNN technique to discriminate between normal skin and malignant skin lesions suggestive of disease. Our method provides classification accuracies comparable to commonly used incremental learning techniques while using less energy, storage space, memory access, and training time (only network updates with new training samples, no network sharing). In a simulation, the segmentation performance of the proposed technique on the ISBI 2017, ISIC 2018, and PH2 datasets reached accuracies of 95.28%, 95.89%, 92.70%, and 98.78%, respectively, on the same dataset and assessed the classification performance. It is accurate 91.67% of the time. The efficiency of the suggested strategy is demonstrated through comparisons with cutting-edge methodologies.

Designing a Deep Neural Network engine for LLC block reuse prediction to mitigate Soft Error in Multicore

Last level cache (LLC), a major contender of chip area, exhibits the highest sensitivity to soft error. Block reuse prediction is used to exhibit selective protection of LLC blocks. However, traditional reuse predictors fail to accurately understand the cache access pattern. This work explores machine learning for block reuse prediction that realizes the features that affect reuse likelihood and therefore, better predicts the reusable blocks. To off-load the reuse calculation from the processing cores, a dedicated reuse predictor engine is designed. This engine implements a deep neural network with unsupervised learning and calculates reuse probabilities offline without affecting the execution path. Priority blocks with high reuse likelihood are protected using replications in shared LLC. Low priority blocks are invalidated to reduce their vulnerable time. A replication aware replacement policy is also developed to protect the replicated/reserved blocks from eviction. The scheme is evaluated in Multi2Sim 5.0 simulation framework with SPEC-CPU benchmarks considering bulk-CMOS, FDSOI and FinFET technologies. The results indicate 41.29% and 43.14% reductions in miss rate, 92.29% and 89.56% reductions in vulnerability, 5.73% and 6.31% increase in write-back rate, 17.55% and 16.27% reductions in average memory access time, 2.17% and 1.54% increase in Instructions-per-cycle (IPC) and 1.31% and 0.77% reductions in normalized-execution-cycle over the baseline (with SECDED protection) for integer and floating point benchmarks respectively. Marginal increase in power consumption by 1.33% and dynamic energy by 0.43% are observed at 6.32% area overhead.

FPGAs memory synchronization and performance evaluation using the open computing language framework

<p>One advantage of the open computing language (OpenCL) software framework is its ability to run on different architectures. Field programmable gate arrays (FPGAs) are a high-speed computing architecture used for computation acceleration. This work develops a set of eight benchmarks (memory synchronization functions, explained in this study) using an OpenCL framework to study the effect of memory access time on overall performance when targeting the general FPGA computing platform. The results indicate the best synchronization mechanism to be adopted to synthesize the proposed design on the FPGA computation architecture. The proposed research results also demonstrate the effectiveness of using a task-parallel model approach to avoid using high-cost synchronization mechanisms within proposed designs that are constructed on the general FPGA computation platform.</p>

Fabrication of Sb2S3/Sb2Se3 heterostructure for potential resistive switching applications

The exponential growth of large data and the widespread adoption of the Internet of Things (IoT) have created significant challenges for traditional Von Neumann computers. These challenges include complex hardware, high energy consumption, and slow memory access time. Researchers are investigating novel materials and device architectures to address these issues by reducing energy consumption, improving performance, and enabling compact designs. A new study has successfully engineered a heterostructure that integrates Sb2Se3 and Sb2S3, resulting in improved electrical properties. This has generated significant interest in its potential applications in resistive switching. In this study, we have demonstrated the fabrication of a device based on Sb2S3/Sb2Se3 heterostructure that exhibits resistive switching behavior. The device has different resistance states that can be switched between high and low resistance levels when exposed to an external bias (−1 V to 0 V to 1 V). It also has good non-volatile memory characteristics, including low power consumption, high resistance ratio (∼102), and reliable endurance (∼103). The device enables faster data processing, reduces energy consumption, and streamlines hardware designs, contributing to computing advancements amidst modern challenges. This approach can revolutionize resistive switching devices, leading to more efficient computing solutions for big data processing and IoT technologies.

Design and Development of Effective Multi-Level Cache Memory Model

An algorithm to determine the effectiveness and efficiency of a multi-level cache was developed in this paper. The developed model was used to test the efficiency rate, the relationships and the performance output level of a computer concerning the cache properties. This research paper showed that the level of cache and access time increases with the absolute hit rate but decreases with the relative hit rate. The number of cache levels varies directly with absolute access time and inversely with relative access time. The level one cache with set associativity of one has the highest access time and as the associativity and cache levels increase the memory access time decreases. The increase in the number of set associativity leads to an increase in cache performance and as well increases the performance speed of a computer.

SWsnn: A Novel Simulator for Spiking Neural Networks.

Spiking neural network (SNN) simulators play an important role in neural system modeling and brain function research. They can help scientists reproduce and explore neuronal activities in brain regions, neuroscience, brain-like computing, and other fields and can also be applied to artificial intelligence, machine learning, and other fields. At present, many simulators using central processing unit (CPU) or graphics processing unit (GPU) have been developed. However, due to the randomness of connections between neurons and spiking events in SNN simulation, this causes a lot of memory access time. To alleviate this problem, we developed an SNN simulator SWsnn based on the new Sunway SW26010pro processor. The SW26010pro processor consists of six core groups, each with 16 MB of local data memory (LDM). LDM has the characteristics of high-speed read and write, which is suitable for performing simulation tasks similar to SNNs. Experimental results show that SWsnn runs faster than other mainstream GPU-based simulators when simulating a certain scale of neural network, showing a strong performance advantage. To conduct larger scale simulations, SWsnn designed a simulation computation based on a large shared model of Sunway processor and developed a multiprocessor version of SWsnn based on this mode, achieving larger scale SNN simulations.

A Bandwidth-Efficient High-Performance RTL-Microarchitecture of 2D-Convolution for Deep Neural Networks

The computation complexity and huge memory access bandwidth of the convolutional layers in convolutional neural networks (CNNs) require specialized hardware architectures to accelerate CNN’s computations while keeping hardware costs reasonable for area-constrained embedded applications. This paper presents an RTL (Register Transfer Logic) level microarchitecture of hardware- and bandwidth-efficient high-performance 2D convolution unit for CNN in deep learning. The 2D convolution unit is made up of three main components including a dedicated Loader, a Circle Buffer, and a MAC (Multiplier-Accumulator) unit. The 2D convolution unit has a 2-stage pipeline structure that reduces latency, increases processing throughput, and reduces power consumption. The architecture proposed in the paper eliminates the reloading of both the weights as well as the input image data. The 2D convolution unit is configurable to support 2D convolution operations with different sizes of input image matrix and kernel filter. The architecture can reduce memory access time and power as well as execution time thanks to the efficient reuse of the preloaded input data, while simplifying hardware implementation. The 2D convolution unit has been simulated and implemented on Xilinx's FPGA platform to evaluate its superiority. Experimental results show that our design is 1.54× and 13.6× faster in performance than the design in [7] and [8], respectively, at lower hardware cost without using any FPGA’s dedicated hardware blocks. By reusing preloaded data, our design achieves a bandwidth reduction ratio between 66.4% and 90.5%.

Using Local Cache Coherence for Disaggregated Memory Systems

Disaggregated memory provides many cost savings and resource provisioning benefits for current datacenters, but software systems enabling disaggregated memory access result in high performance penalties. These systems require intrusive code changes to port applications for disaggregated memory or employ slow virtual memory mechanisms to avoid code changes. Such mechanisms result in high overhead page faults to access remote data and high dirty data amplification when tracking changes to cached data at page-granularity. In this paper, we propose a fundamentally new approach for disaggregated memory systems, based on the observation that we can use local cache coherence to track applications' memory accesses transparently, without code changes, at cache-line granularity. This simple idea (1) eliminates page faults from the application critical path when accessing remote data, and (2) decouples the application memory access tracking from the virtual memory page size, enabling cache-line granularity dirty data tracking and eviction. Using this observation, we implemented a new software runtime for disaggregated memory that improves average memory access time and reduces dirty data amplification1.

Compact optical convolution processing unit based on multimode interference

Convolutional neural networks are an important category of deep learning, currently facing the limitations of electrical frequency and memory access time in massive data processing. Optical computing has been demonstrated to enable significant improvements in terms of processing speeds and energy efficiency. However, most present optical computing schemes are hardly scalable since the number of optical elements typically increases quadratically with the computational matrix size. Here, a compact on-chip optical convolutional processing unit is fabricated on a low-loss silicon nitride platform to demonstrate its capability for large-scale integration. Three 2 × 2 correlated real-valued kernels are made of two multimode interference cells and four phase shifters to perform parallel convolution operations. Although the convolution kernels are interrelated, ten-class classification of handwritten digits from the MNIST database is experimentally demonstrated. The linear scalability of the proposed design with respect to computational size translates into a solid potential for large-scale integration.

Performance aware shared memory hierarchy model for multicore processors

Despite the fact that multicore processors have a better instruction execution speed and lower power consumption, they also encounter a set of design challenges. The appearance of multicore and many core architectures has raised the problem of managing shared hierarchical memory systems. The main focus of this paper is to evaluate the behavior of shared hierarchical memory systems by modeling their response time analytically. Since the gap between the memory and processor speed increases rapidly, it gets more crucial to find an analytical model that includes the significant factors that affect the performance of hierarchical memory systems. The proposed model considers the interdependence between different memory layers and differentiates between the memory response time and memory system time. Moreover, the model evaluates the effect of memory hierarchy on the variance of the memory access time. The existence of a large variance can lead to extremely long wait queues which can dramatically affect the performance of multicore processors

A time domain 2D OaA-based convolutional neural networks accelerator

Convolutional neural networks (CNNs) are widely implemented in modern facial recognition systems for image recognition applications. Runtime speed is a critical parameter for real-time systems. Traditional FPGA-based accelerations require either large on-chip memory or high bandwidth and high memory access time that slow down the network. The proposed work uses an algorithm and its subsequent hardware design for a quick CNN computation using an overlap-and-add-based technique in the time domain. In the algorithm, the input images are broken into tiles that can be processed independently without computing overhead in the frequency domain. This also allows for efficient concurrency of the convolution process, resulting in higher throughput and lower power consumption. At the same time, we maintain low on-chip memory requirements necessary for faster and cheaper processor designs. We implemented CNN VGG-16 and AlexNet models with our design on Xilinx Virtex-7 and Zynq boards. The performance analysis of our design provides 48% better throughput than the state-of-the-art AlexNet and uses 68.85% lesser multipliers and other resources than the state-of-the-art VGG-16.

Enabling memory access isolation in real-time cloud systems using Intel’s detection/regulation capabilities

The increasing interest in adopting cloud technologies for Industry 4.0 and mixed-criticality environments is paving the way for compelling new opportunities and challenges. However, cloud deployments use multicore processors that introduce interference between co-executing applications on different cores due to contention in shared resources, such as the memory subsystem. Such interference can cause critical applications to miss their deadlines.To enable the co-execution of critical and non-critical applications on the same multicore processor, we propose an approach that guarantees memory access time isolation for critical cores, while not jeopardizing the memory bandwidth of the non-critical ones. We prove the viability of our approach using Intel’s resource director technology for memory access detection and regulation. Experiments show that queue occupancy is an excellent metric to estimate the number of interfering cores co-accessing the memory. We also assess the indirect memory bandwidth limitation achievable by applying Intel’s Memory Bandwidth Allocation technology.

A Novel 8T XNOR-SRAM: Computing-in-Memory Design for Binary/Ternary Deep Neural Networks

Deep neural networks (DNNs) and Convolutional neural networks (CNNs) have improved accuracy in many Artificial Intelligence (AI) applications. Some of these applications are recognition and detection tasks, such as speech recognition, facial recognition and object detection. On the other hand, CNN computation requires complex arithmetic and a lot of memory access time; thus, designing new hardware that would increase the efficiency and throughput without increasing the hardware cost is much more critical. This area in hardware design is very active and will continue to be in the near future. In this paper, we propose a novel 8T XNOR-SRAM design for Binary/Ternary DNNs (TBNs) directly supporting the XNOR-Network and the TBN DNNs. The proposed SRAM Computing-in-Memory (CIM) can operate in two modes, the first of which is the conventional 6T SRAM, and the second is the XNOR mode. By adding two extra transistors to the conventional 6T structure, our proposed CIM showed an improvement up to 98% for power consumption and 90% for delay compared to the existing state-of-the-art XNOR-CIM.

Bounding Memory Access Times in Multi-Accelerator Architectures on FPGA SoCs

Modern FPGA System-on-Chips (SoCs) embed large FPGA logics capable of hosting multiple hardware accelerators. Typically, hardware accelerators require direct access to the shared DRAM memory for reaching the high performance demanded by modern applications. In commercial FPGA SoCs, this goal is achieved by interconnecting the hardware accelerators on an interconnect based on AMBA AXI, which is the de-facto industrial standard for on-chip communications. The AXI standard provides great flexibility in the definition of the network topology. Nevertheless, such flexibility generates a significant unpredictability when attempting to bound the hardware accelerators’ response time when executing under contention. This work focus on bounding the worst-case memory access time of hardware accelerators deployed on commercial FPGA SoCs. We propose a modeling and analysis technique to bound the response time of the hardware accelerators and evaluate the schedulability of a system applicable to arbitrary AXI-based bus structures deployed on FPGA SoCs. Our results are validated on real execution traces collected on two popular FPGA SoCs belonging to the Xilinx ZYNQ-7000 and Zynq-Ultrascale+ families and by simulated results.

IMPLY-Based High-Speed Conditional Carry and Carry Select Adders for In-Memory Computing

Big data applications involved with neural networks requires frequent data transfer between memory and processing elements and thus, take a significant portion of energy. One possible method is to realize logic operations inside the memory, often known as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in-memory computing</i> . The memristor employed in a crossbar can also conduct logic operations, making it a promising candidate for enabling such designs. In-memory computing can vastly improve speed efficiency by significantly reducing memory access time and energy. Therefore, efficient operations inside the memory are crucial for data-intensive applications. This paper presents two high-speed adders in memristive technology, which are derived using IMPLY-based logic operations and can be implemented on a crossbar architecture for in-memory computing. One is a carry select adder, which uses efficient ripple carry adders and multiplexers. While the other is a conditional carry adder, designed using a modified half adder and multiplexers. An IMPLY-based multiplexer and modified half adder are proposed as part of these adders. A modified half adder calculates the sum of two inputs and two carry outputs for various carry inputs. Compared to conventional adder designs, the speed of these two adders has been significantly enhanced. In particular, the proposed 32-bit Carry Select Adder and Conditional Carry Adder take 123 and 90 computational steps, respectively. The proposed adders have improved the speed by 30 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> and 49 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> , respectively, compared to the best-existing adder in IMPLY. Further, this paper presents computational steps and the number of memristors required for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$n$</tex-math></inline-formula> -bit adders. The proposed circuits were simulated using the VTEAM model, and energy consumed per each adder is calculated from the simulations.

FPGA-Based Design for Accelerating 3D Convolutional Neural Networks

Three-dimensional convolutional neural networks (3D CNNs) have been shown to be excellent for classification in human action recognition, hyperspectral image classification and many other aspects. However, 3D CNNs often contain convolution kernels of different sizes and a large number of operations, and the numerous operations make 3D CNNs consume a lot of time when executing inference on resource-constrained terminals. Aiming at these two points, this paper proposes a 3D CNN accelerator design based on loop optimization and weight reuse. The design adopts loop tiling, loop unrolling and loop fusion to optimize the direct convolution form of the 3D convolution operation; and guided by the optimized form of the 3D convolution operation, a module that can accelerate different 3D convolution operations is designed; and a memory access method based on weight reuse is also designed to reduce the memory access time of features and weights. Experimental results show that the proposed accelerator not only supports convolution kernels of different sizes, but also has a performance density of 1.59, which outperforms most existing accelerators.

NCDE: In-Network Caching for Directory Entries to Expedite Data Access in Tiled-Chip Multiprocessors

The processing of data-intensive applications, followed by an unprecedented amount of data traffic, drives explosive accesses to the memory subsystem. The overloaded memory subsystem experiences increased data access latency. To expedite data access, a network caching technique that leverages network-on-chip (NoC) virtual channels (VCs) as an expanded memory subsystem has emerged. Previous network caching studies focused on utilizing VCs on the NoC’s local input port as a victim cache to reduce local data access latency. In contrast to previous studies, we explore the opportunity of mitigating problems associated with shared data access via in-network caching for directory entries (NCDE), which can utilize every input port’s VCs to hold directory entries. NCDE exploits VCs as the victim and prefetch buffers of the directory entries, each reducing directory eviction-induced invalidations and simplifying the cache-to-cache (C2C) data transfer. The effectiveness of NCDE was evaluated using a gem5 full-system simulator, and the results show that the average memory access time (AMAT) and workload execution time were reduced by 7.69% and 5.82%, respectively. As a cost for accelerating the data access latency, implementing NCDE incurs a negligible router area overhead of 1.56%.