Chapter Two - Survey on System I/O Hardware Transactions and Impact on Latency, Throughput, and Other Factors

Since first proposed by Gordon Moore (an Intel founder) in 1965, his law [107] that the number of transistors on microprocessors doubles roughly every one to two years has proven remarkably astute. Its corollary, that central processing unit (CPU) performance would also double every two years or so has also remained prescient. Figure 1.1 shows Intel microprocessor data on the number of transistors beginning with the 4004 in 1972. Figure 1.2 indicates that when one includes multi-processor machines and algorithmic development, computer performance is actually better than Moore’s 2-year performance doubling time estimate. Alas, however, in recent years there has developed a disagreeable mismatch between CPU and memory performance: CPUs now outperform memory systems by orders of magnitude according to some reckoning [71]. This is not completely accurate, of course: it is mostly a matter of cost. In the 1980s and 1990s, Cray Research Y-MP series machines had well balanced CPU to memory performance. Likewise, NEC (Nippon Electric Corp.), using CMOS (see glossary, Appendix F) and direct memory access, has well balanced CPU/Memory performance. ECL (see glossary, Appendix F) and CMOS static random access memory (SRAM) systems were and remain expensive and like their CPU counterparts have to be carefully kept cool. Worse, because they have to be cooled, close packing is difficult and such systems tend to have small storage per volume. Almost any personal computer (PC) these days has a much larger memory than supercomputer memory systems of the 1980s or early 1990s. In consequence, nearly all memory systems these days are hierarchical, frequently with multiple levels of cache. Figure 1.3 shows the diverging trends between CPUs and memory performance. Dynamic random access memory (DRAM) in some variety has become standard for bulk memory. There are many projects and ideas about how to close this performance gap, for example, the IRAM [78] and RDRAM projects [85]. We are confident that this disparity between CPU and memory access performance will eventually be tightened, but in the meantime, we must deal with the world as it is.

Memory streaming acceleration for embedded systems with CPU-accelerator cooperative data processing

The 3B20D Processor has been developed to meet the need for very reliable, real-time control of a variety of Bell System applications. To achieve its high-reliability goals, most of the major subsystems within the processor are duplicated, including the Central Processing Unit (CPU). The CPU uses a 32-bit architecture throughout, including the memory and input/output buses. Extensive self-checking logic is employed. The 3B20D CPU is microprogrammed to select dynamically up to four instruction sets. The microstore uses a 64-bit word with up to 16K words of high-speed bipolar PROM or RAM available. This rich emulation capability makes the 3B20D Processor ideal for emulating existing instruction sets and porting existing software. Peripheral units are connected to the CPU via the Direct Memory Access unit (DMA). The DMA controllers provide direct memory transfers between the main store and peripheral devices, reducing the load placed on the central control to process input/output requests.

For more than 50 years, the capabilities of Von Neumann-style information processing systems — in which a "memory" delivers operations and then operands to a dedicated "central processing unit" — have improved dramatically. While it may seem that this remarkable history was driven by ever-increasing density (Moore's Law), the actual driver was Dennard's Law: the amazing realization that each generation of scaled-down transistors could actually perform better, in every way, than the previous generation. Unfortunately, Dennard's Law terminated some years ago, and as a result, Moore's Law is now slowing considerably. In a search for ways to continue to improve computing systems, the attention of the IT industry has turned to Non-Von Neumann algorithms, and in particular, to computing architectures motivated by the human brain. At the same time, memory technology has been going through a period of rapid change, as new nonvolatile memories (NVM) — such as Phase Change Memory (PCM), Resistance RAM (RRAM), and Spin-Torque-Transfer Magnetic RAM (STT-MRAM) — emerge that complement and augment the traditional triad of SRAM, DRAM, and Flash. Such memories could enable Storage-Class Memory (SCM) — an emerging memory category that seeks to combine the high performance and robustness of solid-state memory with the long-term retention and low cost of conventional hard-disk magnetic storage. Such large arrays of NVM can also be used in non-Von Neumann neuromorphic computational schemes, with device conductance serving as the plastic (modifiable) “weight” of each “native” synaptic device. This is an attractive application for these devices, because while many synaptic weights are required, requirements on yield and variability can be more relaxed. However, work in this field has remained highly qualitative in nature, and slow to scale in size. In this talk, we will discuss our recent work on scaling NVM-based neural networks in size while quantitatively assessing engineering tradeoffs [1]. We demonstrate a 3-layer neural network of 164,885 synapses, each implemented with two PCM devices, trained on a subset (5000 examples) of the MNIST database of handwritten digits. A weight-update rule compatible for NVM+selector crossbar arrays is presented, as well as a “G-diamond” concept that illustrates problems created by nonlinearity and asymmetry in NVM conductance response. A neural network (NN) simulator matched to the experimental demonstrator allows extensive tolerancing. NVM-based Neural Networks are found to be highly resilient to random effects (NVM variability, yield, and stochasticity), but highly sensitive to “gradient” effects that act to steer all synaptic weights. Low “learning-rate” is shown to be advantageous for both high accuracy and low training energy. Both the SCM and the neuromorphic applications become more attractive as the NVM arrays become large. However, in order to enable large crossbar arrays, a highly nonlinear access device (AD) is also required (in addition to the NVM devices themselves). We will also review our past work on high-performance ADs based on Cu-containing Mixed-Ionic-Electronic Conduction (MIEC) materials [2]. These devices require only the low processing temperatures of the Back-End-Of-the-Line (BEOL), making them highly suitable for implementing multi-layer cross-bar arrays. MIEC-based ADs offer large ON/OFF ratios (>1e7), a significant voltage margin Vm (over which current <10 nA), and ultra-low leakage (< 10 pA), while also offering the high current densities needed for phase-change memory and the fully bipolar operation needed for high-performance RRAM. [1] G. W. Burr, R. Shelby, C. di Nolfo, J. Jang, R. Shenoy, P. Narayanan, K. Virwani, E. Giacometti, B. Kurdi and H. Hwang, "Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element," IEDM Technical Digest, page 29.5, (2014). [2] R. S. Shenoy, G. W. Burr, K. Virwani, B. Jackson, A. Padilla, P. Narayanan, C. Rettner, R. M. Shelby, D. S. Bethune, K. Raman, M. BrightSky, E. Joseph, P. M. Rice, T. Topuria, A. J. Kellock, B. Kurdi, and K. Gopalakrishnan, "MIEC (Mixed-Ionic-Electronic-Conduction)-based access devices for non-volatile crossbar memory arrays," Semiconductor Science and Technology, 29(10), 104005, (2014).

Computer memory is any physical device capable to store data temporarily or permanently. It covers from fastest, yet most expensive, static random-access-memory (SRAM) to cheapest, but slowest, hard drive disk, while in between there are many other memory technologies that make trade–offs among cost, speed, and power consumption. However, the large volume of memory will experience significant leakage power, especially at advanced CMOS technology nodes, for holding data in volatile memory for fast accesses. The spin–transfer torque magnetic random-access memory (STT–RAM), a novel non-volatile memory– (NVM) based on spintronic devices, has shown the great benefits on power–wall issue compared to traditional volatile memories. In addition, in traditional Von–Neumann architecture, the memory is separated from the central processing unit. As a result, the I/O congestion between memory and processing unit leads to the memory-wall issue, and ultimate solution requires a breakthrough on memory technology. The novel in–memory architecture is the solution of memory–wall issues that both logic operation and data storage are located inside memory. This chapter reviews the existing semiconductor memory technologies and traditional memory architecture first, and then introduces the spintronic memory technologies as well as the in-memory architecture.

Data acquisition and processing systems are essential parts of today’s world. The purpose of this systems is to take measurements from real world characteristics in such a way that they can be processed to obtain information. The scale of these systems varies enormously from the smallest examples found in portable and wearable technology (such as smartphones and tablets) to big systems based on industrial communication buses that acquire and process several gigabytes of information per second. Special cases of these systems are those used in big physics experiments. The nature and purpose of these experiments change considerably from one to another, but all of them need to extract as much information as possible and with the utmost precision from the physic phenomena under study. This leads to a large amount of data to be processed and archived. Moreover, some of the acquired or processed data may be needed for the control of the experiments, so it is necessary to calculate them in real-time and with the lowest possible latency. Additionally, the reliability and availability of these systems must be guaranteed for the correct operation of the experiments, especially for the control and safety systems of the experiment. Another common characteristic in big physics experiments is their organization as a Supervisory Control And Data Acquisition (SCADA) system. The size and complexity of these experiments make necessary the use of these systems that allow dividing the functionality among the different systems while they are communicated for the coordination and synchronization of the actions taken by each one of them. This doctoral dissertation proposes a generic model as reference for the real-time data acquisition and processing solutions in big physics experiments, especially for those used in magnetic confinement fusion experiments. The proposed model tries to address the common requirements for this kind of systems. The model is designed using the technologies present in current experiments, as reconfigurable logic devices based on Field Programmable Gate Arrays (FPGAs), Graphics Processing Units (GPUs), Central Processing Units (CPUs), and architectures based on the industrial version of Peripheral Component Interconnect Express (PCIe) known as PCI eXtensions for Instrumentation Express (PXIe). The SCADA system used for the integration of the data acquisition systems is the Experimental Physics and Industrial Control System (EPICS), chosen for being one most used distributed control system in big physics experiments. xii However, there are some problems derived from the use of these technologies. The integration of the systems in EPICS requires the development of an intermediate application that interfaces between the hardware devices and EPICS. In the case of systems involving FPGA-based devices, the development of these applications becomes more challenging. This is because FPGAs devices can be configured in many different ways, and the development of one application for each possible configuration will increase the maintainability cost of the experiment. It is also necessary to consider that the application must integrate together devices very different from each other, as FPGAs, GPUs, and CPUs, and this must be done in such a way that they can work in collaboration for the data process. However, also important is that the systems developed are easily customizable, as they will be used for playing very different roles in the experiment, and it will be necessary to add custom functionalities, as specific data process or control actions taken by the acquisition device. In response to the problems described, this thesis is focused on the development of a methodology for the implementation of high throughput data acquisition and processing systems and its integration in EPICS. The methodology takes the proposed model system as a reference, and it tries to address the issues that the architecture presents pursuing to ease the implementation of these systems. The proposed methodology is based on the use of a set of software tools specifically designed for this purpose. The developed tools are publicly available and at the disposal of the scientific community under an open software license. Given the above, the main topics of this thesis are the following: • Study of data acquisition and processing systems requirements for big physics experiments as an example of an application where a high throughput is required. • Analysis of the hardware and software architecture of a data acquisition and processing system based on the technologies mentioned previously. • Definition and specification of the methodology and the related development cycle for the implementation of those systems and their integration in EPICS. • Description of the products developed in support of the methodology and for easing the use of the different technologies involved in the system. • Evaluation and validation of the proposed methodology, including actual use cases where the methodology is being applied.

A method has been developed for evaluating central processing unit (CPU) coverage by automatically injecting faults into actual CPU hardware while it is executing relevant test software. A special hardware test fixture is used which contains the CPU under evaluation, program and data memory, and a terminal interface. The test fixture is connected to a microcomputer controller and an in-circuit read only memory (ROM) emulator. The faults injected are stuck-at-1, stuck-at-0, and open on the appropriate integrated circuit pins, plus the altered state of every microprogram memory bit. The effect of each fault is determined by observing the status of the monitor. Data reduction is performed after each run on a separate host computer, and summary results are tabulated. N digital avionics systems the central processing unit (CPU) is obviously the dominant functional element, upon which many other elements are dependent in order to perform correctly related interface and control tasks. Normally, there are arrays of system monitors, in both hardware and software, which have an inherently overlapping capability to detect CPU malfunctions. The combined effectiveness of these monitors determines the total CPU fault coverage. However, since many of these monitors are themselves dependent upon CPU operation, there is a critical topdown relationship between the monitors which must be considered. In particular, a software monitor is of little subsequent value in the presence of a CPU fault unless one of the two following conditions is present. 1) It is actually executed, and it produces the planned fault detection results, and the fault detection is properly communicated to and acted upon by the ultimate hardware control elements in the system. 2) It causes fault detection elsewhere when it is not executed, and the fault detection is properly communicated to and acted upon by the ultimate hardware control elements in the system. In simplest terms, software monitors depend upon a rational CPU; that is, a CPU which is at least able to follow the normal program flow without making addressing or branching errors. If this is not the case, then how can one guarantee that the software monitor is executed? If it is possible to detect the fault of concern without executing the software monitor, then why is the monitor present? The point is that it is critically necessary to have a CPU monitoring scheme which establishes the minimum CPU fault coverage that is required to support all other monitoring activity. Such a monitoring scheme must straddle the hardware-software boundary so that external communication is absolutely guaranteed. In the simple terms introduced in the preceding, such a monitor must be able to detect all faults resulting in irrational CPU operation. In most modern digital avionic systems, the monitor which does these things is the so-called watchdog monitor (WDM) (or alternately the heartbeat or deadman timer monitor). The WDM usually operates in relation to the timer

A method has been developed for evaluating central processing unit (CPU) coverage by automatically injecting faults into actual CPU hardware while it is executing relevant test software. A special hardware test fixture is used which contains the CPU under evaluation, program and data memory, and a terminal interface. The test fixture is connected to a microcomputer controller and an in-circuit read only memory (ROM) emulator. The faults injected are stuck-at-1, stuck-at-0, and open on the appropriate integrated circuit pins, plus the altered state of every microprogram memory bit. The effect of each fault is determined by observing the status of the monitor. Data reduction is performed after each run on a separate host computer, and summary results are tabulated. N digital avionics systems the central processing unit (CPU) is obviously the dominant functional element, upon which many other elements are dependent in order to perform correctly related interface and control tasks. Normally, there are arrays of system monitors, in both hardware and software, which have an inherently overlapping capability to detect CPU malfunctions. The combined effectiveness of these monitors determines the total CPU fault coverage. However, since many of these monitors are themselves dependent upon CPU operation, there is a critical topdown relationship between the monitors which must be considered. In particular, a software monitor is of little subsequent value in the presence of a CPU fault unless one of the two following conditions is present. 1) It is actually executed, and it produces the planned fault detection results, and the fault detection is properly communicated to and acted upon by the ultimate hardware control elements in the system. 2) It causes fault detection elsewhere when it is not executed, and the fault detection is properly communicated to and acted upon by the ultimate hardware control elements in the system. In simplest terms, software monitors depend upon a rational CPU; that is, a CPU which is at least able to follow the normal program flow without making addressing or branching errors. If this is not the case, then how can one guarantee that the software monitor is executed? If it is possible to detect the fault of concern without executing the software monitor, then why is the monitor present? The point is that it is critically necessary to have a CPU monitoring scheme which establishes the minimum CPU fault coverage that is required to support all other monitoring activity. Such a monitoring scheme must straddle the hardware-software boundary so that external communication is absolutely guaranteed. In the simple terms introduced in the preceding, such a monitor must be able to detect all faults resulting in irrational CPU operation. In most modern digital avionic systems, the monitor which does these things is the so-called watchdog monitor (WDM) (or alternately the heartbeat or deadman timer monitor). The WDM usually operates in relation to the timer

In virtual private network (VPN) tunnel mode, the entire original packet, including the header’s five-tuple information, is encrypted, which prevents traditional scheduling algorithms from evenly distributing packets to central processing unit (CPU) cores based on packet header information. To address the need for data security and encrypted packet scheduling, we propose a novel framework, named REFS (receive encrypted flow steering), for accelerated receive encrypted flow steering. This work creatively adopts a new method that allows encrypted packets to be distributed across CPU cores without decrypting them, overcoming limitations of traditional scheduling approaches. It efficiently distributes encrypted packets across CPU cores, enabling dynamic allocation of CPU resources. A key feature of REFS is its ability to perform this distribution without decrypting the packets, which enhances dynamic load balancing and improves system responsiveness. When integrated into the Linux kernel’s VPN functionality, REFS can potentially increase throughput by up to 50% compared to WireGuard, which is a benchmark for kernel-based VPN performance. Upon integration of REFS into userspace, network performance shows significant improvements: throughput doubles, while latency is reduced by 80%.

This paper describes the architecture, functionality, and design of NX-2700 — a digital television (DTV) and media-processor chip from Philips Semiconductors. NX-2700 is the second generation of an architectural family of programmable multimedia processors that supports all eighteen United States Advanced Television Systems Committee (ATSC) [1] formats and is targeted at the high-end DTV market. NX-2700 is a programmable processor with a very powerful, generalpurpose Very LongInstruction Word (VLIW) Central ProcessingUnit (CPU) core that implements many non-trivial multimedia algorithms, coordinates all on-chip activities, and runs a small real-time operating system. The CPU core, aided by an array of autonomous multimedia coprocessors and input-output units with Direct Memory Access (DMA) capability, facilitates concurrent processingof audio, video, graphics, and communication-data.KeywordsCentral Processing UnitCentral Processing Unit CoreDirect Memory AccessInverse Discrete Cosine TransformVery Long Instruction WordThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The Graphical Processing Unit is a specialised piece of hardware that contains many low powered cores, available on both the consumer and industrial market. The original Graphical Processing Units were designed for processing high quality graphical images, for presentation to the screen, and were therefore marketed to the computer games market segment. More recently, frameworks such as CUDA and OpenCL allowed the specialised highly parallel architecture of the Graphical Processing Unit to be used for not just graphical operations, but for general computation. This is known as General Purpose Programming on Graphical Processing Units, and it has attracted interest from the scientific community, looking for ways to exploit this highly parallel environment, which was cheaper and more accessible than the traditional High Performance Computing platforms, such as the supercomputer. This interest in developing algorithms that exploit the parallel architecture of the Graphical Processing Unit has highlighted the need for scientists to be able to analyse proposed algorithms, just as happens for proposed sequential algorithms. In this thesis, we study the abstract modelling of computation on the Graphical Processing Unit, and the application of Graphical Processing Unit-based algorithms in the field of bioinformatics, the field of using computational algorithms to solve biological problems. We show that existing abstract models for analysing parallel algorithms on the Graphical Processing Unit are not able to sufficiently and accurately model all that is required. We propose a new abstract model, called the Abstract Transferring Graphical Processing Unit Model, which is able to provide analysis of Graphical Processing Unit-based algorithms that is more accurate than existing abstract models. It does this by capturing the data transfer between the Central Processing Unit and the Graphical Processing Unit. We demonstrate the accuracy and applicability of our model with several computational problems, showing that our model provides greater accuracy than the existing models, verifying these claims using experiments. We also contribute novel Graphics Processing Unit-base solutions to two bioinformatics problems: DNA sequence alignment, and Protein spectral identification, demonstrating promising levels of improvement against the sequential Central Processing Unit experiments.

In this contribution we present a real-time computation solution for multi-channel histograms at high-performance on Field-Programmable Gate Arrays (FPGAs). Being basic, yet highly useful instruments, histograms find applications in a wide variety of fields, playing a big role in compression and elaboration of large amount of data. Many solutions have already been developed by Academia and Industry, mostly relying on general-purpose Central Processing Units (CPUs) or full-custom Application-Specific Integrated Circuits (ASICs). Notwithstanding being mostly satisfying in terms of ease of use and flexibility (CPUs) on one side, or performance (ASICs) on the other, these solutions have shown to lack in balancing the tradeoff between these features. Another big point of interest is the often needed large storage capability for certain applications. To satisfy these requirements, we present a hybrid hardware and software innovative implementation of a real-time multi-channel histogram generator in an FPGA-based system, helped by a soft processor core implemented in the same FPGA fabric. In this way, the best performance of parallel and temporal computing merge into a firmware/software co-design. This solution features large availability of DDR memory, accessible through a Direct Memory Access (DMA), lower utilization of the precious FPGA resources with respect to the full-FPGA approach, real-time behavior and simplified, yet efficient, interface to the MicroBlaze, the soft core Reduced Instruction Set Computer (RISC), optimized for Xilinx FPGAs. IP-Cores and libraries allow the user-friendly Processing System to be connected to the programmable logic part to exploit its high performance in a flexible way. The system has been successfully tested on Xilinx 28-nm 7-Series devices connected to a 256 MB DDR3 memory, reaching performance in the order of 50 Msps and using a state–of–the–art, FPGA-based, multichannel, Time-to-Digital Converter (TDC) as event generator.

With the explosion of Digital Signal Processor (DSP) applications, there is a constant requirement for increased processing capability. This in turn requires rapid performance scaling in both operations per cycle and cycles per second, both of which result in increased MIPS/MMACS/MFLOPs. The memory system has to sustain the increased frequency and bandwidth demands in order to meet the data requirements of the DSP. Traditionally, DSP system architectures have on-chip addressable RAM, which is accessible by both the central processing unit (CPU) and the direct memory access (DMA). However, RAM frequencies are not scaling along with CPU clock rates, and as a result only relatively small RAM sizes are able to meet the frequency goals. This is in direct contrast to the increasing program size requirements seen by DSP applications, which in turn require even more on-chip RAM. This paper proposes a solution which has caches and RAMs coexisting in a homogeneous environment and working seamlessly together allowing high frequencies while still maintaining the DSP goals of low cost and low power. This multi-level memory system architecture has been implemented on the Texas Instruments (TI) TMS320C6211 C6x DSP.

Power reduction in sensor nodes is an essential technique for long-term operation. We noticed that central processing units (CPUs) accounted for a large volume of power consumption and that CPU functions were usually excessive for sensor node control. Direct memory access (DMA) was substituted for the CPU when iterative routine processing was operated in this research, and the CPU was made to fall asleep while the DMA responsibly controlled tasks. The experimental results revealed that DMA-driven control reduced power consumption to 37% compared with CPU-driven control in 28 MHz operation. We concurrently discovered defects in the existing micro-controller units (MCUs) for further reducing power. We also propose an improved hardware architecture.

The M series of chips produced by Apple has proven a capable and power-efficient alternative to mainstream Intel and AMD ×86 processors for everyday tasks. In addition, the unified design integrating the central processing and graphics processing unit (GPU), have allowed these M series chips to excel at many tasks with heavy graphical requirements without the need for a discrete GPU) in some cases even outperforming discrete GPUs. In this work, we show how the M series chips can be leveraged using the Metal Shading Language (MSL) to accelerate typical array operations in C++. More important, we show how the usage of MSL avoids the typical complexity of compute unified device architecture (CUDA) or OpenACC memory management by allowing the central processing unit (CPU) and GPU to work in unified memory. We demonstrate how performant the M series chips are on standard 1D and 2D array operations such as array addition, single-precision A·X plus Y, and finite-difference stencils, with respect to serial and OpenMP-accelerated CPU code. The reduced complexity of implementing MSL also allows us to accelerate an existing elastic wave equation solver (originally based on OpenMP-accelerated C++) while retaining all CPU and OpenMP functionality without modification. The resulting performance gain of simulating the wave equation is near an order of magnitude for large domain sizes. This gain attained from using MSL is similar to other GPU-accelerated wave-propagation codes with respect to their CPU variants but does not come at much increased programming complexity that prohibits the typical scientific programmer to leverage these accelerators. This result shows how unified processing units can be a valuable tool to seismologists and computational scientists in general, lowering the bar to writing performant codes that leverage modern GPUs.