Software-only Approach Research Articles

Hardware-Accelerated Data Decoding and Reconstruction for Automotive LiDAR Sensors

The automotive industry is facing an unprecedented technological transformation towards fully autonomous vehicles. Optimists predict that, by 2030, cars will be sufficiently reliable, affordable, and common to displace most current human driving tasks. To cope with these trends, autonomous vehicles require reliable perception systems to hear and see all the surroundings, being light detection and ranging (LiDAR) sensors a key instrument for recreating a 3D visualization of the world. However, for a reliable operation, such systems require LiDAR sensors to provide high-resolution 3D representations of the car's vicinity, which results in millions of data points to be processed in real-time. With this article we propose the ALFA-Pi, a data packet decoder and reconstruction system fully deployed on an embedded reconfigurable hardware platform. By resorting to field-programmable gate array (FPGA) technology, ALFA-Pi is able to interface different LiDAR sensors at the same time, while providing custom representation outputs to high-level perception systems. By accelerating the LiDAR interface, the proposed system outperforms current software-only approaches, achieving lower latency in the data acquisition and data decoding tasks while reaching high performance ratios.

A Hardware–Software Co-Design Framework for Real-Time Video Stabilization

Live digital video is a valuable source of information in security, broadcast and industrial quality control applications. Motion jitter due to camera and platform instability is a common artefact found in captured video which renders it less effective for subsequent computer vision tasks such as detection and tracking of objects, background modeling, mosaicking, etc. The process of algorithmically compensating for the motion jitter is hence a mandatory pre-processing step in many applications. This process, called video stabilization, requires estimation of global motion from consecutive video frames and is constrainted by additional challenges such as preservation of intentional motion and native frame resolution. The problem is exacerbated in the presence of local motion of foreground objects and requires robust compensation of the same. As such achieving real-time performance for this computationally intensive operation is a difficult task for embedded processors with limited computational and memory resources. In this work, development of an optimized hardware–software co-design framework for video stabilization has been investigated. Efficient video stabilization depends on the identification of key points in the frame which in turn requires dense feature calculation at the pixel level. This task has been identified to be most suitable for offloading the pipelined hardware implemented in the FPGA fabric due to the involvement of complex memory and computation operations. Subsequent tasks to be performed for the overall stabilization algorithm utilize these sparse key points and have been found to be efficiently handled in the software. The proposed Hardware–Software (HW–SW) co-design framework has been implemented on Zedboard FPGA platform which houses Xilinx Zynq SOC equipped with ARM A9 processor. The proposed implementation scheme can process real-time video stream input at 28 frames per second and is at least twice faster than the corresponding software-only approach. Two different hardware accelerator designs have been implemented using different high-level synthesis tools using rapid prototyping principle and consume less than 50% of logic resources available on the host FPGA while being at least 30% faster than contemporary designs.

HyperCo: Optimizing Network Performance in ARM-Based Mobile Virtualization

In the ARM-based mobile virtualized environment, optimizing both the network throughput and the latency for network-intensive applications is of great importance. Our experimental studies have shown that the context switch between the host and guest triggered by a hypercall results in CPU overload and thus degrades the I/O performance under an intensive workload. To address this problem, we propose the Adaptive Hypercall Coalescing (HyperCo) algorithm, a software-only approach to optimizing the network I/O performance by reducing the number of hypercalls and achieving a trade-off between throughput and latency. We implement HyperCo by modifying the front-end driver of Virtio-net on the KVM/ARM platform, and we carry out extensive performance evaluations, which indicate that the number of hypercalls and the load on the CPU can be significantly reduced. We show that HyperCo can significantly improve the network throughput under an intensive workload while keeping a low penalty by adapting a coalescing interval dynamically at a low frequency of network I/O requests.

Computing with Near Data

One cost that plays a significant role in shaping the overall performance of both single-threaded and multi-thread applications in modern computing systems is the cost of moving data between compute elements and storage elements. Traditional approaches to address this cost are code and data layout reorganizations and various hardware enhancements. More recently, an alternative paradigm, called Near Data Computing (NDC) or Near Data Processing (NDP), has been shown to be effective in reducing the data movements costs, by moving computation to data, instead of the traditional approach of moving data to computation. Unfortunately, the existing Near Data Computing proposals require significant modifications to hardware and are yet to be widely adopted. In this paper, we present a software-only (compiler-driven) approach to reducing data movement costs in both single-threaded and multi-threaded applications. Our approach, referred to as Computing with Near Data (CND), is built upon a concept called "recomputation," in which a costly data access is replaced by a few less costly data accesses plus some extra computation, if the cumulative cost of the latter is less than that of the costly data access. If implemented carefully, CND can successfully trade off data access with computation, and considering the continuously increasing latency gap between the two, doing so can significantly reduce the execution latencies of both sequential and parallel application programs. We i) quantify the intrinsic recomputability of a set of single-threaded and multi-threaded applications, ii) propose a practical, compiler-driven approach that automatically transforms a given application code fragment to a version that employs recomputation, iii) discuss an optimization strategy that increases recomputability; and iv) compare CND, both qualitatively and quantitatively, against NDC. Our experimental analysis of CND reveals that i) the average recomputability across our benchmarks is 51.1%, ii) our compiler-driven strategy is able to exploit 79.3% of the recomputation opportunities presented by our workloads, and iii) our enhancements increase the value of the recomputability metric significantly. As a result, our compiler-driven approach with the proposed enhancements brings an average execution time improvement of 40.1%.

GPUart - An application-based limited preemptive GPU real-time scheduler for embedded systems

Emerging technologies like autonomous driving entail computational intense software solutions. More and more companies accelerate their embedded applications by General Purpose Computing on the Graphics Processing Unit (GPGPU), in order to overcome those computational demands. Unfortunately, Graphics Processing Units (GPUs) severely lack real-time capability, for example controllable preemption support, which limits their applicability in the embedded domain. We therefore present GPUart, a framework for GPU real-time scheduling. GPUart focuses on embedded systems and requires neither hardware nor driver stack extensions. We propose a software-only approach for preemption, based on the fixed preemption point strategy. In contrast to prior work, GPUart enables preemption inside a thread block by adding fixed preemption points. We further propose a portable high-level resource management concept to enable gang scheduling on GPUs. GPUart can schedule GPU workload either under the Gang-Earliest Deadline First (EDF) or Gang-Fixed Task Priority (FTP) policy. A case-study on Nvidia Tegra X1, using real-world engine management applications from Audi AG and Continental Automotive GmbH, shows that only up to 0.28% additional global memory is required to enable interruptible thread blocks. GPUart reduces the worst observed response times by a factor of up to 221, leading to response times without deadline misses.

An Internet of Things Architecture for Cloud-Fit Professional Media Workflow

Flexibility, agility, and scale are key benefits of moving media production infrastructure to the cloud and/or commodity data center platforms, enabling content providers to respond to changing consumer habits. This includes new formats and new propositions, such as personalized immersive social television. Can a lift and shift approach—deploying existing production tools designed for enterprise computing environments into the cloud—deliver these benefits? This paper presents an alternative architecture that is designed to be cloud-fit, using a software-only approach for deploying dynamic software infrastructure. This paper starts by outlining both the benefits and constraints of cloud-based systems, including issues related to virtualized systems being inherently non-realtime. An architecture that overcomes these limitations is then presented, along with an approach for dealing with security and scaling. This consists of a combination of mainly IT tool sets (e.g., monitoring tools, orchestration engines, cloud application programming interfaces, and automated software deployment) with media-specialist software components. An instance of this architecture has been created using an open source Internet of Things wiring tool. This is being trialed as part of the British Broadcasting Corp. (BBC) Northern Ireland’s internet protocol proof-of-concept activity, working jointly with Streampunk Media and Cinegy through Advanced Media Workflow Association Labs. The lab environment has enabled the testing and measurement of aspects of this approach, including gesture-based control (e.g., via smartphone/tablet), HTTPS media transport, fast networking with consumer devices, and virtual infrastructure replacing traditional hardware solutions—including a virtual video mixer. Results from the lab are presented, along with a description and pointer to the open source tools used to produce the results, further analysis, and next steps.

Automated OS-level Device Runtime Power Management

Non-CPU devices on a modern system-on-a-chip (SoC), ranging from accelerators to I/O controllers, account for a significant portion of the chip area. It is therefore vital for system energy efficiency that idle devices can enter a low-power state while still meeting the performance expectation. This is called device runtime Power Management (PM) for which individual device drivers in commodity OSes are held responsible today. Based on the observations of existing drivers and their evolution, we consider it harmful to rely on drivers for device runtime PM. This paper identifies three pieces of information as essential to device runtime PM, and shows that they can be obtained without involving drivers, either by using a software-only approach, or more efficiently, by adding one register bit to each device. We thus suggest a structural change to the current Linux runtime PM framework, replacing the PM code in all applicable drivers with a single kernel module called the central PM agent. Experimental evaluations show that the central PM agent is just as effective as hand-tuned driver PM code. The paper also presents a tool called PowerAdvisor that simplifies driver PM efforts under the current Linux runtime PM framework. PowerAdvisor analyzes execution traces and suggests where to insert PM calls in driver source code. Despite being a best-effort tool, PowerAdvisor not only reproduces hand-tuned PM code from stock drivers, but also correctly suggests PM code never known before. Overall, our experience shows that it is promising to ultimately free driver developers from manual PM.

Automated OS-level Device Runtime Power Management

Non-CPU devices on a modern system-on-a-chip (SoC), ranging from accelerators to I/O controllers, account for a significant portion of the chip area. It is therefore vital for system energy efficiency that idle devices can enter a low-power state while still meeting the performance expectation. This is called device runtime Power Management (PM) for which individual device drivers in commodity OSes are held responsible today. Based on the observations of existing drivers and their evolution, we consider it harmful to rely on drivers for device runtime PM. This paper identifies three pieces of information as essential to device runtime PM, and shows that they can be obtained without involving drivers, either by using a software-only approach, or more efficiently, by adding one register bit to each device. We thus suggest a structural change to the current Linux runtime PM framework, replacing the PM code in all applicable drivers with a single kernel module called the central PM agent. Experimental evaluations show that the central PM agent is just as effective as hand-tuned driver PM code. The paper also presents a tool called PowerAdvisor that simplifies driver PM efforts under the current Linux runtime PM framework. PowerAdvisor analyzes execution traces and suggests where to insert PM calls in driver source code. Despite being a best-effort tool, PowerAdvisor not only reproduces hand-tuned PM code from stock drivers, but also correctly suggests PM code never known before. Overall, our experience shows that it is promising to ultimately free driver developers from manual PM.

IBM Secure Enterprise Desktop

Using software-only approaches makes it is practically impossible to completely secure software applications, as well as corporate information, against determined cyber-criminals. Therefore, in an era where any general-purpose operating system (OS) with end-user access can be hacked, we propose using dedicated security hardware to ensure that only authorized people obtain access to sensitive information. The fundamental principle involves booting the end-user computer from such a trusted mobile device without trusting any software installed on the computer. The device establishes a secure connection to the back-end infrastructure to provide access to the user's OS, e.g., through a remote terminal access or provisioned on the local computer. The solution is very simple to operate, as many corporate employees are not necessarily IT (information technology) savvy. In this paper, we discuss the combination of our dedicated tamper-resistant security boot-token operating user credentials with known defense mechanisms, such as OS virtualization, trusted boot, establishment of client-side and server-side authenticated secure channels to trustworthy back-ends, and client-side storage encryption. This novel combination forms an easy-to-use and highly mobile security solution that addresses security challenges of the BYOD (bring-your-own-device) approach. As a proof point for the latter claims, we report on initial real-world usability tests.

Accelerating sequential programs on commodity multi-core processors

A recently proposed pipelined multithreading (PMT) technique exhibits wide applicability in parallelizing general sequential programs on multi-core processors. However, significant inter-core communication overhead limits PMT performance and prevents its commercial utilization. A simple and effective clustered pipelined multithreading (CPMT) approach is presented to accelerate sequential programs on commodity multi-core processors. This CPMT technique adopts a clustered communication mechanism that can yield very low average communication overhead by eliminating false sharing as well as reducing communication operation and transit delays in the software-only approach. A single-producer/single-consumer concurrent lock-free clusteredQueue algorithm based on a two-level queue structure is also proposed. The accuracy of CPMT is theoretically demonstrated. The performances of the algorithm and CPMT are evaluated on a commodity AMD Phenom four-core processor. The number of enqueue and dequeue times of the algorithm are 20.8 and 23 cycles given an appropriate parameter, respectively. The speedup of CPMT ranges from 13.1% to 119.8% for typical loops extracted from the SPEC CPU 2000 benchmark suite.

Low cost control flow protection using abstract control signatures

The continual trend of shrinking feature sizes and reducing voltage levels makes transistors faster and more efficient. However, it also makes them more susceptible to transient hardware faults. Transient faults due to high energy particle strikes or circuit crosstalk can corrupt the output of a program or cause it to crash. Previous studies have reported that as much as 70% of the transient faults disturb program control flow, making it critical to protect control flow. Traditional approaches employ signatures to check that every control flow transfer in a program is valid. While having high fault coverage, large performance overheads are introduced by such detailed checking. We propose a coarse-grain control flow checking method to detect transient faults in a cost effective way. Our software-only approach is centered on the principle of abstraction: control flow that exhibits simple run-time properties (e.g., proper path length) is almost always completely correct. Our solution targets off-the-shelf commodity embedded systems to provide a low cost protection against transient faults. The proposed technique achieves its efficiency by simplifying signature calculations in each basic block and by performing checking at a coarse-grain level. The coarse-grain signature comparison points are obtained by the use of a region based analysis. In addition, we propose a technique to protect control flow transfers via call and return instructions to ensure all control flow is covered by our technique. Overall, our proposed technique has an average of 11% performance overhead in comparison to 75% performance overhead of previously proposed signature based techniques while maintaining approximately the same degree of fault coverage.

Aikido

Despite a burgeoning demand for parallel programs, the tools available to developers working on shared-memory multicore processors have lagged behind. One reason for this is the lack of hardware support for inspecting the complex behavior of these parallel programs. Inter-thread communication, which must be instrumented for many types of analyses, may occur with any memory operation. To detect such thread communication in software, many existing tools require the instrumentation of all memory operations, which leads to significant performance overheads. To reduce this overhead, some existing tools resort to random sampling of memory operations, which introduces false negatives. Unfortunately, neither of these approaches provide the speed and accuracy programmers have traditionally expected from their tools. In this work, we present Aikido, a new system and framework that enables the development of efficient and transparent analyses that operate on shared data. Aikido uses a hybrid of existing hardware features and dynamic binary rewriting to detect thread communication with low overhead. Aikido runs a custom hypervisor below the operating system, which exposes per-thread hardware protection mechanisms not available in any widely used operating system. This hybrid approach allows us to benefit from the low cost of detecting memory accesses with hardware, while maintaining the word-level accuracy of a software-only approach. To evaluate our framework, we have implemented an Aikido-enabled vector clock race detector. Our results show that the Aikido enabled race-detector outperforms existing techniques that provide similar accuracy by up to 6.0x, and 76% on average, on the PARSEC benchmark suite.

ELI

Direct device assignment enhances the performance of guest virtual machines by allowing them to communicate with I/O devices without host involvement. But even with device assignment, guests are still unable to approach bare-metal performance, because the host intercepts all interrupts, including those interrupts generated by assigned devices to signal to guests the completion of their I/O requests. The host involvement induces multiple unwarranted guest/host context switches, which significantly hamper the performance of I/O intensive workloads. To solve this problem, we present ELI (ExitLess Interrupts), a software-only approach for handling interrupts within guest virtual machines directly and securely. By removing the host from the interrupt handling path, ELI manages to improve the throughput and latency of unmodified, untrusted guests by 1.3x-1.6x, allowing them to reach 97%-100% of bare-metal performance even for the most demanding I/O-intensive workloads.

Aikido

Despite a burgeoning demand for parallel programs, the tools available to developers working on shared-memory multicore processors have lagged behind. One reason for this is the lack of hardware support for inspecting the complex behavior of these parallel programs. Inter-thread communication, which must be instrumented for many types of analyses, may occur with any memory operation. To detect such thread communication in software, many existing tools require the instrumentation of all memory operations, which leads to significant performance overheads. To reduce this overhead, some existing tools resort to random sampling of memory operations, which introduces false negatives. Unfortunately, neither of these approaches provide the speed and accuracy programmers have traditionally expected from their tools. In this work, we present Aikido, a new system and framework that enables the development of efficient and transparent analyses that operate on shared data. Aikido uses a hybrid of existing hardware features and dynamic binary rewriting to detect thread communication with low overhead. Aikido runs a custom hypervisor below the operating system, which exposes per-thread hardware protection mechanisms not available in any widely used operating system. This hybrid approach allows us to benefit from the low cost of detecting memory accesses with hardware, while maintaining the word-level accuracy of a software-only approach. To evaluate our framework, we have implemented an Aikido-enabled vector clock race detector. Our results show that the Aikido enabled race-detector outperforms existing techniques that provide similar accuracy by up to 6.0x, and 76% on average, on the PARSEC benchmark suite.

ELI

Direct device assignment enhances the performance of guest virtual machines by allowing them to communicate with I/O devices without host involvement. But even with device assignment, guests are still unable to approach bare-metal performance, because the host intercepts all interrupts, including those interrupts generated by assigned devices to signal to guests the completion of their I/O requests. The host involvement induces multiple unwarranted guest/host context switches, which significantly hamper the performance of I/O intensive workloads. To solve this problem, we present ELI (ExitLess Interrupts), a software-only approach for handling interrupts within guest virtual machines directly and securely. By removing the host from the interrupt handling path, ELI manages to improve the throughput and latency of unmodified, untrusted guests by 1.3x-1.6x, allowing them to reach 97%-100% of bare-metal performance even for the most demanding I/O-intensive workloads.

A Hardware Virtualization Based Component Sandboxing Architecture

Modern applications comprise multiple components, such as browser plug-ins, often of unknown provenance and quality. Statistics show that failure of such components accounts for a high percentage of software faults. Enabling isolation of such fine-grained components is therefore necessary to increase the robustness and resilience of security-critical and safety-critical computer systems. In this paper, we evaluate whether such fine-grained components can be sandboxed through the use of the hardware virtualization support available in modern Intel and AMD processors. We compare the performance and functionality of such an approach to two previous software based approaches. The results demonstrate that hardware isolation minimizes the difficulties encountered with software based approaches, while also reducing the size of the trusted computing base, thus increasing confidence in the solution's correctness. We also show that our relatively simple implementation has equivalent run-time performance, with overheads of less than 34%, does not require custom tool chains and provides enhanced functionality over software-only approaches, confirming that hardware virtualization technology is a viable mechanism for fine-grained component isolation.

A hybrid hardware--software technique to improve reliability in embedded processors

Numerous methods have been described in research literature with methods to improve reliability of processors by the use of control-flow checking. High performance and code-size penalties cripple the proposed software approaches, while hardware approaches are not scalable and are thus rarely implemented in real embedded systems. In this article, we show that by including control-flow checking as an issue to be considered when designing as embedded processor, we are able to reduce overheads considerably and still provide a scalable solution to this problem. The technique described in this article includes architectural improvements to the processor and binary rewriting of the application. Architectural refinement incorporates additional instructions to the instruction set architecture, while the binary rewriting utilizes these additional instructions into the program flow. Applications from an embedded systems benchmark suite have been used to test and evaluate the system. Our approach increased code size by only 5.55% to 13.5% and reduced performance by just 0.54% to 2.83% for eight different industry standard benchmarks. The additional hardware overhead due to the additional instruction in the design is just 2.70%. In contrast, the state-of-the-art software-only approach required 50% to 150% additional code, and reduced performance by 53.5% to 99.5% when monitoring was inserted. Fault injection analysis demonstrates that our solution is capable of capturing and recovering from all the injected control-flow errors, while the software-only approach detected 87% of the injected control-flow errors.

SoftBound

The serious bugs and security vulnerabilities facilitated by C/C++'s lack of bounds checking are well known, yet C and C++ remain in widespread use. Unfortunately, C's arbitrary pointer arithmetic, conflation of pointers and arrays, and programmer-visible memory layout make retrofitting C/C++ with spatial safety guarantees extremely challenging. Existing approaches suffer from incompleteness, have high runtime overhead, or require non-trivial changes to the C source code. Thus far, these deficiencies have prevented widespread adoption of such techniques. This paper proposes SoftBound, a compile-time transformation for enforcing spatial safety of C. Inspired by HardBound, a previously proposed hardware-assisted approach, SoftBound similarly records base and bound information for every pointer as disjoint metadata. This decoupling enables SoftBound to provide spatial safety without requiring changes to C source code. Unlike HardBound, SoftBound is a software-only approach and performs metadata manipulation only when loading or storing pointer values. A formal proof shows that this is sufficient to provide spatial safety even in the presence of arbitrary casts. SoftBound's full checking mode provides complete spatial violation detection with 67% runtime overhead on average. To further reduce overheads, SoftBound has a store-only checking mode that successfully detects all the security vulnerabilities in a test suite at the cost of only 22% runtime overhead on average.

DMP

Current shared memory multicore and multiprocessor systems are nondeterministic. Each time these systems execute a multithreaded application, even if supplied with the same input, they can produce a different output. This frustrates debugging and limits the ability to properly test multithreaded code, becoming a major stumbling block to the much-needed widespread adoption of parallel programming. In this paper we make the case for fully deterministic shared memory multiprocessing (DMP). The behavior of an arbitrary multithreaded program on a DMP system is only a function of its inputs. The core idea is to make inter-thread communication fully deterministic. Previous approaches to coping with nondeterminism in multithreaded programs have focused on replay, a technique useful only for debugging. In contrast, while DMP systems are directly useful for debugging by offering repeatability by default, we argue that parallel programs should execute deterministically in the field as well. This has the potential to make testing more assuring and increase the reliability of deployed multithreaded software. We propose a range of approaches to enforcing determinism and discuss their implementation trade-offs. We show that determinism can be provided with little performance cost using our architecture proposals on future hardware, and that software-only approaches can be utilized on existing systems.

DMP

Current shared memory multicore and multiprocessor systems are nondeterministic. Each time these systems execute a multithreaded application, even if supplied with the same input, they can produce a different output. This frustrates debugging and limits the ability to properly test multithreaded code, becoming a major stumbling block to the much-needed widespread adoption of parallel programming. In this paper we make the case for fully deterministic shared memory multiprocessing (DMP). The behavior of an arbitrary multithreaded program on a DMP system is only a function of its inputs. The core idea is to make inter-thread communication fully deterministic. Previous approaches to coping with nondeterminism in multithreaded programs have focused on replay, a technique useful only for debugging. In contrast, while DMP systems are directly useful for debugging by offering repeatability by default, we argue that parallel programs should execute deterministically in the field as well. This has the potential to make testing more assuring and increase the reliability of deployed multithreaded software. We propose a range of approaches to enforcing determinism and discuss their implementation trade-offs. We show that determinism can be provided with little performance cost using our architecture proposals on future hardware, and that software-only approaches can be utilized on existing systems.