Design of Embedded Augmented Reality Systems

Abstract

The great majority of current Augmented Reality (AR) applications are built using general purpose processors as development platforms where the processing tasks are executed in software. However, software execution is not always the best solution for the high intensive requirements of the many processing tasks involved in AR, and it inevitably constrains frame rate and latency, which compromises real time operation, and magnifies size and power consumption, hindering mobility. These limitations make the spread of AR applications more difficult. This is particularly remarkable in the case of mobile real time applications. To overcome the aforementioned constraints in the design of embedded AR systems, this chapter presents a hardware/software co-design strategy based on Field Programmable Gate Array (FPGA) devices and Electronic System-Level (ESL) description tools as an alternative to the traditional software-based approach. Modern FPGAs feature millions of gates of programmable logic, with dedicated hardware resources and with the widest range of connectivity solutions. FPGA internal structure makes itself perfectly suitable for exploiting parallelism at several levels. Moreover, because of its flexibility, it is possible to implement not only specific algorithms, but also AD/DA interfaces, controllers, and even several microprocessors, what makes it feasible to build more complex and powerful Systems on a Chip (SoC) with improved performance and reduced costs, size and power consumption. FPGA (re)programmability is also a key factor, which provides not just reduced time to market and design flexibility, but also in-the-field upgradability and intellectual property protection. Thanks to these characteristics, FPGAs are giving rise to a new paradigm in computation named Reconfigurable Computing. ESL, on the other hand, is an emerging electronic design methodology that focuses on building models of the entire system with a high-level language such as C, C++, or MATLAB, which are later used by improved electronic design tools to generate an automated and correct-by-construction implementation of the system. ESL codesign tools allow for developers with little or no prior hardware design skills to implement complex systems composed of mixed software and application-specific hardware modules. The objective of this chapter is to provide a clear vision of the possibilities of FPGA devices and the new development methodologies for embedded AR systems. To do it so, the authors explain the FPGAs key features which make them suitable for the implementation of AR applications. The design flow and tools for hardware description and