Development and implementation of interactive 3D video environment on run-time reconfigurable FPGA platform

Abstract

Video data processing tasks are traditionally performed either through software-based systems when various algorithms must be applied to the data and when time issue is not critical, DSPs -- when certain time constraints are set but when the set of tasks is limited, or ASICs -- when the highest performance is required, the set of tasks is fixed and highly optimized, the data stream doesn't change, and the number of data streams is limited. For a real-time system which must operate on multiple data streams which also can change in time and on which various data processing algorithms must be applied neither of the mentioned approaches can be used. Timing requirements and power limitation does not allow utilization of sequential CPU. ASIC becomes too big to accomodate multiple processing circuits for each algorithm and associated modes. Only Run-Time Reconfigurable (RTR) FPGA approach allows implementation of such a system. The thesis presents a real-time stereo vision system with elements of synthesis of interactive 3-D virtual objects designed and implemented on the FPGA-based reconfigurable platform. FPGA chip integrates a hybrid architecture system with multi-mode and mutli-stream processing ability for critical time tasks and with embedded microprocessor(s) for computing complex algorithms for 3-D objects synthesis for which timing requirements are not so strict. An approach for the formal presentation and processing of the 3-D virtual objects and their transformation is also analyzed and presented in this paper. Architecture synthesis and optimization for a hybrid system are also considered. The experimental results proved the effectiveness of proposed approach: the FPGA-based system-on-chip provides stereo visualization in different modes (actual image and edge detection image), with synthesized 3-D controls (pressed and released buttons).