Introduction

For many years, digital signal processing has been used in movie production environments to carry out pre- and postprocessing of video data. In the case of live broadcast or augmented reality, real-time capability of such a processing system becomes a necessity. Due to the accompanying high computational demands, dedicated FPGA-based accelerators are often utilized. The programmability of FPGAs offers more flexibility than off-the-shelf DSPs, and also provides higher performance when processing massively parallelized data, a typical scenario in digital video applications.

Digital Chameleon is a joint research project conducted at the Institute of Computer Graphics (ICG) and the Institute of Computer and Network Engineering (IDA) at TU Braunschweig to enhance and accelerate an algorithm to digitally retouch textures of garments with real-time capabilities by utilizing FPGA-based hardware. The FlexWAFE library, originally developed during the FlexFilm project, will serve as the basis for application development. It is composed of highly optimized processing and communication elements specifically designed for processing of streaming data. These components are weakly programmable at run-time and thus provide the flexibility needed to implement complex algorithms and to make use of scarce FPGA resources efficiently.

During the course of this project, IDA will focus on the development of an automated design, analysis, and programming environment for FlexWAFE. This environment is crucial for efficient application development with FlexWAFE's highly flexible components. Presently, this task requires tedious, repetitive, and error-prone manual work. In the end, a framework with a level of abstraction significantly above the complexity of a hardware implementation and the requirement of detailed knowledge of FlexWAFE will emerge. This framework will ease the development of dataflow-oriented architectures and help to increase designer productivity. It will be used to implement the computationally intensive preprocessing steps of the texture retouching algorithm that are subject to enhancement.

Project at IDA

At IDA, the main focus of the research, therefore, is on the development of methods to design and analyze weakly programmable systems efficiently. Among these are the following central aspects:

• Development of a dataflow and component model to capture resource usage, timing, and communication behavior, both between components and to/from memory. A strong emphasis is on memory communication, which can be further divided into access to local on-chip and global off-chip memory. These accesses, however, may interfere with each other due to resource sharing. Furthermore, communication and synchronization between components must be considered.

• An application model will serve as a means for application entry. The structure and control of an application shall be captured. A GUI will allow the mapping to the dataflow and component models.

• Analysis: First, a syntactic check verifies the consistency of the application model with respect to its interconnected components. From the component model and communication behavior, a semantic analysis derives the resources, buffers, and synchronization means necessary for an actual implementation. This task will be carried out by a compositional analysis approach, as it has already been successfully put to use in the SymTA/S project.

• Addition of new components necessary for the retouching algorithm to FlexWAFE.

Furthermore, a close collaboration with the Flexelerator project, also ongoing at IDA, exists in the field of control and synchronization of weakly programmable heterogeneous systems.

People at IDA

Daniel Thiele

Funding

Deutsche Forschungsgemeinschaft