Real-time Video Signal Processing Research Articles

Development of Real-time Video Signal Processor

Development of Real-time Video Signal Processor

A Low-Profile Digital Eye-Tracking Oculometer for Smart Eyeglasses.

Wearable eye tracking devices have broad uses in medicine, psychology, augmented & virtual reality and consumer market research. Most mobile eye trackers available today utilize infrared imaging of the pupil and corneal reflections with video cameras. This tracking method requires sophisticated real-time processing of video signals consuming substantial electrical power. This method is thus unsuitable for light weight wearables such as adaptive smart eyeglasses for correction of presbyopia. In this paper we present a low-profile, low-power (7.7 mJ/sample) digital eye tracker oculometer based on infrared sclera tracking. The system is implemented using eight, 24-bit infrared proximity sensors and synchronous infrared LEDs. The pupil location is determined from 32 reflected pulsed light measurements independent of ambient illumination. The digital oculometer is 3.1 mm thick and weighs ~3 g. The tracker mounts adjacent to the tunable lenses in the smart eyeglasses frame. The eye tracker showed a pointing error of 1.3 degrees rms over a vertical and horizontal range of 30 degrees when tested by an observer.

High speed lattice based VLSI architecture of 2D discrete wavelet transform for real-time video signal processing

This paper presents an efficient lattice structure based VLSI architecture of 2D discrete wavelet transform (DWT) for hierarchical image compression, which is scalable to extend to an arbitrary 2D DWT with M laps and J levels. The proposed architecture consists Of four 1D lattice filters, which processes in horizontal and vertical directions at the same time. The proposed lattice structure fits in a VLSI implementation due to its regularity and shows the period of N/sup 2//2 to compute an N/spl times/N image because the even and odd rows are processed simultaneously. Compared to conventional approaches, the proposed architecture shows shorter period to complete 2D DWT while requiring relatively less hardware resources. Therefore, the proposed architecture can be applied in real-time video signal processing such as JPEG-2000 and MPEG4, which require high speed processing. The process schedule using the data dependency graph, performance, and the required hardware cost are discussed.

A high-sensitivity CCD camera system for observations of early universe objects

Abstract The principles of the construction of a new CCD camera system with improved performance in readout noise and charge measurement accuracy for the 6 m telescope are described. Digital real-time processing of video signals applied in the system.

A smart camera application: DSP-based people detection and tracking

As an application of smart cameras, this paper presents an algorithm and its real-time implementation for detecting and tracking multiple people moving in a surveillance scene. The algo- rithm is devised to work for low resolution image frames and any camera angle. First an adaptive background subtraction module is used to obtain potential moving target areas. Second a human body detection module is activated to locate the heads. Third a multitarget tracking module is deployed to resolve ambiguities associated with crosspaths and merged targets. The algorithm is implemented on the Texas Instruments TMS320C6201 fixed-point DSP processor for real-time processing of video signals. This real-time capability allows field deployment for the purpose of collecting surveillance data such as the number of people and their crossing frequency in a certain area. © 2000 SPIE and IS&T. (S1017-9909(00)01802-X)

Architecture of 23GOPS video signal processor with programmable systolic array

This paper describes an architecture of 23GOPS real-time video signal processor. In order to achieve high computational power and high data bandwidth for real-time video signal processing, we adopt a unique architecture based on a programmable systolic array with 90 video processing elements (VPEs). The VPE array realizes high processing ability and high flexibility by a simple structure of the VPE and a time-division multiple-operation scheme. It allows the processor to be applied to various real-time video signal processing like HD-TV (MUSE) decoding. The processor, called the digital filtering array, has been fabricated in 0.35-/spl mu/m CMOS three-metal-layer technology and achieves 23GOPS at 129.6 MHz operating frequency. Four million transistors are integrated in 13.61 mm/spl times/13.07 mm die size.

VLSI implementation of 2-D discrete wavelet transform for real-time video signal processing

This paper presents the architecture and implementation of a single-chip VLSI for the two-dimensional discrete wavelet transform (2-D DWT) decomposition. This nonseparable based architecture uses a parallel-systolic filter structure to compute all the resolution levels of the DWTs, such that the input samples can be processed at the rate of one sample per clock cycle. The chip was fabricated in a 0.6 /spl mu/m CMOS technology and packaged as a 48-pin DIP. For the computation of an N/spl times/N still image with a filter length L, this chip needs N/sup 2/+N clock cycles and N(2L-1) memory storage; for continuous picture such as video signal, its average computation time per picture is about N/sup 2/ only.

High-speed systolic ladder structures for multidimensional recursive digital filters

High-speed multidimensional (MD) digital filtering is very useful for real-time video signal processing such as video image coding, bandwidth compression, sampling rate conversion and the enhancement of television signals. We propose a multilevel approach for designing high-speed systolic ladder structures for MD recursive digital filters. Based on appropriately selected 1-D filter structures for each filter dimension (or level), a large variety of MD systolic filter structures may be derived. In particular, we introduce a new 1-D filter structure that proves the most suitable structure in terms of a systolic ladder implementation, because it leads to MD ladder filter structures possessing such important properties as the shortest critical path (for filters without order augmentation), the canonic number of high-level storage registers (e.g., line and frame registers of images), and local interconnectivity.

Architecture and programming of two generations video signal processors

Programmable video signal processor ICs (VSPs) and dedicated programming tools have been developed for the real-time processing of digital video signals. A large number of applications have been developed with boards containing several of these processors. Currently two implementations of the general architecture exist: VSP1 and VSP2. A single VSP chip contains several arithmetic and logic elements (ALEs) and memory elements. A complete switch matrix implements the unconstrained communication between all elements in a single cycle. The programming of these processors is carried out with signal flow graphs. These signal flow graphs can conveniently express multi-rate algorithms. These algorithms are then mapped onto a network of processors. Mapping is decomposed into delay management, partitioning and scheduling. The solution strategies for the partitioning problem and the scheduling problem are illustrated. Applications with these processors have been made for a number of industrially relevant video algorithms, including the complete processing of next generation fully digital studio TV cameras and several image improvement algorithms in medical applications. Results of the mapping are presented for a number of algorithms in the field of TV processing.

Vlsi programmable digital filter for video signal processing

High speed digital filtering is required in real time video signal processing, as well as high order filters are needed to match television studio signal quality. The hardware complexity involved by such system constraints may be faced by a two-fold approach, concerning both the architecture and the technological aspects of a specific electronics device devoted to the above task. This article deals with a processor especially developed for the purpose of fast digital video signal applications, such as filtering, equalization, interpolation and so on. The nonrecursive transposed F.I.R. (Finite Impulse Response) structure has been selected, which exhibits a linear phase behavior. A novel approach has been developed for the multipliers implementation, by optimizing an EPROM based look-up table storing the products between all video samples and the filter coefficients significant bits, resulting in a programmable system. TheProgrammable Filter Processor has been designed with a high level of parallelism and pipelining and a 1.2 µm CMOS EPROM, single metal technology has been employed for the integration process of the chip. This has been successfully production-tested for 40 Msamples/s throughput rate, thus both allowing to meet most video filtering applications and demonstrating the potentialities of nonvolatile memory technologies in embedded applications. Moreover multiple devices can be interconnected to yield multiprocessor structures for more demanding performances such as, cascaded or longer filters, input signal precision extension, computation improved accuracy, increased throughput rate, and two-dimensional signal processing.

Neural network feature detector for real-time video signal processing.

The application of artificial neural networks to real-time image processing tasks requires the use of dedicated, high performance hardware. A linear array processor called HANNIBAL has been developed which implements the backpropagation neural learning algorithm on-chip. This paper considers the design of a complete neural system which integrates HANNIBAL into an existing image processing environment. The goals for the design of the system have been set partly by the primary application, namely feature recognition, but mainly by the desire for a flexible, high performance hardware tool for the study and evaluation of range of neural image processing applications.

A real-time 2-D median based filter for video signals

A 2-D, multishell median-based filter for real-time video signal processing is presented. The device is able to remove impulse and missing line noise in real-time without blurring sharp video images. A prototype using 75 standard transistor-transistor logic (TTL) chips has been realized on one single speedwire prototype board of size 17*30 cm/sup 2/ and its structure is suitable for VLSI implementation. The statistical performance of the nonlinear filter has been investigated and compared with that of standard 2-D median filters. It is shown that the filter has characteristics superior to those of its median-based counterparts while the hardware implementation is much simpler. >

A real-time video signal processing chip

A real-time video signal processing chip for video data compression which combines block truncation coding (BTC) and a conventional motion compensation (MC) algorithm is presented. The regularity and simplicity of the BTC algorithm strongly suggest that it is quite suitable for an efficient VLSI implementation. It is shown that the proposed BTC-based system is much better than the conventional system in terms of implementation, complexity, and accuracy. Two benchmark sequences are used as test examples. The clock rate of the designed chip is about 47 MHz, which is suitable for most video applications. >

Fast Three-Dimensional Measurement Technique Using a Reference Table. Measurement of a Human Head.

This paper describes a system which enables fast 3-dimensional measurementof a human head by means of a slit ray projection method. The system developed here consists of a laser slit ray projector, two CCD TV cameras, a real-time video signal processor and a 32-bit computer. Due to the employment of this real-time video signal processor and a fast data-processing technique using a reference table, a 3-dimensional measurement of the human head can be Accomplished in 4.3 seconds, and all the image data are converted to 3-dimensional data in one second. Experimental data reveal the applicability of our system.

Fast 3-Dimensional Measuring Technique Using a Look-Up Table. Measurement of a Human Hcad.

This paper describes a system which enables fast 3-dimensional measurement of a human head by means of a slit-ray projection method. The system developed here consists of a laser slit-ray projector, two CCD TV cameras, a real-time video signal processor and a 16-bit computer. Due to the employment of this real-time video signal processor and a fast data-processing technique for using a look-up table, a 3-dimensional measurement of the human head can be accomplished in 4.3 seconds, and all the image data are converted to 3-dimensional data in 1 second. Experimental data reveal the applicability of our system.

特集 画像処理―アルゴリズムとシステム 映像合成のための動ベクトル検出法に関する一検討

A new video synthesis method is proposed. In this method, the key for extracting moving objects is given in the first frame. In the following frames, the key is moved and transformed according to the detected motion-vector of the objects. Then with this transformed key, the objects are extracted automatically and synthesized. A motion-vector detection method suitable for this synthesis is discussed. According to the experiment using motion pictures with pan, tilt, zoom, and rotation, the matching method obtains an error smaller than ±2 pixels with detection points of 4 distant areas on the edge of the objects. It is confirmed that this synthesis method can be achieved at a video-rate on the real-time video signal processing system called the Picot-system.

2-D analog filters for real time video signal processing

A practical hardware design of a two-dimensional (2D) analog filter proposed by M.A. Sid-Ahmed (IEEE Circ. and Syst., vol.36, p.153-4, Jan. 1989) is developed. The structure is implemented using charge coupled device (CCD) analog shift registers and wideband operational amplifiers. The operation of the filter is demonstrated by processing TV video images in real time. The 2D analog approach is evaluated by comparison with a 2D distributed arithmetic digital filter. The analog approach offers realization at lower cost, less power consumption, higher resolution, and inherent true real-time capability independent of filter order. >

A real-time video signal processor suitable for motion picture coding applications

A real-time video signal processor (VSP) system has been developed. The system employs three multiprocessor clusters, each of which has 12 video signal processor modules (VSPMs). A VSPM in a cluster processes its assigned subimage by using an overlap-save technique. Each cluster uses the same multiprocessor structure, in which homogeneous processor modules are connected to input, output, and feedback buses in parallel and two bus switch units. By controlling these units, the clusters can be combined in pipeline and/or parallel forms. Each cluster also uses a variable delay unit which achieves up to one frame delay on the feedback bus. By using this unit, interframe processing can be carried out without using internal data memories in VSPMs for the frame delay. The employment of the bus switches and the variable delay unit increases flexibility for a variety of signal processing algorithms. The system performs 500 million operations per second and is currently used as a real-time evaluation system for low-bit-rate picture encoders. >

A general-purpose programmable video signal processor

A general-purpose digital video signal processor (VSP) and its associated programming support tools are described. Its architecture, tailored to real-time video signal processing, makes the VSP a unique component for flexible and design-cost-effective implementations of real-time video systems. At the lowest architectural level various parallel operating and pipelined processing elements communicate by means of a crossbar switch which provides full communication between the processors and the outside world. Processors and a crossbar switch, constituting a module, are controlled by cyclostatic programs running at 27 MHz. Modularity is exploited at the two higher levels: the number of modules per chip can be geared to the available IC technology, and at the system level the number of identical VSP chips can be geared to the complexity of the respective algorithm. A one-module VSP chip, operating at 27 MHz, offers a processing power of 270 MIPS (million instructions per second) or 1400 MOPS (million operations per second) and a communication bandwidth of 3.2 Gb/s. Sampling rates exceeding 27 MHz can be handled after demultiplexing. Versatile interactive support tools assist in mapping a video algorithm onto the VSP hardware. A description is given of the interactive mapping tools, comprising a signal-flow-graph editor, a hardware editor, a partitioner and a scheduler, all provided with a graphical interface.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A microprogrammable real-time video signal processor (VSP) for motion compensation

A video signal processor (VSP) LSI circuit with a three pipelined architecture has been developed for pattern matching, which is fundamental for the motion compensation necessary for teleconferencing systems. A high-speed arithmetic logic unit with absolute-value calculation capability and a minimum/maximum value detector, which are essential to pattern matching, have been integrated on the VSP LSI. The chip was fabricated with a 2.5- mu m CMOS and double-layer metallization technology. The number of MOSFETs integrated on the 9.91*9.50-mm/sup 2/ chip is about 48000. It operates at a 14.3-MHz clock frequency with a single 5-V power supply and typically consumes 240 mW. An experimental video signal processing system, using a single VSP LSI chip, is discussed. >