Elementary Streams Research Articles

Hint file-based implementation of contents navigation methods for set-top box

In this paper, a hint file scheme for contents navigation algorithms specialized for set-top box client is introduced. As a hint file contains overall information of video/audio properties, video frame, and GOP structure, a server can do streaming without parsing elementary streams. Especially, as our scheme aims at the set-top box, a hint file additionally contains information needed for set-top box requests so that a server can process the requests efficiently. This promises great advantages for service providers for set-top box-based systems.

Efficient Generation of Scalable Transport Stream for High Quality Service in T-DMB

We introduce an advanced terrestrial digital multimedia broadcasting (AT-DMB) system that overcomes the limitation of data transmission rates of T-DMB by doubling it with the same frequency bandwidth. In this letter, we propose an efficient algorithm which generates a scalable transport stream in AT-DMB by multiplexing certain types of elementary streams encoded using scalable video coding and an MPEG- surround audio coder for high-quality multimedia services.

An Embedded and Programmable System Based FPGA for Real Time MPEG Stream Buffer Analysis

The MPEG transport stream is an extremely complex structure using interlinked tables and coded identifiers to separate the programs and the elementary streams. Quality management is therefore a complicated issue and the need to identify the degree of coding degradations in terms of coding and/or transmission errors or system failures is becoming an important criterion for the evaluation of the quality of the MPEG streams. A theoretical decoder (T-STD) defines the verification process based on the proper fill level of an MPEG decoder buffers whose size is defined by the standards in order to obtain an evaluation of the MPEG stream quality. This paper describes a new embedded and programmable solution capable of analysing MPEG streams in real time. The proposed hardware architecture provides a real time continuous buffer analysis of the MPEG stream components and is composed of several modules allowing for simultaneous modeling of the various buffers of the T-STD components (video, audio or system). Real time errors flags are generated when the buffers filling level becomes illegal (overflow, empty buffer, transfer delay, etc.). The architecture has been modeled, validated and simulated using the SystemC and VHDL languages in combination with real MPEG DVB-T streams. A VHDL synthesisable model of our architecture allows an implementation on an field-programmable gate array circuit based on Altera APEX20K1000. The hardware implementation of this configurable T-STD allows a data rate of 232 Mbps and requires only 9738 logical cells and 4,7 kB memory.

A multiplexing algorithm of multiple elementary streams based on virtual buffer control

The paper presents a prototype of virtual decoder of the transport stream's system target decoder (T-STD). By connecting the coding model and decoding model, and feeding the overflow of decoding buffer back to control coding, we have got a self-adaptive coding model, and propose an algorithm of multiplexing multiple elementary streams to a transport stream based on the principle of virtual buffer controlling strategy. The transport stream (TS) which uses this method passes the test of software unzipping and set-top-box (STB) playing, and all of the analyzing parameters which are detected by code analyzer accord with the standard of MPEG-2. Some problems that playing time becomes longer and multiple TS streaming can not be fit for all the players are also analyzed.

MPEG-2 real-time software CODEC for full-duplex transmission application over IP networks

This paper proposes a real-time MPEG-2 software CODEC for full-duplex transmission applications, and evaluates its performance and usefulness. The CODEC consists of a high-speed encodersdecoder, an IP sendersreceiver, and an error recovery controller. Each encodersdecoder is accelerated and optimized by exploiting fast algorithms and instruction-level parallelism. The IP sendersreceiver combination achieves low delay owing to the direct translating of each elementary stream of video and audio into UDPsIP packets. The error recovery controller carries out simple but powerful error tolerance against packet loss over IP networks. This CODEC attains low delay of 99 ms (M = 1, N = 1) to 165 ms (M = 3, N = 3) including input, encoding, transmitting, decoding, and output delays, and maintains a normal frame rate of 30 fps (frames per second) and more than 20 fps even under a fairly heavy network load. It provides sufficiently good performance for use as a real-time full NTSC-size CODEC on a PC of at least 1.2-GHz CPU. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(2): 33–41, 2005; Published online in Wiley InterScience (). DOI 10.1002sscj.20151

MPEG-2 real-time software CODEC for full-duplex transmission application over IP networks

This paper proposes a real-time MPEG-2 software CODEC for full-duplex transmission applications, and evaluates its performance and usefulness. The CODEC consists of a high-speed encoder/decoder, an IP sender/receiver, and an error recovery controller. Each encoder/decoder is accelerated and optimized by exploiting fast algorithms and instruction-level parallelism. The IP sender/receiver combination achieves low delay owing to the direct translating of each elementary stream of video and audio into UDP/IP packets. The error recovery controller carries out simple but powerful error tolerance against packet loss over IP networks. This CODEC attains low delay of 99 ms (M = 1, N = 1) to 165 ms (M = 3, N = 3) including input, encoding, transmitting, decoding, and output delays, and maintains a normal frame rate of 30 fps (frames per second) and more than 20 fps even under a fairly heavy network load. It provides sufficiently good performance for use as a real-time full NTSC-size CODEC on a PC of at least 1.2-GHz CPU. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(2): 33–41, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20151

Efficient Algorithms for Computation of the Burstiness Curve of Video Sources

The burstiness of a video source can be characterized by its burstiness curve. The burstiness curve is useful in the optimal allocation of resources to satisfy a desired quality of service for the video stream in a packet network. In this paper, we present deterministic algorithms for exact computation of the burstiness curve of a video source, for both elementary video streams and MPEG-2 Transport Streams. The algorithms exploit the piecewise linearity of the burstiness curve and compute only the points at which the slope of the burstiness curve changes. We also present approximate versions of these algorithms, which save computational effort by considering only a small number of candidate points at which the slope of the burstiness curve may change. The approximate algorithm was able to compute the burstiness curve of a 2-h long elementary video stream in approximately 10 s, as compared to over 6 h for the exact algorithm, with virtually no loss of accuracy in the computation. The efficiency of the proposed algorithms makes them suitable for quality-of-service (QoS) provisioning not only in off-line environments such as in video-on-demand (VoD) servers, but also in real-time applications such as in live TV distribution systems.

Streaming multimedia over the internet

Media streaming has emerged as an important service offered over the Internet. In response, the ISO/IEC MPEG-4 multimedia standard has been introduced. It encompasses a wide range of tools and technologies for delivering multimedia content. The MPEG-4 standard does not focus only on media compression, it also considers the media packaging and delivery components. One of the most interesting features MPEG-4 is that it is an object-based multimedia technology. An MPEG-4 presentation is comprised of units of aural, visual or audiovisual content, called media objects. Data access in MPEG-4 is always viewed as delivering, storing or accessing elementary streams. MPEG-4 has dedicated a part of the standard to delivery issues by defining a generic platform for the delivery of multimedia information. This platform is called the delivery multimedia integration framework (DMIF). The article gives an overview of DMIF, and describes its communication architecture and the DMIF default signaling protocol (DDSP). The authors also describe their contribution to producing Implementation 1 (IM1), the reference software for MPEG-4. They have contributed the DMIF remote retrieval instance and also an MPEG-4 streaming server that fully supports DMIF.

Efficient algorithms for computation of the loss curve of video sources

The loss curve of a video source characterizes the loss rate of the video stream generated by the source as a function of the allocated buffer size for a given transmission rate. The loss curve is useful in the optimal allocation of resources when the video stream is transmitted over a packet network, so that the desired tradeoff can be reached among the loss rate, bandwidth and the buffer space to be allocated in the network. We present an algorithm for computation of the entire loss curve of an elementary video stream. In contrast to earlier algorithms which employ statistical approaches, our algorithm is deterministic and computes the exact loss curve of the video stream. The algorithm exploits the piecewise linearity of the loss curve and computes only the points at which the slope of the loss curve changes. We also present an extension of the algorithm to MPEG-2 transport streams. The efficiency of the algorithm is demonstrated by results from several example video streams. For example, the algorithm was able to compute the entire loss curve of a 2-h elementary video stream in approximately 11 s on a Sun Ultra-2 workstation.

Metadata driven adaptation in the ADMITS project

The ADMITS project ( Adaptation in Distributed Multimedia IT Systems) is building an experimental distributed multimedia system for investigations into adaptation, which we consider an increasingly important tool for multimedia systems. A number of possible adaptation entities (server, proxy, clients, routers) are being explored, different algorithms for media, component and application-level adaptations are being implemented and evaluated, and experimental data are being derived to gain insight into when, where and how to adapt, and how individual, distributed adaptation steps interoperate and interact with each other. In this paper the “adaptation-chain” of (MPEG-conforming) metadata based adaptation is described: from the creation stage at the server side, through its usage in the network (actually in a proxy), up to the consumption at the client. The metadata are used to steer the adaptation processes. MPEG-conformant metadata, the so-called variation descriptions, are introduced; an example of a complete MPEG-7 document describing temporal scaling of an MPEG-4 video is given. The meta-database designed to store the metadata is briefly discussed. We describe how the metadata can be extracted from MPEG-4 visual elementary streams and initial results from a temporal video scaling experiment are given. We further present how the metadata can be utilized by enhanced cache replacement algorithms in a proxy server in order to realize quality-based caching; experimental results using these algorithms are also given. Finally, an adaptive query and presentation interface to the meta- and media database is outlined.

Implementation of hdtv pes combiner based on horizontal six-block segmentation

We present the hardware design of the packetized elementary stream (PES) combiner in the third generation HDTV encoder in China, which is a key part in the HDTV encoding system. In our design, the input HDTV video signal is divided into six sub-images, and a horizontal six-block segmentation method is implemented in the HDTV encoder. Each of the sub-images is coded by one MP@ML encoding ASIC, which works at a different bitrate. The PES combiner combines all the output bit streams into one HDTV PES. The coding parameters and timing stamps are modified according to the requirements of MPEG-2 MP@HL. All these PES combiner functions are implemented by one complex programmable logic device (CPLD), which makes the whole encoding system compact and stable. The detailed discussions of hardware design are also presented in this paper. Experimental results show that the quality of decoded image is improved over the past two generation encoders, and constant bitrate can be maintained at the same time.

Seamless Audio Splicing for ISO/IEC 13818 Transport Streams

This paper addresses the processing of packetized audio elementary streams (ES) during the splicing of ISO/IEC 13818 (MPEG-2) transport streams (TS). An algorithm is developed that produces a continuous audio ES across the splicing point, avoids all noticeable audio artifacts, and hence provides a seamless audio splice without the need to decode and re-encode audio ESs. Perfect continuity with respect to the presentation time stamps (PTS) and careful management of the audio buffer level at all times lead to a glitch-free audio play-out for an unlimited number of splices. The proposed algorithm parses the packetized audio ES up to the audio access unit (AAU) level and is very simple to implement.

Making MPEG-2 Video Display Agnostic

While in theory the ISO/IEC 13818–2 specification only defines the bit stream syntax and decoding process, implicit assumptions in the specification and common encoding practices often adversely affect the display process. For example, encoding parameters are often set, based on the assumption that the video will be presented only on an interlace-scan device with a fixed refresh rate and fixed aspect ratio. This assumption can result in unsatisfactory presentation on “nontraditional” display devices, such as personal computer monitors. Recently, a proposed amendment to the specification was drafted that allows for supplemental information about the source content to be carried in the video elementary stream. This information does not affect the decoding process, but can be used to generate the most faithful presentation of content on different display devices. This paper describes the history, motivation, status, and intended application of this amendment.

An efficient multiplex architecture for mobile MPEG-4 systems

This paper describes a multiplex architecture for mobile multimedia applications which has been considered as the underlying multiplex system in the MPEG-4 standardization effort (MPEG-4: Coding of audio-visual objects, standardization project of the ISO/IEC JTC1/SC29/WG11). Multimedia information consists of several different kinds of information which differ widely in their requirements on Quality of Service (QoS). The generic data services in today's mobile networks are not designed to comply with these QoS requirements. On the other hand, future object-based multimedia systems (e.g. MPEG-4) will use a multitude of simultaneous data streams. Many of these will have similar requirements on transmission. The proposed multiplexing architecture consists of two layers. One layer serves to bundle elementary data streams of similar QoS requirements, resulting in a small number of data streams with differing QoS requirements. The other layer serves to provide appropriate error protection for each of these streams and to form a single data stream which can be transmitted over an error prone link. Thus multiplexing overhead is minimized while error protection is adapted to the requirements of the individual elementary streams. It is argued that the lower layers dealing with protection and the error-resilient multiplex of the protected streams should be included in the network in order to avoid the problems of end-to-end error control. The system described can thus help to obtain highly efficient multimedia communication with future mobile networks.

An Open Model for Audio Elementary Stream Encoding in a DTV Distribution Chain

This paper addresses issues of audio distribution for modern digital television (DTV) systems focusing on digital broadcast systems. In particular, a model is described for handling audio and video elementary streams in existing distribution networks. The model supports new system architectures that allow audio and video coding to take place at many points in the distribution chain. It covers both physical and logical interface formats, synchronization with associated video signals, and carriage of metadata associated with the audio. The model also demonstrates that many audio features of new digital delivery formats can be exploited by existing broadcast systems with minimal additional equipment investment. It makes use of existing AES and SMPTE standards; however, new standards and recommended practices to support the open model may be desired by the industry. Specific recommendations are discussed. In addition, examples of systems applying the model to DTV broadcast facilities are described, including examples of products that implement the audio coding function using the model.

Realizing the MPEG-4 multimedia delivery framework

MPEG-4 is an emerging international standard based on the encoding of audiovisual data using object description techniques. This novel approach to encoding allows MPEG-4-based multimedia applications to dynamically compose complex scenes from one or more elementary stream objects. These stream objects are carried within potentially hundreds of channels. As a result, the underlying transport system must support the rapid creation and release of such channels with QoS guarantees. In addition to defining the format, structure and rules of composition of these objects, MPEG-4 also specifies a general application and transport delivery framework called the delivery multimedia integration framework (DMIF). DMIF's main purpose is to hide the details of the transport network from the user, as well as to ensure signaling and transport interoperability between end-systems. We describe the first reference implementation of the MPEG-4 DMIF standard for the ISO, including a full specification of the DMIF application interface. The DAI offers the required functionality for realizing multimedia applications without any concerns regarding the selected communication protocol. The realization of DMIF is greatly facilitated by xbind, a broadband kernel that provides connection management, routing, QoS mapping, and intelligent selection of suitable transport protocol stacks. The resulting system, called XDMIF, supports different QoS requirements, and the rapid creation and release of numerous transport channels, along with efficient management of network resources.

Studying solute and particulate sediment transfer in a small Mediterranean mountainous catchment subject to land abandonment

The sediment budget of the small research catchment of Cal Parisa (Vallcebre, Eastern Pyrenees) was studied by hydrological monitoring and assessment of the erosion rates in the major sediment sources. This area is characterized by clayey mudrock prone to landsliding and badland erosion, but the catchment was selected in an area free of major badland features, as a representative of middle mountain regions where a system of terraces and drainage ditches had been built for agricultural use but is now abandoned. Streamwater chemistry is dominated by Ca 2+ and HCO 3 - at concentrations close to calcite saturation. Total dissolved solids show dilution during runoff peaks and positive hysteresis loops that support a slow contribution of subsurface water. Relative dissolved ion concentrations are different for each event analysed. Particulate sediment yield is very low and represents only about 1 per cent of gross erosion in the catchment. Mineralogical analysis of suspended sediments shows an enrichment in calcite because of precipitation. Chemical analysis of suspended sediments, using common one-litre water samples, shows higher contents of Ca, P and Mn in transported sediment than in sediment source areas, attributed to the precipitation of calcite, and enrichment in organic particulate matter during events respectively for the two first elements, whereas enrichment in Mn remains uncertain. Solid matter yield is therefore clearly dominated by dissolved transport as a result of both high calcium bicarbonate concentrations in runoff waters and strong suspended sediment conveyance discontinuities. Land conservation structures are very effective because they are in good condition whereas the soil is covered by dense permanent vegetation. Nevertheless, this state is unstable because the network of drainage ditches needs maintenance; its spontaneous breakdown after abandonment may result in the rearrangement of the elementary stream network and gullying of old fields in hollows. © 1997 John Wiley & Sons, Ltd.

Comparison of MPEG data transferring schemes in ATM networks

In this paper we consider MPEG2 (Moving Pictures Expert Group 2) data transferring schemes using AAL5 (ATM Adaptation Layer 5) connections through which the PS (Program Stream) layer multiplexes PES (Packetized Elementary Stream) streams. We investigate the performance of the multiplexing schemes PS and TS (Transport Stream) in terms of the transfer efficiency. The PS transfer scheme is generally considered for reliable networks whereas the TS transfer scheme is proposed for error-prone networks. In this paper we propose an appropriate criterion that can be used for a decision on which method performs well under which environments for MPEG2 stream transfer. The QoS (Quality of Service) issues concerning the packet length and packet loss are also discussed.

Development of Columbia's video on demand testbed

Abstract This paper describes our progress in developing an advanced video-on-demand (VOD) testbed, which will accommodate various multimedia research and applications such as Electronic News on Demand, Columbia's Video Course Network, and Digital Libraries. Two different prototypes have been completed. The first generation of the testbed was based on a constant-bit-rate (CBR) video server utilizing Ethernet delivery. Contents were encoded and stored as MPEG-2 audio/video elementary streams. Software encoders/decoders were used in content generation and playback. The second generation of the testbed was enhanced with the capability of transmitting true MPEG-2 transport streams over the campus ATM network as well as the wide-area NYNET ATM network. A real-time video pump and a distributed application control protocol (MPEG-2's DSM-CC) have been incorporated. Hardware decoders and set-tops are being incorporated to test wide-area video interoperability. Our VOD testbed also provides an advanced platform for implementing proof-of-concept prototypes of related research. Our current research focus covers video transmission with heterogeneous quality-of-service (QoS) provision, video storage architecture design, content-based video indexing and browsing, multi-resolution (MR) video coding, efficient manipulation of compressed video, and advanced user interfaces. An important aim is to enhance interoperability. Accommodation of practical multimedia applications and interoperability testing with external VOD systems are currently being undertaken.