Content-based Functionalities Research Articles

Object coding based video authentication for privacy protection in immersive communication

Video immersive communication has become a new trend in human-interaction technologies by the information exchanging, which requires the object based coding to provide very low bit rate compression for transmission as well as to protect the privacy of the users. In addition, advanced content-based functionalities in ambient intelligence such as encoder/decoder selection and manipulation of specific objects in a video stream also need the support from object based coding. Unfortunately, due to a malicious entity of modification/replacement/removement of the individual foreground objects and background in the video, effective video authenticity protection is still challenging in some realistic cases. In this paper, we propose an object based coding authentication strategy based on a Chinese remainder theorem for video authenticity protection during transmission. With the watermark generation for single frame and content scalable video coding, we performed the proposed authenticity verification on both original video and sub-video. Based on the efficiency evaluation on video transmission, the proposed approach ensures an applicable authenticity between the foreground objects and their associated background for video immersive applications.

A framework for examining characteristics of computer-based mathematical tasks

Research has shown that affordances of computers may be exploited in the design of mathematical tasks so as to provide interesting and challenging activities. At the same time, opportunities for learning mathematics may be constrained if the design of the task is not appropriate. A tool which allows teachers, researchers and task designers to isolate, a priori, the possibilities and limitations of computer-based tasks is thus desirable. In this paper I develop such a tool in the form of a framework. Key components of this framework are the mathematical focus of the task, its cognitive demands, its technical demands and the affordances of the technology that it exploits. I illustrate the use of this framework by analysing three tasks that were used in a content-based functions course for in-service teachers. This analysis illuminates important pedagogical aspects of these tasks. It also supports the notion that the use of technology in a task may afford the design of high-order thinking activities.

A middleware to enhance current multimedia retrieval systems with content-based functionalities

Nowadays the retrieval of multimedia assets is mainly performed by text-based retrieval systems with powerful and stable indexing mechanisms. Migration from those systems to content-aware multimedia retrieval systems is a common aim for companies from very diverse sectors. In this paper we present a semantic middleware designed to achieve a seamless integration with existing systems. This middleware outsources the semantic functionalities (e.g. knowledge extraction, semantic query expansion,?) that are not covered by traditional systems, thereby allowing the use of complementary content-based techniques. We include a list of key criteria to successfully deploy this middleware, which provides semantic support to many different steps of the retrieval process. Both the middleware and the design criteria are validated by two real complementary deployments in two very different industrial domains.

Enhanced Frame-based Video Coding to Support Content-based Functionalities

This paper presents the enhanced frame-based video coding scheme that enables some useful content-based functionalities such as user-interactivity with video content, assignment of different quality to video objects, and content-based video search and retrieval. The input video source for the proposed enhanced frame-based video encoder consists of rectangular-size video frames and shapes of arbitrarily-shaped video objects in the frames. The rectangular frames are encoded using a scheme similar to the conventional frame-based video coding and the shapes of video objects are encoded using the contour-based vertex coding. It is possible to achieve several useful content-based functionalities by utilizing the shape information in the bitstream at the cost of a very small overhead of shape bits.

ENHANCED FRAME-BASED VIDEO CODING TO SUPPORT CONTENT-BASED FUNCTIONALITIES

This paper presents the enhanced frame-based video coding scheme. The input source video to the enhanced frame-based video encoder consists of a rectangular-sized video and shapes of arbitrarily shaped objects on video frames. The rectangular frame texture is encoded by the conventional frame-based coding technique and the video object's shape is encoded using the contour-based vertex coding. It is possible to achieve several useful content-based functionalities by utilizing the shape information in the bitstream at the cost of a very small overhead to the bit-rate.

An automatic segmentation method for moving objects based on the spatial-temporal information of video

The new MPEG-4 video coding standard enables content-based functions. In order to support the new standard, frames should be decomposed into Video Object Planes (VOP), each VOP representing a moving object. This paper proposes an image segmentation method to separate moving objects from image sequences. The proposed method utilizes the spatial-temporal information. Spatial segmentation is applied to divide each image into connected areas and to find precise object boundaries of moving objects. To locate moving objects in image sequences, two consecutive image frames in the temporal direction are examined and a hypothesis testing is performed with Neyman-Pearson criterion. Spatial segmentation produces a spatial segmentation mask, and temporal segmentation yields a change detection mask that indicates moving objects and the background. Then spatial-temporal merging can be used to get the final results. This method has been tested on several images. Experimental results show that this segmentation method is efficient.

Modified intelligent scissors and adaptive frame skipping for video object segmentation

MPEG-4 introduces the concept of video object to support content-based functionalities. Video object segmentation is a crucial step for object-based coding and manipulation. In this paper, a robust semi- automatic video object segmentation scheme is proposed. To efficiently and accurately define the initial object contour, modified intelligent scissors is proposed on the basis of original intelligent scissors. It can improve about 6–8 times the processing speed with only slight sacrifice of accuracy, which just meets the requirements of initial object extraction for semi-automatic approach. To avoid errors accumulating and propagating during object tracking, an adaptive frame skipping scheme is proposed to decompose video sequence into video clips. For rigid and non-rigid video objects, two different image segmentation algorithms are utilized, and then region-based backward projection technique is adopted to interpolate the video object plane (VOPs) of other frames within every video clip. The proposed approach can cope with occlusion/disocclusion problem to most extent. Experimental results demonstrate the effectiveness and robustness of the method.

Region-Based Image Retrieval Using an Object Ontology and Relevance Feedback

An image retrieval methodology suited for search in large collections of heterogeneous images is presented. The proposed approach employs a fully unsupervised segmentation algorithm to divide images into regions and endow the indexing and retrieval system with content-based functionalities. Low-level descriptors for the color, position, size, and shape of each region are subsequently extracted. These arithmetic descriptors are automatically associated with appropriate qualitative intermediate-level descriptors, which form a simple vocabulary termed object ontology. The object ontology is used to allow the qualitative definition of the high-level concepts the user queries for (semantic objects, each represented by a keyword) and their relations in a human-centered fashion. When querying for a specific semantic object (or objects), the intermediate-level descriptor values associated with both the semantic object and all image regions in the collection are initially compared, resulting in the rejection of most image regions as irrelevant. Following that, a relevance feedback mechanism, based on support vector machines and using the low-level descriptors, is invoked to rank the remaining potentially relevant image regions and produce the final query results. Experimental results and comparisons demonstrate, in practice, the effectiveness of our approach.

Automatic Moving Object Extraction for Content-Based Applications

Rapid developments in the Internet and multimedia applications allow us to access large amounts of image and video data. While significant progress has been made in digital data compression, content-based functionalities are still quite limited. Many existing techniques in content-based retrieval are based on global visual features extracted from the entire image. In order to provide more efficient content-based functionalities for video applications, it is necessary to extract meaningful video objects from scenes to enable object-based representation of video content. Object-based representation is also introduced by MPEG-4 to enable content-based functionality and high coding efficiency. In this paper, we propose a new algorithm that automatically extracts meaningful video objects from video sequences. The algorithm begins with the robust motion segmentation on the first two successive frames. To detect moving objects, segmented regions are grouped together according to their spatial similarity. A binary object model for each moving object is automatically derived and tracked in subsequent frames using the generalized Hausdorff distance. The object model is updated for each frame to accommodate for complex motions and shape changes of the object. Experimental results using different types of video sequences are presented to demonstrate the efficiency and accuracy of our proposed algorithm.

A fast automatic VOP generation using boundary block segmentation

With the increase of multimedia applications and content-based functionalities, efficient video coding methods are necessary. The moving picture experts group-4 (MPEG-4) provides content-based functionalities by introducing the concept of the video object plane (VOP). Each frame of the input sequence is segmented into a number of arbitrarily shaped image regions or VOP's so that each VOP describes a semantically meaningful object or video content of interest. For real-time applications of MPEG-4, a fast automatic object segmentation method of video sequences is needed. We propose a fast automatic VOP generation algorithm composed of two parts: object block segmentation and boundary block segmentation. The former defines block-based object regions in a frame and the latter generates the pixel-based object mask. Block-based object region search and restriction of segmentation regions to boundary blocks reduce a computational load. Experimental results with two test sequences show the effectiveness of the proposed algorithm in terms of the visual quality of segmentation results and the computation time.

Video object segmentation for head–shoulder sequences in the cellular neural networks architecture

MPEG-4 introduces the concept of video object to support content-based functionalities. Video object segmentation is a key step in defining the contents of any video sequences. Head–shoulder sequence (HSS) is typical in video conferencing and surveillance systems, in which real-time performance is required. Since background information can be obtained in advance and pre-stored, video segmentation for HSS can use background information a priori. To avoid the critical selection of threshold for gradient-based method, and to overcome the insufficiency of monochrome intensity-based change detection, an efficient color edge-based change detection scheme (CECD) is utilized in this paper. In order to meet the real-time performance for HSS, it is implemented in the cellular neural networks (CNN) architecture. The algorithm is mainly based on 3 by 3, linear templates. Because of CNN's high parallelism and computational abilities, real-time performance is achieved. Experimental results on several test sequences show the robustness of this approach. It can achieve better spatial accuracy and temporal coherency than COST211 AM.

Performance evaluatio of MPEG-4 visual error resilient tools over a mobile channel

MPEG-4 is the most recent standard for audiovisual representation published by International Organization for Standardization, targeted for future interactive multimedia video communications calling for content-based functionalities, universal access in error prone environments and high coding efficiency. Besides the provisions for content-based functionalities, the MPEG-4 video coding standard will assist the efficient storage and transmission of very low bit rate video in error prone environments. This makes MPEG-4 a good candidate for video coding standard for Terrestrial Trunked Radio Standard (TETRA) video transmission. This paper presents the overall performance of a communication system used to transmit MPEG-4 encoded video over TETRA channels. The video quality that can be achieved at different channel conditions employing different combination of MPEG-4 built-in error resilient tools are presented in this paper.

VLSI architecture design of MPEG-4 shape coding

This paper presents an efficient VLSI architecture design of MPEG-4 shape coding, which is the key technology for supporting the content-based functionality of the MPEG-4 video standard. The real-time constraint of MPEG-4 shape coding leads to a heavy computational bottleneck on today's computer architectures. To overcome this problem, design analysis and optimization of MPEG-4 shape coding are addressed. By utilizing the RISC-based model, computational behaviors of the MPEG-4 shape coding tool are carefully examined and analyzed. The characteristic of a large amount of bit-level data processing and data transfer of MPEG-4 shape coding motivates the optimization of bit-level data operations. Applying data-flow optimization and data reuse techniques, bit-level computation-efficient architectures, such as data-dispatch-based binary-shaped motion estimation, the delay-line model, and configurable context-based arithmetic coding, are designed to accelerate bit-level processing. These hardware blocks are integrated and scheduled in a very efficient data flow to achieve real-time performance for MPEG-4 CPL2 (core profile level 2) specification at 23.5 MHz clock rate. The system architecture is implemented using Verilog HDL and synthesized with a 0.35 /spl mu/m four-layer CMOS standard library.

Automatic segmentation of moving objects in video sequences: a region labeling approach

The emerging video coding standard MPEG-4 enables various content-based functionalities for multimedia applications. To support such functionalities, as well as to improve coding efficiency, MPEG-4 relies on a decomposition of each frame of an image sequence into video object planes (VOP). Each VOP corresponds to a single moving object in the scene. This paper presents a new method for automatic segmentation of moving objects in image sequences for VOP extraction. We formulate the problem as graph labeling over a region adjacency graph (RAG), based on motion information. The label field is modeled as a Markov random field (MRF). An initial spatial partition of each frame is obtained by a fast, floating-point based implementation of the watershed algorithm. The motion of each region is estimated by hierarchical region matching. To avoid inaccuracies in occlusion areas, a novel motion validation scheme is presented. A dynamic memory, based on object tracking, is incorporated into the segmentation process to maintain temporal coherence of the segmentation. Finally, a labeling is obtained by maximization of the a posteriori probability of the MRF using motion information, spatial information and the memory. The optimization is carried out by highest confidence first (HCF). Experimental results for several video sequences demonstrate the effectiveness of the proposed approach.

Fast and automatic video object segmentation and tracking for content-based applications

The new video-coding standard MPEG-4 enables content-based functionality, as well as high coding efficiency, by taking into account shape information of moving objects. A novel algorithm for segmentation of moving objects in video sequences and extraction of video object planes (VOPs) is proposed . For the case of multiple video objects in a scene, the extraction of a specific single video object (VO) based on connected components analysis and smoothness of VO displacement in successive frames is also discussed. Our algorithm begins with a robust double-edge map derived from the difference between two successive frames. After removing edge points which belong to the previous frame, the remaining edge map, moving edge (ME), is used to extract the VOP. The proposed algorithm is evaluated on an indoor sequence captured by a low-end camera as well as MPEG-4 test sequences and produces promising results.

Multiple feature clustering for image sequence segmentation

We present a segmentation scheme in order to develop a method which can identify homogeneous regions to represent higher level objects for content-based functionality. The proposed scheme extracts multiple features, such as motion and texture, on the pixel basis. Different weights are applied to each feature components based on motion confidence measures. The proposed scheme consists of two phases. In the first phase, a multiple feature space is transformed to one-dimensional label space using a self-organizing feature maps (SOFM) neural network clustering method. In the second phase, the neural network outputs are merged in order to generate desired segmentation resolution. Our experimental results and performance analysis show the validity of the proposed scheme.

Photograph retrieval and classification by visual keywords and thesaurus

As we collect more digital images with the advent of digital cameras, we need effective content-based search and categorization functions. In this paper, we propose a novel notion of visual keywords to describe and compare digital visual contents. Visual keywords are visual prototypes extracted from a visual content domain with semantics labels. They can be further abstracted to form visual thesaurus. An image is indexed as a spatial distribution of visual keywords. Both retrieval and classification evaluation tasks on professional natural scene photographs have demonstrated the usefulness of this new methodology.

Rate-scalable object-based wavelet codec with implicit shape coding

In this paper, we present an embedded approach for coding image regions with arbitrary shapes. Our scheme takes a different approach by separating the objects in the transform domain instead of the image domain so that only one transform for the entire image is required. We define a new shape-adaptive embedded zerotree wavelet coding (SA-EZW) technique for encoding the coefficients corresponding to specific objects in gray-scale and color-image segments by implicitly representing their shapes, thereby forgoing the need for separately coding the region boundary. At our decoder, the shape information can be recovered without separate and explicit shape coding. The implicit shape coding enables the bit stream for the object to be fully rate scalable, since no explicit bit allocation is needed for the object shape. This makes it particularly suitable when content-based functionalities are desired in situations where the user bit rate is constrained and enables precise bit-rate control while avoiding the problem of contour coding. We show that our algorithm sufficiently addresses the issue of content-based scalability and improved coding efficiency when compared with the "chroma keying" technique, an implicit shape-coding technique which is adopted by the current MPEG-4 standard.

Automatic video sequence segmentation using object tracking

Emerging video standards address content-based functionalities and interactive multimedia. MPEG-4 segments the frames into video object planes (VOP) with each VOP corresponding to one semantically meaningful object. In this paper a new automatic video sequence segmentation algorithm is presented that can extract such VOPs. A binary model for moving objects is automatically derived and tracked in subsequent frames using the generalised Hausdorff distance. The model is updated every frame to accommodate for rotation and changes in shape. For that an update mechanism consisting of two components is proposed. To improve the segmentation and reduce the computational complexity the stationary background is filtered by a novel technique. Experimental results demonstrate the performance of our proposed algorithm.

A VOP generation tool: automatic segmentation of moving objects in image sequences based on spatio-temporal information

The new MPEG-4 video coding standard enables content-based functionalities. In order to support the philosophy of the MPEG-4 visual standard, each frame of video sequences should be represented in terms of video object planes (VOPs). In other words, video objects to be encoded in still pictures or video sequences should be prepared before the encoding process starts. Therefore, it requires a prior decomposition of sequences into VOPs so that each VOP represents a moving object. This paper addresses an image segmentation method for separating moving objects from the background in image sequences. The proposed method utilizes the following spatio-temporal information. (1) For localization of moving objects in the image sequence, two consecutive image frames in the temporal direction are examined and a hypothesis testing is performed by comparing two variance estimates from two consecutive difference images, which results in an F-test. (2) Spatial segmentation is performed to divide each image into semantic regions and to find precise object boundaries of the moving objects. The temporal segmentation yields a change detection mask that indicates moving areas (foreground) and nonmoving areas (background), and spatial segmentation produces spatial segmentation masks. A combination of the spatial and temporal segmentation masks produces VOPs faithfully. This paper presents various experimental results.