Video Object Plane Research Articles

Discrete Firefly Algorithm for Optimizing Topology Generation and Core Mapping of Network-on-Chip

Network-on-chip (NoC) proves to be the best alternative to replace the traditional bus-based interconnection in Multi-Processor System on a Chip (MPSoCs). Irregular NoC topologies are highly recommended and utilised in various applications as they are application specific. Optimized mapping of the cores in a NoC plays a major role in its performance. Firefly algorithm is a bio-inspired meta-heuristic approach. Discretized firefly algorithm is used in our proposed work. In this work, two optimization algorithms are proposed: Topology Generation using Discrete Firefly Algorithm (TGDFA) and Core Mapping using Discrete Firefly Algorithm (CMDFA) for multimedia benchmark applications, Video Object Plane Decoder (VOPD), Multimedia Window Display (MWD) and MP3 Encoder. The irregular topology generated using TGDFA is mapped using CMDFA onto a reconfigurable mesh with switches in between the routers to make it a fault tolerant one. The proposed TGDFA provides better optimization of communication cost. The speed-up on the average run time of the tasks and the time taken to attain the best solution by the proposed TGDFA is significant. The dynamic energy consumption of core mapping obtained by the proposed CMDFA is lesser when compared to the existing work. Finally, the optimized core mapping is implemented in a cycle accurate NoC simulator: Noxim. It is substantiated experimentally that the proposed CMDFA outperforms the previous work.

Entropy feature and peak-means clustering based slowly moving object detection in head and shoulder video sequences

With the increase in demand for video conferencing and IOT applications, efficient video coding standards are necessary. The performance of MPEG-4 coding scheme depends on the efficiency of the video object plane (VOP) generation methods. In head and shoulder video, such as news reading, video conferencing video sequences, the object has a very little movement in between two consecutive frames. Therefore, traditional segmentation methods could not extract the complete VOP efficiently. In this paper, we propose an efficient spatiotemporal segmentation method to extract the moving object for the generation of VOP in head and shoulder video sequences. First, a motion map of the object at each frame is generated based on the entropy of the temporal change of each pixel. Secondly, each frame is spatially segmented based on peak-means clustering approach. Finally, both motion map and spatial segmentation information are fused to extract the complete shape of the slow moving object. Experimental outcome depicts that the proposed method has highest detection accuracy with average intersection of union (IOU) score of 94.32% per frame and F1 measure of 97.75% per frame.

Fault-Tolerant Application-Specific Topology-Based NoC and Its Prototype on an FPGA

Application-Specific Networks-on-Chips (ASNoCs) are suitable communication platforms for meeting current application requirements. Interconnection links are the primary components involved in communication between the cores of an ASNoC design. The integration density in ASNoC increases with continuous scaling down of the transistor size. Excessive integration density in ASNoC can result in the formation of thermal hotspots, which can cause a system to fail permanently. As a result, fault-tolerant techniques are required to address the permanent faults in interconnection links of an ASNoC design. By taking into account link faults in the topology, this paper introduces a fault-tolerant application-specific topology-based NoC design and its prototype on an FPGA. To place spare links in the ASNoC topology, a meta-heuristic algorithm based on Particle Swarm Optimization (PSO) is proposed. By taking link faults into account in ASNoC design, we also propose an application mapping heuristic and a table-based fault-tolerant routing algorithm. Experiments are carried out for a specific link and any link fault in fault-tolerant topologies generated by our approach and approaches reported in the literature. For the experimentation, we used the multi-media applications Picture-in-Picture (PiP), Moving Pictures Expert Group (MPEG) - 4, MP3Encoder, and Video Object Plane Decoder (VOPD). Experiments are run on software and hardware platforms. The static performance metric communication cost and the dynamic performance metrics network latency, throughput, and router power consumption are examined using software platform. In the hardware platform, the Field Programmable Gate Array (FPGA) is used to validate proposed fault-tolerant topologies and analyze performance metrics such as application runtime, resource utilization, and power consumption. The results are compared with the existing approaches, specifically Ring topology and its modified versions on both software and hardware platforms. The experimental results obtained from software and hardware platforms for a specific link and any link fault show significant improvements in performance metrics using our approach when compared with the related works in the literature.

Delay Bound Optimization in NoC Using a Discrete Firefly Algorithm

The delay bound in system on chips (SoC) represents the worst-case traverse time of on-chip communication. In network on chip (NoC)-based SoC, optimizing the delay bound is challenging due to two aspects: (1) the delay bound is hard to obtain by traditional methods such as simulation; (2) the delay bound changes with the different application mappings. In this paper, we propose a delay bound optimization method using discrete firefly optimization algorithms (DBFA). First, we present a formal analytical delay bound model based on network calculus for both unipath and multipath routing in NoCs. We then set every flow in the application as the target flow and calculate the delay bound using the proposed model. Finally, we adopt firefly algorithm (FA) as the optimization method for minimizing the delay bound. We used industry patterns (video object plane decoder (VOPD), multiwindow display (MWD), etc.) to verify the effectiveness of delay bound optimization method. Experiments show that the proposed method is both effective and reliable, with a maximum optimization of 42.86%.

Bandwidth-Constrained Multi-Objective Segmented Brute-Force Algorithm for Efficient Mapping of Embedded Applications on NoC Architecture

Network-on-chip (NoC) is an emerging alternative to address the communication problem in embedded system-on-chip designs. One of the key and major issues is the optimized mapping of the embedded applications on the underlined NoC platform. In this paper, we propose the bandwidth-constrained multi-objective segmented brute-force mapping (SBMAP) algorithm, which minimizes the communication energy consumption and reduces the computational complexity of the NoC designs. The algorithm generates efficient mapping of the embedded applications on the processing elements of the NoC system by segregating the application into multiple segments. It utilizes the property of modular systematic search, which produces high-performance results with optimized simulation time. We compared the SBMAP algorithm with the state-of-the-art mapping techniques, such as branch and bound (BB), near-optimal mapping (NMAP), and random mapping algorithms for mapping real-world application workloads. The experimental results validated the efficiency of the proposed algorithm against its competitors for most of the performance parameters of the NoC designs. The improvement in energy consumption of the SBMAP algorithm is up to 53% for 2-D mesh and 62% for torus topology as compared with the NMAP, BB, and random algorithms for video object plane decoder, picture in picture, Wi-Fi receiver, and multimedia system real-time application benchmarks.

Comparative analysis of network‐on‐chip simulation tools

Network-on-chip (NoC) is a reliable and scalable communication paradigm deemed as an alternative to classic bus systems in modern systems-on-chip designs. Consequently, one can observe extensive multidimensional research related to the design and implementation of NoC-based systems. A basic requirement for most of these activities is the availability of NoC simulators that enable the study and comparison of different technologies. This study targets the analysis of different NoC simulators and highlights its contributions towards NoC research. Various NoC tools such as NoCTweak, Noxim, Nirgam, Nostrum, BookSim, WormSim, NOCMAP and ORION are evaluated and their strengths and weaknesses are highlighted. The comparative analysis includes methods for estimation of latency, throughput and energy consumption. Further, the exemplary real world application, video object plane decoder is mapped on a 2D mesh NoC using different mapping algorithms under NOCMAP and NoCTweak simulators for comparative analysis of the NoC simulators and their embedded mapping algorithms.

Moving object detection using spatio-temporal multilayer compound Markov Random Field and histogram thresholding based change detection

In this article, we propose a Multi Layer Compound Markov Random Field (MLCMRF) Model to spatially segment different image frames of a given video sequence. The segmented image frames are combined with the change between the frames to detect the moving objects from a video. The proposed MLCMRF uses five Markov models in a single framework, one in spatial direction using color feature, four in temporal direction (using two color features and two edges/line fields). Hence, the proposed MLCMRF is a combination of spatial distribution of color, temporal color coherence and edge maps in the temporal frames. The use of such an edge preserving model helps in enhancing the object boundary in spatial segmentation and hence can detect moving objects with less effect of silhouette. A difference between the frames is used to generate the CDM and is subsequently updated with the previous frame video object plane (VOP) and the spatial segmentation of the consecutive frames, to detect the moving objects from the target image frames. Results of the proposed spatial segmentation approach are compared with those of the existing state-of-the-art techniques and are found to be better.

Comparing key frame selection for one-two hand gesture recognition using different methods

Hand gesture recognition system has gained potential importance in application areas of human-computer interaction, machine vision, etc. Vision-based hand gesture recognition involves visual analysis of hand shape, position and/or motion. Finite State Machine (FSM) and Dynamic Time Warping (DTW) are most prominent methods used for gesture recognition. In this paper, we use object-based key frame selection from video sequence for segmenting Video Object Plane (VOP). Each VOP is a meaningful hand position, where Hausdorff and Euclidean distance are used for shape similarity. The key VOPs are selected on the basis of hand shape changes significantly and subsequently. The core approach of proposed work is that the frames of gesture video sequence are used with FSM and non-linear time alignment methods with key frame selection facility and trajectory features. Experimental results show accuracy and effectiveness of proposed system, for one-handed American and two-handed British sign language gesture recognition.

Weighted Round Robin Configuration for Worst-Case Delay Optimization in Network-on-Chip

We propose an approach for computing the end-to-end delay bound of individual variable bit-rate flows in an First Input First Output multiplexer with aggregate scheduling under weighted round robin (WRR) policy. To this end, we use a network calculus to derive per-flow end-to-end equivalent service curves employed for computing least upper delay bounds (LUDBs) of the individual flows. Since the real-time applications are going to meet guaranteed services with lower delay bounds, we optimize the weights in WRR policy to minimize the LUDBs while satisfying the performance constraints. We formulate two constrained delay optimization problems, namely, minimize-delay and multiobjective optimization. Multiobjective optimization has both the total delay bounds and their variance as the minimization objectives. The proposed optimizations are solved using a genetic algorithm. A video object plane decoder case study exhibits a 15.4% reduction of the total worst case delays and a 40.3% reduction on the variance of delays when compared with round robin policy. The optimization algorithm has low run-time complexity, enabling quick exploration of the large design spaces. We conclude that an appropriate weight allocation can be a valuable instrument for the delay optimization in on-chip network designs.

Hand pose recognition from monocular images by geometrical and texture analysis

One challenging research problem of hand pose recognition is the accurate detection of finger abduction and flexion with a single camera. The detection of flexion movements from a 2D image is difficult, because it involves estimation of finger movements along the optical axis of the camera (z direction). In this paper, a novel approach to hand pose recognition is proposed. We use the concept of object-based video abstraction for segmenting the frames into video object planes (VOPs), as used in MPEG-4, with each VOP corresponding to one semantically meaningful hand position. Subsequently, a particular hand pose is recognized by analyzing the key geometrical features and the textures of the hand. The abduction and adduction movements of the fingers are analyzed by considering a skeletal model. Probabilistic distributions of the geometric features are considered for modeling intra-class abduction and adduction variations. Additionally, gestures differing in flexion positions of the fingers are classified by texture analysis using homogeneous texture descriptors (HTD). Finally, hand poses are classified based on proximity measurement by considering the intra-class abduction and adduction and/or inter-class flexion variations. Experimental results show the efficacy of our proposed hand pose recognition system. The system achieved a 99% recognition rate for one-hand poses and a 97% recognition rate for two-hand poses.

Single change detection‐based moving object segmentation by using Daubechies complex wavelet transform

Research in motion analysis is a challenging field and it has a variety of video surveillance applications. For any video surveillance application, background detection and removal plays an important role in segmentation of the moving objects. This study proposes a new method for segmentation of the moving object, which is based on single change detection applied on Daubechies complex wavelet coefficients of two consecutive frames. The authors have chosen Daubechies complex wavelet transform as it is shift invariant and has a better directional selectivity as compared with real-valued wavelet transforms. Single change detection is a method to obtain video object plane by inter-frame difference of two consecutive frames, and it provides automatic detection of appearances of new objects. The proposed method does not require any other parameter except wavelet coefficients. Segmentation results of the moving objects after applying the proposed method are compared with those obtained after applying other spatial and wavelet domain segmentation methods in terms of visual performance and a number of quantitative measures viz misclassification penalty, relative position-based measure, structural content, normalised absolute error and average difference and the proposed method is found better than the other methods.

Super-resolution video generation algorithm for surveillance applications

Video surveillance is one of the major applications where high-resolution (HR) images are crucial. Since the video camera has limited spatial and temporal resolution, there is a need for super resolution video generation algorithms. In this paper, we have presented a novel technique for activity detection in the surveillance video. To achieve this goal, we have proposed and investigated efficient algorithms for Video Object Plane (VOP) generation, shadow removal from VOP and super-resolved VOP generation, for activity detection from surveillance video. The proposed VOP generation algorithm is computationally efficient and works for both dynamic and static backgrounds. The novel shadow removal algorithm for the VOP is based on texture and its performance has been studied based on average shadow detection and discrimination rates. The proposed super-resolution video generation algorithm has been designed using edge models. The performance of this algorithm has been evaluated using a numerical analysis technique and is found to be better than bi-cubic and bi-linear interpolation techniques.

Moving object detection using Markov Random Field and Distributed Differential Evolution

In this article, we present an algorithm for detecting moving objects from a given video sequence. Here, spatial and temporal segmentations are combined together to detect moving objects. In spatial segmentation, a multi-layer compound Markov Random Field (MRF) is used which models spatial, temporal, and edge attributes of image frames of a given video. Segmentation is viewed as a pixel labeling problem and is solved using the maximum a posteriori (MAP) probability estimation principle; i.e., segmentation is done by searching a labeled configuration that maximizes this probability. We have proposed using a Differential Evolution (DE) algorithm with neighborhood-based mutation (termed as Distributed Differential Evolution (DDE) algorithm) for estimating the MAP of the MRF model. A window is considered over the entire image lattice for mutation of each target vector of the DDE; thereby enhancing the speed of convergence. In case of temporal segmentation, the Change Detection Mask (CDM) is obtained by thresholding the absolute differences of the two consecutive spatially segmented image frames. The intensity/color values of the original pixels of the considered current frame are superimposed in the changed regions of the modified CDM to extract the Video Object Planes (VOPs). To test the effectiveness of the proposed algorithm, five reference and one real life video sequences are considered. Results of the proposed method are compared with four state of the art techniques and provide better spatial segmentation and better identification of the location of moving objects.

Moving object segmentation in Daubechies complex wavelet domain

Motion segmentation is a crucial step in video analysis and is associated with a number of computer vision applications. This paper introduces a new method for segmentation of moving object which is based on double change detection technique applied on Daubechies complex wavelet coefficients of three consecutive frames. Daubechies complex wavelet transform for segmentation of moving object has been chosen as it is approximate shift invariant and has a better directional selectivity as compared to real valued wavelet transform. Double change detection technique is used to obtain video object plane by inter-frame difference of three consecutive frames. Double change detection technique also provides automatic detection of appearance of new objects. The proposed method does not require any other parameter except Daubechies complex wavelet coefficients. Results of the proposed method for segmentation of moving objects are compared with results of other state-of-the-art methods in terms of visual performance and a number of quantitative performance metrics viz. Misclassification Penalty, Relative Foreground Area Measure, Pixel Classification Based Measure, Normalized Absolute Error, and Percentage of Correct Classification. The proposed method is found to have high degree of segmentation accuracy than the other state-of-the-art methods.

Controlling the Bit Rate of Multi-Object Videos With Noncooperative Game Theory

This paper proposes an object-level rate control algorithm to jointly controlling the bit rates of multiple video objects. Utilizing noncooperative game theory, the proposed rate control algorithm mimics the behaviors of players representing video objects. Each player competes for available bits to optimize its visual quality. The algorithm finds an ?optimal solution? in that it conforms to the mixed strategy Nash equilibrium, which is the probability distribution of the actions carried by the players that maximizes their expected payoffs (the number of bits). The game is played iteratively, and the expected payoff of each play is accumulated. The game terminates when all of the available bits for the specific time instant have been distributed to video object planes (VOPs). The advantage of the proposed scheme is that the bidding objects divide the bits among themselves automatically and fairly, according to their encoding complexity, and with an overall solution that is strategically optimal under the given circumstances. To minimize buffer fluctuation and avoid buffer overflow and underflow, a proportional-integral-derivative (PID) control based buffer policy is utilized.

Crinkle: A heuristic mapping algorithm for network on chip

In this paper, a heuristic mapping algorithm which maps tasks, using priority lists and the crinkle moving pattern is proposed. To evaluate this algorithm, a set of real (i.e. Video Object Plan Decoder) and random applications have been used and the results have been compared. By reducing the number of hops between IP cores, the energy consumption and the completion time of the application (time which all tasks in the task graph execute wholly) have been optimized. Compared to other mapping algorithms, the algorithm execution time (due to its low complexity) is considerably lower.

Chain-Mapping for mesh based Network-on-Chip architecture

Mapping of IP cores on a given platform is one of the three aspects of Network-on-Chip design. Mapping priority of IP cores is mostly based on a single communication in previously proposed algorithms. In this paper we present Chain-Mapping (CHMAP), as an algorithm for mapping cores onto a mesh-based Network-on-Chip architecture. The main aim of the algorithm is to produce chains of connected cores in order to introduce a new method to prioritize IP core which helps us to have more efficient mapping. Proposed algorithm and previous researches were compared on two real applications, i.e. Video object plan decoder (VOPD) and MPEG-4 and results were reported.

3-D Shape-Adaptive Directional Wavelet Transform for Object-Based Scalable Video Coding

This paper presents a 3-D shape-adaptive directional wavelet coding technique for object-based scalable video coding. This technique includes 3-D object-based directional threading and extensions of weighted adaptive lifting (WAL) scheme. The 3-D object-based directional threading, which unifies the concept of temporal motion threading and 2-D spatial directional threading, provides the opportunities to align a series of video object planes to form a 3-D video object and exploit the spatio-temporal correlation inside the 3-D video object. The WAL scheme, which is extended from 2-D frame-based image coding to 3-D object-based video coding, is employed to decompose the 3-D video object into a 3-D multispatio-temporal resolution video object pyramid for object-based scalable video coding. Experimental results show that the proposed 3-D shape-adaptive directional wavelet coding technique consistently outperforms MPEG-4 and other wavelet-based schemes for coding arbitrarily shaped video objects.

ローカルエリアテレビ会議ネットワークにおける遅延予測モデル

The communication environment is diverse. Therefore, we proposed a VOP decision-making algorithm, which can be used to adaptively select an arrangement of VOPs (Video Object Planes). This algorithm properly encodes movies depending on the power of the CPU. However, it is difficult to estimate the transmission latency when a movie is streamed in real-time. We estimate the time cost of transmitting when sending/receiving MPEG-4 objects in a one-to-one MPEG communication environment. Our results show that our method can be used to accurately estimate encoding and decoding latency.

MPEG‐4 video transmission over UMTS mobile networks

AbstractThis paper presents an overview [1] of the MPEG‐4 (Motion Picture Experts Group) coded video file structure and the VOP (Video Object Plane) types followed by classification of errors and introduction of the NEWPRED mechanism in MPEG‐4 for error concealment. An UEP (Unequal Error Protection) scheme is described with a discussion on how the proposed scheme affects the UMTS (Universal Mobile Telecommunications System) network efficiency. (© 2008 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)