Future Multimedia Applications Research Articles

Suitability of SDN and MEC to facilitate digital twin communication over LTE-A

Haptic is the modality that complements traditional multimedia, i.e., audiovisual, to evolve the next wave of innovation at which the Internet data stream can be exchanged to enable remote skills and control applications. This will require ultra-low latency and ultra-high reliability to evolve the mobile experience into the era of Digital Twin and Tactile Internet. While the 5th generation of mobile networks is not yet widely deployed, Long-Term Evolution (LTE-A) latency remains much higher than the 1 ms requirement for the Tactile Internet and therefore the Digital Twin. This work investigates an interesting solution based on the incorporation of Software-defined networking (SDN) and Multi-access Mobile Edge Computing (MEC) technologies in an LTE-A network, to deliver future multimedia applications over the Tactile Internet while overcoming the QoS challenges. Several network scenarios were designed and simulated using Riverbed modeler and the performance was evaluated using several time-related Key Performance Indicators (KPIs) such as throughput, End-2-End (E2E) delay, and jitter. The best scenario possible is clearly the one integrating MEC and SDN approaches, where the overall delay, jitter, and throughput for haptics- attained 2 ms, 0.01 ms, and 1000 packets per second. The results obtained give clear evidence that the integration of, both SDN and MEC, in LTE-A indicates performance improvement, and fulfills the standard requirements in terms of the above KPIs, for realizing a Digital Twin/Tactile Internet-based system.

A ResNet deep learning based facial recognition design for future multimedia applications

CNN deep-learning models are the most trusted computational approaches in industrial, medical, and intelligent applications. In this research work, a deep micro-facial emotion has been recognised to employ CNN approach. Multi-class feature extraction has been initiated by the 2D-ResNet convolutional neural network. In order to detect the maskable images of facial emotions, a 2D-ResNet CNN multi-class classifier has been proposed. This novel model has reduced overfitting and controversial problems, which are verified on the Python 3.7.0 software tool. The public dataset JAFFE (open source) has been used to train the 2D-ResNet deep learning model. The flask, mysql, fastai, and jsonify python packages are compatible and feasible with the 2D-ResNet Deep Learning architecture. The proposed method practices better performance metrics in terms of accuracy of 99.3%, recall of 99.12%, F1 score of 0.98% and sensitivity of 99.16%, which are figured out from the confusion matrix, where only 5 failures have been observed when tested with 1 lakh images.

Implementation of Cognitive Radio Network Testbed for Multimedia Communications

A cognitive radio is a form of wireless communication in which a transceiver can intelligently detect which communication channels are in use and which are not, and instantly move into vacant channels while avoiding occupied ones. This intelligently avoids interference amongst users and provides the ability to use up all available bandwidth on the RF spectrum. There are many ways in which the communication can be demonstrated. This paper examines how we have implemented a cognitive radio network testbed using software defined radios (SDRs) for multimedia communications, where components that have been traditionally implemented in hardware are instead implemented by means of software on a personal computer or embedded system to communicate and transfer a file between each other. We attempt to demonstrate potential use of SDRs for future multimedia applications.

Hybrid relay selection‐based scheme for UWB BANs combining MB‐OFDM and decode‐and‐forward cooperative architectures

Wireless body area networks (WBANs) is now a well-accepted technology that provides not only efficient and remote health-care services but also targets the integration of future multimedia applications, in which high data rate is compulsory. UWB communication is one of the key solutions that afford such rates, and its extended version, multiband OFDM (MB-OFDM) has recently shown its efficiency with WBANs. However, increasing the data rates may come at the expense of the data reliability which is a critical issue in biomedical applications. Cooperative diversity has proven its ability in offering such robustness to channel errors while avoiding to use multiple antennas. Indeed, a single antenna terminal is sufficient to achieve diversity benefits by sharing its physical resources with cooperating single antenna relays. Aiming at providing a viable compromise between the data rate and accuracy in low-profile UWB body networks, to incorporate a decode-and-forward-based cooperative architecture in MB-OFDM system by resorting to opportunistic relaying approach is proposed, which retains only the best on-body relay to forward the information to the destination, thereby significantly reducing the complexity.

Coexistence of WiFi and LiFi toward 5G: concepts, opportunities, and challenges

Smart phones, tablets, and the rise of the Internet of Things are driving an insatiable demand for wireless capacity. This demand requires networking and Internet infrastructures to evolve to meet the needs of current and future multimedia applications. Wireless HetNets will play an important role toward the goal of using a diverse spectrum to provide high quality-of-service, especially in indoor environments where most data are consumed. An additional tier in the wireless HetNets concept is envisioned using indoor gigabit small-cells to offer additional wireless capacity where it is needed the most. The use of light as a new mobile access medium is considered promising. In this article, we describe the general characteristics of WiFi and VLC (or LiFi) and demonstrate a practical framework for both technologies to coexist. We explore the existing research activity in this area and articulate current and future research challenges based on our experience in building a proof-of-concept prototype VLC HetNet.

A cognitive approach for effective coding and transmission of 3D video

Future multimedia applications will rely on the transmission of 3D video contents within heterogeneous fruition scenarios, and as a matter of fact, the reliable delivery of 3D video signals proves to be a crucial issue in such communications. To this purpose, multimedia communication experts have been designing cross-layer strategies to improve the quality of the perceived 3D experience. This article presents a new cross-layer strategy, called Cognitive Source Coding (CSC), that defines a new 3D video system able to identify the different elements of the 3D scene and choose the most appropriate coding strategy.

System design of full HD MVC decoding on mesh-based multicore NoCs

Future multimedia applications such as full HD (1920 × 1080) multiview video coding (MVC) present great challenges on computing architectures. Even if with the state-of-the-art ASIC technology which can process single view HD decoding, dealing with multiple views would require times of computation capacity in proportion to the number of views, which is difficult to achieve. In this paper, we explore the system-level design space for full HD MVC applications mapped onto mesh-based multicore Network-on-Chip (NoC) architectures. To this end, we establish a simulation framework capable of simulating the combination of communication networks with computing cores. We investigate two task assignment schemes: picture-level assignment and view-level assignment. With an eight-view MVC decoding, we explore the design options with respect to network size, single-core performance and link bandwidth under both task assignment schemes. Our studies show that, to achieve a certain decoding performance, the computation capability and communication capacity should be balanced in the system. Also, to realize the eight-view HD decoding, the system only requires twice or less than twice of the single-core processing capacity required by single view decoding, thanks to the parallel computation and communication enabled by the multicore NoC architectures. Our results exhibit feasibility and potential of efficiently implementing the full HD MVC decoding on multicore NoC architectures.

A 36 fps SXGA 3-D Display Processor Embedding a Programmable 3-D Graphics Rendering Engine

In this paper, a 3D display processor embedding a programmable 3D graphics rendering engine is proposed. The proposed processor combines a 3D graphics rendering engine and a 3D image synthesis engine to support both true realism and interactivity for the future multimedia applications. Using high coherence between 3D graphics data and 3D display inputs, both pipelines are merged by sharing buffers such that a 3D display engine directly uses the output of a 3D graphics rendering engine. The merged architecture has synergetic coupling effects such as freely providing various rendering effects to 3D images and easily computing disparities without complex extraction processes. In the 3D image synthesis engine, we adopt view interpolation algorithm and propose real-time synthesis method, pixel-by-pixel process. The view interpolation algorithm reduces the number of images to be rendered, resulting in the reduction of external memory size to 64.8% compared to conventional synthesis process. The proposed pixel-by-pixel process synthesizes 3D images at 36 fps through bandwidth reduction of 26.7% and decreases internal memory size to 64.2% compared to typical image-by-image process. The 3D graphics rendering engine is programmable and supports the instruction sets of the latest 3D graphics standard APIs, Pixel Shader 3.0 and OpenGL|ES 2.0. The die contains about 1.7 M transistors, occupies 5 mm times 5 mm in 0.18 mum CMOS and dissipates 379 mW at 1.85 V.

Opportunistic WiMAX–GSM coexistence

In future multi-media applications dynamic access to spectral resources will play an important role. However, regulation of spectrum will only change very slowly and spectrum rights holders might not be willing to share their resources without gaining something out of it. Therefore the authors analyse the suitability of WiMAX as an overlay system on GSM where available resources will be auctioned among several competitors. The authors design a cross-layer approach to facilitate dynamic resource allocation. It is demonstrated that, with minor changes of the original WiMAX frame and a small signalling overhead which is currently unused, GSM resources can be allocated efficiently without decreasing system performance. Thus, this adapted WiMAX system used in opportunistic coexistence to GSM offers a suitable alternative to broadband service access.

Clustering-Based Multichannel MAC Protocols for QoS Provisionings Over Vehicular Ad Hoc Networks

Making the best use of the dedicated short range communications multichannel architecture, we propose a cluster-based multichannel communications scheme that can support not only public-safety message delivery but also a wide range of future multimedia (e.g., video/audio) and data (e.g., e-maps, road/vehicle traffic/weather information) applications. Our proposed scheme integrates clustering with contention-free and/or -based medium access control (MAC) protocols. In our scheme, the elected cluster-head vehicle functions as the coordinator to collect/deliver real-time safety messages within its own cluster and forward the consolidated safety messages to the neighboring cluster heads. In addition, the cluster-head vehicle controls channel assignments for cluster-member vehicles transmitting/receiving nonreal-time traffics, which makes the wireless channels more efficiently utilized for vehicle-to-vehicle (V2V) nonreal-time data transmissions. Our scheme uses the contention-free MAC within a cluster and the contention-based IEEE 802.11 MAC among cluster-head vehicles such that the real-time delivery of safety messages can be guaranteed. Under our proposed scheme, we develop an analytical model to study the delay for the consolidated safety messages transmitted by the cluster-head vehicles. Based on this analytical model, we derive the desirable contention-window size, which can best balance the tradeoff between the delay of safety messages and the successful rate of delivering safety messages. The extensive simulation results show that, under various highway traffic scenarios, our proposed scheme can efficiently support the nonreal-time traffics while guaranteeing the real-time delivery of the safety messages.

A Cross Layer Scheduling Framework for Supporting Bursty Data Applications in WCDMA Networks

In future wireless networks multimedia applications are expected to finally dominate the overall traffic volume. Shared channels are more suitable for the transmission of this type of traffic, as they are able to periodically adjust their transmission rate. In this paper, we introduce a cross-layer framework for WCDMA based networks which aims to make the packet scheduling procedure more efficient. In addition to that, we further propose a traffic scheduling scheme which serves the connections not only according to their delay sensitivity, but also according to the predicted state of their wireless channel. The efficiency of the proposed scheme, in terms of average packet delay and channel utilization is verified via simulations.

P2P Object-based adaptivE Multimedia Streaming (POEMS)

Peer-to-peer (P2P) paradigm has recently gained tremendous attraction and is widely used for content distribution and sharing. The future multimedia communication applications have to support the user's needs, the terminal capabilities, the content specification and the underlying networking technologies. They should be network-aware, topology-aware, and end-user-centric. Thus, in this paper, we use the characteristics of the object-based encoding scheme and P2P network topology to propose adaptive content delivery architecture for P2P networks. We propose an efficient mechanism for transmission of real-time content over P2P networks, called POEMS (P2P Object-based adaptivE Multimedia Streaming). This object-based audio-visual quality adaptive mechanism over P2P networks is media-aware, network-aware, and user-centric that is carried-out through (1) selection of appropriate sending peers willing to participate in the streaming mechanism, (2) organization of sending peers by constructing an overlay network to facilitate content delivery and adaptation, (3) dynamicity management of peers when some peer enters or leaves the system to maintain an acceptable level of perceived video quality, and (4) ensuring the end-to-end QoS (Quality of Services) by orchestrating the overall streaming mechanism. The obtained results demonstrate that combining content adaptation using object-based encoding and advance network-aware peers selection based on peer monitoring leads to intelligent, efficient, and large-scale support of multimedia services over complex network architectures.

Shared distribution of digital products using multicast

Broadband connectivity enables effective distribution of interactive multimedia content over the Internet. The successful deployment of future multimedia applications with high bandwidth requirements will depend on business models that efficiently allocate network resources based on user demands and preferences. We present a market-based allocation framework to complement existing network protocols in a scalable and feasible manner. The model exploits the redundancy of the network to share digital products among hierarchies of communities that share common demand profiles. Quality of service-based multicast is used as the underlying network protocol. While significant advances have been made in terms of the technological protocols, corresponding business models have not been explored extensively. We outline such a model accounting for the current market structure. The highlights of the business model we propose are shared distribution, hierarchy of communities and bundling of products by local distributors. Our model leads to the NP-hard problem of computation of Steiner arborescences, and we outline feasible solution heuristics.

A knowledge-based framework for multimedia adaptation

Personalized delivery of multimedia content over the Internet opens new business perspectives for future multimedia applications and thus plays an important role in the ongoing MPEG-7 and MPEG-21 multimedia standardization efforts. Based on these standards, next-generation multimedia services will be able to automatically prepare the digital content before delivery according to the client's device capabilities, the network conditions, or even the user's content preferences. However, these services will have to deal with a variety of different end user devices, media formats, as well as with additional metadata when adapting the original media resources. In parallel, an increasing number of commercial or open-source media transformation tools will be available, capable of exploiting such descriptive metadata or dealing with new media formats; thus it is not realistic that a single tool will support all possible transformations. In this paper, we present a novel, fully knowledge-based approach for building such multimedia adaptation services, addressing the above mentioned issues of openness, extensibility, and concordance with existing and upcoming standards. In our approach, the original media is transformed in multiple adaptation steps performed by an extensible set of external tools, where the construction of adequate adaptation sequences is solved in an Artificial Intelligence planning process. The interoperability issue is addressed by exploiting standardized Semantic Web Services technology. This technology allows us to express tool capabilities and execution semantics in a declarative and well-defined form. In this context, existing multimedia standards serve as a shared domain ontology. The presented approach was implemented and successfully evaluated in an official ISO/IEC MPEG (Moving Picture Experts Group) Core Experiment and is currently under further evaluation by the standardization body.

Optimisation du cache sibling à l'aide d'une architecture pair à pair

Nowadays, the data search concept on a peer to peer (P2P) network is more and more widely used. A new approach of this technique is the use of peer to peer technology for streaming video multimedia applications.The throughputs on future-generation wireless networks will be able to dispatch several multimedia streams simultaneously and their capabilities to handle download and upload throughputs will provide real interactivity in future multimedia applications. We have developed a cache sibling architecture called PACS whose issue is to increase the data exchange between caches. In this article, we intend to deal with document exchanges between caches thanks to a P2P platform.

Study and simulation of the infrared WLAN IrDA: an alternative to the radio

The dramatic development of radio solutions for wireless communications tends to blur the wider use that can be made of infrared transmissions in a room, a workshop or even a building. For some local applications for which a wireless part often constitutes the end point of global networks, the infrared medium can be a real alternative to the radio medium. Infrareds offer undeniable advantages, and are a technological niche in the field of in-house wireless LAN. Here the characteristics of this technology are illustrated and compared with better known radio solutions. As the world of communications is constantly readjusting to new standards, the essential characteristics of IrDA, the most significant standard are presented. The IrDA standard keeps changing to include new aspects. It now features a set of layers for a complete infrared local network providing the usual functionality of the different OSI layers. Thus, baud rates of IrDA products have increased from 115200 bps to 4 Mbps, (even 30 Mbps) paving the way for future multimedia applications demanding high baud rates. Many new modelisations and simulations with OPNET have facilitated the study and the analysis of this IR standard.

Improving user satisfaction in adaptive multicast video

Adaptability is the most promising feature to be applied in future robust multimedia applications. In this paper, we propose the Direct Algorithm to improve the degree of satisfaction at heterogeneous receivers in multi-layered multicast video environments. The algorithm relies on a mechanism that dynamically controls the rates of the video layers and is based on feedback control packets sent by the receivers. The algorithm also addresses scalability issues by implementing a merging procedure at intermediate nodes in order to avoid packet implosion at the source in the case of large multicast groups. The proposed scheme is optimized to achieve high global video quality and reduced bandwidth requirements. We also propose the Direct Algorithm with a virtual number of layers. The virtual layering scheme induces intermediate nodes to keep extra states of the multicast session, which reduces the video degradation for all the receivers. The results show that the proposed scheme leads to improved global video quality at heterogeneous receivers with no cost of extra bandwidth.

The communication revolution goes on--global high-speed networks: setting the pace for future multimedia applications.

The communication revolution goes on--global high-speed networks: setting the pace for future multimedia applications.

Architecture Concepts for Multimedia Signal Processing

Architectural concepts are presented aimed at future multimedia processing schemes. Starting from an analysis of current and future multimedia applications, specific computational requirements are derived. It will be shown that multimedia applications benefit from an exhaustive and flexible exploitation of parallelism. Three architectural concepts—reconfigurable computing, simultaneous multithreading, and associative controlling—are presented, and their potential to increase further the performance on future multimedia applications is investigated.

Retrieval of Commercials by Semantic Content: The Semiotic Perspective

Video information processing and retrieval is a key aspect of future multimedia technologies and applications. Commercial videos encode several planes of expression through a rich and dense use of colors, editing effects, viewpoints and rhythms, which are exploited together to attract potential purchasers. Databases of commercials can be accessed in order to analyze how a commercial has been developed, retrieve commercials similar to an example, catalog commercials according to the kind of message conveyed to the user. In this paper, we present a system allowing the retrieval of commercial streams based on their salient semantics. Semantics is regarded from the semiotics perspective: collections of signs and semantic features like colors, editing effects, motion, etc. are used as basic blocks with which the meaning of a commercial is constructed. In our system, it is possible to retrieve commercials according to both the meaning they convey and to their similarity to examples.