Multimedia Processing Research Articles

Performance of a Partial Discrete Wavelet Transform Based Path Merging Compression Technique for Wireless Multimedia Sensor Networks

Wireless multimedia sensor network is well-known for its constraints in the field of multimedia processing in terms of processing power, bandwidth etc. Data processing is always a challenge in such network. Exploiting low overhead compression accompanied by proper routing and aggregation is a challenge for such energy starved multihop network. In this paper, we propose an efficient path merging protocol for wireless multimedia sensor network with randomly deployed nodes. Any partial discrete wavelet transform based compression technique can be plugged into this path merging protocol to reduce redundant data transmission in a significant manner by appropriate aggregation of data packets from merging paths. The design feasibility and the simulation results prove supremacy of our protocol over state-of-the-art competing schemes in terms of maintaining a trade-off between energy consumption and reconstruction quality.

A Perception-Driven Transcale Display Scheme for Space Image Sequences

With the rapid development of multimedia technology, the way of obtaining high-quality motion reproduction for space targets has attracted much attention in recent years. This paper proposes a Perception-driven Transcale Display Scheme, which significantly improves the awareness of multimedia processing. This new scheme contains two important modules, transcale description based on visual saliency and perception-driven display of space image sequences. The former concentrates on describing the transcle feature of space targets, including three algorithms, attention region computing, frame rate conversion, and image resolution resizing. On this basis, the latter focuses on high-quality display of space movements under different scales, including three algorithms, namely, target trajectory computing, space transcale display, and space movement display. Extensive quantitative and qualitative experimental evaluations demonstrate the effectiveness of the proposed scheme.

Editorial Introduction to the Special Issue on Multimedia Big Data for Extreme Events

The four papers in this special section examine the use of multimedia Big Data for leveraging big events such as natural disasters. Leveraging technological advances and using Big Data approaches for information management and processing in extreme events have attracted much research attention. Multimedia is increasingly becoming the most important and valuable source for insight and information on extreme events. It covers from everyone’s experiences to everything happening in the world. There will be lots of multimedia big data in extreme events—surveillance video, disaster images, social media, voice and video, to name a few, only if their volumes grow to the extent that the traditional multimedia processing and analysis systems may not handle effectively. As such, multimedia big data in extreme events are spurring on tremendous amounts of research and development of related technologies and applications. The papers in this issue is intended to provide a premier forum for researchers working on multimedia big data for extreme events to present their recent research results. It also provides an important opportunity for multidisciplinary studies connecting multimedia computing to disaster information management.

Priority-Based Cloud Computing Architecture for Multimedia-Enabled Heterogeneous Vehicular Users

In recent days, vehicles have been equipped with smart devices that offer various multimedia-related applications and services, such as smart driving assistance, traffic congestions, weather forecasting, road safety alarms, and many entertainment and comfort applications. Thus, these smart vehicles produce a large amount of multimedia-related data that require fast and real-time processing. However, due to constrained computing and storage capacities, such huge amounts of multimedia-related data cannot be processed in on-board standalone devices. Thus, multimedia cloud computing (MCC) has emerged as an economical and scalable computing technology that can process multimedia-related data efficiently while providing improved Quality of Service (QoS) to vehicular users from anywhere, at any time and on any device, at reduced costs. However, there are certain challenges, such as fast service response time and resource cost optimization, that can severely affect the performance of the MCC. Therefore, to tackle these issues, in this paper, we propose a dynamic priority-based architecture for the MCC. In the proposed scheme, we divide multimedia processing into four different subphases, while computing resources to each computing server are assigned dynamically, according to the workload, in order to process multimedia tasks according to the multimedia user Quality of Experience (QoE) requirements. The performance of the proposed scheme is evaluated in terms of service response time and resource cost optimization using the CloudSim simulator.

Watermarking Techniques for Three Dimensional (3D) Mesh Authentication in Spatial Domain

Due to lightning growth in multimedia processing technologies and frequent usage of Internet, media contents often get exposed to malicious interventions. Hence it is very important to verify the authenticity and integrity of the digital contents by using proper security mechanisms. Thanks to the advancement in computer graphics processing technologies due to which higher dimensional rendering of real life objects has become apparent. Lately, three dimensional (3D) representations are more preferred in different application domains for better visualization and understanding. While talking about security related to 3D models, digital watermarking has come out to be an effective and prominent solution among various other authentication mechanisms. The purpose of this study is to present an in-depth review and investigation of the authentication based watermarking techniques for 3D models in spatial domain. The methods are classified based on various attributes that effect the design and working of an authentication protocol. The study also discusses different types of geometrical attacks that may occur to a 3D model and methodologies that help in detecting and locating the attacks. The paper concludes with a set of remarks and some of the research issues and challenges which are needed to be studied further.

RETRACTED: LTE: An enhanced hybrid domain downlink scheduling

Abstract The third Generation Partnership Project (3GPP) has successfully launched 4G data on mobile communication by utilizing the Long Term Evolution (LTE). The multimedia processing such as video calling/conferencing, video transmission/reception in mobile communication are in need of multiple user support, speed, high data transmission rate, low data loss rate and high throughput. In this paper an enhanced scheduling algorithm known as, EM-LWDF (Modified Largest Weighted Delay First) is proposed in combination of time/frequency scheduling for LTE. The proposed scheduling algorithm is evaluated under various simulation scenarios and compared with conventional algorithm such as PF, M-LWDF, EXP/PF, and proposed +LWDF. The evaluation is made in terms of system throughput, fairness index and Packet Loss Rate (PLR) and the evaluation results shows that the proposed scheduling algorithm shows remarkable performance improvement.

Design of a Gabor Filter HW Accelerator for Applications in Medical Imaging

The Gabor filter (GF) has been proved to show good spatial frequency and position selectivity, which makes it a very suitable solution for visual search, object recognition, and, in general, multimedia processing applications. GFs prove useful also in the processing of medical imaging to improve part of the several filtering operations for their enhancement, denoising, and mitigation of artifact issues. However, the good performances of GFs are compensated by a hardware complexity that traduces in a large amount of mapped physical resources. This paper presents three different designs of a GF, showing different tradeoffs between accuracy, area, power, and timing. From the comparative study, it is possible to highlight the strength points of each one and choose the best design. The designs have been targeted to a Xilinx field-programmable gate array (FPGA) platform and synthesized to 90-nm CMOS standard cells. FPGA implementations achieve a maximum operating frequency among the different designs of 179 MHz, while 350 MHz is obtained from CMOS synthesis. Therefore, 86 and 168 full-HD ( $1920 \times 1080$ ) f/s could be processed, with FPGA and std_cell implementations, respectively. In order to meet space constraints, several considerations are proposed to achieve an optimization in terms of power consumption, while still ensuring real-time performances.

Cloud computing and big data: Technologies and applications

Cloud computing and big data: Technologies and applications

Super resolution reconstruction algorithm of video image based on deep self encoding learning

Super resolution reconstruction of video image is a research hotspot in the field of image processing, and it is widely used in video surveillance, image processing, criminal analysis and other fields. Super resolution image reconstruction can reconstruct a high resolution image from low resolution images, and this technology has become a research hotspot in the field of image processing. In recent years, deep learning has been developed rapidly in the field of multimedia processing, and image super resolution restoration technology based on deep learning has gradually become the mainstream technology. In view of the existing image super-resolution algorithm problems, such as more parameters, larger amount of calculation, longer training time, blurred image texture, we use the deep self-coding learning method to improve it. We analyze the advantages and disadvantages of the existing technology from the network type, network structure, training methods and so on, and sort out the development of the technology. The experimental results show that the improved network model achieves better super-resolution results, and the subjective visual effect and objective evaluation index are improved obviously, and the image sharpness and edge sharpness are improved obviously.

Opinions of Teacher Candidates on the Usage of Mobile Applications in the Multimedia Development Processes

<p class="0abstract">In order that mobile applications can be employed successfully in education, the opinions of teachers for using these applications must be identified and improved. For these reasons, the purpose of this study is to identify the opinions of teacher candidates, who are teachers of the future, on the usage of mobile applications in the process of multimedia project development. This study which was conducted with the purpose of identifying the opinions of teacher candidates on usage of mobile applications in education as means of communication employed qualitative method. Results is that using mobile applications in multimedia project development process attracted the attention of students and ensured effective communication.</p><p class="0abstract"> </p>

Blind visual quality assessment for image super-resolution by convolutional neural network

Image super-resolution aims to increase the resolution of images with good visual experience. Over the past decades, there have been many image super-resolution algorithms proposed for various multimedia processing applications. However, how to evaluate the visual quality of high-resolution images generated by image super-resolution methods is still challenging. In this paper, a Convolutional Neural Network is designed to predict the visual quality of image super-resolution. The proposed network consists of two convolutional layers, two pooling layers including average, min and max pooling, three fully connected layers and one regression layer. The contribution of the proposed method is twofold. The first one is that we propose a the deep convolutional neural network to extract the high-level intrinsic features more effectively than the hand-crafted features for super-resolution images, which can be used to estimate the image quality accurately. The other is that we divide the super-resolution image into small patches, to consider the local information for the visual quality assessment of super-resolution image as well as increase the number of training data for the deep neural network. Experimental results show that the proposed metric can obtain better performance than other existing ones in visual quality assessment of image super-resolution.

Multimedia image data processing on smartphone for authentication

The spread of smartphones is rapidly increasing. Such smartphones mainly handle image and video data. In particular, there are many security problems that may occur at this time when there are many accesses to multimedia image data. But most systems use various technologies for user authentication such as identification/password (ID/PW), the certificate, One-Time Password (OTP), fingerprint/iris recognition, etc. When using ID/PW technology, the system is vulnerable, if ID/PW is disclosed to a hacker. The certificate or fingerprint/iris recognition technologies are more secure than ID/PW technology, but they require more cost, and they are complex to build. In this paper, we propose the method for enhancing user authentication. The proposed method improves security at a low cost using the user’s authentication information and the user’s smartphone IMEI information. In particular, IMEI information can be seen to provide a fast right and secure authentication mechanism to perform user authentication of multimedia image processing. In our research method, it is much safer and superior responsiveness than authentication methods using only user ID / Password on smartphones.

Outsourcing Set Intersection Computation Based on Bloom Filter for Privacy Preservation in Multimedia Processing

With the development of cloud computing, the advantages of low cost and high computation ability meet the demands of complicated computation of multimedia processing. Outsourcing computation of cloud could enable users with limited computing resources to store and process distributed multimedia application data without installing multimedia application software in local computer terminals, but the main problem is how to protect the security of user data in untrusted public cloud services. In recent years, the privacy-preserving outsourcing computation is one of the most common methods to solve the security problems of cloud computing. However, the existing computation cannot meet the needs for the large number of nodes and the dynamic topologies. In this paper, we introduce a novel privacy-preserving outsourcing computation method which combines GM homomorphic encryption scheme and Bloom filter together to solve this problem and propose a new privacy-preserving outsourcing set intersection computation protocol. Results show that the new protocol resolves the privacy-preserving outsourcing set intersection computation problem without increasing the complexity and the false positive probability. Besides, the number of participants, the size of input secret sets, and the online time of participants are not limited.

A High-Performance and Hardware-Efficient PCIe Transmission for a Multi-Channel Video Using Command Caching and Dynamic Splicing on FPGA

With the increasing requirements of transmitting larger amounts of diverse video and image data, the PCI express (PCIe) interface has been extensively used in high-speed digital systems for multimedia processing and communication. However, the existing PCIe transmission methods are not efficient to transmit multi-channel videos on FPGA processors, as they waste PCIe bandwidth and hardware resources. In this paper, an efficient PCIe transmission method for multi-channel video is presented on an FPGA processor. First, to reduce the programmed I/O (PIO) write latency, the command caching method is adopted. The traditional DMA descriptors are transformed into DMA commands, which are smaller, and the command buffer is used to store these commands. Second, a dynamic splicing mechanism is proposed for data transmission and commands the scheduling separately. The multi-type data is transmitted concurrently in this dynamic splicing mechanism. Moreover, the adjacent commands are spliced into one command, which reduces the DMA initiation times. Finally, the direct kernel memory access technique and timer self-feedback monitor technique are applied to improve the transmission efficiency and enhance the system reliability. The experimental results show that the transmission efficiency of the proposed method is increased by nearly 30% compared with other conventional methods, and the highest measured transfer rate is 1631 Mbytes/s on write and 1582 Mbytes/s on read, which reaches 84.4% of the theoretical maximum.

RSE-PoW: a Role Symmetric Encryption PoW Scheme with Authorized Deduplication for Multimedia Data

The explosive growth of global data and the gradual evolution of multimedia processing pattern promote us to enter the era of heterogeneous multimedia data. To improve the storage efficiency and reduce the management expenditure of these massive multimedia data in heterogeneous networks, the direct way is multimedia data deduplication. However, it arises serious privacy concerns and poses new security challenges, such as privacy leakage, the side-channel attack and unauthorized access. To solve these issues, we propose a novel role symmetric encryption (RSE) algorithm and a RSE-based proof of ownership (RSE-PoW) scheme for secure deduplication in hierarchical heterogeneous environments, which is based on the role symmetric encryption, proof of ownership and bloom filter. The proposed scheme is the first solution to prevent the privacy leakage and resist against the side-channel attacks, meanwhile, achieve the authorized deduplication effectively for multimedia data in heterogeneous network. Security analysis and performance evaluation show the security and efficiency of our proposed scheme.

Towards a smart selection of resources in the cloud for low‐energy multimedia processing

SummaryNowadays, image and video processing applications have become widely used in many domains related to computer vision. Indeed, they can come from cameras, smartphones, social networks, or from medical devices. Generally, these images and videos are used for illustrating people or objects (cars, trains, planes, etc) in many situations such as airports, train stations, public areas, sport events, and hospitals. Thus, image and video processing algorithms have got increasing importance, they are required from various computer vision applications such as motion tracking, real time event detection, database (images and videos) indexation, and medical computer‐aided diagnosis methods. The main inconvenient of image and video processing applications is the high intensity of computation and the complex configuration and installation of the related materials and libraries. In this paper, we propose a new framework that allows users to select in a smart and efficient way the computing units (CPU or/and GPU) in a cloud‐based platform, in case of processing one image (or one video in real time) or many images (or videos). This framework enables to affect the local or remote computing units for calculation after analyzing the type of media and the algorithm complexity. The framework disposes of a set of selected CPU and GPU‐based computer vision methods, such as image denoising, histogram computation, features descriptors (SIFT, SURF), points of interest extraction, edges detection, silhouette extraction, and sparse and dense optical flow estimation. These primitive functions are exploited in various applications such as medical image segmentation, videos indexation, real time motion analysis, and left ventricle segmentation and tracking from 2D echocardiography. Experimental results showed a global speedup ranging from 5× to 273×(compared to CPU versions) as result of the application of our framework for the above‐mentioned methods. In addition to these performances, the parallel and heterogeneous implementations offered lower power consumption as result of the fast treatment.

A Saliency Prior Context Model for Real-Time Object Tracking

Real-time object tracking has wide applications in time-critical multimedia processing areas such as motion analysis and human–computer interaction. It remains a hard problem to balance between accuracy and speed. In this paper, we present a fast real-time context-based visual tracking algorithm with a new saliency prior context (SPC) model. Based on the probability formulation, the tracking problem is solved by sequentially maximizing the computed confidence map of target location in each video frame. To handle the various cases of feature distributions generated from different targets and their contexts, we exploit low-level features as well as fast spectral analysis for saliency to build a new prior context model. Then, based on this model and a spatial context model learned online, a confidence map is computed and the target location is estimated. In addition, under this framework, the tracking procedure can be accelerated by the fast Fourier transform. Therefore, the new method generally achieves a real-time running speed. Extensive experiments show that our tracking algorithm based on the proposed SPC model achieves real-time computation efficiency with overall best performance comparing with other state-of-the-art methods.

Retrieving real world clothing images via multi-weight deep convolutional neural networks

Clothing images are abundantly available from the Internet, especially from the e-commercial platform. Retrieving those images is of importance for commercial and social applications and has recently been received tremendous attention from communities, such as multimedia processing and computer vision. However, the large variations in clothing of their appearance and style, and even the large quantity of multiple categories and attributes make those problems challenging. Furthermore, for real world images their labels provided by shop retailers from webpages are largely erroneous or incomplete. And the imbalance among those image categories prevents the effective learning. To overcome those problems, in this paper, we adopt a multi-task deep learning framework to learn the representation. And we propose multi-weight deep convolutional neural networks for imbalance learning. The topology of this network contains two groups of layers, shared layers at the bottom and task dependent ones at the top. Furthermore, category-relevant parameters are incorporated to regularize the backward gradients for categories. Mathematical proof shows its relationship to regulating the learning rates. Experiments demonstrate that our proposed joint framework and multi-weight neural networks can effectively learn robust representations and achieve better performance.

Antipiracy-Aware IP Chipset Design for CE Devices: A Robust Watermarking Approach [Hardware Matters

Intellectual property (IP) chip sets are indispensable components of consumer electronics (CE) products, such as set-top boxes, digital televisions, digital video discs (DVDs), tablets, digital cameras, and audio-video receivers. They represent several years of investment, research, and development through expensive infrastructure. Watermarking in IP chip sets for the protection of CE devices against false claims of ownership, piracy, and counterfeiting has proven to be a promising solution. However, the design process of a watermarked (antipiracy-aware) IP chip set is complex, and no published work exists in the literature to introduce a formal design methodology. This column presents a formal design approach for antipiracy-aware IP chip sets for CE devices. Using robust multivariable signature-encoding methodology, decoded watermarking constraints are embedded into the formal architectural synthesis design steps of an IP chip set. Each step of the IP chip set design is lucidly introduced with the aid of a real-life benchmark from the domain of multimedia and digital signal processing.

Perancangan Multimedia Interaktif Tentang Kesenian Jaranan Turangga Yaksa Sebagai Media Pembelajaran Di SDN 3 Pringapus Kabupaten Trenggalek

Pemahaman budaya daerah perlu dimiliki setiap orang sejak dini. Pada SDN 3 Pringapus mempelajari kesenian jaranan Turangga Yaksa untuk mengenalkan kepada siswanya. Akan tetapi proses belajar yang hanya terpaku pada buku serta kurangnya faktor interaktif, sehingga menurunnya minat belajar siswa. Dengan semakin berkembangnya teknologi informasi, upaya untuk meningkatkan minat belajar anak dapat dilakukan dengan berbagai media, salah satunya adalah menggunakan media interaktif. Oleh karena itu dirancang sebuah multimedia interaktif menggunakan software pengolah multimedia. Karakter yang digunakan adalah karakter anak lengkap dengan pakaian jaranan dengan muka yang ceria, warna yang dominan hijau, dan warna dasar serta teks yang sederhana supaya mudah dibaca untuk memahami materi yang disampaikan oleh siswa. Setelah proses selesai maka selanjutnya pengujian efektivitas produk dengan cara penyebaran kuisioner. Hasil yang didapat dalam penelitian melalui penyebaran kuisioner yang telah dilakukan di SDN III Pringapus kecamatan Dongko kabupaten Trenggalek, dapat ditarik kesimpulan bahwa 93% media pembelajaran ini dikatakan efektif.