Collections Of Multimedia Data Research Articles

Weakly supervised classification through manifold learning and rank-based contextual measures

Over the last decade, significant advances have been achieved by machine learning approaches, notably in supervised learning scenarios. Supported by the advent of deep learning and comprehensive training sets, the accuracy achieved on classification tasks has improved significantly. Simultaneously, we have experienced massive growth in multimedia data and applications, which have become ubiquitous in several domains. However, with the increase in multimedia data collections, significant bottlenecks associated with the lack of labeled data emerged. To surpass this critical issue, developing methods capable of exploiting the unlabeled data and operating under weak supervision has become imperative. This work proposes a rank-based model capable of using contextual information encoded in the unlabeled data to perform weakly supervised classification. We evaluated the proposed weakly supervised approach on multimedia classification tasks with and without manifold learning algorithms, considering several combinations of rank correlation measures and classifiers. An experimental evaluation was conducted on 6 public image datasets considering different features, including convolutional neural networks and visual transformers. Positive gains were achieved compared to supervised and semi-supervised baselines for the same amount of labeled data. For instance, the proposed approach with manifold learning enhanced the accuracy of the Optimum-Path Forest (OPF) classifier from 71.77% to 83.24% when applied to the Flowers dataset and ResNet features. Among the conclusions, this work reveals that rank-based correlation measures and manifold learning can be used for a more effective labeled set expansion.

Music Recommendation via Hypergraph Embedding.

In recent years, we have witnessed an ever wider spread of multimedia streaming platforms (e.g., Netflix, Spotify, and Amazon). Hence, it has become more and more essential to provide such systems with advanced recommendation facilities, in order to support users in browsing these massive collections of multimedia data according to their preferences and needs. In this context, the modeling of entities and their complex relationships (e.g., users listening to topic-based songs or authors creating different releases of their lyrics) represents the key challenge to improve the recommendation and maximize the users' satisfaction. To this end, this is the first study to leverage the high representative power of hypergraph data structures in combination with modern graph machine learning techniques in the context of music recommendation. Specifically, we propose hypergraph embeddings for music recommendation (HEMR), a novel framework for song recommendation based on hypergraph embedding. The hypergraph data model allows us to represent seamlessly all the possible and complex interactions between users and songs with the related characteristics; meanwhile, embedding techniques provide a powerful way to infer the user-song similarities by vector mapping. We have experimented the effectiveness and efficiency of our approach with respect to the state-of-the-art most recent music recommender systems, exploiting the Million Song dataset. The results show that HEMR significantly outperforms other state-of-the-art techniques, especially in scenarios where the cold-start problem arises, thus making our system a suitable solution to embed within a music streaming platform.

Postgraduate Student Depression Assessment by Multimedia Gait Analysis

In recent years, mental health (especially depression) of university students has aroused general concern. Fast detection of students at risk of depression using multimedia data is a challenge. However, existing methods require the cooperation of participants such as using their speech or facial expression, which are inconvenient to collect and difficult for large-scale screening. In this article, we propose an integrated gait assessment framework that contains the collection and analysis of multimedia data to assess risk of depression for postgraduate students. First, the rigid-body representation is realized by analyzing kinetic energy (KE) and potential energy (PE) generated during walking. Then, we use the fast Fourier transform to analyze KE and PE in the frequency domain for extracting the joint energy feature. Compared with the conventional methods, our method has significantly increased the objectivity of depression assessment in both clinical theory and practice.

DeepDiffusion: Unsupervised Learning of Retrieval-Adapted Representations via Diffusion-Based Ranking on Latent Feature Manifold

Unsupervised learning of feature representations is a challenging yet important problem for analyzing a large collection of multimedia data that do not have semantic labels. Recently proposed neural network-based unsupervised learning approaches have succeeded in obtaining features appropriate for classification of multimedia data. However, unsupervised learning of feature representations adapted to content-based matching, comparison, or retrieval of multimedia data has not been explored well. To obtain such retrieval-adapted features, we introduce the idea of combining diffusion distance on a feature manifold with neural network-based unsupervised feature learning. This idea is realized as a novel algorithm called DeepDiffusion (DD). DD simultaneously optimizes two components, a feature embedding by a deep neural network and a distance metric that leverages diffusion on a latent feature manifold, together. DD relies on its loss function but not encoder architecture. It can thus be applied to diverse multimedia data types with their respective encoder architectures. Experimental evaluation using 3D shapes and 2D images demonstrates versatility as well as high accuracy of the DD algorithm. Code is available at https://github.com/takahikof/DeepDiffusion

Animasi Interaktif Klasifikasi Jangkauan Dan Topologi Jaringan Komputer Berbasis Android Sebagai Media Belajar

Classification of range and topology has its own types and characteristics. There are many types of outreach classifications and topologies, as well as process methods in learning in schools that are less interactive so that students are bored in the learning process, especially for students of SMK Computer and Network Engineering Department. In practice, making LAN cables requires equipment that is quite expensive, making it difficult to carry out these practical activities outside of school. So that students find it difficult to develop their abilities, especially in the vocational field. With current technological developments, of course, it can be used in the world of education, one of the uses is by making educational applications that can provide a new atmosphere in learning. Interactive animation can be a solution in overcoming the practice of making LAN cables and an explanation of Network Topology. This interactive animation was made using Construct 2 software, developed using multimedia development methods and data collection techniques in the form of observations, interviews and literature studies. The purpose of making interactive applications is so that students can understand the range classification and topology well, and can practice installing LAN cables through interactive animation simulation software as a substitute for school facilities.

PCA and projection pursuits on high dimensional data reduction

The flood of multimedia data has affected various fields of research, including multimedia retrieval, database management, data mining, machine learning, social media analysis, image processing and so on. Many digital multimedia data collections are made in various fields and application contexts not only in advertising or journalism, but also often used in the context of cultural heritage applications, web search, or geographical information. Digital multimedia data is created in various formats and media modalities and available worldwide every day. The higher the quantity of data, the higher the complexity, diversity, dimensions and multy modality, so that multimedia data grows exponentially. To reduce the impact of high-dimensional data, data reduction is carried out. Data reduction plays an important role in classification. Therefore the selection of data reduction techniques must be done carefully. Principal Component Transform is widely used in data reduction. This analysis attempts to eliminate correlations between bands and further determines the optimal linear combination of original data dimensions for variations in the value of data pixels. The mathematical principle of Principal Component depends on the decomposition of eigenvalues from the covariance matrix of the data matrix. When multimedia data is smaller or covered by data noise, Principal Component is no longer efficient at reducing. To overcome this, Projection Pursuit was introduced using sphering matrices and invariant translations in the transformations. In Projection Pursuits, only data that is considered interesting will be transformed. Optimization of reduced data selection is based on the maximum value of the projection index generated. The projection index used is Skewness and Kurtosis because it is considered in accordance with the data used, the CASI image of Bogor city.

Data Collection Strategies for Sharing Lived Experiences: Low-Income Mothers’ Perceptions of Text (SMS) and Multimedia (MMS) Data Collection

The purpose of this study was to explore low-income lone mothers’ perceptions of their engagement with a text (SMS) and multimedia message (MMS) qualitative study. Study participants were asked to submit text and pictures via SMS and MMS that represented their reflections, observations, and experiences accessing community support services over a 6-week period. After engaging in the study, participants were asked to complete an evaluation questionnaire. The resoundingly positive feedback received in the evaluation—and the researchers’ satisfaction with the data collected—suggests that researchers should consider employing SMS and MMS data collection strategies to gain open-ended insights into the daily experiences of marginalized groups. This method may be particularly well suited to feminist research designs and research with populations underrepresented in the literature due to barriers presented by traditional data collection strategies.

Image collections clustering in large databases on the basis of recurrent optimization

The following paper considers methods for clustering large amounts of data and proposes a modification of the density-based approach to clustering multimedia objects with disturbance. The analysis of the existing DENCLUE method is carried out, and the matrix influence function is introduced, which makes it possible to effectively use this approach in the analysis of multidimensional objects, the collections of images, video and multimedia data in particular. The introduced matrix form makes it possible to increase the speed of clustering due to the absence of vectorization-devectorization of the initial data.

A Mobile Multimedia Data Collection Scheme for Secured Wireless Multimedia Sensor Networks

Wireless Multimedia Sensor Networks (WMSNs) produce enormous amounts of big multimedia data. Due to large size, Multimedia Sensor Nodes (MSNs) cannot store generated multimedia data for a long time. In this scenario, mobile sinks can be utilized for data collection. However, due to vulnerable nature of wireless networks, there is a need for an efficient security scheme to authenticate both MSNs and mobile sinks. In this paper, we propose a scheme to protect an underlying WMSN during mobile multimedia data collection. The proposed scheme is a two-layer scheme. At the first layer, all MSNs are distributed into small clusters, where each cluster is represented by a single Cluster Head (CH). At the second layer, all CHs verify identities of mobile sinks before sharing multimedia data. Authentication at both layers ensures a secure data exchange. We evaluate the performance of proposed scheme through extensive simulation results. The simulation results prove that the proposed scheme performs better as compared to existing state-of-the-art approaches in terms of resilience and handshake duration. The proposed scheme is also analyzed in terms of authentication rate, data freshness, and packet delivery ratio, and has shown a better performance.

Unsupervised Similarity Learning through Rank Correlation and kNN Sets

The increasing amount of multimedia data collections available today evinces the pressing need for methods capable of indexing and retrieving this content. Despite the continuous advances in multimedia features and representation models, to establish an effective measure for comparing different multimedia objects still remains a challenging task. While supervised and semi-supervised techniques made relevant advances on similarity learning tasks, scenarios where labeled data are non-existent require different strategies. In such situations, unsupervised learning has been established as a promising solution, capable of considering the contextual information and the dataset structure for computing new similarity/dissimilarity measures. This article extends a recent unsupervised learning algorithm that uses an iterative re-ranking strategy to take advantage of different k -Nearest Neighbors (kNN) sets and rank correlation measures. Two novel approaches are proposed for computing the kNN sets and their corresponding top- k lists. The proposed approaches were validated in conjunction with various rank correlation measures, yielding superior effectiveness results in comparison with previous works. In addition, we also evaluate the ability of the method in considering different multimedia objects, conducting an extensive experimental evaluation on various image and video datasets.

DDSV: Optimizing Delay and Delivery Ratio for Multimedia Big Data Collection in Mobile Sensing Vehicles

The large number of mobile-sensing vehicles traveling in cities offer a novel solution to the collection of vast amounts of multimedia data packets. When a vehicle passes through the data center (DC), the collected multimedia data packets will be transmitted to the DC. Due to the mobile characteristic of vehicular sensor networks, the main challenge lies in how to improve the multimedia data delivery ratio and balance the data packet collections. In this paper, in consideration of delay and delivery factors, a novel routing method is proposed to optimize multimedia data collections in mobile sensing vehicles (DDSVs). This method targets at balancing multimedia data collections, improving the delivery ratio of the multimedia data, and reducing the delay ratio in Internet of Things (IoT) networks. In the DDSV scheme, two rules are designed for improving the collection of multimedia data in the IoT. These rules pertain to: 1) data and 2) vehicular priorities. First, different regions hold different priorities of data packet transmission, which can improve the delivery ratio in the suburban areas and reduce the delay ratio. Meanwhile, this scheme is capable of guaranteeing the balance of multimedia data collection. Second, the vehicular priority is proportional to the probability of a vehicle reaching a DC. Therefore, the data should be forwarded to vehicles with higher priorities, that is, the vehicles which are more likely to pass by the DC. By using these two rules, the DDSV scheme can improve the performances of the multimedia data delivery ratio, compared with the conventional optimal vehicular data forwarding scheme. In the simulation experiments, the DDSV scheme utilizes multidatasets of Beijing city, where the average delay for data collection can be decreased by 17.3% in general, and by 41.8% in the suburban areas; the average data delivery ratio can be improved by 16.9% in comparison to the previous studies.

Guest Editorial: Large-Scale Multimedia Data Retrieval, Classification, and Understanding

The papers in this special section focus on multimedia data retrieval and classification via large-scale systems. Today, large collections of multimedia data are explosively created in different fields and have attracted increasing interest in the multimedia research area. Large-scale multimedia data provide great unprecedented opportunities to address many challenging research problems, e.g., enabling generic visual classification to bridge the well-known semantic gap by exploring large-scale data, offering a promising possibility for in-depth multimedia understanding, as well as discerning patterns and making better decisions by analyzing the large pool of data. Therefore, the techniques for large-scale multimedia retrieval, classification, and understanding are highly desired. Simultaneously, the explosion of multimedia data puts urgent needs for more sophisticated and robust models and algorithms to retrieve, classify, and understand these data. Another interesting challenge is, how can the traditional machine learning algorithms be scaled up to millions and even billions of items with thousands of dimensionalities? This motivated the community to design parallel and distributed machine learning platforms, exploiting GPUs as well as developing practical algorithms. Besides, it is also important to exploit the commonalities and differences between different tasks, e.g., image retrieval and classification have much in common while different indexing methods evolve in a mutually supporting way.

A Novel Probability based Approach for Optimized Prefetching

As the World Wide Web carries on to grow up rapidly in size and popularity, web traffic and network bottlenecks are more important issues in the networked world. The continued enhancement in demand for items on the World Wide Web causes severe overloading in many sites, network congestion, delay in perceived latency and network bottleneck. Many users have no patience in waiting more than a few seconds for downloading a web page, that's why Web traffic reduction system is very necessary in today World Wide Web for accessing the websites efficiently with the facility of existing networks. Web caching is an effective method to improve the performance of the World Wide Web but in today's World Wide Web caching method alone is not enough because of World Wide Web has grown quickly from a simple information-sharing mechanism to a rich collection of dynamic objects and multimedia data. The web prefetching is used to improve the performance of the proxy server. Prefetching predict web object that is expected to be requested in the near future and store them in advance, thus the response time of the user request is reduced. To improve the performance of the proxy server, this paper proposed a new framework which combines the caching system and prefetching technique and also optimize the prefetching with the help of probability. In this paper, we use the dataset for the experiment which is collected from ircache.net proxy server and give the result with the comparison of other technique of prefetching.

Research of an Adaptive Aggregation Routing Algorithm in Wireless Sensor Networks

At present, most aggregation routing algorithms for WSNs assume that aggregation expends are so little as to be neglected. But with the growing demand for the collection of multimedia data, the aggregation expends are much larger and can’t be neglect. In view of this problem, this paper proposes an adaptive aggregation routing algorithm with the minimal energy consumption (RMEAAT). Firstly, it constructs an -balancing spanning tree on the basis of SPT and MST as the initial communication path. And then it defines the aggregation benefit with consideration of aggregation expend and transmission expend, and data are adaptively judged whether to aggregate at every node of -balancing spanning tree according to aggregation benefit in the process of transmission. Correspondingly, the initial transmission path is gradually improved in terms of the aggregation judgment. The simulation results show that RMEAAT algorithm has better performance in WSNs with different aggregation expends and fusion degrees.

Ethnography goes online: towards a user-centred methodology to research interpersonal communication on the internet

Ethnographic research is increasingly concerned with how the internet operates within our everyday life. This article attempts to offer a methodological contribution of online communication and an exploration of initial empirical data generated with this methodology. The article calls for a specification of how ethnography can be applied appropriately to the study of relationships online. It departs from the real versus virtual dichotomy, offering a user-centred methodology to study interpersonal communications on the internet. It suggests the use of three main strategies to pay tribute to the characteristics of uses online: multi-situated, online and offline, and flexible and multimedia data collection methods. This approach facilitates a holistic analysis of the way in which social information and communication technologies operate within society in everyday life. It deals with the problem of defining the setting of research online and proposes an expanded ethnography. The article specifies details of this methodology for research into interpersonal communications and emotions online. It does so by drawing on empirical data generated in a study on everyday life and emotions on the internet. Epistemic questions related to this methodological approach will also be discussed. Overall, the exemplification suggests that the methodological approach proposed here is able to capture the uses and understandings of the internet.

Guest Editor's Introduction: Special Section on the IEEE Symposium on Visual Analytics Science and Technology (VAST)

Guest Editor's Introduction: Special Section on the IEEE Symposium on Visual Analytics Science and Technology (VAST)

"You Tube and I Find" - Personalizing Multimedia Content Access

Recent growth in broadband access and proliferation of small personal devices that capture images and videos has led to explosive growth of multimedia content available everywhere - from personal disks to the Web. While digital media capture and upload has become nearly universal with newer device technology, there is still a need for better tools and technologies to search large collections of multimedia data and to find and deliver the right content to a user according to her current needs and preferences. A renewed focus on the subjective dimension in the multimedia lifecycle, from creation, distribution, to delivery and consumption, is required to address this need beyond what is feasible today. Integration of the subjective aspects of the media itself - its affective, perceptual, and physiological potential (both intended and achieved), together with those of the users themselves will allow for personalizing the content access, beyond today's facility. This integration, transforming the traditional multimedia information retrieval (MIR) indexes to more effectively answer specific user needs, will allow a richer degree of personalization predicated on user intention and mode of interaction, relationship to the producer, content of the media, and their history and lifestyle. In this paper, we identify the challenges in achieving this integration, current approaches to interpreting content creation processes, to user modelling and profiling, and to personalized content selection, and we detail future directions. The structure of the paper is as follows: In Section I, we introduce the problem and present some definitions. In Section II, we present a review of the aspects of personalized content and current approaches for the same. Section III discusses the problem of obtaining metadata that is required for personalized media creation and present eMediate as a case study of an integrated media capture environment. Section IV presents the MAGIC system as a case study of capturing effective descriptive data and putting users first in distributed learning delivery. The aspects of modelling the user are presented as a case study in using user's personality as a way to personalize summaries in Section V. Finally, Section VI concludes the paper with a discussion on the emerging challenges and the open problems.

Enabling MPEG‐7 structural and semantic descriptions in retrieval applications

AbstractThe MPEG‐7 standard supports the description of both the structure and the semantics of multimedia; however, the generation and consumption of MPEG‐7 structural and semantic descriptions are outside the scope of the standard. This article presents two research prototype systems that demonstrate the generation and consumption of MPEG‐7 structural and semantic descriptions in retrieval applications. The active system for MPEG‐4 video object simulation (AMOS) is a video object segmentation and retrieval system that segments, tracks, and models objects in videos (e.g., person, car) as a set of regions with corresponding visual features and spatiotemporal relations. The region‐based model provides an effective base for similarity retrieval of video objects. The second system, the Intelligent Multimedia Knowledge Application (IMKA), uses the novel MediaNet framework for representing semantic and perceptual information about the world using multimedia. MediaNet knowledge bases can be constructed automatically from annotated collections of multimedia data and used to enhance the retrieval of multimedia.

Distributed and reactive query planning in R-MAGIC: an agent-based multimedia retrieval system

We present the planning scheme for a cooperative agent-based multimedia retrieval architecture that integrates a heterogeneous set of repositories into a coherent information system. The agents in the system collaborate in context of a conceptual query to formulate unique retrieval strategies for the different collections. The retrieval plan makes need-based use of independent content analysis tools available on the network. The retrieval strategies for the repositories so formulated satisfy the specified constraints on quality of results and the response time requirements. The retrieval plan is reactively updated based on the retrieval performance at the individual repositories. We present some experimental results to show the effectiveness of the planning scheme for repositories with different characteristics and the scalability of the architecture. We present a prototype implementation of this architecture that integrates a set of dissimilar collections of multimedia data on Indian cultural heritage. A comparison of the retrieval results with some existing Internet search tools proves the effectiveness of the architecture.

Providing definition and temporal structure for multimedia data

Currently, multimedia (MM) applications are prepared in an ad hoc fashion. Images or video are usually produced by the MM author. The MM application itself is composed with a specific combination of a hardware platform and authoring software, and it typically contains its own specific data. Today, MM data are usually associated with a particular platform or application, but future MM developers will use repositories of reusable data shared among various applications and across various platforms. The layered multimedia data model (LMDM) emphasisizes the sharing of data and components by dividing the process of MM application development into smaller, more manageable pieces. In particular, the LMDM emphasizes the conceptual separation, manipulation, and presentation of data. When combined with a collection of MM data, the LMDM enables description of a wide variety of MM compositions. This paper describes the first two layers of the LMDM, those concerned with data definition and with the specification of temporal structures that can be reused by various MM presentations. Several examples demonstrate the advantages of the layered paradigm: modular, reusable components and isolation of platform dependencies.