High-dimensional Multimedia Data Research Articles

Large-Scale Subspace Clustering by Independent Distributed and Parallel Coding.

Subspace clustering is a popular method to discover underlying low-dimensional structures of high-dimensional multimedia data (e.g., images, videos, and texts). In this article, we consider a large-scale subspace clustering (LS2C) problem, that is, partitioning million data points with a millon dimensions. To address this, we explore an independent distributed and parallel framework by dividing big data/variable matrices and regularization by both columns and rows. Specifically, LS2C is independently decomposed into many subproblems by distributing those matrices into different machines by columns since the regularization of the code matrix is equal to a sum of that of its submatrices (e.g., square-of-Frobenius/ l1 -norm). Consensus optimization is designed to solve these subproblems in a parallel way for saving communication costs. Moreover, we provide theoretical guarantees that LS2C can recover consensus subspace representations of high-dimensional data points under broad conditions. Compared with the state-of-the-art LS2C methods, our approach achieves better clustering results in public datasets, including a million images and videos.

Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry, and Fusion

With the development of web technology, multi-modal or multi-view data has surged as a major stream for big data, where each modal/view encodes individual property of data objects. Often, different modalities are complementary to each other. This fact motivated a lot of research attention on fusing the multi-modal feature spaces to comprehensively characterize the data objects. Most of the existing state-of-the-arts focused on how to fuse the energy or information from multi-modal spaces to deliver a superior performance over their counterparts with single modal. Recently, deep neural networks have been exhibited as a powerful architecture to well capture the nonlinear distribution of high-dimensional multimedia data, so naturally does for multi-modal data. Substantial empirical studies are carried out to demonstrate its advantages that are benefited from deep multi-modal methods, which can essentially deepen the fusion from multi-modal deep feature spaces. In this article, we provide a substantial overview of the existing state-of-the-arts in the field of multi-modal data analytics from shallow to deep spaces. Throughout this survey, we further indicate that the critical components for this field go to collaboration, adversarial competition, and fusion over multi-modal spaces. Finally, we share our viewpoints regarding some future directions in this field.

Intelligent computational techniques for multimodal data

This guest editorial introduces the special issue on “Intelligent Computational Techniques for Multimodal Data”. The goal of this special issue was to solve a variety of real-life problems which have uncertainty, imprecision, vagueness, resulting in high performance applications or prototypes for real time system. The special issue touched different hot topics related to Computer Vision, Computational Biology, Multimedia data mining, High-dimensional multimedia data, Deep convolution network, Deep semantic preserving hashing, High-dimensional multimedia classification, Deep CNN and extended residual units, particle swarm optimization, Cyberbullying detection on social multimedia, Multimedia detection algorithm of malicious nodes, Multimedia based fast face recognition, Diversifying personalized mobile multimedia application, Multimedia image edge extraction algorithm, Rumour veracity detection.

High-dimensional multimedia classification using deep CNN and extended residual units

Processing multimedia data has emerged as a key area for the application of machine learning methods Building a robust classification model to use in high dimensional space requires the combination of a deep feature extractor and a powerful classifier. We present a new classification pipeline to facilitate multimedia data analysis based on convolutional neural network and the modified residual network which can integrate with the other feedforward network style in an endwise training fashion. The proposed residual network is producing attention-aware features. We proposed a unified deep CNN model to achieve promising performance in classifying high dimensional multimedia data by getting the advantages of the residual network. In every residual module, up-down and vice-versa feedforward structure is implemented to unfold the feedforward and backward process into a unique process. The hybrid proposed model evaluated on four datasets and have been shown promising results which outperform the previous best results. Last but not the least, the proposed model achieves detection speeds that are much faster than other approaches.

Early Diagnosis of Alzheimer's Disease Based on Resting-State Brain Networks and Deep Learning.

Computerized healthcare has undergone rapid development thanks to the advances in medical imaging and machine learning technologies. Especially, recent progress on deep learning opens a new era for multimedia based clinical decision support. In this paper, we use deep learning with brain network and clinical relevant text information to make early diagnosis of Alzheimer's Disease (AD). The clinical relevant text information includes age, gender, and ApoE gene of the subject. The brain network is constructed by computing the functional connectivity of brain regions using resting-state functional magnetic resonance imaging (R-fMRI) data. A targeted autoencoder network is built to distinguish normal aging from mild cognitive impairment, an early stage of AD. The proposed method reveals discriminative brain network features effectively and provides a reliable classifier for AD detection. Compared to traditional classifiers based on R-fMRI time series data, about 31.21 percent improvement of the prediction accuracy is achieved by the proposed deep learning method, and the standard deviation reduces by 51.23 percent in the best case that means our prediction model is more stable and reliable compared to the traditional methods. Our work excavates deep learning's advantages of classifying high-dimensional multimedia data in medical services, and could help predict and prevent AD at an early stage.

Multimedia annotation via semi-supervised shared-subspace feature selection

With the rapid development of social network and computer technologies, we always confront with high-dimensional multimedia data. It is time-consuming and unrealistic to organize such a large amount of data. Most existing methods are not appropriate for large-scale data due to their dependence of Laplacian matrix on training data. Normally, a given multimedia sample is usually associated with multiple labels, which are inherently correlated to each other. Although traditional methods could solve this problem by translating it into several single-label problems, they ignore the correlation among different labels. In this paper, we propose a novel semi-supervised feature selection method and apply it to the multimedia annotation. Both labeled and unlabeled samples are sufficiently utilized without the need of graph construction, and the shared information between multiple labels is simultaneously uncovered. We apply the proposed algorithm to both web page and image annotation. Experimental results demonstrate the effectiveness of our method.

Linear Subspace Ranking Hashing for Cross-Modal Retrieval.

Hashing has attracted a great deal of research in recent years due to its effectiveness for the retrieval and indexing of large-scale high-dimensional multimedia data. In this paper, we propose a novel ranking-based hashing framework that maps data from different modalities into a common Hamming space where the cross-modal similarity can be measured using Hamming distance. Unlike existing cross-modal hashing algorithms where the learned hash functions are binary space partitioning functions, such as the sign and threshold function, the proposed hashing scheme takes advantage of a new class of hash functions closely related to rank correlation measures which are known to be scale-invariant, numerically stable, and highly nonlinear. Specifically, we jointly learn two groups of linear subspaces, one for each modality, so that features' ranking orders in different linear subspaces maximally preserve the cross-modal similarities. We show that the ranking-based hash function has a natural probabilistic approximation which transforms the original highly discontinuous optimization problem into one that can be efficiently solved using simple gradient descent algorithms. The proposed hashing framework is also flexible in the sense that the optimization procedures are not tied up to any specific form of loss function, which is typical for existing cross-modal hashing methods, but rather we can flexibly accommodate different loss functions with minimal changes to the learning steps. We demonstrate through extensive experiments on four widely-used real-world multimodal datasets that the proposed cross-modal hashing method can achieve competitive performance against several state-of-the-arts with only moderate training and testing time.

Learning high-dimensional multimedia data

The internet revolution has made information acquisition easy and cheap and has been producing massive high-dimensional multimedia data, including text, audio, images, animation, video, etc. High-dimensional multimedia data bring new opportunities to modern society and challenges to researchers of the multimedia domain as well. The goal of this special issue is bridging the gap between machine learning methods and the real requirement of high-dimensional multimedia domain, aiming at gaining insight into the relationship between the current multimedia and the past ones, and also accurately predicting the future trends of multimedia data. Specifically, this special issue targeted the most recent technical progresses on learning techniques for high-dimensional multimedia data

Learning in high-dimensional multimedia data: the state of the art

During the last decade, the deluge of multimedia data has impacted a wide range of research areas, including multimedia retrieval, 3D tracking, database management, data mining, machine learning, social media analysis, medical imaging, and so on. Machine learning is largely involved in multimedia applications of building models for classification and regression tasks, etc., and the learning principle consists in designing the models based on the information contained in the multimedia dataset. While many paradigms exist and are widely used in the context of machine learning, most of them suffer from the `curse of dimensionality', which means that some strange phenomena appears when data are represented in a high-dimensional space. Given the high dimensionality and the high complexity of multimedia data, it is important to investigate new machine learning algorithms to facilitate multimedia data analysis. To deal with the impact of high dimensionality, an intuitive way is to reduce the dimensionality. On the other hand, some researchers devoted themselves to designing some effective learning schemes for high-dimensional data. In this survey, we cover feature transformation, feature selection and feature encoding, three approaches fighting the consequences of the curse of dimensionality. Next, we briefly introduce some recent progress of effective learning algorithms. Finally, promising future trends on multimedia learning are envisaged.

A Real-Time 3D Visualization Framework for Multimedia Data Management, Simulation, and Prediction

Geographic Information Systems (GISs), which map spatiotemporal event data on geographical maps, have proven to be useful in many applications. Time-based Geographic Information Systems (GISs) allow practitioners to visualize collected data in an intuitive way. However, while current GIS systems have proven to be useful in post hoc analysis and provide simple two-dimensional geographic visualizations, their design typically lacks the features necessary for highly targeted real-time surveillance with the goal of spread prevention. This paper outlines the design, implementation, and usage of a 3D framework for real-time geospatial temporal visualization. In this case study, using livestock movements, the authors show that the framework is capable of tracking and simulating the spread of epidemic diseases. Although the application discussed in this paper relates to livestock disease, the proposed framework can be used to manage and visualize other types of high-dimensional multimedia data as well.

Tertiary hash tree-based index structure for high dimensional multimedia data

Dominant features for the content-based image retrieval usually have high-dimensionality. So far, many researches have been done to index such values to support fast retrieval. Still, many existing indexing schemes are suffering from performance degradation due to the curse of dimensionality problem. As an alternative, heuristic algorithms have been proposed to calculate the answer with ‘high probability’ at the cost of accuracy. In this paper, we propose a new hash tree-based indexing structure called tertiary hash tree for indexing high-dimensional feature data. Tertiary hash tree provides several advantages compared to the traditional extendible hash structure in terms of resource usage and search performance. Through extensive experiments, we show that our proposed index structure achieves outstanding performance.