Social Multimedia Content Research Articles

Social media interaction and analytics for enhanced educational experiences

In this paper we present an innovative system based on social media and high quality multimedia content delivery designed to enhance the educational experiences of students in cultural centers and museums. The system brings together students that are viewing an educational movie with a panel of experts that may be geographically dispersed, and engages them in an exciting social media experience via which they can interact with each other as well as with the remote experts. We employ appropriate methodologies for monitoring and analysing social network activity in order to extract valuable information from the social network activity and use it to seamlessly annotate in real-time the presented educational movie using anthropocentric semantic extraction based on MPEG-7. In this way, the multimedia content of the presented movie can be easily reviewed by multiple geographically dispersed individuals, and impoved afterwards according to the received feedback, thus maximising the benefit for students, teachers and content producers alike.

Context-adaptive blocking for protecting personal information exposed to social multimedia content

With the development of many devices that support high-speed wired and wireless Internet, it has been universalized to upload and download various kinds of social multimedia information. However, image data including sensitive personal information are easily exposed to Internet users. In this study, the algorithm is introduced to make it possible to exclude background regions from various types of input color images, to robustly detect the target regions including personal information, and then effectively protect the detected target regions through context-adaptative blocking. In this study, the background regions are first excluded from input color images, and then only the target regions including personal information are robustly segmented on the basis of human skin color. Subsequently, the detected target regions are effectively blocked in the way of selecting regional blurring adaptively in line with surrounding situations, and therefore it is possible to prevent personal information from being exposed. The experimental result of this study reveals that the proposed method robustly blocks the target object regions including personal information of input images in line with surrounding situations. The algorithm proposed in this study is expected to be applied practically to many fields associated with image processing, such as video surveillance, video monitoring, and target object covering.

Scenario-based cyber attack·defense education system on virtual machines integrated by web technologies for protection of multimedia contents in a network

Security issues have been raised with the tremendous growth of social multimedia content evolving content distribution and social interaction. To protect the social multimedia contents in a network, information security education has been actively studied, and various educational systems are being provided. Even when trainees have completed the training that they can practice, such as DDoS(Distributed Denial of Services) response and database outbreak attack, most of the education methods are based on the theory or advance notification method designed as part of the scenario beforehand and it is difficult to apply these to actual situations. In addition, building new systems and learning to use them for each scenario lowers the training effectiveness. In this paper, we investigate various cases of cyber incident in the content distribution networks. Based on research, we develop Meltdown, Mirai malware, Carbanak APT(Advanced Persistent Threats), and Ransomware scenarios that assume a real multimedia contents distribution situation. We also build and store individual virtual environments for each scenario, integrating them into VMWare ESXI, so that attack and defense practices can be conducted similar to real world networks. The deployed environment is integrated with the Web through the VM API(Virtual Machine Application Programming Interfaces). When the trainee finishes the training, one utilizes the snapshot function of the virtual machine to return the virtual environment to the pre-training state. Trainees do not need to build a virtual environment for each scenario, but they are able to conduct training in an environment similar to a real network by calling a virtual environment with a simple operation from a web browser. Therefore, it is expected that the convenience of trainees and the efficiency of the education will be increased.

A product reputation framework based on social multimedia content

PurposeThe purpose of this paper is to bring together the textual and multimedia opinions, since the use of social data has become the new trend that enables to gather the product reputation traded in social media. Integrating a product reputation process into the companies' strategy will bring several benefits such as helping in decision-making regarding the current and the new generation of the product by understanding the customers’ needs. However, image-centric sentiment analysis has received much less attention than text-based sentiment detection.Design/methodology/approachIn this work, the authors propose a multimedia content-based product reputation framework that helps in detecting opinions from social media. Thus, in this case, the analysis of a certain publication is made by combining their textual and multimedia parts.FindingsTo test the effectiveness of the proposed framework, a case study based on YouTube videos has been established, as it brings together the image, the audio and the video processing at the same time.Originality/valueThe key novelty is the implication of multimedia content in addition of the textual one with the goal of gathering opinions about a certain product. The multimedia analysis brings together facial sentiment detection, printed text analysis, opinion detection from speeches and textual opinion analysis.

Guest Editorial Trustworthiness in Social Multimedia Analytics and Delivery

The papers in this special issue focus on trustworthiness in multimedia communications. Recently, social multimedia content is being delivered to users with a high quality of experience (QoE) with the advance of multimedia technologies and social networks. However, as a huge amount of social users have various demands to exchange and share multimedia content with each other, it becomes a new challenge for the current social multimedia analytics and delivery to deal with the various attacks perpetrated by malicious users or through spam contents. Therefore, the trust and risk management for social multimedia content based on the social tie of users become of prime importance to face the unpredicted threats and subsequent damage. This Special Section aims to provide a premier forum for researchers working on the trust-based social multimedia analytics and delivery. It also provides the opportunity for both academic and industrial researchers to discuss recent results and provide solutions to the above-mentioned challenges.

Sentiment Analysis of Text Based on Bidirectional LSTM With Multi-Head Attention

Recent years, many scientists address the research on text sentiment analysis of social media due to the exponential growth of social multimedia content. Natural language ambiguities and indirect sentiments within the social media text have made it hard to classify by using traditional machine learning approaches, such as support vector machines, naive Bayes, hybrid models and so on. This article aims to investigate the sentiment analysis of social media Chinese text by combining Bidirectional Long-Short Term Memory (BiLSTM) networks with a Multi-head Attention (MHAT) mechanism in order to overcome the deficiency of Sentiment Analysis that is performed with traditional machine learning. BiLSTM networks, not only solve the long-term dependency problem, but they also capture the actual context of the text. Due to the fact that the MHAT mechanism can learn the relevant information from a different representation subspace by using multiple distributed calculations, the purpose is to add influence weights to the constructed text sequence. The results of the numerical experiments show that the proposed model achieves better performance than the existing well-established methods.

Learning from Collective Intelligence

Feature representation for visual content is the key to the progress of many fundamental applications such as annotation and cross-modal retrieval. Although recent advances in deep feature learning offer a promising route towards these tasks, they are limited in application domains where high-quality and large-scale training data are expensive to obtain. In this article, we propose a novel deep feature learning paradigm based on social collective intelligence, which can be acquired from the inexhaustible social multimedia content on the Web, in particular, largely social images and tags. Differing from existing feature learning approaches that rely on high-quality image-label supervision, our weak supervision is acquired by mining the visual-semantic embeddings from noisy, sparse, and diverse social image collections. The resultant image-word embedding space can be used to (1) fine-tune deep visual models for low-level feature extractions and (2) seek sparse representations as high-level cross-modal features for both image and text. We offer an easy-to-use implementation for the proposed paradigm, which is fast and compatible with any state-of-the-art deep architectures. Extensive experiments on several benchmarks demonstrate that the cross-modal features learned by our paradigm significantly outperforms others in various applications such as content-based retrieval, classification, and image captioning.

Multimodal Analysis and Prediction of Persuasiveness in Online Social Multimedia

Our lives are heavily influenced by persuasive communication, and it is essential in almost any type of social interaction from business negotiation to conversation with our friends and family. With the rapid growth of social multimedia websites, it is becoming ever more important and useful to understand persuasiveness in the context of social multimedia content online. In this article, we introduce a newly created multimedia corpus of 1,000 movie review videos with subjective annotations of persuasiveness and related high-level characteristics or attributes (e.g., confidence). This dataset will be made freely available to the research community. We designed our experiments around the following five main research hypotheses. First, we study if computational descriptors derived from verbal and nonverbal behavior can be predictive of persuasiveness. We further explore combining descriptors from multiple communication modalities (acoustic, verbal, para-verbal, and visual) for predicting persuasiveness and compare with using a single modality alone. Second, we investigate how certain high-level attributes, such as credibility or expertise, are related to persuasiveness and how the information can be used in modeling and predicting persuasiveness. Third, we investigate differences when speakers are expressing a positive or negative opinion and if the opinion polarity has any influence in the persuasiveness prediction. Fourth, we further study if gender has any influence in the prediction performance. Last, we test if it is possible to make comparable predictions of persuasiveness by only looking at thin slices (i.e., shorter time windows) of a speaker's behavior.

Leveraging Cross-Domain Social Media Analytics to Understand TV Topics Popularity

The way we watch television is changing with the introduction of attractive Web activities that move users away from TV to other media. The social multimedia and user-generated contents are dramatically changing all phases of the value chain of contents (production, distribution and consumption). We propose a concept-level integration framework in which users' activities on different social media are collectively represented, and possibly enriched with external knowledge, such as information extracted from the Electronic Program Guides, or available ontological domain knowledge. The integration framework has a knowledge graph as its core data model. It keeps track of active users, the television events they talk about, the concepts they mention in their activities, as well as different relationships existing among them. Temporal relationships are also captured to enable temporal analysis of the observed activity. The data model allows different types of analysis and the definition of global metrics in which the activity on different media concurs with the measure of success.

Joint optimization of admission control and rate adaptation for video sharing over multirate wireless community cloud

Emerging wireless community cloud enables usergenerated video content to be shared and consumed in a social context. However, the nature of shared wireless medium and time-varying channels seriously limits the quality of service (QoS), partially owing to the lack of mechanisms for effectively utilizing multi-rate channel resources. In this paper, the joint optimization of admission control and rate adaptation is proposed, resulting in a bandwidth-aware rate-adaptive admission control (BRAC) scheme to provide bandwidth guarantee for sharing social multimedia contents. The analytical approach leads to the following major contributions: (1) a bandwidth-aware rate selection (BRS) algorithm to optimally meet the bandwidth requirement of the data session and channel conditions at the physical layer; (2) a routing-coupled rate adaption and admission control algorithm to admit data sessions with bandwidth guarantee. Moreover, extensive numerical simulations suggest that BRAC is efficient and effective in meeting the bandwidth requirements for sharing social multimedia contents. These insights will shed light on communication system implementation for multimedia content sharing over multirate wireless community cloud.

Exploiting concept drift to predict popularity of social multimedia in microblogs

Microblogging services such as Twitter and Plurk allow users to easily access and share different types of social multimedia (e.g. images and videos) in the cyber world. However, the massive amount of information available causes information overload, which prevents users from quickly accessing popular and important digital content. This paper studies the problem of predicting the popularity of social multimedia content embedded in short microblog messages. A property of social multimedia is that it can be continuously re-shared, thus its popularity may revive or evolve over time. We exploit the idea of concept drift to capture this property. We formulate the problem using a classification-based approach and propose to tackle two tasks, re-share classification and popularity score classification. Two categories of features are devised and extracted, including information diffusion and explicit multimedia meta information. We develop a concept drift-based popularity predictor by ensembling multiple trained classifiers from social multimedia instances in different time intervals. The key idea lies in dynamically determining the ensemble weights of classifiers. Experiments conducted on Plurk and Twitter datasets show the high accuracy of the popularity classification and the results on detecting popular social multimedia are promising.

Secure social multimedia content distribution based on social network analysis

Multimedia fingerprinting is an effective technique to trace traitor who redistribute copyrighted social multimedia contents. However, several traitors can make an average collusion attack with their fingerprinted copies to avoid to be traced by digital fingerprints; furthermore, the commercial value of the colluded fingerprinted copy is often time-sensitive in social networks. The earlier the colluded content being redistributed in social network, the more profit the colluder will make from it. In this paper, a new desynchronisation fingerprinting is proposed. Random grid templates are used to produce desynchronised copies. This research has dealt with random distortions due to local geometric transformations of random grid template of image based on chaos, and then the processed fingerprinted content is distributed using social network analysis. The experimental results show that the average collusion will produce collude copy with low commercial value, and although colluders can try to use image registration to resynchronisation with expensive computational cost, and the visual quality does not improve with the increase of the number of copies.

Secure social multimedia content distribution based on social network analysis

Multimedia fingerprinting is an effective technique to trace traitor who redistribute copyrighted social multimedia contents. However, several traitors can make an average collusion attack with their fingerprinted copies to avoid to be traced by digital fingerprints; furthermore, the commercial value of the colluded fingerprinted copy is often time-sensitive in social networks. The earlier the colluded content being redistributed in social network, the more profit the colluder will make from it. In this paper, a new desynchronisation fingerprinting is proposed. Random grid templates are used to produce desynchronised copies. This research has dealt with random distortions due to local geometric transformations of random grid template of image based on chaos, and then the processed fingerprinted content is distributed using social network analysis. The experimental results show that the average collusion will produce collude copy with low commercial value, and although colluders can try to use image registration to resynchronisation with expensive computational cost, and the visual quality does not improve with the increase of the number of copies.

When Location Meets Social Multimedia

Coming with the popularity of multimedia sharing platforms such as Facebook and Flickr, recent years have witnessed an explosive growth of geographical tags on social multimedia content. This trend enables a wide variety of emerging applications, for example, mobile location search, landmark recognition, scene reconstruction, and touristic recommendation, which range from purely research prototype to commercial systems. In this article, we give a comprehensive survey on these applications, covering recent advances in recognition and mining of geographical-aware social multimedia. We review related work in the past decade regarding to location recognition, scene summarization, tourism suggestion, 3D building modeling, mobile visual search and city navigation. At the end, we further discuss potential challenges, future topics, as well as open issues related to geo-social multimedia computing, recognition, mining, and analytics.

Using Social Multimedia Content to Inform Emergency Planning of Recurring and Cyclical Events in Local Communities

AbstractSocial media has been widely adopted for assisting the efforts in emergency response and recovery, but it has been underutilized for emergency planning purposes. Emergency planning in a local community context must leverage accessible and free resources such as social media, because it is largely a volunteer enterprise. We describe our fieldwork with local annual festival emergency planning teams that led to the design of the Community Incident Report (CIR). CIR is a novel emergency planning system that externalizes community knowledge on persisting issues and common mitigation strategies by integrating police reports, local crisis information, and social multimedia content to foster citizens’ awareness of local emergency information and to assist emergency planners in planning for recurring and cyclical events. We provide a use case analysis of CIR and its evaluation with 20 local residents, and discuss how it could be extended to inform emergency planning for other community events and local municipalities that share similar characteristics.

High-efficient video compression for social multimedia distribution

To reduce data transmission bandwidth and storage space, the compression of original video is a prerequisite for social multimedia content uploading and distribution. The paper proposes an efficient early detection algorithm of all-zero blocks to eliminate redundant computations in H.264/AVC video encoding. First, for DCT-like integer transforms, a general sufficient condition of early detection of all-zero blocks is given based on the sum of absolute differences of the prediction error block, which is a direct derivation of Sousa's method. Then, by analyzing the dynamic ranges of different frequency components, we further refine the sufficient condition of early detection of all-zero blocks for DCT-like integer transforms and apply it to $$4\times 4$$4×4 and $$8\times 8$$8×8 DCT-like integer transforms in H.264/AVC. Experimental results show that the proposed algorithm for early detection of all-zero blocks outperforms Sousa's method in complexity reduction, where computations in transform, quantization, inverse quantization and inverse transform are significantly reduced. In addition, the proposed general sufficient condition of early detection of all-zero blocks for H.264/AVC can also be easily extended to other forms of DCT-like integer transforms. As a result, better audience experience and more efficient analysis can be provided for social multimedia applications.

Triangulating Social Multimedia Content for Event Localization using Flickr and Twitter

AbstractThe analysis of social media content for the extraction of geospatial information and event‐related knowledge has recently received substantial attention. In this article we present an approach that leverages the complementary nature of social multimedia content by utilizing heterogeneous sources of social media feeds to assess the impact area of a natural disaster. More specifically, we introduce a novel social multimedia triangulation process that uses both Twitter and Flickr content in an integrated two‐step process: Twitter content is used to identify toponym references associated with a disaster; this information is then used to provide approximate orientation for the associated Flickr imagery, allowing us to delineate the impact area as the overlap of multiple view footprints. In this approach, we practically crowdsource approximate orientations from Twitter content and use this information to orient Flickr imagery accordingly and identify the impact area through viewshed analysis and viewpoint integration. This approach enables us to avoid computationally intensive image analysis tasks associated with traditional image orientation, while allowing us to triangulate numerous images by having them pointed towards the crowdsourced toponym location. The article presents our approach and demonstrates its performance using a real‐world wildfire event as a representative application case study.

Social Multimedia Computing

The explosive growth of social multimedia content on the Internet is revolutionizing content distribution and social interaction. It has even led to a new research area, called social multimedia computing.

GameSense: game-like in-image advertising

Considering the continuously increasing availability and accessibility of multimedia contents via social networking sites, our research addresses how to monetize the social multimedia contents with an efficient advertising approach. This paper presents a novel game-like advertising system called GameSense, which is driven by the compelling contents of online images. The contextually relevant ads (i.e., product logos) are embedded at appropriate positions within the online games, which are created on the basis of online images. The ads are selected based on multimodal relevance, i.e. text relevance, user relevance and visual content similarity. The game is able to provide viewers rich experience and thus promotes the embedded ads to provide more effective advertising. GameSense represents one of the first attempts toward effective online mashup applications which connect a photo-sharing site with an advertising agency. The effectiveness of GameSense is evaluated over a large-scale real world image set.