Pervasive Video Research Articles

Pervasive Video and Audio

The articles in this special section focus on pervasive video and audio. For a long time, cameras and microphones have been studied to give machines the ability to see and hear the world around them. The remarkable progress in artificial intelligence (AI) in recent years has pushed this perception ability to a new high. AI-enabled cameras and microphones are now able to replicate human-like reasoning and cognition, be it detecting cancers better than a human doctor, identifying people’s sexual orientation, or understanding human speech. Naturally, these interminable possibilities across a wide range of verticals, ranging from public safety, infrastructure efficiency, entertainment, education, and health, have led to the deployment of billions of cameras and microphones around us—in our homes, workplaces, and cities.

Low Quality for High Quality: Exploiting 2K Frames for Supporting Efficient 4K-Quality Pervasive Video Streaming Applications

Emerging cloud-based video streaming applications suggest limitations on current video streaming systems and emphasize the need to preserve and exchange high-quality contents while minimizing processing latency, given that both are critical in perpetuating a high user-perceived quality. This article presents 4KODEC, a codec for supporting real-time high-quality videos for cloud-based video streaming. 4KODEC is designed to induce minimal latency when encoding and decoding 4K video frames, by exploiting the fact that many regions (tiles) of a 4K frame can be recovered from their 2K counterparts given the low-complexity embedded in most tiles. Furthermore, 4KODEC is designed to effectively identify complex tiles, so that original 4K representations can be selectively exchanged to maintain high quality. We present evaluations using four different 4K-quality online games to show that 4KODEC reduces the processing latency by nearly half while improving perceptual quality when compared to widely used H.264/265 codec-based 4K video streaming implementations.

Pervasive Computing at the Edge

Today the infrastructure needed to support pervasive computing and the Internet of Things (IoT) is unparalleled as entirely new classes of applications and systems emerge. For example, pervasive systems designed to augment human cognition with tasks such as face recognition must operate at “superhuman speeds,” delivering insights to help with human decision-making within very strict and narrow time limits. Similarly, the emergence of pervasive video analytics demands processing of very large volumes of video data in near-real-time. In general, the field of pervasive computing is rapidly changing in the face of major advances in sensing, data processing techniques, and wearable computing. The ever increasing data rates of high-speed networks also factor into the design tradeoff to decide if computing should be local, or remote.

Multicast and Broadcast Services Over Mobile Networks: A Survey on Standardized Approaches and Scientific Outcomes

Due to the increasing demand for pervasive video and broadcast-like applications, multicast and broadcast communications are expected to assume an important role in upcoming 5G systems. The current research trend is trying to reuse, extend, or adapt reference transmission strategies already designed for the conventional 4G technology. Nevertheless, apart the two reference and standardized methodologies, i.e., multimedia broadcast/multicast service and single cell-point to multipoint, many technical extensions and novel solutions were published in the literature so far. Therefore, in order to provide a clear overview on available solutions (already standardized or simply extending conventional approaches), the present work provides a comprehensive survey on network architectures, communication protocols, transmission strategies, and optimization algorithms to improve the performance of multicast communications over mobile radio systems. The core of the conducted study represents a structured taxonomy, able to properly classify scientific contributions based on their reference standard, targeted goal, addressed methodology, considered application domain, and obtained results. Taking into account this taxonomy, more than one hundred of scientific contributions are presented, classified, and reviewed. The study of the state-of-the-art is further enhanced with the discussion on important lessons learned, which clearly highlight the pros and cons of any investigated approach. A focus is also provided on the main issues on long term evolution multicasting that need to be better investigated, and determine the possible future research directions on this subject. The final goal of this paper is to support research activities devoted to the identification of promising methodologies, that efficiently support the delivery of real-time and on-demand video contents in a TV-like fashion.

Understanding trust in privacy-aware video surveillance systems

Recent advances in pervasive video surveillance systems pave the way for a comprehensive surveillance of every aspect of our lives, hence, leading us to a state of dataveillance. Computerized and interconnected systems of cameras could be used to profile, track and monitor individuals for the sake of security. Notwithstanding, these systems clearly interfere with the fundamental right of the individuals to privacy. Most literature on privacy in video surveillance systems concentrates on the goal of detecting faces and other regions of interest and in proposing different methods to protect them. However, the trustworthiness of those systems and, by extension, of the privacy they provide are mostly neglected. In this article, we define the concept of trustworthy privacy-aware video surveillance system. Moreover, we assess the techniques proposed in the literature according to their suitability for such a video surveillance system. Finally, we describe the properties that a deployment of a trustworthy video surveillance system must fulfill.

A Deep Awareness Framework for Pervasive Video Cloud

Context-awareness for big data applications is different from that of traditional applications in that it is getting challenging to obtain the contexts from big data due to the complexity, velocity, variety, and other aspects of big data, especially big video data. The awareness of contexts in big data is more difficult, and should be more in-depth than that of classical applications. Therefore, in this paper, we propose an in-depth context-awareness framework for a pervasive video cloud in order to obtain underlying contexts in big video data. In this framework, we propose an approach that combines the historical view with the current view to obtain meaningful in-depth contexts, where deep learning techniques are used to obtain raw context data. We have conducted initial evaluations to show the effectiveness of the proposed approach in terms of performance and also the accuracy of obtaining the contexts. The evaluation results show that the proposed approach is effective for real-time context-awareness in a pervasive video cloud.

Video over IEEE802.11 wireless LAN: A brief survey

With relatively high transmission capacity and usually unconstrained connections, IEEE802.11 WLANs provide the ideal infrastructure for pervasive video content sharing and communications. However, the delivery of high-performance video streams over 802.11 WLANs remains a challenging task due to the inherent characteristics of compressed video and dynamic channels. In this paper, we present a brief survey of various recent innovations that have been developed to enhance the Quality of Service (QoS) performance for video over WLANs. Based on the application scenarios, the solutions have focused mainly on three network layers, that is, Application layer (APP), Media Access Control layer (MAC), and Physical layer (PHY). After reviewing the video compression technology, we first examine various single-layer solutions for video over WLANs. We then discuss several cross-layer solutions that take advantage of mutual interactions between different network layers. Finally, several technical issues beyond QoS performance, including energy and security, are also addressed. We conclude that the application of video over WLANs will continue to increase in future.

QoE management for video conferencing applications

We introduce a framework for managing the QoE of videos coded with the H.264 codec and transmitted by video conferencing applications through limited bandwidth networks. We focus our study on the medium-motion videos with QCIF, CIF, and VGA resolutions, the most pervasive video formats used by video conferencing applications across the Internet and cellular telephony systems. Using subjective tests for measuring the level of video quality perceived by end users, we expose the relation between the main influential video parameters and the quality experienced by end users. Furthermore, after investigating the effect of different frame rates and compression levels on video streaming bit rate, and consequently on QoE, we propose a QoE control mechanism for limited-bandwidth situations. A congestion control technique is also introduced in this paper and used in simulations for verifying the efficiency of the proposed QoE management algorithm and to implement this algorithm for practical applications.

A two-stage multimodal speaker location-aware approach in pervasive computing

Location-aware computing is important in pervasive computing and intelligent video surveillance. We propose a two-stage multimodal approach to locate the active speaker in intelligent environments. Firstly, human voice is captured as audio cue to find the approximate orientation of current speaker. Secondly, the colour feature of mouth region is extracted as visual cue to detect continuous mouth motion that identifies the active speaker. The speaking recognition is conducted by a well-trained Hidden Markov Model based on colour feature of mouth region during continuous motion. Experiments show that the proposed multimodal approach is effective for speaker localisation in intelligent indoor environments.

Power and Distortion Optimization for Pervasive Video Coding

This paper addresses the problem of how to efficiently manage the power consumption while preserving high visual quality performance in video coding for pervasive video applications. We define an optimization problem that corresponds to the minimization of both the power consumption and video distortion of a video encoder. A joint rate-complexity-distortion(R-C-D) model is developed to capture and control the general R-C-D behavior of video coding. Based on the analysis of the proposed model, optimization strategies are developed to achieve good video quality while maximizing the battery service life. We demonstrate how the observations and optimization strategies can be utilized in a practical parameterized encoder under various preferences of quality and power. We perform simulations on an instruction set simulator to exhibit the effectiveness of the proposed scheme.

City as Screen / Body as Movie

Let's start with a photograph taken by Hiroko Masuke--of a billboard featuring an ad for the television miniseries The Andromeda Strain that was printed in the New York Times a year ago. (1) The billboard includes a large, horizontal poster for the series, along with a video display showing clips embedded within the poster. What's not visible, however, is a small video camera (made by the company Quividi) that records passersby as they look at the billboard. The company has developed what it calls the automated audience measurement solution, which documents visitors who look at the billboard, channeling the information into a database, from which it decodes the data. It examines factors such as the overall height of viewers, as well as facial features, including cheekbone height and the measurement of between the nose and chin. The goal is to determine gender and age, and although the company says it does not yet factor for race, it plans to soon. This essay takes as its topic urban video and screens, umbrella terms that include the multiple cameras and projection surfaces that characterize the topology of contemporary urban space. At any given time in an urban setting we are participants within an information space--as subjects surveilled and captured by video cameras; as users of cell phones and PDAs that allow for multiple and layered interactions with data, from telephone usage to web-based activities, photography, and video capture; as users of ATM screens; as viewers of the media displays on terminals at train stations and airports; as readers of assorted screen-based texts; as viewers of video ads; and as subjects interacting with a broad range of specific media streams. In these capacities, we exist as both networked subject and object, viewer and viewed. We are imbricated within networks, often without our knowledge, while interacting with others. In short, we continually negotiate innumerable interfaces, are hailed as subjects and enact our subjectivities in the midst of continual media flux. Urban screens are just one aspect of a larger set of screens that build a network around us. As such, screens are no longer simply dedicated to display; instead, they become components of what has been described as augmented space and mixed reality, (2) namely physical spaces overlaid with data through various devices and technologies. Academic interest in the urban networked sphere stems from the specific, yet convergent, media from which its iterations derive, namely the traditions of cinema, video, public art, and an emerging participatory culture, some of whose participants borrow from and extend the tactical and politicized activist media practices of earlier decades. However, urban screens also intersect with other categories and disciplines: * as integral extensions of the built environment, they are discussed in the field of architectural theory and debates about the intersection of the digital and material as articulated by architects designing and constructing actual buildings; * because screens often display advertising, they are also included in discussions about corporate branding and the interface between markets and individuals; * because they dot the urban landscape, sometimes hovering between private and public space, these screens serve as the focal point for discussions about political power and public as a necessary component of democracy. However, in the context of this particular issue of Afterimage, I want to address what kinds of literacies are being shaped by pervasive video in public space. In spaces rife with advertising, what rhetorical strategies are deployed by users of this space, whether corporations, governments, artists, and others, to speak to viewers? And how are everyday viewers responding with their own images in public? As cities become screens, and bodies become movies, where do we situate empowerment and literacy? …

At Home with Video Cameras

ABSTRACTThis article presents a study in which French families have been observed over one week each, with the aim of contributing to the understanding of home activities through video ethnography. This study also seeks to inform the design of home technology (ubiquitous computing) by examining the details of daily life. We used an ethnomethodogically informed reflexive approach to collected video data that meant including household members' orientations to the recording equipment in our analysis. This shows how examining these orientations is valuable not only in terms of understanding family activities at home but also in terms of informing the design of new technology. We demonstrate how the presence of recording equipment can play a role in the routine accomplishment of home life. Some implications of these analyses for the design of home technology are discussed, by focusing on the potential significance of introducing pervasive video cameras in the domestic sphere.

From Disciplinary Gaze to Biological Gaze: Genetic Crime Thrillers and Biogovernance

Imagine an America where violent crime rates have quadrupled over the past century, inexorably rising to a level that would be incomprehensible to previous generations. Imagine that the nature of violent crime seems to have changed, tak­ing a turn toward casual sociopathy where the representative crime has become calculated serial murder rather than a one-off crime of passion. Imagine the resulting level of public fear, rampant to the point where civic-mindedness is replaced by a fortress mentality, walled communities, pervasive video surveil­lance, and electronic monitoring as the public sphere is virtually abandoned as too risky. In this imaginary America, the fearful public turns angrily upon its gov­ernments, demanding more decisive action and inquiring as to why traditional practices of crime control are not controlling crime.