Fast Video Research Articles

Blockchain-enabled deep semantic video-to-video summarization for IoT devices

The rapid development of multimedia technologies gave birth to large video data that demands effective and fast video summarization methods along with ability to assure the authenticity of digital data. Video summarization provides consolidated view of original video in short and compact form of a video. The summarized videos should be transmitted to remote IoT devices through secure channels. To prevent data manipulation and to verify the transmitted data, we exploited the blockchain technology. This paper focuses on different steps of keyframe extraction from a video followed by secure and trusted transmission to the consumers. The proposed system enables the users to summarize a video based on human and objects as parameters. Cryptographic hashes are used with blockchain, hashes are generated from summarized video blocks, signed and transmitted through blockchain. Cumulus blockchain technique is utilized to ensure the video integrity. System allows remote users to get tamper-proofed summarized video of their business sites or any sensitive premises on their smartphones.

Complexity reduction method for High Efficiency Video Coding encoding based on scene-change detection and image texture information

Recently, as demand for high-quality video and realistic media has increased, High Efficiency Video Coding has been standardized. However, High Efficiency Video Coding requires heavy cost in terms of computational complexity to achieve high coding efficiency, which causes problems in fast coding processing and real-time processing. In particular, High Efficiency Video Coding inter-coding has heavy computational complexity, and the High Efficiency Video Coding inter prediction uses reference pictures to improve coding efficiency. The reference pictures are typically signaled in two independent lists according to the display order, to be used for forward and backward prediction. If an event occurs in the input video, such as a scene change, the inter prediction performs unnecessary computations. Therefore, the reference picture list should be reconfigured to improve the inter prediction performance and reduce computational complexity. To address this problem, this article proposes a method to reduce computational complexity for fast High Efficiency Video Coding encoding using information such as scene changes obtained from the input video through preprocessing. Furthermore, reference picture lists are reconstructed by sorting the reference pictures by similarity to the current coded picture using Angular Second Moment, Contrast, Entropy, and Correlation, which are image texture parameters from the input video. Simulations are used to show that both the encoding time and coding efficiency could be improved simultaneously by applying the proposed algorithms.

Real-time forest smoke detection using hand-designed features and deep learning

In this paper, a fast and accurate video smoke detection algorithm is proposed to extract a powerful feature representation of forest fire. Because of the irregular shape, the variation of colors and hard-to-depict texture of smoke, it is incredibly challenging to detect smoke quickly and accurately in a complex environment. We proposes a smoke detection algorithm that combines manual features and deep learning features, an approach that differs from those of existing detection methods. First, a manual design algorithm is used to extract the suspected smoke area, which is then employed as input to the improved very small deep neural network SqueezeNet model to achieve smoke detection. Extensive detection experiments are conducted, and the results showed that the proposed method is able to differentiate smoke from other challenging objects. Compared with the existing forest fire algorithms, the proposed method has higher classification accuracy and speed, with a more comprehensive application range and lower requirements for hardware devices, and these features are worthy of promotion.

Pruning 3D Filters For Accelerating 3D ConvNets

Many methods have been proposed to accelerate 2D ConvNets by removing redundant parameters. However, few efforts are devoted to the problem of accelerating 3D Convolutional Networks. The 3D ConvNets, which are mainly designed for extracting spatiotemporal features, have been widely used in many video analytics tasks, such as action recognition and scene analysis. In this paper, we focus on accelerating 3D ConvNets for two motivations: (1) Fast video processing techniques are in dire need due to the explosive growth of video data; (2) Compared with individual images, video data consist of consecutively similar frames, thus are inherently more redundant. In this paper, we present a novel algorithm to dramatically accelerate 3D ConvNets by pruning redundant convolutional filters, while preserving the discriminative power of the networks. Specifically, we formulate the filter pruning from 3D ConvNets as a subset selection problem where each filter is regarded as a candidate. Determinantal Point Processes (DPPs) are employed to discriminatively select the filter candidates which are informative and yet diverse. We evaluate our method using two popular 3D networks, C3D and Pseudo-3D, on Sports-1 M dataset for video classification. Extensive experimental results demonstrate both the efficiency and performance advantages of our method. We also show that the proposed method can be easily generalized to 2D ConvNets pruning with promising experimental results on VGGnet and ResNet.

Coalescence Characteristics of Side‐by‐Side Growing Bubbles in Carboxymethyl Cellulose Solutions

AbstractNew experimental data are presented on the coalescence dynamics of twin bubbles at two adjacent nozzles in carboxymethyl cellulose (CMC) solutions. The coalescence process was recorded by using a fast video technique and the images were then analyzed to evaluate the regime, efficiency, and patterns of bubble coalescence. The results reveal that the coalescence process contains the four periods of spherical expansion, rapid mergence, overall growth, and instant departure. The coalescence efficiency always passes through five stages with the gas flow rate; it grows with the CMC solution concentration, but decreases with the surfactant concentration, except at very low gas flow rates. The bubble coalescence pattern strongly depends on the nozzle spacing, but conditionally on the gas flow rate and the solution properties.

A Fast 4K Video Frame Interpolation Using a Multi-Scale Optical Flow Reconstruction Network

Recently, video frame interpolation research developed with a convolutional neural network has shown remarkable results. However, these methods demand huge amounts of memory and run time for high-resolution videos, and are unable to process a 4K frame in a single pass. In this paper, we propose a fast 4K video frame interpolation method, based upon a multi-scale optical flow reconstruction scheme. The proposed method predicts low resolution bi-directional optical flow, and reconstructs it into high resolution. We also proposed consistency and multi-scale smoothness loss to enhance the quality of the predicted optical flow. Furthermore, we use adversarial loss to make the interpolated frame more seamless and natural. We demonstrated that the proposed method outperforms the existing state-of-the-art methods in quantitative evaluation, while it runs up to 4.39× faster than those methods for 4K videos.

Fast surveillance video indexing & retrieval with WiFi MAC address tagging

<span>Conventional public safety surveillance video camera systems required 24/7 monitoring of security officers with video wall display installed in the control room. When a crime or incident is reported, all the recorded surveillance video streams nearby the incident area are playback simultaneously on video wall to help locate the target person. The security officers can fast forward the video playback to speed up the video search, but it requires massive manpower if there are hundreds of video streams required to be examined on the video wall. One of the possible solutions is through a suitable video indexing and retrieval technique to prioritize the video frames that need to be processed. This paper presents a WiFi sniffer enabled surveillance camera, with 3-stage WiFi frame inspection filter and the use of collected WiFi signal strength for filtering, to tag the collected WiFi MAC addresses to the surveillance video frames according to the time of the MAC address is sniffed. Additional metadata (WiFi MAC address of smartphone) collected during the occurrence of the incident can be used to prioritize the retrieving of surveillance video frames for subsequent image processing. </span>

Animated Stickies: Fast Video Projection Mapping onto a Markerless Plane through a Direct Closed-Loop Alignment

This paper presents a fast projection mapping method for moving image content projected onto a markerless planar surface using a low-latency Digital Micromirror Device (DMD) projector. By adopting a closed-loop alignment approach, in which not only the surface texture but also the projected image is tracked by a camera, the proposed method is free from a calibration or position adjustment between the camera and projector. We designed fiducial patterns to be inserted into a fast flapping sequence of binary frames of the DMD projector, which allows the simultaneous tracking of the surface texture and a fiducial geometry separate from a single image captured by the camera. The proposed method implemented on a CPU runs at 400 fps and enables arbitrary video contents to be "stuck" onto a variety of textured surfaces.

Symmetry Encoder-Decoder Network with Attention Mechanism for Fast Video Object Segmentation

Semi-supervised video object segmentation (VOS) has obtained significant progress in recent years. The general purpose of VOS methods is to segment objects in video sequences provided with a single annotation in the first frame. However, many of the recent successful methods heavily fine-tune the object mask in the first frame, which decreases their efficiency. In this work, to address this issue, we propose a symmetry encoder-decoder network with the attention mechanism for video object segmentation (SAVOS) requiring only one forward pass to segment the target object in a video. Specifically, the encoder generates a low-resolution mask with smoothed boundaries, while the decoder further refines the details of the segmentation mask and integrates lower level features progressively. Besides, to obtain accurate segmentation results, we sequentially apply the attention module on multi-scale feature maps for refinement. We conduct several experiments on three challenging datasets (i.e., DAVIS 2016, DAVIS 2017, and SegTrack v2) to show that SAVOS achieves competitive performance against the state-of-the-art.

Fast Video Compression Method based on Wavelet Transform and Simple Efficient Search Algorithm

Fast Video Compression Method based on Wavelet Transform and Simple Efficient Search Algorithm

Time-Sync Video Tag Extraction Using Semantic Association Graph

Time-sync comments (TSCs) reveal a new way of extracting the online video tags. However, such TSCs have lots of noises due to users’ diverse comments, introducing great challenges for accurate and fast video tag extractions. In this article, we propose an unsupervised video tag extraction algorithm named Semantic Weight-Inverse Document Frequency (SW-IDF). Specifically, we first generate corresponding semantic association graph (SAG) using semantic similarities and timestamps of the TSCs. Second, we propose two graph cluster algorithms, i.e., dialogue-based algorithm and topic center-based algorithm, to deal with the videos with different density of comments. Third, we design a graph iteration algorithm to assign the weight to each comment based on the degrees of the clustered subgraphs, which can differentiate the meaningful comments from the noises. Finally, we gain the weight of each word by combining Semantic Weight (SW) and Inverse Document Frequency (IDF). In this way, the video tags are extracted automatically in an unsupervised way. Extensive experiments have shown that SW-IDF (dialogue-based algorithm) achieves 0.4210 F1-score and 0.4932 MAP (Mean Average Precision) in high-density comments, 0.4267 F1-score and 0.3623 MAP in low-density comments; while SW-IDF (topic center-based algorithm) achieves 0.4444 F1-score and 0.5122 MAP in high-density comments, 0.4207 F1-score and 0.3522 MAP in low-density comments. It has a better performance than the state-of-the-art unsupervised algorithms in both F1-score and MAP.

Fast Video Frame Correlation Analysis for Vehicular Networks by Using CVS–CNN

In vehicular communication systems, due to limited computation power of vehicles, low-cost sampling technologies, such as compressed video sensing (CVS), have been proposed. However, after one-time coarse compressive sampling, it is difficult to obtain accurate temporal correlation between video frames. To address this issue, this paper proposes a correlation analysis model in the measurement domain by combining CVS and convolutional neural network (CNN), which is termed as “CVS-CNN.” Specifically, to analyze the temporal correlation of video frames in the measurement domain, we use CNN as a substitute for the pseudo-inverse transform of the measurement matrix and establish the correlation between the measurements of the block to be estimated and those of the neighboring non-overlapping blocks. The network parameters are trained to minimize the loss between the predicted and true measurements, and are assigned to the non-overlapping image blocks. The various experimental results demonstrate that the proposed CVS-CNN method significantly outperforms similar methods of analyzing the video frame correlation in accuracy, process speed, and robustness. This result indicates that the proposed method can be used in many potential applications, such as self-driving systems and in-car warning systems.

Experimental study of the dynamic mechanism on gas bubbles migration, fragment, coalescence and trapping in a porous media

Immiscible gas injection recovery is one of the most important methods used for EOR. However, it is very difficult to directly observe the flow of gas bubbles in the reservoir stratum. In our experimental work, we investigated visually the morphological configurations of gas bubbles migration, fragment, coalescence and trapping in different liquid-simulated porous media packs of glass beads. Also, the volume fraction of the trapped gas bubbles in the drainage process were measured. A fast video recording technique was used for the visual observations of the morphological characteristics of the rising gas bubbles in the saturated porous media. The obtained experimental images recorded reveal three types of the rising migration bubbles namely: free migration in the pore throat, attached wall migration on the beads and deformation migration. Additionally, coalescence and fragment have an interdependency for a rising gas bubble. The adsorption and capillary trappings are both two kinds of important trapping mechanisms, which determine the volume fraction of the trapped bubbles. The volume fraction of the trapped bubbles increases with an increment in the gas injection time and rates with the trapped bubbles fluctuating upward and downward due to the more coalescence and fragment after a long gas injection time/period. Lastly, the volume fraction of the trapped bubbles increases with a decrease in the inclination angles and porosities, which influences the areal sweep efficiency for enhanced liquid recovery in a porous media.

Dağıtık Sıkıştırmalı Video Algılama için Basit ve Uyarlanabilir bir Veri Hızı Kontrolü Tasarımı

Providing fast video encoding, Compressive Sensing has been very suitable for the schemes which require power-constraint devices such as in wireless sensor networks. However, resulted data load on the network is extremely high and accomplishing rate control on these compressive sensing codecs by a simple scheme is crucial. In this paper we propose a simple scheme of data rate control mechanism for an existing compressive sensing codec for distributed video coding which has been reported as an efficient framework in terms of video reconstruction quality. Our approach does not use a feedback channel from decoder which avoids encoder to wait for high-complexity optimization problem. Embedding the proposed scheme to the codec architecture, we have obtained ~43% gain in data rate with an acceptable decrease (6%) in Video PSNR.

A Novel Dual-band Video Fusion Algorithm Using Fast Lookup-Tables: Toward Naturalistic Color

This paper presents an algorithm for fast dual-band video fusion with good naturalistic color using lookup tables. We establish a color lookup table based on the methods of frame extraction, pseudo-color mapping and color transfer; then we fill up the missing values of the lookup table using the Euclidean distance among its elements. From this lookup table, the luminance value and chromatic values of each pixel in the output frames are retrieved, and consequently the final color fused video is attained. Experiments show that the algorithm is a fast and practical approach to fuse dual-channel videos with stable color appearance and perceived high naturalness.

Fast distributed video deduplication via locality-sensitive hashing with similarity ranking

The exponentially growing amount of video data being produced has led to tremendous challenges for video deduplication technology. Nowadays, many different deduplication approaches are being rapidly developed, but they are generally slow and their identification processes are somewhat inaccurate. Till now, there is rare work that studies the generic hash-based distributed framework and the efficient similarity ranking strategy for video deduplication. This paper proposes a flexible and fast distributed video deduplication framework based on hash codes. It is able to support the hash table indexing using any existing hashing algorithm in a distributed environment and can efficiently rank the candidate videos by exploring the similarities among the key frames over multiple tables using MapReduce strategy. Our experiments with a popular large-scale dataset demonstrate that the proposed framework can achieve satisfactory video deduplication performance.

Fast video encoding based on random forests

Machine learning approaches have been increasingly used to reduce the high computational complexity of high-efficiency video coding (HEVC), as this is a major limiting factor for real-time implementations, due to the decision process required to find optimal coding modes and partition sizes for the quad-tree data structures defined by the standard. This paper proposes a systematic approach to reduce the computational complexity of HEVC based on an ensemble of online and offline Random Forests classifiers. A reduced set of features for training the Random Forests classifier is proposed, based on the rankings obtained from information gain and a wrapper-based approach. The best model parameters are also obtained through a consistent and generalizable method. The proposed Random Forests classifier is used to model the coding unit and transform unit-splitting decision and the SKIP-mode prediction, as binary classification problems, taking advantage from the combination of online and offline approaches, which adapts better to the dynamic characteristics of video content. Experimental results show that, on average, the proposed approach reduces the computational complexity of HEVC by 62.64% for the random access (RA) profile and 54.57% for the low-delay (LD) main profile, with an increase in BD-Rate of 2.58% for RA and 2.97% for LD, respectively. These results outperform the previous works also using ensemble classifiers for the same purpose.

Low-Complexity Texture Video Coding Based on Motion Homogeneity for 3D-HEVC

Three-dimensional extension of the high efficiency video coding (3D-HEVC) is an emerging international video compression standard for multiview video system applications. Similar to HEVC, a computationally expensive mode decision is performed using all depth levels and prediction modes to select the least rate-distortion (RD) cost for each coding unit (CU). In addition, new tools and intercomponent prediction techniques have been introduced to 3D-HEVC for improving the compression efficiency of the multiview texture videos. These techniques, despite achieving the highest texture video coding efficiency, involve extremely high-complex procedures, thus limiting 3D-HEVC encoders in practical applications. In this paper, a fast texture video coding method based on motion homogeneity is proposed to reduce 3D-HEVC computational complexity. Because the multiview texture videos instantly represent the same scene at the same time (considering that the optimal CU depth level and prediction modes are highly multiview content dependent), it is not efficient to use all depth levels and prediction modes in 3D-HEVC. The motion homogeneity model of a CU is first studied according to the motion vectors and prediction modes from the corresponding CUs. Based on this model, we present three efficient texture video coding approaches, such as the fast depth level range determination, early SKIP/Merge mode decision, and adaptive motion search range adjustment. Experimental results demonstrate that the proposed overall method can save 56.6% encoding time with only trivial coding efficiency degradation.

A Fast Traffic Video Stabilization Method Based on Trajectory Derivatives

This paper aims to solve the stabilization problem of traffic videos, which are mostly captured by the cameras mounted on the vehicles. Compared with normal videos captured with handheld cameras, traffic videos often face more challenges, such as dynamic scenes, dominant foreground objects, and significant parallax. Conventional methods often regard videos stabilization as an optimization problem with complex constraints and thus cost much computation time. To address the above issues, we propose a fast method by making use of trajectory derivatives at adjacent frames. When the parallax is not serious or the inter-frame time is short enough, the non-rigid characteristic of camera jitter can be ignored. Then, we simplify the perspective transformation to affine transformation between adjacent frames, i.e., the trajectory derivative holds linear with respect to the position of the trajectory. We combine our method with a feedback-based foreground trajectory judgment strategy and significantly speed up the processing speed. The new algorithm can satisfy the real-time requirement, which is critical for real applications, at the cost of tolerable stabilization performance degradation.

Adaptive Cluster based Model for Fast Video Background Subtraction

Background subtraction (BGS) is one of the impor�tant steps in many automatic video analysis applications. Several researchers have attempted to address the challenges due to illumination variation, shadow, camouflage, dynamic changes in the background and bootstrapping requirement. In this paper, a method to perform BGS using dynamic clustering is proposed. A background model is generated using the K0 -means algorithm. The normalized γ corrected distance values and an automatic threshold value is used to perform the background subtraction. The background models are updated online to handle slow illu�mination changes. The experiment was conducted on CDNet2014 dataset. The experimental results show that the proposed method is fast and performs well for baseline, camera-jitter and dynamic background categories of video.