Video Block Research Articles

PVBLiF: A Pseudo Video-Based Blind Quality Assessment Metric for Light Field Image

Going beyond traditional 2D imaging is not only an emerging trend of imaging technology, but also the key to a more immersive user experience. Light Field Image (LFI) is a typical high-dimensional imaging format, and the quality evaluation of which is very challenging but necessary. In this paper, we propose a novel Pseudo Video-based Blind quality assessment metric for Light Field image (PVBLiF). In contrast to most previous Light Field Image Quality Assessment (LF-IQA) metrics, in which different types of 2D representations derived from LFI are used for quality assessment indirectly, our metric exploits a more intuitive 3D representation, named Pseudo Video Block Sequence (PVBS), to evaluate the perceptual quality of LFI. For this purpose, we first divide the LFI into a massive number of non-overlapping PVBSs, which simultaneously contain spatial and angular information of LFI. Then, we propose a novel network (named PVBSNet) based on Convolutional Neural Networks (CNNs) to extract the spatio-angular features of PVBS and further evaluate the PVBS quality. The proposed PVBSNet consists of four stages: multi-information division, intra-feature extraction, cross-feature fusion, and quality regression. Finally, a Saliency- and Variance-guided Pooling (SVPooling) method is presented to integrate all the PVBS quality into the overall quality of LFI. The proposed PVBLiF metric has been extensively evaluated on three widely-used LFI datasets: Win5-LID, NBU-LF1.0, and SHU. Experimental results demonstrate that our proposed PVBLiF metric outperforms state-of-the-art metrics and is capable of highly approximating the performance of human observers. The source code of PVBLiF is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ZhengyuZhang96/PVBLiF</uri> .

Кинопедагогика как средство современного освоения математики в школе

Relevance. The article provides a theoretical analysis of the requirements for an educational film from the point of view of psychology and pedagogy. The analysis of Russian and foreign studies in the field of perception of sound and video, as well as the semantic content of the educational film showed that they research general characteristics and mechanisms of processing for auditory and visual information in creating an image of reality. The main studies consider the focus of attention and semantic and cognitive attitudes. Objective. The research aims to create a video course that allows a student with problems mastering mathematics to overcome the "learned helplessness", to find meaning in studying algebra and geometry, to believe that learning mathematics is easy and interesting. Methods. Content analysis, subject-subject dialogue, assessment and forecast of changes, conclusions from general to particular and from particular to general, comparative analysis. Results. The project "Matstream" considering the main topics of algebra that students of primary school study in a series of video tutorials is presented. Taking into account the cognitive characteristics of modern fifth-ninth grade students is implemented in the presentation of educational material. Three blocks of videos are aimed at forming an attitude to the perception of abstract-logical information expressed in mathematical symbols, specific knowledge, skills and abilities. Conclusion. In creating educational audiovisual products, it is necessary to take into account motivational, emotional, and cognitive components. Within the framework of the Matstream project, a program of assistance in teaching mathematics to schoolchildren is being implemented. It also involved developing attitudes to the perception of abstract-logical language of mathematical algorithms and overcoming difficulties in mastering mathematics.

Methodological concept of distance learning at the stage of basic training at the Maritime University

The article presents an innovative approach to the educational and methodological support of the educational process in a distance format. The purpose of the study is associated with the identification of such a structure and content of thematic complexes of e-learning tools, the use of which at the stage of basic training of specialists at the Maritime University would optimally meet the objectives of improving the quality of basic knowledge and competencies, the level of digital and professional literacy of industry personnel. The scientific novelty of the study is the development of the methodological concept of distance learning in the direction of increasing the efficiency of the educational process based on the thematic electronic modules developed by the author, consisting of subject-specific video blocks, to different forms of training sessions united by one topic. The author reveals the structure, content and technological features of the implementation of video blocks in the process of educational activities of students in lectures, practical and laboratory classes, as well as in the process of evaluating the success of mastering the course being studied. The article presents the qualitative results of the study, which show the effectiveness of the developed methodological concept of distance learning in the cycle of basic academic disciplines in terms of improving the level of basic training of students as a fundamental basis for professional knowledge and competencies.

Long Jump Action Recognition Based on Deep Convolutional Neural Network.

Long jump is a test item of national student physical health monitoring, which can reflect the quality of students' lower limb strength. Long jump is a highly technical activity, which includes four basic movements: running aid, jumping, vacating, and landing. Many students have problems with the technical aspects, resulting in test scores that do not objectively reflect the true physical condition of the students, which affects the accuracy of the test results. From the perspective of rapid diagnostic feedback of students' long jump movements, we design and develop a long jump movement recognition method based on deep convolutional neural network. In this paper, we firstly summarize the traditional visual action recognition algorithm, then apply 3D convolution to extract the spatiotemporal features of long jump action from three directions of the video block, and fuse the spatiotemporal features of the three directions in different ways to achieve feature complementation; finally, using the multimodality of long jump action data, we use 3D convolutional neural network to train the RGB images and then train the depth. This joint training method can accelerate the convergence speed and improve the accuracy of the network on both depth and edge images. The experiments compared the recognition effects of the tandem fusion of features, the maximum fusion, and the multiplicative fusion in the scoring layer, and the highest accuracy of 82.3% was achieved by the tandem fusion of features with the fusion of three modalities.

Multi-Connection Based Scalable Video Streaming in UDNs: A Multi-Agent Multi-Armed Bandit Approach

Scalable video coding (SVC) has received much attention for video transmission over wireless due to its flexibility. However, most previous work only considered SVC video streaming from a single base station (BS). At present, the densification of BSs enables a user equipment (UE) to connect to multiple BSs in ultra-dense networks (UDNs). In this paper, we consider the problem of SVC video streaming in a UDN, which allows different layers of a video block to be downloaded from different BSs. An optimization problem is formulated aiming to maximize the quality of experience (QoE) of users by selecting the optimal connection strategy and optimal number of video layers. Considering the complexity, to efficiently solve the problem in a distributed manner, the problem of choosing connection strategy is formulated as a multi-agent multi-armed bandit (MA-MAB) problem with only few information exchange. Each user can adapt its connection strategy in a distributed self-learning system. To obtain the optimal arm for the MA-MAB problem, we propose a multi-user arm decision algorithm. To avoid large computation and handover costs, we adopt the same connection strategy for the entire video sequence. Then for each video block, with the given connection strategy, the number of video layers is adjusted adaptively according to dynamic network conditions. Finally, based on the above designs, we provide the SVC-based video downloading scheme to obtain an approximate optimal solution to the original optimization problem. Extensive simulations and comparisons show the feasibility and superiority of the proposed scheme.

Design of Personalized VR Short Video Content Distribution Algorithm based on Artificial Neural Network

In order to improve the video quality, reduce the number of video jams, and improve the video transcoding rate, a new personalized distribution algorithm of VR short video content based on artificial neural network is proposed. BP neural network is used to compress the original video and determine the execution mode and cache location of VR short video cache file; Transcoding cached high bit rate VR short video stream to generate low bit rate video stream to meet the needs of different network bandwidth; Build a video distribution model, and build a multicast distribution tree based on this model, that is, add some relay servers to minimize the video distribution cost; Finally, through the algorithm of minimizing the distribution cost of VR short video, the bandwidth loss and response delay are effectively reduced to achieve the goal of minimizing the distribution cost. The experimental results show that the video blocking times of this method are always less than six times, which effectively reduces the video blocking times. The PSNR value is high, the increase is large, up to 0.5, and the video transcoding rate is improved, up to 92%.

Retraction Note to: Risk assessment of extreme rainfall climate change and sports stadium sports based on video summarization algorithm

Based on the research of existing video summarization algorithms, this paper proposes a video summarization algorithm based on block sparse representation. This algorithm is mainly used to record multi-screen videos with similar content and to create video advertisements for multiple videos. First, the video is divided into multiple blocks according to the similarity, and the size of the block is controlled, and then, multiple video blocks are embedded in the space, and representative video blocks are selected through sparse display. This study is based on the annual heavy rainfall in a specific area of the Loess Plateau in 2020 and takes the factors that affect the risk of college sports as the research object. The goal of this article is to evaluate and determine the degree of sports risk in colleges and universities through the establishment of a fuzzy comprehensive evaluation model, determine the weight of risk indicators, and determine the risk prevention system. On this basis, this article puts forward some suggestions: improve students’ self-awareness, improve safety awareness, strengthen the safety guidance of physical education teachers, increase the importance of school leaders’ safety, improve the school’s risk prevention mechanism, and strengthen the emergency response capabilities of ordinary medical staff, strengthen the management of medical staff, strengthen the management of school sports facilities, build and improve student sports injury insurance, make contributions to the management of sports risk prevention in colleges and universities, and lay the foundation for the construction of a healthy campus. Therefore, this paper uses the existing abstract video technology to describe and study the extreme rainfall climate in a certain place and conduct a narrative research on the sports risk assessment in university stadiums.

Video block and FABEMD features for an effective and fast method of reporting near-duplicate and mirroring videos

Near-duplicate video content has taken the large storage space in the age of big data. Without respecting the copyright ethic, social media users mirror, resize, and/or hide certain online video content and re-upload it as new data. This research aims to avoid the complex and high-dimensional matching and present an efficient approach for detecting near-duplicate videos, this detection is based on feature extraction using visual, motion, and high-level features. Fast and adaptive bidimensional empirical mode decomposition is used to preserve the relevant data to the furthest extent possible during the low/high-frequency transition and vice-versa. In addition, for a generic model, the invariant moments are added to the aforementioned features in order to reinforce them against different video transformations such as rotating and scaling. Furthermore, the video frames are divided into blocks with a fixed number of features, this set of features is represented by a signature, where its mean and standard deviation represents a single video map allowing easy similarity computation. The F1-score and accuracy are used to evaluate the results of this study; the relevant results are ranked by Top_{1} for the best result, and the five top-ranked results are presented by Top_{5}. Further, our result of Top_{1} reached over 80% on F1-score, with a difference of ±4% from the Top_{5} results, and it is over 90% on Accuracy using different datasets, such as UCF11, UCF50, and HDMB51.

Hyperspectral imagery denoising using minimum noise fraction and VBM3D

Hyperspectral imagery denoising is a classical problem in hyperspectral remote sensing applications. In our previous work, we combined principal component analysis with wavelet shrinkage, and good denoising results were obtained. We combine minimum noise fraction (MNF) with video block matching and 3D filtering (VBM3D), which is a powerful video denoising method. After MNF transform, we automatically select k0, the number of spectral band images as a threshold for denoising. We reduce the noise in spectral band images of MNF transformed data from k0 to the last band image and do not denoise the first k0 − 1 spectral band images. Finally, we perform an inverse MNF transform to obtain the denoised data cubes. We compare our MNF + VBM3D method with different denoising methods such as bivariate wavelet shrinkage (BivShrink), non-local means, SURELET, and block matching and 3D filtering (BM3D). Experimental results demonstrate that MNF + VBM3D achieves the best denoising results among almost all methods for three testing data cubes and with different noise levels.

EEG neurofeedback improves cycling time to exhaustion

ObjectiveThe role of the brain in endurance performance is frequently debated; surprisingly, few investigations have attempted to improve endurance performance by directly targeting brain activity. One promising but untested approach to modifying brain activity is electroencephalogram (EEG) neurofeedback. Consequently, our experiment is the first to examine an EEG neurofeedback intervention for whole-body endurance performance. MethodWe adopted a two-part experiment. The first consisted of a randomized parallel controlled design. Forty participants were allocated to three experimental groups; increase relative left cortical activity (NFL), increase relative right (NFR), and passive control (CON). They performed a depleting cognitive task, followed by either six 2-min blocks of EEG neurofeedback training (NFL or NFR) or time-matched videos of the neurofeedback display (CON). Next, they performed a time-to-exhaustion (TTE) test on a cycle-ergometer. We then tested participants of NFL and NFR groups in an additional experimental visit and administered the opposite neurofeedback training within a fully repeated-measures protocol. ResultsEEG neurofeedback modified brain activity as expected. As hypothesized, the NFL group cycled for over 30% longer than the other groups in the parallel controlled design, NFL: 1382 ± 252 s, NFR: 878 ± 167, CON: 963 ± 117 s. We replicated this result in the repeated-measures design where NFL: 1167 ± 831 s performed 11% longer than NFR: 1049 ± 638 s. There were no differences in pre-exercise fatigue, vigor or self-control; area under the curve group-differences for perceived effort were interpreted within a goal persistence framework. ConclusionThe brief EEG neurofeedback intervention elicited greater relative left frontal cortical activity and enhanced endurance exercise performance.

No‐reference quality assessment of dynamic sports videos based on a spatiotemporal motion model

This paper proposes an approach to improve the performance of no-reference video quality assessment for sports videos with dynamic motion scenes using an efficient spatiotemporal model. In the proposed method, we divide the video sequences into video blocks and apply a 3D shearlet transform that can efficiently extract primary spatiotemporal features to capture dynamic natural motion scene statistics from the incoming video blocks. The concatenation of a deep residual bidirectional gated recurrent neural network and logistic regression is used to learn the spatiotemporal correlation more robustly and predict the perceptual quality score. In addition, conditional video block-wise constraints are incorporated into the objective function to improve quality estimation performance for the entire video. The experimental results show that the proposed method extracts spatiotemporal motion information more effectively and predicts the video quality with higher accuracy than the conventional no-reference video quality assessment methods.

RETRACTED: Development and application of VR course resources based on embedded system in open education

Virtual Reality VR technology offers a unique approach to enhance the use of complex 3D graphics and visualization environments. The study conducted using virtual reality (VR) technology integrated into the computer-aided design network; VR is a Web-based learning environment. Virtual reality technology is expected to be a traditional 2D computer graphics (CG) CG extended to a 3D real-time simulation program. Virtual reality technology can be used to promote participation and student learning. With VR education, change the delivery of educational content. No user interaction is well suited to see it and is very suitable to create a real or imaginary virtual world. Immersive virtual reality (VR) applications require ultra-high data rate and low latency to run smoothly, designed to improve multiuser wireless VR video (depth of learning support system) VR experiences. It has been proposed streaming: quality and low-latency video block submission. Scenario, VR is that it can describe some fields to be widely applied to education. One of the main features of VR is that it allows for multisensory interaction in the visualization space. This combination of visualization and interactive cosmetic VR of multisensory is, ideally, to see whether suitable for effective learning and try to explain this effect by the benefits of active learning from experience. Also, may want to consider some of the defenders and the application of VR education related to technology-related lottery.

No-Reference Nonuniform Distorted Video Quality Assessment Based on Deep Multiple Instance Learning

Each part of a nonuniform distorted video (NUDV) has a unique distortion degree. When NUDV blocks are used as inputs, traditional machine-learning-based video quality assessment (VQA) methods frequently do not work effectively. Because these methods directly assign the label of the entire video to blocks, causing the unreliability of labels. We creatively propose video bag, a collection of video blocks, to deal with this unreliability. We develop a novel multiple instance learning (MIL) based model, VQA-MIL, which dynamically adjusts the weights by a block-wise attention module and enriches the features of video bags by a MI Pooling layer. Furthermore, we apply the mixup data-augmentation strategy to address the lack of human labels in common video datasets. We test our method on LIVE and CSIQ, and on a relatively large-scale dataset, named NUDV-KT, that we have collected. Results show that our method outperforms popular state-of-the-art no-reference VQA methods on NUDVs.

Burning condition recognition of rotary kiln based on spatiotemporal features of flame video

In the coal-fire industry, recognition of burning condition is vital for combustion control and optimization as it can provide early warning of abnormal conditions in the combustion system. We propose an effective model based on spatiotemporal features extracted from flame video for burning condition recognition especially for unsteady state condition. This model extracts features from both the spatial and temporal dimensions based on multiple adjacent frames to capture the appearance and motion information of the flame. Two new three-dimensional (3D) descriptors are developed to characterize the dynamic characteristics of the flame video stream. A computationally simple and fast dynamic texture descriptor, 3DBLBP, is designed to extract flame dynamic texture and motion from three adjacent frames of a video block, and a dynamic structure descriptor, HOPC-TOP is developed by extracting 3D phase congruency information of video clip from three orthogonal planes to capture structure and motion characterizes of flame. The two descriptors are combined to extract spatiotemporal features from flame clip for burning condition recognition. Experiment results show the proposed framework can achieve a high recognition accuracy in the real-world data, thus verifying the effectiveness of our proposed framework for the recognition of burning conditions in a rotary kiln.

Exploring Absolute Differences Arithmetic Operators for Power and Area-Efficient SAD Hardware Architectures

Managing the energy requirements of video encoders has been an important research topic throughout the latest years, given the limited resources of battery-powered systems. Sum of Absolute Differences (SAD) stands out among the most costly steps in the video encoding process compliant with the High Efficiency Video Coding (HEVC) standard. This metric is mainly used to explore temporal redundancies during the inter prediction stage of the encoder, and it is applied by adding the absolute differences of the colocalized pixels of two video blocks, as a means of measuring the similarity between them. SAD architectures are usually designed by using an adder tree, with its first level consisting of subtractors and absolute operators. This paper explores various structures of absolute operators in the context of SAD architectures, in order to define the most suitable implementation for a power-efficient SAD module. Besides the analysis of several different models, we exploit the use of pipelining, and the impact of varying block input bitwidth, to determine which versions scale better with the increase of input size. We have synthesized the architectures for ASIC CMOS technology using real-input vectors taking the delays into consideration, with an ST 65 nm standard cells library, and compared them with the default absolute operator macrofunction from the synthesis tool.

Investigation of filtering of rain streaks affected video sequences under various quantisation parameter in HEVC encoder using an enhanced V‐BM4D algorithm

This study proposes an enhanced video block matching four-dimensional (V-BM4D) denoising algorithm for highly correlated rain streaks pattern video sequence in high-efficiency video coding (HEVC) encoder. Removal of structural rain streaks from high definition (HD) video sequences are challenging issues in real-time video application. Temporal information of original video sequences is distorted among the successive video frames due to the presence of natural degradation parameters such as fog, smog and rain streaks. Rain streaks are highly correlated structural noise pattern, which affects the nature of video frame compared to other natural degradations. Existing HEVC encoder is employed with in-loop filtering to protect the temporal information of original video sequences from rain streaks pattern. However, in-loop filtering is unable to remove the highly directional oriented rain pattern from video frames. To retain temporal information among the successive video frames, enhanced block matching denoising block is adopted in HEVC coder. The proposed enhanced VBM4D- based in-loop filtering eliminates the various level of rain streaks from the HD video sequences by temporal information prediction in 4D platform. Experimental results demonstrate that the proposed denoising algorithm yields better peak signal to noise ratio value and provides better bit rate saving for various HEVC configuration.

Foreground Extraction and Motion Recognition Technology for Intelligent Video Surveillance

With the rapid development of computer technology and network technology, it has become possible to build a large-scale networked video surveillance system. The video surveillance system has become a new type of infrastructure necessary for modern cities. In this paper, the problem of foreground extraction and motion recognition in intelligent video surveillance is studied. The three key sub-problems, namely the extraction of motion foreground in video, the deblurring of motion foreground and the recognition of human motion, are studied and corresponding solutions are proposed. A background modeling technique based on video block is proposed. The background is modeled at the block level, which greatly reduces the spatial complexity of the algorithm. It solves the problem that the traditional Gaussian model (GMM) moving target enters the static state and is integrated into the background process. The target starts to move for a long time and there are ghosts and other problems, which reduce the processing efficiency of the lifting algorithm. The test results on the Weizmann dataset show that the proposed algorithm can achieve high human motion recognition accuracy and recognition with low computational complexity. The rate can reach 100%; the local constrained group sparse representation classification (LGSRC) model is used to classify it. The experimental results on Weizmann, KTH, UCF sports and other test datasets confirm the validity of the algorithm in this chapter. KNN, SRC voting classification accuracy.

Exploring high-order adder compressors for power reduction in sum of absolute differences architectures for real-time UHD video encoding

The sum of absolute difference (SAD) calculation is one of the most computing-intensive operations in video encoders compatible with recent standards, such as high-efficiency video coding (HEVC). SAD hardware architectures employ an adder tree to accumulate the coefficients from the absolute difference between two video blocks. This paper employs high-order adder compressors (HOAC) structures into SAD hardware architectures to achieve ultra-high definition (UHD) encoding in real time, using block sizes compatible with HEVC. The proposed HOAC architectures are power-efficient and enable low-power SAD hardware accelerators. Our throughput analysis shows that the HOAC-based SAD hardware architecture is capable of encoding UHD 4K ( $$3840\times 2160$$ ) videos in real-time at 60 frames per second. The architectures were entirely designed as dedicated ASIC blocks and were synthesized to ST 65 nm CMOS standard cells. Synthesis results show that SAD architectures using 64-2, 32-2, 16-2 and 8-2 compressors built from 4-2 compressors are significantly more efficient in terms of circuit area and total power dissipation when compared with SAD architectures using conventional adders selected by a commercial logic synthesis tool.

Blockchain-enabled deep semantic video-to-video summarization for IoT devices

The rapid development of multimedia technologies gave birth to large video data that demands effective and fast video summarization methods along with ability to assure the authenticity of digital data. Video summarization provides consolidated view of original video in short and compact form of a video. The summarized videos should be transmitted to remote IoT devices through secure channels. To prevent data manipulation and to verify the transmitted data, we exploited the blockchain technology. This paper focuses on different steps of keyframe extraction from a video followed by secure and trusted transmission to the consumers. The proposed system enables the users to summarize a video based on human and objects as parameters. Cryptographic hashes are used with blockchain, hashes are generated from summarized video blocks, signed and transmitted through blockchain. Cumulus blockchain technique is utilized to ensure the video integrity. System allows remote users to get tamper-proofed summarized video of their business sites or any sensitive premises on their smartphones.

DeepBinaryMask: Learning a binary mask for video compressive sensing

In this paper, we propose an encoder-decoder neural network model referred to as DeepBinaryMask for video compressive sensing. In video compressive sensing one frame is acquired using a set of coded masks (sensing matrix) from which a number of video frames, equal to the number of coded masks, is reconstructed. The proposed framework is an end-to-end model where the sensing matrix is trained along with the video reconstruction. The encoder maps a video block to compressive measurements by learning the binary elements of the sensing matrix. The decoder is trained to map the measurements from a video patch back to a video block via several hidden layers of a Multi-Layer Perceptron network. The predicted video blocks are stacked together to recover the unknown video sequence. The reconstruction performance is found to improve when using the trained sensing mask from the network as compared to other mask designs such as random, across a wide variety of compressive sensing reconstruction algorithms. Finally, our analysis and discussion offers insights into understanding the characteristics of the trained mask designs that lead to the improved reconstruction quality.