Video Frame Image Research Articles

Animation Design Based on 3D Visual Communication Technology

The depth synthesis of image texture is neglected in the current image visual communication technology, which leads to the poor visual effect. Therefore, the design method of film and TV animation based on 3D visual communication technology is proposed. Collect film and television animation videos through 3D visual communication content production, server processing, and client processing. Through stitching, projection mapping, and animation video image frame texture synthesis, 3D vision conveys animation video image projection. In order to ensure the continuous variation of scaling factors between adjacent triangles of animation and video images, the scaling factor field is constructed. Deep learning is used to extract the deep features and to reconstruct the multiframe animated and animated video images based on visual communication. Based on this, the frame feature of video image under gray projection is identified and extracted, and the animation design based on 3D visual communication technology is completed. Experimental results show that the proposed method can enhance the visual transmission of animation video images significantly and can achieve high-precision reconstruction of video images in a short time.

Motion Enhanced Model Based on High-Level Spatial Features

Action recognition has become a current research hotspot in computer vision. Compared to other deep learning methods, Two-stream convolutional network structure achieves better performance in action recognition, which divides the network into spatial and temporal streams, using video frame images as well as dense optical streams in the network, respectively, to obtain the category labels. However, the two-stream network has some drawbacks, i.e., using dense optical flow as the input of the temporal stream, which is computationally expensive and extremely time-consuming for the current extraction algorithm and cannot meet the requirements of real-time tasks. In this paper, instead of the dense optical flow, the Motion Vectors (MVs) are used and extracted from the compressed domain as temporal features, which greatly reduces the extraction time. However, the motion pattern that MVs contain is coarser, which leads to low accuracy. In this paper, we propose two strategies to improve the accuracy: firstly, an accumulated strategy is used to enhance the motion information and continuity of MVs; secondly, knowledge distillation is used to fuse the spatial information into the temporal stream so that more information (e.g., motion details, colors, etc.) is obtainable. Experimental results show that the accuracy of MV can be greatly improved by the strategies proposed in this paper and the final recognition for human actions accuracy is guaranteed without using optical flow.

Criminal Persons Recognition Using Improved Feature Extraction Based Local Phase Quantization

Facial recognition is a trending technology that can identify or verify an individual from a video frame or digital image from any source. A major concern of facial recognition is achieving the accuracy on classification, precision, recall and F1-Score. Traditionally, numerous techniques involved in the working principle of facial recognition, as like Principle Component Analysis (PCA), Linear Discriminant Analysis (LDA), Subspace Decomposition Method, Eigen Feature extraction Method and all are characterized as instable, poor generalization which leads to poor classification. But the simplified method is feature extraction by comparing the particular facial features of the images from the collected dataset namely Labeled faces in the wild (LFW) and Olivetti Research Laboratory (ORL) dataset. In this paper, the feature extraction is based on local phase quantization with directional graph features for an effective optimal path and the geometric features. Further, Person Identification based deep neural network (PI-DNN) has proposed are expected to provide a high recognition rate. Various performance metrics, such as recognition rate, classification accuracy, accuracy, precision, recall, F1-score is evaluated. The proposed method achieves high-performance values when it is compared with other existing methods. The novelty of this paper explains in understanding the various features of different types of classifiers used. It is mainly developed to recognize the human faces in the crowd, and it is also deployed for criminal identification.

Real-time Vehicle Surveillance System Based on Image Processing and Short Message Service

This paper proposes a real-time vehicle surveillance system based on image processing approach tailored with short message service. A background subtraction, color balancing, chain code based shape detection, and blob filtering are used to detect suspicious moving human around the parked vehicle. Once detected, the developed system will generate a warning notification to the owner by sending a short message to his mobile phone. The current frame of video image will also be stored and be sent to the owner e-mail for further checking and investigation. Last stored image will be displayed in a centralized monitoring website, where the status of the vehicle also can be monitored at the same time. When necessary, the stored images can be used during investigation process to assist the authority to take further legal actions.

Deep Learning-Based Multitarget Motion Shadow Rejection and Accurate Tracking for Sports Video

The effect is tested in various specific scenes of sports videos to complete the multitarget motion multitarget tracking detection application applicable to various specific scenes within sports videos. In this paper, deep neural networks are applied to sports video multitarget motion shadow suppression and accurate tracking to improve tracking performance. After the target frame selection is determined, the tracker uses an optical flow method to estimate the limits of the target sports video multitarget motion based on the sports video multitarget motion of the target object between frames. The detector first scans each sports video image frame one by one, observing the previously discovered and learned image frame subregions one by one until the current moment that is highly like the target to be tracked. The preprocessed remote sensing images are converted into grayscale images, the histogram is normalized, and the appropriate height threshold is selected in combination with the regional growth function to realize the rejection of sports video multitarget motion shadow and establish the sports video multitarget network model. The distance and direction of the precise target displacement are determined by frequency-domain vectors and null domain vectors, and the target action judgment mechanism is formed by decision learning. Finally, comparing with other shadow rejection and precision tracking algorithms, the proposed algorithm achieves greater advantages in terms of accuracy and time consumption.

Using Sensor Network in Motion Detection Based on Deep Full Convolutional Network Model

Aiming at the shortcomings of traditional moving target detection methods in complex scenes such as low detection accuracy and high complexity, and not considering the overall structure information of the video frame image, this paper proposes a moving-target detection based on sensor network. First, a low-power motion detection wireless sensor network node is designed to obtain motion detection information in real time. Secondly, the background of the video scene is quickly extracted by the time domain averaging method, and the video sequence and the background image are channel-merged to construct a deep full convolutional network model. Finally, the network model is used to learn the deep features of the video scene and output the pixel-level classification results to achieve moving target detection. This method not only can adapt to complex video scenes of different sizes but also has a simple background extraction method, which effectively improves the detection speed.

Mean-shift Object Tracking Algorithm with Systematic Sampling Technique

Mean shift is a fast object tracking algorithm that only considers pixels in an object area, hence its relatively small computational load. This algorithm is suitable for use in real-time conditions in terms of execution time. The use of histograms causes this algorithm to be relatively resistant to rotation and changes in object size. However, its resistance to lighting changes is not optimal. This study aims to improve the performance of the algorithm under lighting changes and reduce its processing time. The proposed technique involves the use of sampling techniques to reduce the number of iterations, optimization of candidate search object locations using simulated annealing, and addition of tolerance parameter to optimize object location search and area-based weighting instead of the Epanechnikov kernel. The results of the one-tail t-test with two independent sample groups reveal that the average performance of the proposed algorithm is significantly better than that of the traditional mean-shift algorithm in terms of resistance to lighting changes and processing time per video frame. In the test involving 999 frames of video images, the average processing time of the proposed algorithm is 83.66 ms, whereas that of the traditional mean-shift algorithm is 116.86 ms.

Automatic Pharyngeal Phase Recognition in Untrimmed Videofluoroscopic Swallowing Study Using Transfer Learning with Deep Convolutional Neural Networks

Background: Video fluoroscopic swallowing study (VFSS) is considered as the gold standard diagnostic tool for evaluating dysphagia. However, it is time consuming and labor intensive for the clinician to manually search the recorded long video image frame by frame to identify the instantaneous swallowing abnormality in VFSS images. Therefore, this study aims to present a deep leaning-based approach using transfer learning with a convolutional neural network (CNN) that automatically annotates pharyngeal phase frames in untrimmed VFSS videos such that frames need not be searched manually. Methods: To determine whether the image frame in the VFSS video is in the pharyngeal phase, a single-frame baseline architecture based the deep CNN framework is used and a transfer learning technique with fine-tuning is applied. Results: Compared with all experimental CNN models, that fine-tuned with two blocks of the VGG-16 (VGG16-FT5) model achieved the highest performance in terms of recognizing the frame of pharyngeal phase, that is, the accuracy of 93.20 (±1.25)%, sensitivity of 84.57 (±5.19)%, specificity of 94.36 (±1.21)%, AUC of 0.8947 (±0.0269) and Kappa of 0.7093 (±0.0488). Conclusions: Using appropriate and fine-tuning techniques and explainable deep learning techniques such as grad CAM, this study shows that the proposed single-frame-baseline-architecture-based deep CNN framework can yield high performances in the full automation of VFSS video analysis.

Campus Violence Detection Based on Artificial Intelligent Interpretation of Surveillance Video Sequences

Campus violence is a common social phenomenon all over the world, and is the most harmful type of school bullying events. As artificial intelligence and remote sensing techniques develop, there are several possible methods to detect campus violence, e.g., movement sensor-based methods and video sequence-based methods. Sensors and surveillance cameras are used to detect campus violence. In this paper, the authors use image features and acoustic features for campus violence detection. Campus violence data are gathered by role-playing, and 4096-dimension feature vectors are extracted from every 16 frames of video images. The C3D (Convolutional 3D) neural network is used for feature extraction and classification, and an average recognition accuracy of 92.00% is achieved. Mel-frequency cepstral coefficients (MFCCs) are extracted as acoustic features, and three speech emotion databases are involved. The C3D neural network is used for classification, and the average recognition accuracies are 88.33%, 95.00%, and 91.67%, respectively. To solve the problem of evidence conflict, the authors propose an improved Dempster–Shafer (D–S) algorithm. Compared with existing D–S theory, the improved algorithm increases the recognition accuracy by 10.79%, and the recognition accuracy can ultimately reach 97.00%.

Offset Detection of Grate Trolley’s Side Plate Based on YOLOv4

Side plate offset is one of the grate system faults. If it is not dealt with in time, some accidents will occur and economic losses will be made. Aiming at the problems like time‐consuming, labour‐wasting, and low intelligent by the side plate offset detection method manually, an autoside plate offset detection method is proposed, based on You Only Look Once version 4 (YOLOv4). Two cameras were fixed to collect the image information of the grate trolley’s side plate. With reference to the grate trolley’s operation, the offset judgment rules were set. YOLOv4 object detection algorithm was used to detect the side plate and trolley’s chassis frame in video frame images. A baseline was set according to the position information of the trolley’s chassis frame output by detection, and then, the position intervals between side plates and the baseline could be determined by calculation. According to the judgment rules, the scheme in this paper could detect the offset fault of the trolley’s side plate timely, and an alarm would be made automatically when faults are detected. Our video images of the trolley’s side plate were collected and sorted in Baogang Group sintering plant for testing. In this experiment, no error judgment was made, and the average detection and judgment time was 0.024 s. In this paper, rather than manually, the real‐time automatic detection was realized to detect the offset fault of the trolley’s side plate so as to provide a new solution for offset detection of the grate trolley’s side plate.

Optimization Research on Deep Learning and Temporal Segmentation Algorithm of Video Shot in Basketball Games.

The analysis of the video shot in basketball games and the edge detection of the video shot are the most active and rapid development topics in the field of multimedia research in the world. Video shots' temporal segmentation is based on video image frame extraction. It is the precondition for video application. Studying the temporal segmentation of basketball game video shots has great practical significance and application prospects. In view of the fact that the current algorithm has long segmentation time for the video shot of basketball games, the deep learning model and temporal segmentation algorithm based on the histogram for the video shot of the basketball game are proposed. The video data is converted from the RGB space to the HSV space by the boundary detection of the video shot of the basketball game using deep learning and processing of the image frames, in which the histogram statistics are used to reduce the dimension of the video image, and the three-color components in the video are combined into a one-dimensional feature vector to obtain the quantization level of the video. The one-dimensional vector is used as the variable to perform histogram statistics and analysis on the video shot and to calculate the continuous frame difference, the accumulated frame difference, the window frame difference, the adaptive window's mean, and the superaverage ratio of the basketball game video. The calculation results are combined with the set dynamic threshold to optimize the temporal segmentation of the video shot in the basketball game. It can be seen from the comparison results that the effectiveness of the proposed algorithm is verified by the test of the missed detection rate of the video shots. According to the test result of the split time, the optimization algorithm for temporal segmentation of the video shot in the basketball game is efficiently implemented.

An AIoT Monitoring System for Multi-Object Tracking and Alerting

Pig farmers want to have an effective solution for automatically detecting and tracking multiple pigs and alerting their conditions in order to recognize disease risk factors quickly. In this paper, therefore, we propose a novel monitoring system using an Artificial Intelligence of Things (AIoT) technique combining artificial intelligence and Internet of Things (IoT). The proposed system consists of AIoT edge devices and a central monitoring server. First, an AIoT edge device extracts video frame images from a CCTV camera installed in a pig pen by a frame extraction method, detects multiple pigs in the images by a faster region-based convolutional neural network (RCNN) model, and tracks them by an object center-point tracking algorithm (OCTA) based on bounding box regression outputs of the faster RCNN. Finally, it sends multi-pig tracking images to the central monitoring server, which alerts them to pig farmers through a social networking service (SNS) agent in cooperation with an oneM2M-compliant IoT alerting method. Experimental results showed that the multi-pig tracking method achieved the multi-object tracking accuracy performance of about 77%. In addition, we verified alerting operation by confirming the images received in the SNS smartphone application.

Pose recognition in sports scenes based on deep learning skeleton sequence model

Human skeleton extraction is a basic problem in the field of computer vision. With the rapid progress of science and technology, it has become a hot issue in the field of target detection such as pedestrian recognition, behavior monitoring, and pedestrian gesture recognition. In recent years, due to the development of deep neural networks, modeling of human joints in acquired images has made progress in skeleton extraction. However, most models have low modeling accuracy, poor real-time performance, and poor model availability. problem. Aiming at the above-mentioned human target detection problem, this paper uses the deep learning skeleton sequence model gesture recognition method in sports scenes to study, aiming to provide a gesture recognition method with strong noise resistance, good real-time performance and accurate model. This article uses motion video frame images to train the VGG16 network. Using the network to extract skeleton information can strengthen the posture feature expression, and use HOG for feature extraction, and use the Adam algorithm to optimize the network to extract more posture features, thereby improving the posture of the network Recognition accuracy. Then adjust the hyperparameters and network structure of the basic network according to the training results, and obtain the key poses in the sports scene through the final classifier.

Reliability Analysis of Multi-Objective Spatio-Temporal Segmentation of Human Motion in Video Sequences

In view of the problem of uneven distribution of edge contour of multi-target human motion image in video sequence, which leads to the decline of target detection ability, an algorithm of multi-target spatial-temporal segmentation of human motion in video sequence based on edge contour feature detection and block fusion is proposed. Firstly, a multi-target spatial-temporal detection model of human motion in video sequence was constructed, extracting video image frame sequence, using discrete frame fusion method to segment and fuse moving target image, matching moving multi-target in video sequence, secondly segmenting motion features in moving target image, combining with SURF algorithm (speeded up robust features, accelerated robust features) to detect and extract human motion objects in video sequence. The experimental results show that the gray histogram of human motion multi-target space-time segmentation is close to the original image histogram, and the detection and recognition ability of human motion target is improved.

Flume experiments test grain-size distribution of onshore tsunami deposits

We conducted flume experiments to examine the effects of grain size on the distribution of tsunami deposits, using well-sorted quartz sand with median diameters (D50) of 0.064, 0.134, 0.215, and 0.250 mm. In these experiments, we investigated deposits transported onto a flat terrestrial area by a single tsunami-like bore without a strong backwash flow. The post-tsunami distributions of the coarser (D50 = 0.215 and 0.250 mm) and finer (D50 = 0.064 and 0.134 mm) sands differed markedly. The amount of the deposited coarser sand decreased landward across the entire terrestrial area, whereas that of the deposited finer sand was either approximately uniform across the terrestrial area or increased gradually landward. Time-series of flow depth and sediment transport observed via composited video frame images showed that suspension of the coarser sand did not reach the height of the maximum flow depth, and decreased with distance from the sediment source, and that the sediment settled gradually across the terrestrial surface with the amount deposited decreasing landward. Conversely, the finer sand was densely suspended up to the maximum flow depth and was transported over longer distances across the terrestrial area with the amount deposited being relatively uniform or increasing slightly landward. Advection lengths that we calculated on the basis of our experimental parameters supported the dependence on grain size of the distribution patterns of onshore deposits shown by our flume experiments. Moreover, advection lengths that we calculated for the 2011 Tohoku-oki tsunami from previously reported tsunami parameters were consistent with a deposit distribution similar to that observed in the field after the tsunami. Although we ignored some key factors (e.g., geomorphology and sediment availability), our flume experiments showed that the distribution patterns of identical tsunami flows differ for sediments of different grain size.

Saliency-dependent adaptive remeshing for cloth simulation

We propose a method for simulating cloth with meshes dynamically refined according to visual saliency. It is a common belief that it is preferable for the regions of an image being viewed to have more details than others. For a certain scene, a low-resolution cloth mesh is first simulated and rendered into images in the preview stage. Pixel saliency values of these images are predicted according to a pre-trained saliency prediction model. These pixel saliencies are then translated to a vertex saliency of the corresponding meshes. Vertex saliency, together with camera positions and a number of geometric features of surfaces, guides the dynamic remeshing for simulation in the production stage. To build the saliency prediction model, images extracted from various videos of clothing scenes were used as training data. Participants were asked to watch these videos and their eye motion was tracked. A saliency map is generated from the eye motion data for each extracted video frame image. Image feature vectors and map labels are sent to a Support Vector Machine for training to obtain a saliency prediction model. Our method greatly reduces the number of vertices and faces in the clothing model, and generates a speed-up of more than 3 × for scenes with single dressed character, while for multi-character scenes the speed-up is increased to more than 5×. The proposed technique can work together with view-dependency for offline simulation.

Daytime observation of low-brightness stars (8m–10m) and space objects with a video camera having image summation

This paper presents the results of recording stars and space objects under daytime conditions, using a video camera with a silicon photodetector array. Algorithms for summing up to 300 frames of video images and two-point correction of spatial (geometrical) noise of the array are used in real time in the camera. They made it possible to reduce temporal and spatial noise, to increase the visualization threshold of magnitude, and to observe low-brightness stars and space objects with magnitudes of 8m–10m under daytime conditions from various sites in the northern hemisphere of the sky.

An augmented reality image registration method based on improved ORB

In the process of Augmented Reality (AR) image registration, the traditional ORB (oriented FAST and rotated BRIEF) algorithm has low registration rate and poor real-time performance. In this paper, an improved AR image registration method based on improved ORB is proposed. Firstly, the calibration image and video frame image feature points are obtained by the improved FAST feature detection algorithm. Then, the binary descriptor of BRISK, which using the custom domain sampling pattern is used for feature description, and the scale invariance of the traditional ORB algorithm is improved. Finally, the random sampling consistency (RANSAC) algorithm is used to eliminate the wrong matching point pairs and optimize the feature matching. Experiments show that compared with the AR image registration method described by the traditional ORB algorithm and the FREAK feature, the registration rate of the proposed algorithm is increased by 1.1% and 8.4%, and the generation time is reduced by 0.13s and 0.12s, respectively. The experimental results show that the AR image registration method proposed in this paper can obtain higher feature point registration rate, and has better real-time performance, which can better meet the application needs of AR image registration.

CCTV High-Speed Analysis Algorithm for Real-time Monitoring of Building Access

The most important information required to help people escape from disasters such as fires is knowing the number of people that are trapped in the building and their mobility situation. For this purpose, CCTV can be installed and used at entrances, but they do not need to rely on visual monitoring. This paper describes the system of automatically detecting access personnel information from video frame images collected from the CCTV in real time. We propose to improve the region growing method commonly used in the digital image processing field without the degradation of the recognition rate. The proposed method is based on a pyramid-type hierarchical area structure, and is designed to compress the resolution of the top layer to 1/16 of the bottom layer. In this hierarchical structure, the top layer is a casting that primarily serves as a trigger to decide whether to proceed with further analysis and, if necessary, a lower level analysis. In addition, CPU consumption time was reduced by ensuring that only areas related to the high-level triggered areas are processed at the lower-level layers. As a result of the experiment, the proposed layer area analysis method was able to reduce the execution time to 13.96% and the overall processing time to 69.92% over the existing one-layer analysis method.

Digital video intrusion intelligent detection method based on narrowband Internet of Things and its application

This paper proposes a digital video intrusion detection method based on Narrow Band Internet of Things (NB-IoT), and establishes a digital video intrusion detection system based on NB-IoT network and SVM intelligent classification algorithm. Firstly, the image is preprocessed by gradation processing and threshold transformation to extract the HOG feature extraction of human intrusion behavior in digital video frame images. Then, combined with the human intrusion HOG feature data, the SVM intelligent algorithm is used to classify the human intrusion behavior, so as to accurately classify the movements of walking, jumping, running and waving in video surveillance. Finally, the performance analysis of the algorithm finds that the classification time, classification accuracy and classification false positive rate of the model are tested. The classification time is 40.8 s, the shortest is 27 s, the classification accuracy is 87.65%, and the lowest is 83.7%. The false detection rate is up to 15%, both of which are less than 20%, indicating that the classification method has good accuracy and stability. Comparing the algorithm with other algorithms, the intrusion sensitivity, intrusion specificity and training speed of the model are 93.6%, 94.3%, and 19 s, respectively, which are better than other methods, which indicates that the model has good detection performance in the experimental stage.