Inter-frame Difference Research Articles

Small Aerial Target Detection Using Trajectory Hypothesis and Verification

Due to cluttered backgrounds, long-range imaging, rapid relative motion, etc., the detection of small aerial targets remains a challenge. Existing works have illustrated the importance of temporal cues for robust small aerial target detection. However, the strong assumptions about the temporal cues give such methods high computational complexity and weak adaptability. To address this problem, we treat trajectory verification as a classification problem using convolutional neural networks. A novel small aerial target detection method that uses trajectory hypothesis and verification is proposed in this study. First, the IMU assisted inter-frame difference is used to extract target candidates from each single image. Then, based on the continuous and smoothness characteristics of the target trajectory, hypotheses of the required length are generated by linking target candidates in adjacent frames. Finally, the trajectory hypotheses are sent to a trained classification convolution neural network for verification. The detected targets are traced back from the verified trajectory. Furthermore, trajectory merge is conducted to link adjacent trajectory segments. The proposed method can achieve real-time processing. Experimental results on public datasets show that the proposed method performs better than existing methods.

(Retracted) Dynamic scene images-assisted intelligent control method for industrialized feeding through deep vision learning

In a recirculating aquaculture system (RAS), feeding is an important factor affecting the growth of breeding objects. The traditional feeding methods relied on manual experience, which resulted in high labor costs and bait waste. To deal with these challenges, this paper proposes a dynamic scene images-assisted intelligent control method for industrial feeding through deep vision learning. First, a feeding video is processed according to the interframe difference method to obtain the image of the feeding state of the fish. Then, a modified VGG16 model is developed to determine the feeding state of the fish, transform it into a binary classification problem, and calculate the feeding frequency of the fish. After that, residual bait detection is deployed by adapting the YOLOv5 model. The results of the feeding frequency and the residual bait detection are then used to develop an intelligent feeding strategy to improve the growth rate of the fish and the conversion rate of the bait. Experimental tests on real-world scene images showed that the accuracy of identifying the feeding state by the modified VGG16 model reaches 92.4%. Through the verification of the medium-size RAS, compared with the traditional feeding method, the intelligent feeding method significantly saves manpower and reduce 15% of bait waste.

Ice detection and tracking by the infrared thermography based on the electro-impulse de-icing system

An infrared thermography method was proposed, which exploited the electro-impulse de-icing system as electromagnetic excitation to detect ice shape and track ice motion. The coupling physical fields of circuit, electromagnetic, stress and heat were expounded theoretically and numerical simulated to verify the feasibility of the measurement. With the use of image processing for ice samples of different system parameters and shapes, the original ice edge was obtained at discharge voltages of 100–200 V. As the residual icing state discussed at discharge voltages of 350–450 V, the trajectory of fragmented ice was tracked by interframe difference and centroid method, and de-icing effect was evaluated by the ratio of the Euclidean distance of the fragmented ice centroids as well. The proposed method presents a novel research insight into integrating ice detection and anti/de-icing technology of aircrafts. It can also be referenced for infrared thermography detection and fragmented ice tracking via heat sources.

Detection of Rockfalls on Tunnel Faces Using Extracted Moving Objects, Excavation Point Estimation, and Generation of Trajectory Images

Background: Tunnels are constructed at various locations for infrastructural purposes. During tunnel construction, industrial accidents have occurred due to rockfalls at tunnel faces. In addition, it has been confirmed that large rockfalls are precursors to tunnel collapses. Visual monitoring is currently used to detect such rockfalls. However, there are limitations to visual monitoring, and monitoring equipment is needed. Objective: In existing research, the inter-frame difference method of image processing and laser measurement methods have been proposed. However, there are difficulties in monitoring the entirety of the tunnel face using these methods. Thus, we propose methods that can overcome these difficulties and accurately detect rockfalls and identify where they occur. Methods: In this study, we propose a method for detecting rockfalls by combining the extraction of moving objects on a tunnel face and the estimation of excavation points. To identify the location of the rockfalls, rockfall trajectory images were generated. Results: Through experiments conducted during excavation, it was confirmed that the proposed method could correctly identify and detect only rockfalls in real time and identify the locations where they occurred. Conclusion: In this study, only rockfalls of 52 mm x 71 mm in size are detected in real time. It can enable workers to evacuate and prevent industrial accidents. In addition, by identifying the location of rockfalls, it is possible to know the danger level of the tunnel face.

Night Vision Anti-Halation Method Based on Infrared and Visible Video Fusion.

In order to address the discontinuity caused by the direct application of the infrared and visible image fusion anti-halation method to a video, an efficient night vision anti-halation method based on video fusion is proposed. The designed frame selection based on inter-frame difference determines the optimal cosine angle threshold by analyzing the relation of cosine angle threshold with nonlinear correlation information entropy and de-frame rate. The proposed time-mark-based adaptive motion compensation constructs the same number of interpolation frames as the redundant frames by taking the retained frame number as a time stamp. At the same time, considering the motion vector of two adjacent retained frames as the benchmark, the adaptive weights are constructed according to the interframe differences between the interpolated frame and the last retained frame, then the motion vector of the interpolated frame is estimated. The experimental results show that the proposed frame selection strategy ensures the maximum safe frame removal under the premise of continuous video content at different vehicle speeds in various halation scenes. The frame numbers and playing duration of the fused video are consistent with that of the original video, and the content of the interpolated frame is highly synchronized with that of the corresponding original frames. The average FPS of video fusion in this work is about six times that in the frame-by-frame fusion, which effectively improves the anti-halation processing efficiency of video fusion.

Reliability and validity analysis of personality assessment model based on gait video.

Personality affects an individual’s academic achievements, occupational tendencies, marriage quality and physical health, so more convenient and objective personality assessment methods are needed. Gait is a natural, stable, and easy-to-observe body movement that is closely related to personality. The purpose of this paper is to propose a personality assessment model based on gait video and evaluate the reliability and validity of the multidimensional model. This study recruited 152 participants and used cameras to record their gait videos. Each participant completed a 44-item Big Five Inventory (BFI-44) assessment. We constructed diverse static and dynamic time-frequency features based on gait skeleton coordinates, interframe differences, distances between joints, angles between joints, and wavelet decomposition coefficient arrays. We established multidimensional personality trait assessment models through machine learning algorithms and evaluated the criterion validity, split-half reliability, convergent validity, and discriminant validity of these models. The results showed that the reliability and validity of the Gaussian process regression (GPR) and linear regression (LR) models were best. The mean values of their criterion validity were 0.478 and 0.508, respectively, and the mean values of their split-half reliability were all greater than 0.8. In the formed multitrait-multimethod matrix, these methods also had higher convergent and discriminative validity. The proposed approach shows that gait video can be effectively used to evaluate personality traits, providing a new idea for the formation of convenient and non-invasive personality assessment methods.

FastFusion: Real-Time Indoor Scene Reconstruction with Fast Sensor Motion

Real-time 3D scene reconstruction has attracted a great amount of attention in the fields of augmented reality, virtual reality and robotics. Previous works usually assumed slow sensor motions to avoid large interframe differences and strong image blur, but this limits the applicability of the techniques in real cases. In this study, we propose an end-to-end 3D reconstruction system that combines color, depth and inertial measurements to achieve a robust reconstruction with fast sensor motions. We involved an extended Kalman filter (EKF) to fuse RGB-D-IMU data and jointly optimize feature correspondences, camera poses and scene geometry by using an iterative method. A novel geometry-aware patch deformation technique is proposed to adapt the changes in patch features in the image domain, leading to highly accurate feature tracking with fast sensor motions. In addition, we maintained the global consistency of the reconstructed model by achieving loop closure with submap-based depth image encoding and 3D map deformation. The experiments revealed that our patch deformation method improves the accuracy of feature tracking, that our improved loop detection method is more efficient than the original method and that our system possesses superior 3D reconstruction results compared with the state-of-the-art solutions in handling fast camera motions.

PFD-SLAM: A New RGB-D SLAM for Dynamic Indoor Environments Based on Non-Prior Semantic Segmentation

Now, most existing dynamic RGB-D SLAM methods are based on deep learning or mathematical models. Abundant training sample data is necessary for deep learning, and the selection diversity of semantic samples and camera motion modes are closely related to the robust detection of moving targets. Furthermore, the mathematical models are implemented at the feature-level of segmentation, which is likely to cause sub or over-segmentation of dynamic features. To address this problem, different from most feature-level dynamic segmentation based on mathematical models, a non-prior semantic dynamic segmentation based on a particle filter is proposed in this paper, which aims to attain the motion object segmentation. Firstly, GMS and optical flow are used to calculate an inter-frame difference image, which is considered an observation measurement of posterior estimation. Then, a motion equation of a particle filter is established using Gaussian distribution. Finally, our proposed segmentation method is integrated into the front end of visual SLAM and establishes a new dynamic SLAM, PFD-SLAM. Extensive experiments on the public TUM datasets and real dynamic scenes are conducted to verify location accuracy and practical performances of PFD-SLAM. Furthermore, we also compare experimental results with several state-of-the-art dynamic SLAM methods in terms of two evaluation indexes, RPE and ATE. Still, we provide visual comparisons between the camera estimation trajectories and ground truth. The comprehensive verification and testing experiments demonstrate that our PFD-SLAM can achieve better dynamic segmentation results and robust performances.

Quantitative Evaluation of Vitreous Opacities Using Motion Video Through Slit-Lamp Examination

This study proposed a quantitative evaluation method for vitreous opacities using motion video. The proposed method focused on moving turbidity in the vitreous. The moving turbidity appeared as an inter-frame difference, which was calculated from two consecutive frames. Therefore, the degree of vitreous opacity was estimated using this inter-frame difference. The proposed method was applied in the experiments to actual motion videos obtained using slit-lamp examination. The effectiveness of method was confirmed using the t-test and linear discriminant method.

Detecting falling rocks by estimating excavation points using single color markers

Tunnels have been constructed in various places for transportation and lifelines. During tunnel construction, the industrial accidents have occurred due to falling rocks from the tunnel face. A large amount of falling rocks is confirmed in the precursor of tunnel collapse. Therefore, it is necessary to detect falling rocks to prevent industrial accidents and to grasp of the situation on tunnel face. As conventional methods, the inter-frame difference method and the laser measurement method were proposed. However, those methods were difficult to monitor the entire tunnel face and detected moving objects other than falling rocks. In this paper, we propose a falling rocks detection method that combines the moving object detection method on the tunnel face and the estimation method of excavation points using single color markers. It was confirmed that only falling rocks were detected during excavation experiment.

Retracted] Design of Sports Image Contour Feature Acquisition System Based on the Background Subtraction Method

Moving target detection (MTD) is one of the emphases and difficulties in the field of computer vision and image processing. It is the basis of moving target tracking and behavior recognition. We propose two methods are improved and fused, respectively, and the fusion algorithm is applied to the complex scene for MTD, so as to improve the accuracy of MTD in complex and hybrid scenes. Using the main idea of the three‐frame difference image method, the background difference method and the interframe difference method are combined to make their advantages complementary to overcome each other’s weaknesses. The experimental results show that the method can be well adapted to the situation of periodic motion interference in the background, and it can adapt to the situation of sudden background changes.

An Adaptive Space Target Detection Algorithm

In space target monitoring system, due to the interference of noise, stray light and the limitations of the image acquisition system, there are nonuniform background and smear phenomena in star images. Moreover, there are also a large number of background stars in star images. All these factors lead to the difficulty of space target detection. In order to improve the detection performance, an adaptive space target detection algorithm is proposed in this letter. Firstly, in order to suppress the complex background interference in the star image, a window adaptive bidirectional one-dimensional (1-D) median filtering method is proposed, whose window size is set based on the maximum star size in the first star image. Then, background correction is performed by bidirectional 1-D median filter. Secondly, in order to improve the reliability of space target segmentation, an improved inter-frame difference method is proposed, which consists of two parts: 1) the bright line residue is eliminated based on a horizontal 1-D median filter; 2) the target segmentation threshold is adaptively set based on the Kalman filter method. Compared with existing algorithms, our proposed method can detect space targets quickly with low false alarm rate and high detection rate under complex background conditions.

Study on the Automatic Basketball Shooting System Based on the Background Subtraction Method

There are many drawbacks such as clustering, background updating, inaccurate testing results, and low anti-interference performance in traditional moving target detection theory. In our study, a background subtraction method to automatically capture the basketball shooting trajectory was used to eliminate the drawbacks of the fixed-point shooting system such as cumbersome installation and time and manpower consumption. It also can improve the accuracy and efficiency of moving target detection. We also synthetically compared to common methods including the optical flow method and interframe difference method. Results showed that the background subtraction method has better accuracy with an accuracy rate over about 90% than the interframe subtraction method (88%) and the optimal flow method (85%) and presents excellent robustness with considering variable speed and nonrigid objects. Meanwhile, the automatic detection system for basketball shooting based on background subtraction is built by coupling background subtraction with detection characteristics. The system detection speed built is further accelerated, and the image denoising is improved. The trajectory error rate is about 0.3, 0.4, and 0.5 for the background subtraction method, interframe subtraction method, and optimal flow method, respectively.

Detection of Body Behavior Characteristics in Sports Training Based on Grey Relational Model

In order to deal with the problems of low detection accuracy and efficiency existing in traditional detection methods of body behavior characteristics in sports training, this paper proposes a new detection method of body behavior characteristics in sports training based on the grey correlation model. According to this method, the foreground binary image is obtained by the interframe difference method, and the action information and contour information are extracted, firstly. And then, the behavior feature descriptor is obtained. After the description of body behavior characteristics in sports training is completed, the grey correlation coefficient is calculated, and the complete observation equation of time delay estimation of image grey value under complex sports training background is established to complete the detection of body behavior characteristics. The experimental results show that, compared with the traditional methods, the detection accuracy and efficiency of the text method can always maintain a high level, indicating that the practical application performance of this method is strong. In fact, the grey analysis does not attempt to find the best solution but does provide techniques for determining a good solution.

The Acquisition of Position and Orientation of the Conveyor Belt Workpiece Based on the Inter Frame Difference in ROI in Camera Video

Conveyor belt transfer is a widely used transportation means in industry and agriculture, with the help of the robot arms the workpiece on the belt can be picked and placed, replacing human sorters for production lines work. The position and orientation of the workpiece are important for grabbing by the robot arms. The goal of the paper was to investigate the acquisition of the position and orientation of the conveyor belt workpiece by means of the camera video overhead looking down the belt. The proposed method is the inter frame difference in nature, using the conveyor belt background as the first frame, but the other frames were not used wholly as usually, only an ROI all around the conveyor belt in the camera video was chosen, and the inter frame difference was carried out in the ROI. The ROI was of the same width as that of the belt in the video which was known in advance, while the length of the ROI was arbitrary, so one pixel in the frame was scaled to the actual length conveniently. Every read frame behind the background was computed the difference with the background in such ROI, and the four vertexes coordinates of the rectangle workpiece image on the belt were obtained when it passed the ROI, and then the distance apart from the right belt boundary was calculated due to the proportional relation between the width of workpiece and that of the ROI. Two kind workpiece orientation on the belt toward the left and right were judged using the same obtained four vertexes coordinates by means of Euclidian length, and the tilt angle was calculated by arc tangent function in favour of two narrow sides of rectangle workpiece grab. The actual test showed that the method of obtaining the position and orientation of workpiece on the belt proposed in the paper could be realized correctly.

Experimental Discussion on Fire Image Recognition Based on Feature Extraction

Video image-based fire detection technology can overcome some shortcomings of traditional fire detection, and has a good development prospect. This paper summarizes the basic principles of image-based fire detection, and analyzes the main features of fire combustion images. According to these features, firstly, the interframe difference method and the watershed algorithm are used to extract the suspected fire image area which may occur. Then, the features of flame image in early fire stage, such as increasing flame area, fluttering edge, irregular shape and flame color, are used as fire recognition criteria. Meanwhile, various image processing technologies and algorithms are used to extract the four main features of the fire, so as to eliminate various sources of interference and further determine whether a fire has occurred. Finally, a variety of different fuels were selected under indoor conditions to simulate fire experiments under different conditions, and the video was recorded. Fire recognition experiments were carried out using experimental videos and some videos found on the Internet. The experimental results show that the extraction and further recognition of suspected fire areas are both effective. However, the experimental simulation environment is relatively simple, and many theoretical and practical problems need to be further studied and solved.

Vision-based real-time vehicle detection and vehicle speed measurement using morphology and binary logical operation

In recent trends, digital information to the industrial integration for the intelligent transportation system (ITS) field is gaining importance for the researcher, academia, and industrial persons. Visual information helps to manage traffic systems in the industrial forum to build smart cities in developed countries. This paper presents vision-based real-time vehicle detection and Vehicle Speed Measurement (VSM) using morphology operation and binary logical process for an unplanned traffic scenario using image processing techniques. Vehicle detection and VSM help to reduce the number of accidents and improve road network efficiency. The bounding box size for vehicle detection is flexible according to vehicles' sizes on the road. We test this system with different colors and dimensions for a selected Region of Interest (ROI). The ROI sets using the two-line approach. Here, we compare the proposed system with the inter-frame difference method and the blob analysis method with recall, precision, and F1 performance parameters.

Development of a novel motion capture and gait analysis system for rat locomotion

This paper proposes a marker-based motion capture system for rat motion detection and gait analysis. Motion capture in small animals such as rodents is more challenging than in humans because of their small bodies and rapid motion. Existing algorithms have poor applicability in rat motion capture in environments outside the studio. Moreover, gait analysis targeting on the rat is not performed by existing motion capture software. Our method consists of four procedures. First, Region of Interest (ROI) is extracted from the background using the inter-frame difference method and a depth filter. Second, a double-threshold marker detection method is used to detect markers in ROI and a marker shape filter is used to classify the markers. Third, a marker corrector is designed to modify missing and incorrect markers. Finally, a deep learning network is used to analyse the gait trajectory to classify rat as healthy, injured, or rehabilitated. The experimental result shows that marker recognition accuracy is 99.33%, higher than that of most existing software. The validation accuracy of the network is 100% and the loss is 0.0001. This method is conductive to the development of motion capture systems for small animals and research into the gait kinematics of rodents.

Crowd estimation using key‐point matching with support vector regression

The crowd behaviour understanding and density estimation are some of the fast-growing fields in video surveillance. There are many techniques (detection and regression) that are used as the method of crowd analysis and estimation. In the present approach, SVR (support vector regression) is used as the basic analysis technique and the novel key-point matching with SURF (speedup robust feature) is used as the feature extractor for moving objects in the video. The traditional linear regression methods used mainly key-point as one of the statistical features instead of matching with consecutive frames, but we used the magnitude of the optical flow for foreground object extraction instead of inter-frame difference. The combination of the optical flow of foreground objects and key-point matching generates new features apart from conventional features such as areas and corners. In this new approach, key-point pairing with linear regression is tested with the PETS2009 dataset, and performance is compared with the existing approaches.

Optimizing Synchronization of Tennis Professional League Live Broadcast Based on Wireless Network Planning

The live broadcast of the common league mainly adopts the design of streaming media data block by block. The secondary processing of streaming data is carried out on the server-side. It results in a higher delay during a live broadcast. The higher delay has numerous disadvantages during broadcasting for delay-sensitive applications. We have to solve the delay problems during a live broadcast. Therefore, in this paper, we optimize the synchronization of the tennis professional league live broadcast based on wireless network planning. In the proposed scheme, the 3D detection target model is first constructed, and the background is extracted from the moving video image. The extraction is performed using the interframe difference elimination algorithm, and the target motion trajectory is predicted. Then the phase difference characteristics of the image foreground trajectory are analyzed by the Hilbert transform to detect target missing points. By observing the vertex structure of the target missing point, the phase Fourier transform frame is constructed. Finally, the synchronization of the league live broadcast is encoded and decoded. Based on the timestamp, the synchronization optimization of the tennis professional league live broadcast is completed. The experimental results show that the synchronization optimization method has good synchronization, high resolution, short reaction time, and good detection effect.