Video Dataset Research Articles

The SPECTRAL Perfusion Arm Clamping dAtaset (SPECTRALPACA) for video-rate functional imaging of the skin

Spectral imaging has the potential to become a key technique in interventional medicine as it unveils much richer optical information compared to conventional RBG (red, green, and blue)-based imaging. Thus allowing for high-resolution functional tissue analysis in real time. Its higher information density particularly shows promise for the development of powerful perfusion monitoring methods for clinical use. However, even though in vivo validation of such methods is crucial for their clinical translation, the biomedical field suffers from a lack of publicly available datasets for this purpose. Closing this gap, we generated the SPECTRAL Perfusion Arm Clamping dAtaset (SPECTRALPACA). It comprises ten spectral videos (∼20 Hz, approx. 20,000 frames each) systematically recorded of the hands of ten healthy human participants in different functional states. We paired each spectral video with concisely tracked regions of interest, and corresponding diffuse reflectance measurements recorded with a spectrometer. Providing the first openly accessible in human spectral video dataset for perfusion monitoring, our work facilitates the development and validation of new functional imaging methods.

Temporally Consistent Enhancement of Low-Light Videos via Spatial-Temporal Compatible Learning

Temporal inconsistency is the annoying artifact that has been commonly introduced in low-light video enhancement, but current methods tend to overlook the significance of utilizing both data-centric clues and model-centric design to tackle this problem. In this context, our work makes a comprehensive exploration from the following three aspects. First, to enrich the scene diversity and motion flexibility, we construct a synthetic diverse low/normal-light paired video dataset with a carefully designed low-light simulation strategy, which can effectively complement existing real captured datasets. Second, for better temporal dependency utilization, we develop a Temporally Consistent Enhancer Network (TCE-Net) that consists of stacked 3D convolutions and 2D convolutions to exploit spatial-temporal clues in videos. Last, the temporal dynamic feature dependencies are exploited to obtain consistency constraints for different frame indexes. All these efforts are powered by a Spatial-Temporal Compatible Learning (STCL) optimization technique, which dynamically constructs specific training loss functions adaptively on different datasets. As such, multiple-frame information can be effectively utilized and different levels of information from the network can be feasibly integrated, thus expanding the synergies on different kinds of data and offering visually better results in terms of illumination distribution, color consistency, texture details, and temporal coherence. Extensive experimental results on various real-world low-light video datasets clearly demonstrate the proposed method achieves superior performance to state-of-the-art methods. Our code and synthesized low-light video database will be publicly available at https://github.com/lingyzhu0101/low-light-video-enhancement.git.

Training deep learning based dynamic MR image reconstruction using open-source natural videos

To develop and assess a deep learning (DL) pipeline to learn dynamic MR image reconstruction from publicly available natural videos (Inter4K). Learning was performed for a range of DL architectures (VarNet, 3D UNet, FastDVDNet) and corresponding sampling patterns (Cartesian, radial, spiral) either from true multi-coil cardiac MR data (N = 692) or from synthetic MR data simulated from Inter4K natural videos (N = 588). Real-time undersampled dynamic MR images were reconstructed using DL networks trained with cardiac data and natural videos, and compressed sensing (CS). Differences were assessed in simulations (N = 104 datasets) in terms of MSE, PSNR, and SSIM and prospectively for cardiac cine (short axis, four chambers, N = 20) and speech cine (N = 10) data in terms of subjective image quality ranking, SNR and Edge sharpness. Friedman Chi Square tests with post-hoc Nemenyi analysis were performed to assess statistical significance. In simulated data, DL networks trained with cardiac data outperformed DL networks trained with natural videos, both of which outperformed CS (p < 0.05). However, in prospective experiments DL reconstructions using both training datasets were ranked similarly (and higher than CS) and presented no statistical differences in SNR and Edge Sharpness for most conditions.The developed pipeline enabled learning dynamic MR reconstruction from natural videos preserving DL reconstruction advantages such as high quality fast and ultra-fast reconstructions while overcoming some limitations (data scarcity or sharing). The natural video dataset, code and pre-trained networks are made readily available on github.

A Long-Term Video Tracking Method for Group-Housed Pigs

Pig tracking provides strong support for refined management in pig farms. However, long and continuous multi-pig tracking is still extremely challenging due to occlusion, distortion, and motion blurring in real farming scenarios. This study proposes a long-term video tracking method for group-housed pigs based on improved StrongSORT, which can significantly improve the performance of pig tracking in production scenarios. In addition, this research constructs a 24 h pig tracking video dataset, providing a basis for exploring the effectiveness of long-term tracking algorithms. For object detection, a lightweight pig detection network, YOLO v7-tiny_Pig, improved based on YOLO v7-tiny, is proposed to reduce model parameters and improve detection speed. To address the target association problem, the trajectory management method of StrongSORT is optimized according to the characteristics of the pig tracking task to reduce the tracking identity (ID) switching and improve the stability of the algorithm. The experimental results show that YOLO v7-tiny_Pig ensures detection applicability while reducing parameters by 36.7% compared to YOLO v7-tiny and achieving an average video detection speed of 435 frames per second. In terms of pig tracking, Higher-Order Tracking Accuracy (HOTA), Multi-Object Tracking Accuracy (MOTP), and Identification F1 (IDF1) scores reach 83.16%, 97.6%, and 91.42%, respectively. Compared with the original StrongSORT algorithm, HOTA and IDF1 are improved by 6.19% and 10.89%, respectively, and Identity Switch (IDSW) is reduced by 69%. Our algorithm can achieve the continuous tracking of pigs in real scenarios for up to 24 h. This method provides technical support for non-contact pig automatic monitoring.

Challenges in multi-centric generalization: phase and step recognition in Roux-en-Y gastric bypass surgery

PurposeMost studies on surgical activity recognition utilizing artificial intelligence (AI) have focused mainly on recognizing one type of activity from small and mono-centric surgical video datasets. It remains speculative whether those models would generalize to other centers.MethodsIn this work, we introduce a large multi-centric multi-activity dataset consisting of 140 surgical videos (MultiBypass140) of laparoscopic Roux-en-Y gastric bypass (LRYGB) surgeries performed at two medical centers, i.e., the University Hospital of Strasbourg, France (StrasBypass70) and Inselspital, Bern University Hospital, Switzerland (BernBypass70). The dataset has been fully annotated with phases and steps by two board-certified surgeons. Furthermore, we assess the generalizability and benchmark different deep learning models for the task of phase and step recognition in 7 experimental studies: (1) Training and evaluation on BernBypass70; (2) Training and evaluation on StrasBypass70; (3) Training and evaluation on the joint MultiBypass140 dataset; (4) Training on BernBypass70, evaluation on StrasBypass70; (5) Training on StrasBypass70, evaluation on BernBypass70; Training on MultiBypass140, (6) evaluation on BernBypass70 and (7) evaluation on StrasBypass70.ResultsThe model’s performance is markedly influenced by the training data. The worst results were obtained in experiments (4) and (5) confirming the limited generalization capabilities of models trained on mono-centric data. The use of multi-centric training data, experiments (6) and (7), improves the generalization capabilities of the models, bringing them beyond the level of independent mono-centric training and validation (experiments (1) and (2)).ConclusionMultiBypass140 shows considerable variation in surgical technique and workflow of LRYGB procedures between centers. Therefore, generalization experiments demonstrate a remarkable difference in model performance. These results highlight the importance of multi-centric datasets for AI model generalization to account for variance in surgical technique and workflows. The dataset and code are publicly available at https://github.com/CAMMA-public/MultiBypass140.

A self-supervised spatio-temporal attention network for video-based 3D infant pose estimation

General movement and pose assessment of infants is crucial for the early detection of cerebral palsy (CP). Nevertheless, most human pose estimation methods, in 2D or 3D, focus on adults due to the lack of large datasets and pose annotations on infants. To solve these problems, here we present a model known as YOLO-infantPose, which has been fine-tuned, for infant pose estimation in 2D. We further propose a self-supervised model called STAPose3D for 3D infant pose estimation based on videos. We employ multi-view video data during the training process as a strategy to address the challenge posed by the absence of 3D pose annotations. STAPose3D combines temporal convolution, temporal attention, and graph attention to jointly learn spatio-temporal features of infant pose. Our methods are summarized into two stages: applying YOLO-infantPose on input videos, followed by lifting these 2D poses along with respective confidences for every joint to 3D. The employment of the best-performing 2D detector in the first stage significantly improves the precision of 3D pose estimation. We reveal that fine-tuned YOLO-infantPose outperforms other models tested on our clinical dataset as well as two public datasets MINI-RGBD and YouTube-Infant dataset. Results from our infant movement video dataset demonstrate that STAPose3D effectively comprehends the spatio-temporal features among different views and significantly improves the performance of 3D infant pose estimation in videos. Finally, we explore the clinical application of our method for general movement assessment (GMA) in a clinical dataset annotated as normal writhing movements or abnormal monotonic movements according to the GMA standards. We show that the 3D pose estimation results produced by our STAPose3D model significantly boost the GMA prediction performance than 2D pose estimation. Our code is available at github.com/wwYinYin/STAPose3D.

English Speaking Assessment Algorithm Based on Deep Learning

English as a Foreign Language (EFL) students perform when speaking in public. An increasingly globalized world, effective public speaking is critical, but EFL students struggle to perform it, despite importance of qualities such as eye contact, speech pauses, there is presently no objective examination of such elements. A summative assessment has historically been the predominant form of evaluation in college English speaking assessments. Exam-centric teaching has considerable negative effect on foreign language training. In this research work, English Speaking Assessment Algorithm Based on Deep Learning (ESA-NEGCN-NBOA) is proposed. Initially, input video data are gathered from the multiple video dataset (MVD).The input video data is then pre-processed using Deep Attentional Guided Image Filtering (DAGIF) to remove presence of signal-dependent noise and improve lack of pixels from the regions and enhanced the video data. The data that has been pre-processed is utilized to Feature extraction using New General Double Integral Transform (NGDIT), which extract the significant attributes such as mel-frequency cepstral coefficients, energy, speech rate and pitch. Then NEGCN is proposed to improve students spoken English performance by assessmenting the English speakers. In general, NEGCN doesn’t express some adaption of optimization approaches for determining optimal parameters to promise exact improvement of assessment. Therefore, NBOA is proposed to enhance weight parameter of NEGCN for English speaking assessments, which precisely assess the English speaking. Performance measures such as accuracy, assessment error, evaluation time, pretest and posttest are examined when the proposed ESA-NEGCN-NBOA method is put into practice. The proposed ESA-NEGCN-NBOA method attains21.36%, 23.42% and 19.29% higher accuracy, 23.36%, 18.42% and 28.27% lower evaluation error, 20.36%, 27.42%, 28.17% lesser evaluation time analysed with existing techniques, likes innovative strategy towards oral English assessment utilizing machine learning, data mining, blockchain methods(IST-OEA-ML), machine learning assessment system for spoken English depend on linear predictive coding (AS-SE-LPC-ML), multimodal transfer learning for oral presentation assessment (MM-TL-OPA) respectively.

Efficient Structure from Motion for Large-Size Videos from an Open Outdoor UAV Dataset.

Modern UAVs (unmanned aerial vehicles) equipped with video cameras can provide large-scale high-resolution video data. This poses significant challenges for structure from motion (SfM) and simultaneous localization and mapping (SLAM) algorithms, as most of them are developed for relatively small-scale and low-resolution scenes. In this paper, we present a video-based SfM method specifically designed for high-resolution large-size UAV videos. Despite the wide range of applications for SfM, performing mainstream SfM methods on such videos poses challenges due to their high computational cost. Our method consists of three main steps. Firstly, we employ a visual SLAM (VSLAM) system to efficiently extract keyframes, keypoints, initial camera poses, and sparse structures from downsampled videos. Next, we propose a novel two-step keypoint adjustment method. Instead of matching new points in the original videos, our method effectively and efficiently adjusts the existing keypoints at the original scale. Finally, we refine the poses and structures using a rotation-averaging constrained global bundle adjustment (BA) technique, incorporating the adjusted keypoints. To enrich the resources available for SLAM or SfM studies, we provide a large-size (3840 × 2160) outdoor video dataset with millimeter-level-accuracy ground control points, which supplements the current relatively low-resolution video datasets. Experiments demonstrate that, compared with other SLAM or SfM methods, our method achieves an average efficiency improvement of 100% on our collected dataset and 45% on the EuRoc dataset. Our method also demonstrates superior localization accuracy when compared with state-of-the-art SLAM or SfM methods.

A novel multi-scale violence and public gathering dataset for crowd behavior classification

Dependable utilization of computer vision applications, such as smart surveillance, requires training deep learning networks on datasets that sufficiently represent the classes of interest. However, the bottleneck in many computer vision applications lies in the limited availability of adequate datasets. One particular application that is of great importance for the safety of cities and crowded areas is smart surveillance. Conventional surveillance methods are reactive and often ineffective in enable real-time action. However, smart surveillance is a key component of smart and proactive security in a smart city. Motivated by a smart city application which aims at the automatic identification of concerning events for alerting law-enforcement and governmental agencies, we craft a large video dataset that focuses on the distinction between small-scale violence, large-scale violence, peaceful gatherings, and natural events. This dataset classifies public events along two axes, the size of the crowd observed and the level of perceived violence in the crowd. We name this newly-built dataset the Multi-Scale Violence and Public Gathering (MSV-PG) dataset. The videos in the dataset go through several pre-processing steps to prepare them to be fed into a deep learning architecture. We conduct several experiments on the MSV-PG datasets using a ResNet3D, a Swin Transformer and an R(2 + 1)D architecture. The results achieved by these models when trained on the MSV-PG dataset, 88.37%, 89.76%, and 89.3%, respectively, indicate that the dataset is well-labeled and is rich enough to train deep learning models for automatic smart surveillance for diverse scenarios.

Performance Comparison of Five Methods Available in ImageJ for Bird Counting and Detection from Video Datasets

Bird monitoring is an important approach to studying the diversity and abundance of birds, especially during migration, as it can provide core data for bird conservation purposes. The previous methods for bird number estimation are largely based on manual counting, which suffers from low throughput and a high error rate. In this study, we aimed to provide an alternative bird-counting method from video datasets by using five available ImageJ methods: Particle Analyzer, Find Maxima, Watershed segmentation, TrackMate, and trainable WEKA segmentation. The numbers of birds and their XY coordinates were extracted from videos to conduct a side-by-side comparison with the manual counting results, and the three important criteria of the sensitivity, precision, and F1 score were calculated for the performance evaluation. From the tests, which we conducted for four different cases with different bird numbers or flying patterns, TrackMate had the best overall performance for counting birds and pinpointing their locations, followed by Particle Analyzer, Find Maxima, WEKA, and lastly, Watershed, which showed low precision in most of the cases. In summary, five ImageJ-based counting methods were compared in this study, and we validated that TrackMate obtains the best performance for bird counting and detection.

Temporal action segmentation for video encryption

Videos contain temporal information, enabling them to capture the dynamic changes of actions and provide richer visual effects. Traditional video encryption methods involve decomposing videos into frames and encrypting them frame by frame, which results in significant resource consumption. This paper proposes a video encryption method based on temporal action segmentation. This methodology involves the identification and extraction of pivotal frames from a video dataset, followed by the encryption of these significant key frames. This approach serves to enhance the efficacy of the video encryption algorithm. The method consists of three modules. The first module uses temporal action segmentation to classify video frames and extract important frames for the second module’s input. The second module encrypts the extracted key frames using a chaos-based encryption algorithm, thereby reducing the time cost of video encryption. The third module outputs the encrypted video. During the encryption process, a large amount of key stream is required. To address this, the paper introduces a new pseudo-random sequence generation method called two-dimensional Gramacy&Lee map (2D-GLM). Comprehensive comparative analysis clearly demonstrates that compared to other systems, 2D-GLM exhibits superior performance and can generate a large number of high-performance pseudo-random sequences. The proposed algorithm is tested on GTEA, and the simulation results demonstrate that it can accomplish video encryption tasks with high security.

The Chinese customers and service staff interactive affective system (CCSIAS): introduction to a multimodal stimulus dataset.

To research the emotional interaction between customers and service staff, single-modal stimuli are being used to activate subjects' emotions while multimodal emotion stimuli with better efficiency are often neglected. This study aims to construct a multimodal emotion stimuli database (CCSIAS) with video records of real work status of 29 service staff and audio clips of interactions between customers and service staff by setting up wide-angle cameras and searching in company's Ocean Engine for 15 consecutive days. First, we developed a tool to assess the emotional statuses of customers and service staff in Study 1. Second, 40 Masters and PhD students were invited to assess the audio and video data to evaluate the emotional states of customers and service staff in Study 2, using the tools developed in Study 1. Third, 118 participants were recruited to test the results from Study 2 to ensure the stability of the derived data. The results showed that 139 sets of stable emotional audio & video data were constructed (26 sets were high, 59 sets were medium and 54 sets were low). The amount of emotional information is important for the effective activation of participants' emotional states, and the degree of emotional activation of video data is significantly higher than that of the audio data. Overall, it was shown that the study of emotional interaction phenomena requires a multimodal dataset. The CCSIAS (https://osf.io/muc86/) can extend the depth and breadth of emotional interaction research and can be applied to different emotional states between customers and service staff activation in the fields of organizational behavior and psychology.

CLRNet: A Cross Locality Relation Network for Crowd Counting in Videos.

In this article, we propose a new cross locality relation network (CLRNet) to generate high-quality crowd density maps for crowd counting in videos. Specifically, a cross locality relation module (CLRM) is proposed to enhance feature representations by modeling local dependencies of pixels between adjacent frames with an adapted local self-attention mechanism. First, different from the existing methods which measure similarity between pixels by dot product, a new adaptive cosine similarity is advanced to measure the relationship between two positions. Second, the traditional self-attention modules usually integrate the reconstructed features with the same weights for all the positions. However, crowd movement and background changes in a video sequence are uneven in real-life applications. As a consequence, it is inappropriate to treat all the positions in reconstructed features equally. To address this issue, a scene consistency attention map (SCAM) is developed to make CLRM pay more attention to the positions with strong correlations in adjacent frames. Furthermore, CLRM is incorporated into the network in a coarse-to-fine way to further enhance the representational capability of features. Experimental results demonstrate the effectiveness of our proposed CLRNet in comparison to the state-of-the-art methods on four public video datasets. The codes are available at: https://github.com/Amelie01/CLRNet.

HFSCCD: A Hybrid Neural Network for Fetal Standard Cardiac Cycle Detection in Ultrasound Videos.

In the fetal cardiac ultrasound examination, standard cardiac cycle (SCC) recognition is the essential foundation for diagnosing congenital heart disease. Previous studies have mostly focused on the detection of adult CCs, which may not be applicable to the fetus. In clinical practice, localization of SCCs needs to recognize end-systole (ES) and end-diastole (ED) frames accurately, ensuring that every frame in the cycle is a standard view. Most existing methods are not based on the detection of key anatomical structures, which may not recognize irrelevant views and background frames, results containing non-standard frames, or even it does not work in clinical practice. We propose an end-to-end hybrid neural network based on an object detector to detect SCCs from fetal ultrasound videos efficiently, which consists of 3 modules, namely Anatomical Structure Detection (ASD), Cardiac Cycle Localization (CCL), and Standard Plane Recognition (SPR). Specifically, ASD uses an object detector to identify 9 key anatomical structures, 3 cardiac motion phases, and the corresponding confidence scores from fetal ultrasound videos. On this basis, we propose a joint probability method in the CCL to learn the cardiac motion cycle based on the 3 cardiac motion phases. In SPR, to reduce the impact of structure detection errors on the accuracy of the standard plane recognition, we use XGBoost algorithm to learn the relation knowledge of the detected anatomical structures. We evaluate our method on the test fetal ultrasound video datasets and clinical examination cases and achieve remarkable results. This study may pave the way for clinical practices.

Aligning Correlation Information for Domain Adaptation in Action Recognition.

Domain adaptation (DA) approaches address domain shift and enable networks to be applied to different scenarios. Although various image DA approaches have been proposed in recent years, there is limited research toward video DA. This is partly due to the complexity in adapting the different modalities of features in videos, which includes the correlation features extracted as long-range dependencies of pixels across spatiotemporal dimensions. The correlation features are highly associated with action classes and proven their effectiveness in accurate video feature extraction through the supervised action recognition task. Yet correlation features of the same action would differ across domains due to domain shift. Therefore, we propose a novel adversarial correlation adaptation network (ACAN) to align action videos by aligning pixel correlations. ACAN aims to minimize the distribution of correlation information, termed as pixel correlation discrepancy (PCD). Additionally, video DA research is also limited by the lack of cross-domain video datasets with larger domain shifts. We, therefore, introduce a novel HMDB-ARID dataset with a larger domain shift caused by a larger statistical difference between domains. This dataset is built in an effort to leverage current datasets for dark video classification. Empirical results demonstrate the state-of-the-art performance of our proposed ACAN for both existing and the new video DA datasets.

EFFICIENT VIDEO COMPRESSION USING HEVC AND NON-LINEAR CONVOLUTIONAL MOBILEVNET BASED RATE-DISTORTION OPTIMIZATION

With the surge in demand for high-quality video content over various platforms, efficient video compression techniques have become indispensable. High-Efficiency Video Coding (HEVC) has been a cornerstone, yet further enhancements are essential for optimal compression. Despite HEVC’s advancements, achieving optimal compression while maintaining video quality remains challenging. Additionally, existing methods often overlook the computational complexity, hindering real-time applications. We propose a novel approach integrating HEVC with Non-Linear Convolutional MobileNet (NLCM) for enhanced compression efficiency. Our method employs a rate-distortion optimization framework, leveraging the capabilities of both HEVC and NLCM to achieve superior compression performance. NLCM provides adaptive filtering, enhancing spatial and temporal correlations, while HEVC ensures high compression efficiency. Through experimentation on standard video datasets, our method demonstrates significant improvements over existing techniques. Compared to HEVC alone, our approach achieves up to 30% reduction in bitrate at equivalent perceptual quality levels. Moreover, computational complexity is reduced by 15%, enabling real- time applications without compromising performance. The proposed method exhibits competitive results across various resolutions and frame rates, making it versatile for diverse video compression scenarios.

Exposing Fake Faces Through Deep Neural Networks Combining Content and Trace Feature Extractors

Abstract: In recent times, the proliferation of free deep learning-based software has facilitated the emergence of convincing facial swaps in videos, commonly referred to as ‘DeepFake’ (DF) videos. ‘Deep learning’ has improved the realism and accessibility of creating fake digital video content, which was previously attainable through traditional visual effects. These AIgenerated media, often referred to as DF, present a dual challenge: their creation is relatively straightforward using AI tools, yet their detection poses a significant hurdle. We address this challenge by employing Convolutional Neural Networks (CNNs) and ‘Recurrent Neural Networks’ (RNNs) to identify ‘DFs’. Specifically, our system utilizes a CNN to obtain frame level characteristics and apply them to train an RNN capable of identifying temporal inconsistencies introduced by DF creation tools. We evaluate our approach on a substantial dataset of fake videos and demonstrate competitive performance with a straightforward architecture.

VIDEO FORGERY IDENTIFICATION APPLICATION

The rapid advancement of deepfake technology has raised significant concerns about the authenticity of visual content. Deep fake videos, which convincingly superimpose one person's likeness onto another's, can be used for both harmless entertainment and malicious misinformation. Detecting these manipulated videos is crucial for maintaining trust and accuracy in media. In this study, we propose a novel approach for deep fake video detection using deep learning techniques, coupled with a user-friendly graphical user. Our methodology involves training a deep neural network using a large dataset of both real and deep fake videos. These features are then fed in to a series of convolutional and recurrent layers to capture both spatial and temporal information. Keywords: Superimpose, Misinformation, Neural Network, convolutional, spatial.

An Angle-Oriented Approach to Transferring Speech to Gesture for Highly Anthropomorphized Embodied Conversational Agents

Realistic co-speech gestures are important to anthropomorphize ECAs, as nonverbal behavior improves expressiveness of their speech greatly. However, the existing approaches to generating co-speech gestures with sufficient details (including fingers, etc.) in 3D scenarios are indeed rare. Additionally, they hardly address the problem of abnormal gestures, temporal–spatial coherence and diversity of gesture sequences comprehensively. To handle abnormal gesture issues, we put forward an angle conversion method to remove body part length from the original in-the-wild video dataset via transferring coordinates of human upper body key points into relative deflection angles and pitch angles. We also propose a neural network called HARP with encoder–decoder architecture to transfer MFCC featured speech audio into aforementioned angles on the basis of CNN and LSTM. The angles then can be rendered as corresponding co-speech gestures. Compared with the other latest approaches, the co-speech gestures generated by HARP are proved to be almost as good as the real person, i.e., they have strong temporal–spatial coherence, diversity, persuasiveness and credibility. Our approach puts finer control on co-speech gestures than most of the existing works by handling all key points of the human upper body. It is more feasible for industrial application, since HARP can be adaptive to any human upper body model. All related code and evidence videos of HARP can be accessed at https://github.com/drrobincroft/HARP .

Smoke Detection with Dual Convolutional Networks From Infrared Frames

In many smoke detection fields, such as fire detection in confined space of aircraft cargo hold, the false alarm rate is still high. In the closed and dark environment of the confined space cabin, the traditional video smoke detection method is difficult to find the fire early because of the limitation of lighting conditions. The advantage of fire detection based on infrared video image is that it does not need lighting conditions and has better performance in dark environment. There is a rapid temperature rise process in the confined space at the beginning of the fire, which is more easily captured by infrared cameras. However, there is little research on infrared frame detection methods in confined space. Therefore, based on the limited space environment of aviation industry, this paper studies the smoke detection problem under the infrared framework, and proposes a high-precision fire and smoke image detection algorithm based on infrared double convolution neural network. By modeling the texture features of neural network and infrared smoke frames, and using video frames as an auxiliary means to increase the number of available training images, the problem of insufficient infrared video data sets is solved. The experimental results show that the detection effect of this method is better than other comparison algorithms in limited space, and the detection false alarm rate is effectively reduced.