Set Of Videos Research Articles

Inter-rater and test-retest reliabilities of lumbar stiffness measurement in the postero-anterior direction using a portable algometer and the Kinovea program.

Back pain negatively impacts a person's quality of life and can cause major disability or even death. The measurement of spinal stiffness can be utilized as a promising tool to guide therapeutic decisions regarding physical therapy that result in effective back pain management. This study aimed to determine the reliability of instrumented postero-anterior (PA) stiffness assessment of the lumbar spine in asymptomatic participants by novice assessors using a portable algometer and the Kinovea program. Thirty asymptomatic participants aged 18-25 years were enrolled in this study. Two novice assessors examined the participants for lumbar spinal stiffness at L1-L5 for two consecutive days. The algometer was applied to measure the PA force that applied to each lumbar. The stiffness assessment of each lumbar region was recorded as a video. The 600 data sets of assessment videos were imported into the Kinovea program to perform displacement measurements of each lumbar level. Spinal displacement values at 15 N were defined by graph plotting between force and displacement. The spinal stiffness values were defined by slope calculation. Both variables were analyzed for inter-rater and test-retest reliabilities using intra-class correlation coefficients (ICCs) and standard error of measurement (SEM). Bland-Altman analysis was applied to assess the inter-rater and test-retest systematical bias and limits of agreement of measuring displacement and stiffness. The inter-rater reliability of measuring the displacement and the stiffness of L1-L5 was moderate to good (displacement ICCs: 0.67-0.83, stiffness ICCs: 0.60-0.83). The test-retest reliability of measuring the displacement and stiffness of L1-L5 were moderate to good, ICCs: 0.57-0.86 and ICCs: 0.51-0.88, respectively. The inter-rater analysis's Bland-Altman plot showed that the systematic bias was 0.83 when measuring displacement and 0.20 when measuring stiffness and the bias of both parameters were in both directions. While the test-retest systematically biased measurements of displacement and stiffness were -0.26 mm and 0.22 N/mm, respectively, and the bias of both parameters were in both directions. The moderate-to-good inter-rater and test-retest reliabilities of the portable instrumented spinal stiffness assessment using a digital algometer and the Kinovea program by novice assessors were demonstrated in this study. Bland-Altman analysis showed that measuring stiffness was more stable and had less systematic bias than measuring displacement. To figure out how reliable the device is in general, more comprehensive studies should be comparatively conducted in the future on subgroups of patients with normal vertebra, hypomobile or hypermobile conditions.

Designing Patient-Centered Health Education Materials for Radiation Dermatitis in Breast Cancer Patients: A Pilot Study and Single-Center Clinical Trial.

For breast cancer patients receiving radiotherapy (RT), radiation dermatitis is the most commonly experienced discomfort. However, patients usually do not participate in the design of health education materials regarding radiation dermatitis. We designed a pilot study to investigate the unmet need of breast cancer patients who would be receiving RT at our department and created a patient-centered educational video. We initiated a single-arm clinical trial to evaluate the effectiveness of the video in improving patient and their family's knowledge of radiation dermatitis, alleviating their anxiety, and helping patients improve self-care skills during RT course. The pilot study was executed in a radiation oncology department of a tertiary medical center in Taiwan. We designed a questionnaire to explore new breast cancer patients' idea of a comprehensive pre-RT health education. Enrolled patients would fill out an electronic questionnaire during their first visit at our outpatient clinic. They were asked to select at most two of the following items they considered most important for pre-RT health education: (1) The mechanism behind RT; (2) Self-care skills for irradiated skin; (3) Activities to be avoided during RT course; (4) The toxicity grading of radiation dermatitis. Given the above information, we created a 4-minute video set in a clinical scenario of a patient receiving pre-RT health education from a nurse. A single-arm clinical trial (IRB approval number: CMUH111-REC2-121) is currently recruiting. The video was launched on YouTube (Link: https://youtu.be/O1uTXpS_ed0) to give patients easy access to the video. Enrolled patients would complete Skindex-16 survey (translated into Chinese) at 3 timepoints: during their first visit at our clinic, in the middle of RT course, and at the first post-RT follow-up visit. Cronbach's alpha test was applied to determine the internal consistency of the questionnaires. The pilot study included 19 patients with an average age of 46.6 (range: 34 to 69) years old. All patients' primary language was Chinese Mandarin. Most patients (84%) received at least secondary education. We found by surprise that RTOG skin toxicity grading was considered very important for most patients (79%), while less than half (42%) found the mechanism behind RT important. The internal validity of the Chinese version of Skindex-16 was good (Cronbach's alpha score = 0.87). For the recruiting trial, we have enrolled 10 patients and planned to close the trial upon enrolling 50 participants. The study result should be available before August 2023. We suggest to actively involve patients in designing health education materials for patient-centered care. Toxicity grading of radiation dermatitis should be included in pre-RT health education for breast cancer patients. Whether the educational video helps reduce patients' anxiety and the incidence of severe radiation dermatitis will soon be answered by our ongoing clinical trial.

Do a pedagogical agent's clothing and an animated video's setting affect learning?

Pedagogical agents are often used to enhance social cues in learning materials. The inclusion of pedagogical agents raises several design questions, for example on what kind of clothing the agent should wear. Further, it is not yet clear how the setting of an animated learning video (i.e., the digitally created background) affects learning. In an online experiment (N = 200), we investigated whether creating thematically appropriate clothing and setting has some added value in that it improves learning outcomes in comparison to more neutral assets. Whereas all participants acquired knowledge from the animated video, there were no main effects of clothing and setting for any of the dependent variables, but an interaction for learning outcomes (), indicating that the appropriately dressed agent worked better combined with the inappropriate setting than with the appropriate setting. Overall, given those non-significant main effects and the small effect size of the interaction, there seem to be some degrees of freedom for designers of pedagogical agents in animated learning videos. However, these degrees of freedom may be limited to at least moderate (i.e., neutral) levels of appropriateness.

Lagrangian motion magnification with double sparse optical flow decomposition

Microexpressions are fast and spatially small facial expressions that are difficult to detect. Therefore, motion magnification techniques, which aim at amplifying and hence revealing subtle motion in videos, appear useful for handling such expressions. There are basically two main approaches, namely, via Eulerian or Lagrangian techniques. While the first one magnifies motion implicitly by operating directly on image pixels, the Lagrangian approach uses optical flow (OF) techniques to extract and magnify pixel trajectories. In this study, we propose a novel approach for local Lagrangian motion magnification of facial micro-motions. Our contribution is 3-fold: first, we fine tune the recurrent all-pairs field transforms (RAFT) for OFs deep learning approach for faces by adding ground truth obtained from the variational dense inverse search (DIS) for the OF algorithm applied to the CASME II video set of facial micro expressions. This enables us to produce OFs of facial videos in an efficient and sufficiently accurate way. Second, since facial micro-motions are both local in space and time, we propose to approximate the OF field by sparse components both in space and time leading to a double sparse decomposition. Third, we use this decomposition to magnify micro-motions in specific areas of the face, where we introduce a new forward warping strategy using a triangular splitting of the image grid and barycentric interpolation of the RGB vectors at the corners of the transformed triangles. We demonstrate the feasibility of our approach by various examples.

Constraint-Aware Multi-Technique Approximate High-Level Synthesis for FPGAs

Numerous approximate computing (AC) techniques have been developed to reduce the design costs in error-resilient application domains, such as signal and multimedia processing, data mining, machine learning, and computer vision, to trade-off computation accuracy with area and power savings or performance improvements. Selecting adequate techniques for each application and optimization target is complex but crucial for high-quality results. In this context, Approximate High-Level Synthesis (AHLS) tools have been proposed to alleviate the burden of hand-crafting approximate circuits by automating the exploitation of AC techniques. However, such tools are typically tied to a specific approximation technique or a difficult-to-extend set of techniques whose exploitation is not fully automated or steered by optimization targets. Therefore, available AHLS tools overlook the benefits of expanding the design space by mixing diverse approximation techniques toward meeting specific design objectives with minimum error. In this work, we propose an AHLS design methodology for FPGAs that automatically identifies efficient combinations of multiple approximation techniques for different applications and design constraints. Compared to single-technique approaches, decreases of up to \(30\% \) in mean squared error and absolute increases of up to \(6.5\% \) in percentage accuracy were obtained for a set of image, video, signal processing and machine learning benchmarks.

A deep learning pipeline for automated classification of vocal fold polyps in flexible laryngoscopy.

To develop and validate a deep learning model for distinguishing healthy vocal folds (HVF) and vocal fold polyps (VFP) on laryngoscopy videos, while demonstrating the ability of a previously developed informative frame classifier in facilitating deep learning development. Following retrospective extraction of image frames from 52 HVF and 77 unilateral VFP videos, two researchers manually labeled each frame as informative or uninformative. A previously developed informative frame classifier was used to extract informative frames from the same video set. Both sets of videos were independently divided into training (60%), validation (20%), and test (20%) by patient. Machine-labeled frames were independently verified by two researchers to assess the precision of the informative frame classifier. Two models, pre-trained on ResNet18, were trained to classify frames as containing HVF or VFP. The accuracy of the polyp classifier trained on machine-labeled frames was compared to that of the classifier trained on human-labeled frames. The performance was measured by accuracy and area under the receiver operating characteristic curve (AUROC). When evaluated on a hold-out test set, the polyp classifier trained on machine-labeled frames achieved an accuracy of 85% and AUROC of 0.84, whereas the classifier trained on human-labeled frames achieved an accuracy of 69% and AUROC of 0.66. An accurate deep learning classifier for vocal fold polyp identification was developed and validated with the assistance of a peer-reviewed informative frame classifier for dataset assembly. The classifier trained on machine-labeled frames demonstrates improved performance compared to the classifier trained on human-labeled frames.

Background subtraction challenges in motion detection using Gaussian mixture model: a survey

<p>Motion detection is becoming prominent for computer vision applications. The background subtraction method that uses the Gaussian mixture model (GMM) is utilized frequently in camera or video settings. However, there is still more work that needs to be done to develop a reliable, accurate and high-performing technique due to various challenges. The degree of difficulty for this challenge is primarily determined by how the object to be detected is defined. It could be influenced by the changes in the object posture or deformations. In this context, we describe and bring together the most significant challenges faced by the background subtraction techniques based on GMM for dealing with a crucial background situation. Therefore, the findings of this study can be used to identify the most appropriate GMM version based on the crucial background situation.</p>

A new mixed reality tool for training in minimally invasive robotic-assisted surgery

Robotic-assisted surgery (RAS) is developing an increasing role in surgical practice. Therefore, it is of the utmost importance to introduce this paradigm into surgical training programs. However, the steep learning curve of RAS remains a problem that hinders the development and widespread use of this surgical paradigm. For this reason, it is important to be able to train surgeons in the use of RAS procedures. RAS involves distinctive features that makes its learning different to other minimally invasive surgical procedures. One of these features is that the surgeons operate using a stereoscopic console. Therefore, it is necessary to perform RAS training stereoscopically. This article presents a mixed-reality (MR) tool for the stereoscopic visualization, annotation and collaborative display of RAS surgical procedures. The tool is an MR application because it can display real stereoscopic content and augment it with virtual elements (annotations) properly registered in 3D and tracked over time. This new tool allows the registration of surgical procedures, teachers (experts) and students (trainees), so that the teacher can share a set of videos with their students, annotate them with virtual information and use a shared virtual pointer with the students. The students can visualize the videos within a web environment using their personal mobile phones or a desktop stereo system. The use of the tool has been assessed by a group of 15 surgeons during a robotic-surgery master’s course. The results show that surgeons consider that this tool can be very useful in RAS training.

Quality, reliability, and popularity of YouTube videos on urticaria: a cross-sectional analysis.

Recently, YouTube has been frequently used by non-physicians and physicians to share information and experiences about diseases. This study aims to compare videos uploaded to YouTube about urticaria in terms of quality, reliability, usefulness, and popularity based on the uploaded source. YouTube was searched using the keyword "urticaria". From the search returns, we selected the first 60 videos created by physicians and the first 60 videos created by non-physicians. We compared the two sets of videos in terms of reliability, usefulness, quality, and popularity using the Video Power Index (VPI) score, DISCERN score, global quality score (GQS) and usefulness score. The mean DISCERN score, GQS, and usefulness score were statistically significantly higher (P<0.001) for videos uploaded by physicians than those uploaded by non-physicians. Although not statistically significant, the median number of likes and comments, the rate of views, and the VPI scores were higher for videos uploaded by the non-physician group. The mean DISCERN scores, GQS, and usefulness scores were statistically significantly higher (P<0.001) for videos uploaded by specialist physicians (dermatologists and allergists) than for those uploaded by non-specialists. Specialist physicians mentioned medical treatment for the condition more than non-specialists, which was statistically significant (P<0.001). YouTube videos about urticaria uploaded by physicians, especially specialists, are of higher quality, reliable and useful, but videos uploaded by non-physicians are more popular and have higher viewing rates.

MARN: Multi-level Attentional Reconstruction Networks for Weakly Supervised Video Temporal Grounding

Video temporal grounding is a challenging task in computer vision that involves localizing a video segment semantically related to a given query from a set of videos and queries. In this paper, we propose a novel weakly-supervised model called the Multi-level Attentional Reconstruction Networks (MARN), which is trained on video-sentence pairs. During the training phase, we leverage the idea of attentional reconstruction to train an attention map that can reconstruct the given query. At inference time, proposals are ranked based on attention scores to localize the most suitable segment. In contrast to previous methods, MARN effectively aligns video-level supervision and proposal scoring, thereby reducing the training-inference discrepancy. In addition, we incorporate a multi-level framework that encompasses both proposal-level and clip-level processes. The proposal-level process generates and scores variable-length time sequences, while the clip-level process generates and scores fix-length time sequences to refine the predicted scores of the proposal in both training and testing. To improve the feature representation of the video, we propose a novel representation mechanism that utilizes intra-proposal information and adopts 2D convolution to extract inter-proposal clues for learning reliable attention maps. By accurately representing these proposals, we can better align them with the textual modalities, and thus facilitate the learning of the model. Our proposed MARN is evaluated on two benchmark datasets, and extensive experiments demonstrate its superiority over existing methods.

A survey on infrared image &amp; video sets

A survey on infrared image &amp; video sets

NDMFCS: An automatic fruit counting system in modern apple orchard using abatement of abnormal fruit detection

Automatic fruit counting is an important task for growers to estimate yield and manage orchards. Although many deep-learning-based fruit detection algorithms have been developed to improve performance of automatic fruit counting systems, abnormal fruit detection has often been caused by these algorithms detecting non-target fruits that have similar growth characteristics to target fruits. For abnormal fruit detection, detected fruits in the back row of the tree were defined as DFBRT, while detected fruits on the ground were defined as DFG. Both of them would result in a higher number of fruits counting than the ground truth. This study proposes an automatic fruit counting system called NDMFCS (Normal Detection Matched Fruit Counting System) to solve this problem for improving fruit counting accuracy in modern apple orchard. NDMFCS consists of three sub-systems, i.e. object detection based on You Only Look Once Version 4-tiny (YOLOv4-tiny), abatement of abnormal fruit detection based on threshold, and fruit counting based on trunk tracking and identity document (ID) assignment. YOLOv4-tiny was selected to implement detection of fruits and trunks, whose output is confidence and pixel coordinates of detected object. The DFBRT and DFG were abated by thresholds to improve detection performance of fruit. This meant that detected fruits were removed when their distance from camera is further than a distance threshold or the confidence of fruit detection is less than a confidence threshold. Finally, fruit counting was implemented by trunk tracking and ID assignment, where each fruit was assigned a unique tracking ID. Results on 10 sets of original videos indicated that average fruit detection precision was improved from 89.1% to 93.3% after abatement of abnormal fruit detection. Also, Multiple Object Tracking Accuracy and Multiple Object Tracking Precision were improved on average by 4.2% and 3.3%, respectively, while average ID Switch Rate was decreased on average by 1.1%. And average fruit counting accuracy was improved to 95.0% by 4.2%. Coefficient of determination (R2) was 0.97, which indicated the number of fruits counted by NDMFCS was near to the ground truth. These results demonstrate that the abatement of abnormal fruit detection can improve performance of apple counting, which has the potential to provide a technical support for estimating fruit yield in modern apple orchards.

Toward a Real-Time Analysis of Column Height by Visible Cameras: An Example from Mt. Etna, in Italy

Volcanic plume height is one the most important features of explosive activity; thus, it is a parameter of interest for volcanic monitoring that can be retrieved using different remote sensing techniques. Among them, calibrated visible cameras have demonstrated to be a promising alternative during daylight hours, mainly due to their low cost and low uncertainty in the results. However, currently these measurements are generally not fully automatic. In this paper, we present a new, interactive, open-source MATLAB tool, named ‘Plume Height Analyzer’ (PHA), which is able to analyze images and videos of explosive eruptions derived from visible cameras, with the objective of automatically identifying the temporal evolution of eruption columns. PHA is a self-customizing tool, i.e., before operational use, the user must perform an iterative calibration procedure based on the analysis of images of previous eruptions of the volcanic system of interest, under different eruptive, atmospheric and illumination conditions. The images used for the calibration step allow the computation of ad hoc expressions to set the model parameters used to recognize the volcanic plume in new images, which are controlled by their individual characteristics. Thereby, the number of frames used in the calibration procedure will control the goodness of the model to analyze new videos/images and the range of eruption, atmospheric, and illumination conditions for which the program will return reliable results. This also allows improvement of the performance of the program as new data become available for the calibration, for which PHA includes ad hoc routines. PHA has been tested on a wide set of videos from recent explosive activity at Mt. Etna, in Italy, and may represent a first approximation toward a real-time analysis of column height using visible cameras on erupting volcanoes.

AUDIO SUMMARIZATION IN REAL TIME FOR PODCAST, SPEECHES AND AUDIOBOOKS

The majority of public works are carried out online as a result of the coronavirus disease (COVID19) pandemic. Many job interviews, primary health care consultations, and company meetings are conducted entirely online, and universities all over the world have switched to online teaching. Applications for online meetings like Microsoft Team and Google Meet are widely available on the market. We approach a topic with numerous practical applications in this work. A method that takes recorded video as input and produces identical written or audio summaries as output is the subject of this paper. Additionally, the proposed method can be used to create meeting minutes, subtitle or scribe entertainment videos, and create lecture notes from lecture videos. The audio track of a video is converted into text by the proposed system. We also used a text summarization algorithm to create text summaries. Users of the system have the option of using a text summary or making an audio output that matches the text summary. The proposed strategy is carried out in Python and the proposed conspire is assessed utilizing brief recordings got from YouTube. The proposed method is manually validated on a set of uploaded videos because there are neither specific datasets or benchmarks available to evaluate efficacy nor any benchmarks to use in related studies.

A data-driven investigation of human action representations

Understanding actions performed by others requires us to integrate different types of information about people, scenes, objects, and their interactions. What organizing dimensions does the mind use to make sense of this complex action space? To address this question, we collected intuitive similarity judgments across two large-scale sets of naturalistic videos depicting everyday actions. We used cross-validated sparse non-negative matrix factorization to identify the structure underlying action similarity judgments. A low-dimensional representation, consisting of nine to ten dimensions, was sufficient to accurately reconstruct human similarity judgments. The dimensions were robust to stimulus set perturbations and reproducible in a separate odd-one-out experiment. Human labels mapped these dimensions onto semantic axes relating to food, work, and home life; social axes relating to people and emotions; and one visual axis related to scene setting. While highly interpretable, these dimensions did not share a clear one-to-one correspondence with prior hypotheses of action-relevant dimensions. Together, our results reveal a low-dimensional set of robust and interpretable dimensions that organize intuitive action similarity judgments and highlight the importance of data-driven investigations of behavioral representations.

Still Rings Movements Recognition in Gymnastics Sport Based on Deep Learning

The methods of detecting objects and tracking their movements are among the methods that are relied upon in many fields, whether medical or industrial, and others. One of these areas that will rely on deep learning meth-ods in discovering and distinguishing the player's movements is the sports field and is very useful in games in which the player's degree depends on the accura-cy of the performance of the movement, such as the gymnastics game, where it was applied to the static ring gymnastics game, where the distinction of move-ments was discovered the stability in this game is based on a convolutional neu-ral network. Models. The neural network was trained on five of the most im-portant stability movements in this game after creating the data set based on a set of videos of tournaments held in the period from 2016-2022, where an aver-age of 1500 images were obtained for each stability movement, which was di-vided into 80% for training and 20 % for testing, after training the convolution-al neural network model, it was applied to a group of video clips for different tournaments. Many criteria were adopted to measure the efficiency of the model after training and practical application, which showed the efficiency of the pro-posed system.

Computer Vision Enabled Segmentation of Kidney Stones During Ureteroscopy and Laser Lithotripsy.

Objective: To evaluate the performance of computer vision models for automated kidney stone segmentation during flexible ureteroscopy and laser lithotripsy. Materials and Methods: We collected 20 ureteroscopy videos of intrarenal kidney stone treatment and extracted frames (N = 578) from these videos. We manually annotated kidney stones on each frame. Eighty percent of the data were used to train three standard computer vision models (U-Net, U-Net++, and DenseNet) for automatic stone segmentation during flexible ureteroscopy. The remaining data (20%) were used to compare performance of the three models after optimization through Dice coefficients and binary cross entropy. We identified the highest performing model and evaluated automatic segmentation performance during ureteroscopy for both stone localization and treatment using a separate set of endoscopic videos. We evaluated performance of the pixel-based analysis using area under the receiver operating characteristic curve (AUC-ROC), accuracy, sensitivity, and positive predictive value both in previously recorded videos and in real time. Results: A computer vision model (U-Net++) was evaluated, trained, and optimized for kidney stone segmentation during ureteroscopy using 20 surgical videos (mean video duration of 22 seconds, standard deviation ±13 seconds). The model showed good performance for stone localization with both digital ureteroscopes (AUC-ROC: 0.98) and fiberoptic ureteroscopes (AUC-ROC: 0.93). Furthermore, the model was able to accurately segment stones and stone fragments <270 μm in diameter during laser fragmentation (AUC-ROC: 0.87) and dusting (AUC-ROC: 0.77). The model automatically annotated videos intraoperatively in three cases and could do so in real time at 30 frames per second (FPS). Conclusion: Computer vision models demonstrate strong performance for automatic stone segmentation during ureteroscopy. Automatically annotating new videos at 30 FPS demonstrate the feasibility of real-time application during surgery, which could facilitate tracking tools for stone treatment.

Remote monitoring system using slow-fast deep convolution neural network model for identifying anti-social activities in surveillance applications

Remote monitoring is the process that monitors and observes information from a distance utilizing sensors or electronic types of equipment. Remote monitoring is used in real-time applications like traffic, forest, military, shops, and hospitals to determine abnormal activities. Earlier research has done video processing methods based on computer vision techniques, but the computational complexity regarding time and memory is high. This paper designs and implements a novel Slow-Fast Convolution Neural Network (SF–CNN) to identify, detect, and classify abnormal behaviours from a surveillance video. The proposed CNN architecture learns the video frames automatically, obtains the most appropriate properties about various objects' behaviour from a large set of videos. The learning process of SF-CNN is carried out in two ways, such as slow learning and fast learning. The slow learning process is enabled when the frame rate is less, and the rapid learning process is enabled when the frame rate is high. Both the learning processes learn spatial and temporal information from the input video. Different objects, such as humans, vehicles, and animals, are detected and recognized according to their actions. All the videos have normal and abnormal activities that vary in various contexts. The proposed SF-CNN architecture provides an end-to-end solution to dealing with multiple constraints abnormal movements. The experiment is carried out on several benchmark datasets, and the performance of the SF-CNN architecture is evaluated. The proposed approach obtained 99.6% of accuracy, which is higher than the other existing techniques.

A Fast and Accurate Real-Time Vehicle Detection Method Using Deep Learning for Unconstrained Environments

Deep learning-based classification and detection algorithms have emerged as a powerful tool for vehicle detection in intelligent transportation systems. The limitations of the number of high-quality labeled training samples makes the single vehicle detection methods incapable of accomplishing acceptable accuracy in road vehicle detection. This paper presents detection and classification of vehicles on publicly available datasets by utilizing the YOLO-v5 architecture. This paper’s findings utilize the concept of transfer learning through fine tuning the weights of the pre-trained YOLO-v5 architecture. To employ the concept of transfer learning, extensive data sets of images and videos of the congested traffic patterns were collected by the authors. These datasets were made more comprehensive by pointing various attributes, for instance high- and low-density traffic patterns, occlusions, and different weather circumstances. All of these gathered datasets were manually annotated. Ultimately, the improved YOLO-v5 structure becomes accustomed to any difficult traffic patterns. By fine-tuning the pre-trained network through our datasets, our proposed YOLO-v5 has exceeded several other traditional vehicle detection methods in terms of detection accuracy and execution time. Detailed simulations performed on the PKU, COCO, and DAWN datasets demonstrate the effectiveness of the proposed method in various challenging situations.

Deep-Learning-Based Sequence Causal Long-Term Recurrent Convolutional Network for Data Fusion Using Video Data

The purpose of AI-Based schemes in intelligent systems is to advance and optimize system performance. Most intelligent systems adopt sequential data types derived from such systems. Realtime video data, for example, are continuously updated as a sequence to make necessary predictions for efficient system performance. The majority of deep-learning-based network architectures such as long short-term memory (LSTM), data fusion, two streams, and temporal convolutional network (TCN) for sequence data fusion are generally used to enhance robust system efficiency. In this paper, we propose a deep-learning-based neural network architecture for non-fix data that uses both a causal convolutional neural network (CNN) and a long-term recurrent convolutional network (LRCN). Causal CNNs and LRCNs use incorporated convolutional layers for feature extraction, so both architectures are capable of processing sequential data such as time series or video data that can be used in a variety of applications. Both architectures also have extracted features from the input sequence data to reduce the dimensionality of the data and capture the important information, and learn hierarchical representations for effective sequence processing tasks. We have also adopted a concept of series compact convolutional recurrent neural network (SCCRNN), which is a type of neural network architecture designed for processing sequential data combined by both convolutional and recurrent layers compactly, reducing the number of parameters and memory usage to maintain high accuracy. The architecture is challenge-able and suitable for continuously incoming sequence video data, and doing so allowed us to bring advantages to both LSTM-based networks and CNNbased networks. To verify this method, we evaluated it through a sequence learning model with network parameters and memory that are required in real environments based on the UCF-101 dataset, which is an action recognition data set of realistic action videos, collected from YouTube with 101 action categories. The results show that the proposed model in a sequence causal long-term recurrent convolutional network (SCLRCN) provides a performance improvement of at least 12% approximately or more to be compared with the existing models (LRCN and TCN).