Video Sequences Research Articles

Cleft Lip and Palate Classification Through Vision Transformers and Siamese Neural Networks

This study introduces a novel approach for the diagnosis of Cleft Lip and/or Palate (CL/P) by integrating Vision Transformers (ViTs) and Siamese Neural Networks. Our study is the first to employ this integration specifically for CL/P classification, leveraging the strengths of both models to handle complex, multimodal data and few-shot learning scenarios. Unlike previous studies that rely on single-modality data or traditional machine learning models, we uniquely fuse anatomical data from ultrasound images with functional data from speech spectrograms. This multimodal approach captures both structural and acoustic features critical for accurate CL/P classification. Employing Siamese Neural Networks enables effective learning from a small number of labeled examples, enhancing the model’s generalization capabilities in medical imaging contexts where data scarcity is a significant challenge. The models were tested on the UltraSuite CLEFT dataset, which includes ultrasound video sequences and synchronized speech data, across three cleft types: Bilateral, Unilateral, and Palate-only clefts. The two-stage model demonstrated superior performance in classification accuracy (82.76%), F1-score (80.00–86.00%), precision, and recall, particularly distinguishing Bilateral and Unilateral Cleft Lip and Palate with high efficacy. This research underscores the significant potential of advanced AI techniques in medical diagnostics, offering valuable insights into their application for improving clinical outcomes in patients with CL/P.

TempDiff: Enhancing Temporal‐awareness in Latent Diffusion for Real‐World Video Super‐Resolution

AbstractLatent diffusion models (LDMs) have demonstrated remarkable success in generative modeling. It is promising to leverage the potential of diffusion priors to enhance performance in image and video tasks. However, applying LDMs to video super‐resolution (VSR) presents significant challenges due to the high demands for realistic details and temporal consistency in generated videos, exacerbated by the inherent stochasticity in the diffusion process. In this work, we propose a novel diffusion‐based framework, Temporal‐awareness Latent Diffusion Model (TempDiff), specifically designed for real‐world video super‐resolution, where degradations are diverse and complex. TempDiff harnesses the powerful generative prior of a pre‐trained diffusion model and enhances temporal awareness through the following mechanisms: 1) Incorporating temporal layers into the denoising U‐Net and VAE‐Decoder, and fine‐tuning these added modules to maintain temporal coherency; 2) Estimating optical flow guidance using a pre‐trained flow net for latent optimization and propagation across video sequences, ensuring overall stability in the generated high‐quality video. Extensive experiments demonstrate that TempDiff achieves compelling results, outperforming state‐of‐the‐art methods on both synthetic and real‐world VSR benchmark datasets. Code will be available at https://github.com/jiangqin567/TempDiff

Content-Adaptive Bitrate Ladder Estimation in High-Efficiency Video Coding Utilizing Spatiotemporal Resolutions

The constant increase in multimedia Internet traffic in the form of video streaming requires new solutions for efficient video coding to save bandwidth and network resources. HTTP adaptive streaming (HAS), the most widely used solution for video streaming, allows the client to adaptively select the bitrate according to the transmission conditions. For this purpose, multiple presentations of the same video content are generated on the video server, which contains video sequences encoded at different bitrates with resolution adjustment to achieve the best Quality of Experience (QoE). This set of bitrate–resolution pairs is called a bitrate ladder. In addition to the traditional one-size-fits-all scheme for the bitrate ladder, context-aware solutions have recently been proposed that enable optimum bitrate–resolution pairs for video sequences of different complexity. However, these solutions use only spatial resolution for optimization, while the selection of the optimal combination of spatial and temporal resolution for a given bitrate has not been sufficiently investigated. This paper proposes bit-ladder optimization considering spatiotemporal features of video sequences and usage of optimal spatial and temporal resolution related to video content complexity. Optimization along two dimensions of resolution significantly increases the complexity of the problem and the approach of intensive encoding for all spatial and temporal resolutions in a wide range of bitrates, for each video sequence, is not feasible in real time. In order to reduce the level of complexity, we propose a data augmentation using a neural network (NN)-based model. To train the NN model, we used seven video sequences of different content complexity, encoded with the HEVC encoder at five different spatial resolutions (SR) up to 4K. Also, all video sequences were encoded using four frame rates up to 120 fps, presenting different temporal resolutions (TR). The Structural Similarity Index Measure (SSIM) is used as an objective video quality metric. After data augmentation, we propose NN models that estimate optimal TR and bitrate values as switching points to a higher SR. These results can be further used as input parameters for the bitrate ladder construction for video sequences of a certain complexity.

Toward comprehensive short utterances manipulations detection in videos

AbstractIn a landscape increasingly populated by convincing yet deceptive multimedia content generated through generative adversarial networks, there exists a significant challenge for both human interpretation and machine learning algorithms. This study introduces a shallow learning technique specifically tailored for analyzing visual and auditory components in videos, targeting the lower face region. Our method is optimized for ultra-short video segments (200-600 ms) and employs wavelet scattering transforms for audio and discrete cosine transforms for video. Unlike many approaches, our method excels at these short durations and scales efficiently to longer segments. Experimental results demonstrate high accuracy, achieving 96.83% for 600 ms audio segments and 99.87% for whole video sequences on the FakeAVCeleb and DeepfakeTIMIT datasets. This approach is computationally efficient, making it suitable for real-world applications with constrained resources. The paper also explores the unique challenges of detecting deepfakes in ultra-short sequences and proposes a targeted evaluation strategy for these conditions.

An efficient convolution neural network method for copy-move video forgery detection

Unmanned systems play a pivotal role in military surveillance, critical infrastructure protection, law enforcement, search and rescue operations, and border security, showcasing their multifaceted importance across diverse applications. Video fraud detection is integral to multimedia security, where our task involves the precise identification of modified segments within video sequences. Current approaches to video fraud detection often rely on manual feature selection and models tailored to detect specific tampering types, such as copy-move or splicing. The general representation powers of deep learning models and the connection of multiple forensic characteristics are still not fully explored. This research uses a convolutional neural network (CNN) model to identify copy-move video forgeries. Copy-move forgery is a type of video tampering whereby a portion of the video is copied and pasted somewhere different in the same video to cover an essential video characteristic. The method that is being proposed involves dividing the video into individual frames, extracting features from each frame by using a CNN model that has already been trained, and then utilizing these features to train a new CNN model that would classify each frame as either legitimate or fabricated. The proposed method effectively detects copy-move video forgery with an exceptionally high accuracy rate, exceeding current methods of accuracy and computational effort. The proposed method outperformed all other approaches on the SULFA, GRIP, and VTD datasets. The model's accuracy was 85.42 %, 86.16 %, and 81.87 %, respectively, with the shortest times recorded being 9.6 sec, 11.4 sec, and 13.7 sec, respectively. Consequently, specialists can employ the suggested method as a machine-learning instrument for detecting fake videos in real-time.

Pose-oriented scene-adaptive matching for abnormal event detection

For intelligent surveillance systems, abnormal event detection automatically analyses surveillance video sequences and detects abnormal objects or unusual human actions at the frame level. Due to the lack of labelled data, most approaches are semi-supervised based on reconstruction or prediction methods. However, these methods may not generalize well to unseen scene contexts. To address this issue, we present a novel self and mutual scene-adaptive matching method for abnormal event detection. In the framework, we propose synergistic pose estimation and object detection, which effectively integrates human pose and object detection information to improve pose estimation accuracy. Then, the poses are resized to reduce the spatial distance between the source and target domains. The improved pose sequences are further fed into a spatio-temporal graph convolutional network to extract the geometric features. Finally, the features are embedded in a clustering layer to classify action types and compute normality scores. The training data is taken from the training part of common video anomaly detection datasets: UCSD PED1 & PED2, CHUK Avenue, and ShanghaiTech Campus. The proposed framework is evaluated on video sequences with unseen scene contexts in the UCSD PED2 and ShanghaiTech Campus datasets. The detection accuracy and efficiency are also evaluated in detail, and the proposed method for abnormal event detection achieves the highest AUC performance, 84.6%, on the ShanghaiTech Campus dataset and relatively high AUC performance, 96.9% and 74.8%, on UCSD PED2 & PED1 datasets. Compared with other state-of-the-art works, the performance analysis and results confirm the robustness and effectiveness of our proposed framework for cross-scene abnormal event detection.

Visual Communication Method of Multi frame Film and Television Special Effects Images Based on Deep Learning

For the dynamic film and television special effects industry, creating visually stunning and valuable graphics requires the integration of excellent deep-learning algorithms. This paper presents an enhanced version of the 3-D Convolutional Neural Network (3-D CNN), specifically tailored to meet the demanding needs of multi-frame television and film special effects. The version’s key feature is its efficient data handling through coupled precision training, significantly increasing computational efficiency while reducing memory needs. This method, which combines floating-issue operations of 16 and 32 bits, is ideal for efficiently processing large amounts of high-resolution video data. The proposed 3-D CNN structure excels in extracting and analysing complex spatiotemporal capabilities from video sequences, capturing the spatial and temporal nuances crucial in film imagery. This capability is important for accomplishing excessive fidelity in visual results, ensuring seamless integration with live motion pics. Incorporating a cutting-edge attention mechanism inspired by the aid of transformer-based architectures, the model specialises inside the maximum pertinent components of video frames to enhance the quality and realism of outcomes. Furthermore, the model boasts an excessive-resolution processing characteristic, permitting simultaneous capture of first-rate records and broader scene context. This guarantees consistency and realism in outcomes, from complicated textures to the overarching visual narrative. Advanced regularisation strategies are hired to prevent overfitting, permitting the model to generalise efficiently through numerous film manufacturing conditions. The state-of-the-art 3-D information augmentation techniques make the model robust and prepare it to deal with an intensive variety of challenging situations involving computer special effects. Real-time processing competencies make the model a game-changer for on-set visible outcomes and adjustments. Designed for seamless integration with the famous CGI software program software utility, it allows a harmonious aggregate of AI and ingenious creativity. Acknowledging the environmental impact of high-powered computing, the model consists of power-efficient computational techniques that align with sustainable computing practices, lower operational costs, and are related to processing complex video statistics. model boasts an excessive-resolution processing characteristic, permitting simultaneous capture of first-rate records and broader scene context. This guarantees consistency and realism in outcomes, from complicated textures to the overarching visual narrative. Advanced regularisation strategies are hired to prevent overfitting, permitting the model to generalize efficiently through numerous film manufacturing conditions. The state-of-the-art 3-D information augmentation techniques, in addition, make the model robust, and prepare it to deal with an intensive variety of computer special effects challenging situations.

AI-smartphone markerless motion capturing of hip, knee, and ankle joint kinematics during countermovement jumps.

Recently, AI-driven skeleton reconstruction tools that use multistage computer vision pipelines were designed to estimate 3D kinematics from 2D video sequences. In the present study, we validated a novel markerless, smartphone video-based artificial intelligence (AI) motion capture system for hip, knee, and ankle angles during countermovement jumps (CMJs). Eleven participants performed six CMJs. We used 2D videos created by a smartphone (Apple iPhone X, 4K, 60 fps) to create 24 different keypoints, which together built a full skeleton including joints and their connections. Body parts and skeletal keypoints were localized by calculating confidence maps using a multilevel convolutional neural network that integrated both spatial and temporal features. We calculated hip, knee, and ankle angles in the sagittal plane and compared it with the angles measured by a VICON system. We calculated the correlation between both method's angular progressions, mean squared error (MSE), mean average error (MAE), and the maximum and minimum angular error and run statistical parametric mapping (SPM) analysis. Pearson correlation coefficients (r) for hip, knee, and ankle angular progressions in the sagittal plane during the entire movement were 0.96, 0.99, and 0.87, respectively. SPM group-analysis revealed some significant differences only for ankle angular progression. MSE was below 5.7°, MAE was below 4.5°, and error for maximum amplitudes was below 3.2°. The smartphone AI motion capture system with the trained multistage computer vision pipeline was able to detect, especially hip and knee angles in the sagittal plane during CMJs with high precision from a frontal view only.

Improved RNN Model for Real-Time Human Activity Recognition

This study focuses on the automatic detection of human actions in video streams. The requirement to detect what human activities happen in videos is recognition of human action due to significant differences in people's visual and motion appearance and actions, camera perspective shifts, moving background, occlusions, noise, and a massive amount of video data. The human activity recognition challenge involves identifying physical activities carried out by individuals or groups based on traces of movements, including gestures, actions, interactions, and group activities. The detection of concepts usually requires additional annotations for the training dataset. In this paper, useful methods for categorizing human action recognition are discussed. The current models are an accurate deep learning method that is based on models that have been changed to be more useful. The large disparities that result from the backdrop and the size of the objects have prevented the identification of activities in videos from being fully and effectively addressed. The main objective is to achieve better accuracy for the Long Short-Term Memory (LSTM) method, which was used to improve the Recurrent Neural Networks (RNN) model. In this paper, LSTM is used to come up with models for different action recognition tasks. The model was made better by making the LSTM have four layers and putting 128 units, 64 units, 32 units, and 16 units in each layer, respectively. In addition, the performance evaluation of deep learning-based approaches has been compared to other related works. Therefore, an improved approach to RNN is proposed to recognize human actions. To classify the videos, a multilayer RNN with a specific type of LSTM is used to extract features from video sequences. The UCF-101 and UCF Sports human action recognition datasets are utilized in this study for both training and assessment. Test findings demonstrate that the suggested strategy achieved increased accuracy. Finally, the enhanced RNN model's total model accuracy in the UCF-101 dataset is 93.78% and 95.70% for the UCF Sport dataset.

Recognition of Real-Time Video Activities Using Stacked Bi-GRU with Fusion-based Deep Architecture

Recognizing and understanding human activities in real-time videos is a challenging task due to the complex nature of video data and the need for efficient and accurate analysis. This research pioneers a breakthrough in video activity recognition by introducing a robust framework leveraging the power of a stacked Bidirectional Long Short-Term Memory (Bi-LSTM) and Gated Recurrent Unit (GRU) architecture, harmonized within a fusion-based deep model. The stacked Bi-LSTM-GRU model capitalizes on its dual recurrent architecture, capturing nuanced temporal dependencies within video sequences. The fusion-based deep architecture synergizes spatial and temporal features, enabling the model to discern intricate patterns in human activities. To further enhance the discriminative power of the model, we introduce a fusion module in the proposed deep architecture. The fusion module integrates multi-modal features extracted from different levels of the network hierarchy, allowing for a more comprehensive representation of video activities. We demonstrate the efficacy of our approach through rigorous experimentation on UCF50, UCF101, and HMDB51 datasets. In experiments on the UCF50 dataset, our model achieves an accuracy of 97.01% and 95.86% on training and validation sets respectively, showcasing its proficiency in discerning activities across a diverse range of scenarios. The evaluation extends to the UCF101 dataset, where the proposed approach achieves a competitive accuracy of 97.62% and 96.93% on training and validation sets, surpassing previous benchmarks by a margin of approx 1%. Further-more, on the challenging HMDB51 dataset, the model demonstrates a robust accuracy of 89.71%and 88.88% on training and validation sets, solidifying its efficacy in intricate action recognition tasks.

Quantification of Video Sequences of the Microcirculation: A Comparison Between Automated Analysis Using Analysis Manager and Manual Analysis Using Capillary Mapper.

Microcirculatory disturbances can contribute to organ dysfunction in patients undergoing major surgeries and critical illness. Incident dark field imaging (CytoCam, Braedius Medical BV, Huizen, Netherlands) provides direct visualization of the microcirculation. To utilize this method in daily clinical practice, automated image analysis is essential. This study aims to compare the automated analysis of recorded microcirculation video sequences using CytoCamTools V2 Analysis Manager (Braedius Medical BV) with established manual analysis using Capillary Mapper (Version 1.4.5, University Hospital Münster, Germany) as reference method. Sublingual microcirculation video sequences were recorded in patients undergoing laparotomy at four time points (before surgery, 2 and 6 h after surgery, and on the first postoperative day) using incident dark field imaging. Agreement between automated and manual analysis of total vessel density (TVD), perfused vessel density (PVD), and proportion of perfused vessels (PPV) was compared using intraclass correlation (ICC) and Bland-Altman method. A total of 336 videos from 30 patients were analyzed. The ICC between the two measurement methods was 0.13 for TVD, 0.14 for PVD, and 0.16 for PPV. Bland-Altman analysis showed mean differences (95% limits of agreement) of 10.46 mm/mm2 (-1.73-22.65 mm/mm2) for TVD, 8.25 mm/mm2 (-9.88-26.39 mm/mm2) for PVD, and - 3.96% (-59.58%-51.65%) for PPV. Automated microcirculatory analysis using the Analysis Manager did not show clinically acceptable agreement with manual analysis using Capillary Mapper. Consequently, automated video analysis using the Analysis Manager does not appear to be a suitable approach. ClinicalTrials.gov identifier: DRKS00020264.

On the Security of Selectively Encrypted HEVC Video Bitstreams

With the growing applications of video, ensuring its security has become of utmost importance. Selective encryption (SE) has gained significant attention in the field of video content protection due to its compatibility with video codecs, favorable visual distortion, and low time complexity. However, few studies consider SE security under cryptographic attacks. To fill this gap, we analyze the security concerns of encrypted bitstreams by SE schemes and propose two known plaintext attacks (KPAs). Then the corresponding defense is presented against the KPAs. To validate the effectiveness of the KPA, it is applied to attack two existing SE schemes with superior visual degradation in HEVC videos. Firstly, the video sequences are encoded using H.265/HEVC. During encoding, the selected syntax elements are recorded. Secondly, the recorded syntax elements are imported into the HEVC decoder. By utilizing the encryption parameters and the imported data in the decoder, it becomes possible to reconstruct a significant portion of the original syntax elements before encryption. Finally, the reconstructed syntax elements are compared with the encrypted syntax elements in the decoder, allowing the design of a pseudo-key stream (PKS) through the inverse of the encryption operations. The PKS is used to decrypt the existing SE scheme, and the experimental results provide evidence that the two existing SE schemes are vulnerable to the proposed KPAs. In the case of single bitstream estimation (SBE), the average correct rate of key stream estimation exceeds 93%. Moreover, with multi-bitstream complementation (MBC), the average estimation accuracy can be further improved to 99%.

Focus on reflective competence: A triangulation to examine youth soccer coaches' reflective competence

Introduction & Purpose Coaching is recognized as a complex socio-educational interaction (Jones, 2007), a view supported by the German Football Association (DFB). The DFB emphasizes reflection in the development of football coaches, seeing it as essential for the quality of coaching and the holistic development of youth players. Reflective competence (RC) is seen as a key component of professionalization (DFB, 2022), defined as the ability and willingness to comprehensively deal with one’s own knowledge and actions in breadth and to achieve a certain depth in the reflexive and transformative engagement with issues (Krieg & Kreis, 2014). Studies show that coaches learn primarily through observation and experience (Cushion, et al., 2012), which risks perpetuating outdated behaviors. Against this background, our study asks: What (breadth) and how (depth) do youth coaches reflect on their knowledge and coaching practice? Method Seven coaches from DFB selection centers in Bavaria participated in the study. Their training sessions were videotaped and used for reflection in stimulated recall interviews (Dempsey, 2010). Coaches were asked about their understanding of reflection using narrative prompts and to reflect independently on their training sessions using the video sequences. Analysis included categorizing the understanding and relevance of reflection, as well as the breadth and depth of reflection, taking into account individual differences. Result Coaches understood reflection as “thinking about oneself” or “looking back” to consciously justify or improve practices. They showed a high level of willingness to reflect, especially valuing peer feedback. In terms of breadth, coaches mostly reflected on corrective feedback behaviors, error patterns, or training alternatives. However, the depth of reflection was limited; coaches often moved to transformative reflection without deeply analyzing issues, lacking detailed descriptions against related assumptions, previous experiences, or theoretical explanations. Discussion Coaches focused primarily on optimizing practices rather than examining issues in depth. The breadth of reflection is consistent with the DFB's shift from coach-dominated to player-dominated training practices. Conclusion Our study highlights the critical role of reflection in the professional development of youth soccer coaches at DFB selection centers in Bavaria. While coaches show a significant willingness to engage in reflective practices, there is a notable deficiency in the depth of their reflection. This highlights the need for enhanced reflective training programs that encourage deeper, more comprehensive engagement with coaching practices and theoretical knowledge. By fostering such reflective competence, coaches can contribute to a more nuanced and effective coaching methodology, ultimately benefiting the holistic development of youth players.

Thermal video-based hand gestures recognition using lightweight CNN

AbstractHand gesture recognition has gained a lot of attention in computer vision due to multiple applications. Further, most of the existing works utilized RGB data for hand gesture recognition. However, RGB cameras mainly depend on lighting, angles, and other factors including skin color which impacts the accuracy. Thus, we propose a methodology for video hand gesture recognition using thermal data in this work. Initially, we created a dataset of short video sequences captured from a thermal camera. Thereafter, a lightweight convolutional neural network model (CNN) is proposed for hand gesture recognition. Further, the performance of the proposed CNN model is evaluated on different sizes of the dataset consisting of 15, 10, and 5 frames per sequence. Results show that the proposed model achieves an accuracy of $$97\% \pm (0.05)$$, $$96\% \pm (0.05)$$, and $$87\% \pm (0.1)$$ on the dataset consisting of 15, 10, and 5 frames per sequence, respectively.

Annealing Temporal–Spatial Contrastive Learning for multi-view Online Action Detection

In open-world scenarios, the analysis of action events from multiple viewpoints is crucial for achieving a holistic understanding, a concept that resonates with human perception. However, achieving a synthesis of information from multiple viewpoints presents a challenge, as the data can induce inter-view regularization, complicating the learning process. This paper is the first to delve into Online Action Detection (OAD) through a multi-view lens, underscoring the value of cross-observation in enriching view-level information. By harnessing the spatiotemporal dynamics inherent in multi-view video sequences, an Annealing Temporal–Spatial Contrastive Learning (ATSCL) consisting of Annealing Temporal Contrastive Learning (ATCL) and Spatial Contrastive Learning (SCL) is proposed, optimized for compatibility with RNN-based models. ATCL employs an annealing temporal loss to uncover the intrinsic video structures via a temporal annealing sampling mechanism. Concurrently, SCL utilizes a spatial loss to draw representations from various viewpoints closer together, mitigating the regularization effects. The ATSCL liberates training multi-view OAD from the stringent requirements of synchronized training videos, enabling the execution of OAD tasks asynchronously. Experiments demonstrate that the RNN-based models realize an average improvement of 5.92% on the DAHLIA dataset, 3.36% on the IKEA ASM dataset, 2.92% on the BREAKFAST dataset and 0.6% (mAP) on the THUMOS’14 dataset following the integration of the ATSCL framework, underscoring the ATSCL’s efficacy across different RNN structures.

To evaluate the impact of watching a video sequence in a virtual reality helmet and on a TV screen on a person’s postural stability

The paper presents an analysis of changes in postural stability when a person is presented with a video sequence in a virtual reality helmet and from a TV screen. Postural stability was assessed using a computer stabilometer complex. Changes in the stabilometric indicators compared with control tests (before viewing) were shown for both cases (watching videos on the screen and in a virtual reality helmet). Besides, viewing a video sequence in a virtual reality helmet had a greater impact on the instability. While watching a video from a TV screen and in a virtual reality helmet, the contribution of visual information to maintaining balance in the sagittal plane decreased. However, while watching from the TV screen, the contribution of vestibular information for posture control increased. When viewed with virtual reality helmet, the contribution of somatosensory information and the cerebellum increased. The results may suggest that virtual reality requires more conscious corrective mechanisms to stabilize posture.

A survey of video-based human action recognition in team sports

Over the past few decades, numerous studies have focused on identifying and recognizing human actions using machine learning and computer vision techniques. Video-based human action recognition (HAR) aims to detect actions from video sequences automatically. This can cover simple gestures to complex actions involving multiple people interacting with objects. Actions in team sports exhibit a different nature compared to other sports, since they tend to occur at a faster pace and involve more human-human interactions. As a result, research has typically not focused on the challenges of HAR in team sports. This paper comprehensively summarises HAR-related research and applications with specific focus on team sports such as football (soccer), basketball and Australian rules football. Key datasets used for HAR-related team sports research are explored. Finally, common challenges and future work are discussed, and possible research directions identified.

SMART CROWD MONITORING SYSTEM USING IOT-BASED YOLO-GHOST

Internet of Things (IoT) devices offer a smart, sophisticated, real-time surveillance solution for public spaces. However, due to instantaneous lighting changes and varying viewing angles, counting and tracking the people in crowded scenes is a challenging problem. To combat these issues, a novel YOLO-CROWD is proposed for a smart crowd-monitoring system using YOLO-GHOST. Initially, an Internet Protocol (IP) based camera was used to monitor and capture crowds in a video sequence. The captured video sequences are converted into frames and passed to the server via the Internet. The recorded frames are given to the YOLO-GHOST classification module to perform people detection and counting. Finally, the detected output results are transferred to the surveillance monitoring center's server. The YOLO-CROWD technique is simulated by using MATLAB. The effectiveness of the proposed YOLO-CROWD technique is assessed using evaluation metrics such as accuracy, precision, recall, sensitivity, F1-score, and mean average precision. The experimental results show that the accuracy of the YOLO-CROWD has increased to up to 99.95 %, proving that its intended use is for accurate crowd detection. The detection accuracy of the proposed method is 84.9 %, 87.58 %, 93.91 %, and 97.72 % better than existing EABeD, LCDnet, CDEM-M, and Public Vision, respectively.

Automated MRI Video Analysis for Pediatric Neuro-Oncology: An Experimental Approach

Over the past year, there has been a significant rise in interest in the application of open-source artificial intelligence models (OSAIM) in the field of medicine. An increasing number of studies focus on evaluating the capabilities of these models in image analysis, including magnetic resonance imaging (MRI). This study aimed to investigate whether two of the most popular open-source AI models, namely ChatGPT 4o and Gemini Pro, can analyze MRI video sequences with single-phase contrast in sagittal and frontal projections, depicting a posterior fossa tumor corresponding to a medulloblastoma in a child. The study utilized video files from single-phase contrast-enhanced head MRI in two planes (frontal and sagittal) of a child diagnosed with a posterior fossa tumor, type medulloblastoma, confirmed by histopathological examination. Each model was separately provided with the video file, first in the sagittal plane, analyzing three different sets of commands from the most general to the most specific. The same procedure was applied to the video file in the frontal plane. The Gemini Pro model did not conduct a detailed analysis of the pathological change but correctly identified the content of the video file, indicating it was a brain MRI, and suggested that a specialist in the field should perform the evaluation. Conversely, ChatGPT 4o conducted image analysis but failed to recognize that the content was MRI. The attempts to detect the lesion were random and varied depending on the plane. These models could not accurately identify the video content or indicate the area of the neoplastic change, even after applying detailed queries. The results suggest that despite their widespread use in various fields, these models require further improvements and specialized training to effectively support medical diagnostics.

Video Anomaly Detection via self-supervised and spatio-temporal proxy tasks learning

Video Anomaly Detection (VAD) aims to identify events in videos that deviate from typical patterns. Given the scarcity of anomalous samples, previous research has primarily focused on learning regular patterns from datasets exclusively containing normal behaviors, and treating deviations from these patterns as anomalies. However, most of these methods are constrained by coarse-grained modeling approaches that renders them incapable of learning highly-discriminative features, which are necessary to effectively distinguish between the subtle differences between normal and abnormal behaviors. To better capture these features, we propose an innovative method. Initially, pseudo-anomalous samples for appearance and motion are generated through geometric transformations (2D rotations) and the scrambling of video sequences. Subsequently, a dual-branch network featuring spatio-temporal decoupling is proposed, in which the spatial and temporal branches each handle a specific proxy task. These tasks are designed to distinguish between normal and pseudo-anomalous samples, involving operations such as predicting patch-based 2D rotation angles and classifying video frame triplets as total-anomaly, left-anomaly, right-anomaly, and non-anomaly. Our approach employs an end-to-end training methodology, without relying on pre-trained models (except for the object detector). Evaluations on the UCSD Ped2, Avenue, and ShanghaiTech datasets show that our method achieved AUC scores of 99.1%, 91.9%, and 81.1%, respectively, demonstrating its effectiveness. The code is publicly accessible at the following link: https://spatio-temporal-tasks.