Frame Detection Research Articles

Noise-Perception Multi-Frame Collaborative Network for Enhanced Polyp Detection in Endoscopic Videos

The accurate detection and localization of polyps during endoscopic examinations are critical for early disease diagnosis and cancer prevention. However, the presence of artifacts and noise, along with the high similarity between polyps and surrounding tissues in color, shape, and texture complicates polyp detection in video frames. To tackle these challenges, we deployed multivariate regression analysis to refine the model and introduced a Noise-Suppressing Perception Network (NSPNet) designed for enhanced performance. NSPNet leverages wavelet transform to enhance the model’s resistance to noise and artifacts while improving a multi-frame collaborative detection strategy for dynamic polyp detection in endoscopic videos, efficiently utilizing temporal information to strengthen features across frames. Specifically, we designed a High-Low Frequency Feature Fusion (HFLF) framework, which allows the model to capture high-frequency details more effectively. Additionally, we introduced an improved STFT-LSTM Polyp Detection (SLPD) module that utilizes temporal information from video sequences to enhance feature fusion in dynamic environments. Lastly, we integrated an Image Augmentation Polyp Detection (IAPD) module to improve performance on unseen data through preprocessing enhancement strategies. Extensive experiments demonstrate that NSPNet outperforms nine SOTA methods across four datasets on key performance metrics, including F1Score and recall.

Cost vs. Performance: Raspberry Pi and ESP32 in Face Detection Tasks

A cost-effective, scalable solution that leverages low-cost embedded systems for computationally intensive tasks like face detection is critical for the growing demands of IoT applications, motivating the development of innovative hybrid architectures. This paper proposes such a framework, combining the ESP32 CAM as a distributed image capture unit with light preprocessing and the Raspberry Pi as a centralized processing unit. By addressing the limitations of standalone implementations—low computational capability of the ESP32 CAM and high deployment costs of the Raspberry Pi—our framework achieves a balance between affordability and performance. We present the analysis of the devices, evaluating metrics such as frame rate, detection accuracy, and cost-efficiency. The results highlight the potential of the hybrid system to significantly lower costs while enabling scalable, real-time face detection in IoT scenarios. This study contributes to the ongoing research by proposing an adaptable, resource-optimized framework suitable for diverse use cases, from smart surveillance to retail monitoring, paving the way for more efficient IoT-based vision systems.

Damage detection of an asymmetrical frame via Sparse Bayesian Learning with a PSO algorithm

Damage detection of an asymmetrical frame via Sparse Bayesian Learning with a PSO algorithm

Position and heading estimation for indoor navigation of a micro aerial vehicle using vanishing point

Abstract Indoor navigation for micro aerial vehicles (MAVs) is challenging in GPS signal-obstructed indoor corridor environments. Position and heading estimation for a MAV is required to navigate without colliding with obstacles. The connected components algorithm and k-means clustering algorithm have been integrated for line and vanishing point detection in the corridor image frames to estimate the position and heading of the MAV. The position of the vanishing point indicates the position of the MAV (centre, left or right) in the corridor. Furthermore, the Euclidean distance between the image centre and mid-pixel coordinates at the last row of the image and the detected vanishing point pixel coordinates in the successive corridor image frames are used to compute the heading of the MAV. When the MAV deviates from the corridor centre, the position and heading measurement can send a suitable control signal to the MAV and align the MAV at the centre of the corridor. When compared with a grid-based vanishing point detection method heading accuracy of ±1⋅5°, the k-means clustering-based vanishing point detection is suitable for real-time heading measurement for indoor MAVs with an accuracy of ±0⋅5°.

Attention-Based Lightweight YOLOv8 Underwater Target Recognition Algorithm.

Underwater object detection is highly complex and requires a high speed and accuracy. In this paper, an underwater target detection model based on YOLOv8 (SPSM-YOLOv8) is proposed. It solves the problems of high computational complexities, slow detection speeds and low accuracies. Firstly, the SPDConv module is utilized in the backbone network to replace the standard convolutional module for feature extraction. This enhances computational efficiency and reduces redundant computations. Secondly, the PSA (Polarized Self-Attention) mechanism is added to filter and enhance the polarization of features in the channel and spatial dimensions to improve the accuracy of pixel-level prediction. The SCDown (spatial-channel decoupled downsampling) downsampling mechanism is then introduced to reduce the computational cost by decoupling the space and channel operations while retaining the information in the downsampling process. Finally, MPDIoU (Minimum Point Distance-based IoU) is used to replace the CIoU (Complete-IOU) loss function to accelerate the convergence speed of the bounding box and improve the bounding box regression accuracy. The experimental results show that compared with the YOLOv8n baseline model, the SPSM-YOLOv8 (SPDConv-PSA-SCDown-MPDIoU-YOLOv8) detection accuracy reaches 87.3% on the ROUD dataset and 76.4% on the UPRC2020 dataset, and the number of parameters and amount of computation decrease by 4.3% and 4.9%, respectively. The detection frame rate reaches 189 frames per second on the ROUD dataset, thus meeting the high accuracy requirements for underwater object detection algorithms and facilitating lightweight and fast edge deployment.

Near-Infrared Light and OpenCV as Components for Low-Cost Airborne Microplastic Detection Machine

Abstract. Microplastic, resulting from the breakdown of larger plastic objects, poses substantial threats to ecosystems and human health. This research addresses the escalating environmental concern of airborne microplastics by developing a low-cost detection machine employing infrared sensors. The near-infrared light is attached to the machine for detection using the sensors. Different types of microplastic, specifically less than 5mm, are used. There are 3 types of plastics used for detection, 30 pieces each of polyethylene, polystyrene, and polyester microplastics. Each type of microplastic is tested for detection in different time frames, 1, 2, 3, 4, and 5 minutes each, and the three types of plastics are combined randomly for the last detection with the same number of minutes. Because of the final results, it can determine the effectiveness of the sensor in detecting airborne microplastics in terms of the number of correctly detected microplastics. In conclusion, this research advances our capacity to detect airborne microplastics, contributing to environmental conservation and public health protection. The detection machine's development and the insights gained from the study have broader implications for standardizing methodologies, advancing technology, and fostering collaborative efforts in addressing the challenges posed by airborne microplastics.

A Two-Stream Method for Human Action Recognition Using Facial Action Cues.

Human action recognition (HAR) is a critical area in computer vision with wide-ranging applications, including video surveillance, healthcare monitoring, and abnormal behavior detection. Current HAR methods predominantly rely on full-body data, which can limit their effectiveness in real-world scenarios where occlusion is common. In such situations, the face often remains visible, providing valuable cues for action recognition. This paper introduces Face in Action (FIA), a novel two-stream method that leverages facial action cues for robust action recognition under conditions of significant occlusion. FIA consists of an RGB stream and a landmark stream. The RGB stream processes facial image sequences using a fine-spatio-multitemporal (FSM) 3D convolution module, which employs smaller spatial receptive fields to capture detailed local facial movements and larger temporal receptive fields to model broader temporal dynamics. The landmark stream processes facial landmark sequences using a normalized temporal attention (NTA) module within an NTA-GCN block, enhancing the detection of key facial frames and improving overall recognition accuracy. We validate the effectiveness of FIA using the NTU RGB+D and NTU RGB+D 120 datasets, focusing on action categories related to medical conditions. Our experiments demonstrate that FIA significantly outperforms existing methods in scenarios with extensive occlusion, highlighting its potential for practical applications in surveillance and healthcare settings.

Deep Learning-Enabled Optimization of Polyp Frame Detection and Segmentation

Polyp segmentation is a method for identifying and characterizing polyps, or abnormal tissue development, as seen in frame images from medical procedures such as endoscopic or colonoscopy frame images. Polyp segments are reliant on high-quality imaging data, which can be intermittently accessible, causing lower detection and precision in segmentation. To overcome these challenges, we propose a novel deep learning (DL) method called intelligent chameleon swarm optimized-adaptive fully convolutional neural network (ICSO-AFConvNet) for polyp segmentation and frame detection. Initially, we gather the dataset named (KVasir-SEG), which contains a thousand frame images, and then contrast limited adaptive histogram equalization (CLAHE) technique, is used to preprocess the data. Experimental findings on the KVasir-SEG dataset reveal that ICSO-AFConvNet outperforms previous approaches in measures such as Dice, Interception over union (IoU), Recall and Precision. These findings show that the suggested ICSO-AFConvNET technique has tough generalization power and learning ability, making it an appealing alternative for practical applications with significant data variances.

Improved YOLOv8 for Gas-Flame State Recognition under Low-Pressure Conditions

This paper introduces a lightweight flame detection algorithm, enhancing the accuracy and speed of gas-flame state recognition in low-pressure environments using an improved YOLOv8n model. This method effectively resolves the aforementioned problems. Firstly, GhostNet is integrated into the backbone to form the GhostConv module, reducing the model’s computational parameters. Secondly, the C2f module is improved by integrating RepGhost, forming the C2f_RepGhost module, which performs deep convolution, extends feature dimensions, and simplifies the inference structure. Additionally, the CBAM attention mechanism is added to enhance the model’s ability to capture fine-grained features of flames in both channel and spatial dimensions. The replacement of CIoU with WIoU improves the sensitivity and accuracy of the model’s regression loss. Experimental results on a simulated dataset of the theoretical testbed indicate that compared to the original model, the proposed improvements achieve good performance in low-pressure flame state detection. The model’s parameter count is reduced by 12.64%, the total floating-point operations are reduced by 12.2%, and the detection accuracy is improved by 21.2%. Although the detection frame rate slightly decreases, it still meets real-time detection requirements. The experimental results demonstrate that the feasibility and effectiveness of the proposed algorithm have been significantly improved.

YOLO-based thyroid ultrasound for assisted diagnosis

Abstract Real-time intelligent diagnosis of thyroid ultrasound images using deep learning techniques can mitigate the impact of human subjective factors on diagnostic outcomes. Three thousand and fifty-five thyroid ultrasound images were acquired from 205 selected patients aged 10 to 77 years who underwent ultrasound examinations between 2019 and 2023. Each image contained at least one area of thyroid nodules, totaling 3, 088 nodules, including 1, 752 benign and 1, 336 malignant nodules. YOLOV8 is the baseline model for developing an end-to-end architecture for thyroid nodule detection. This architecture automatically identifies nodule lesions in ultrasound images, classifies them as benign or malignant, and enables real-time detection in video frames. The experimental results on 611 clinical thyroid ultrasound images demonstrate that our method can accurately diagnose benign and malignant nodal lesions in thyroid ultrasound with 88.1% and 84.6% accuracy, respectively. The mean average accuracy is as high as 91.1%, indicating a 5.1% improvement compared to the baseline model.

Lightweight Algorithm for Rail Fastener Status Detection Based on YOLOv8n

To improve the accuracy of rail fastener detection and deploy deep learning models on mobile platforms for fast real-time inference, this paper proposes a defect detection model for rail fasteners based on an improved YOLOv8n. Considering the significant aspect ratio differences of rail fasteners, we designed the EIOU+ as the regression box loss function. The model is compressed and trained using an improved channel-wise knowledge distillation (CWD+) approach to address the challenge of accurately recognizing minor defects in rail fasteners. We introduced a feature extraction module to design a feature extraction network as the distillation teacher model (YOLOv8n-T) and a lightweight cross-stage partial bottleneck with two convolutions and a fusion module (C2f) to improve the YOLOv8n backbone network as the distillation student model (YOLOv8n-S). Experiments conducted on data collected from actual rail lines demonstrate that after CWD+ distillation training, the model’s mean detection accuracy (IOU = 0.5) reached 96.3%, an improvement of 2.7% over the original YOLOv8n algorithm. The recall rate increased by 4.5%, the precision by 2.7%, the number of floating-point operations decreased by 13%, and the detection frame rate frames per second (FPS) increased by 6.1 frames per second. Compared with other one-stage object detection algorithms, the CWD+ distilled model achieves the precise real-time detection of rail fastener conditions.

Deep learning based enhanced secure emergency video streaming approach by leveraging blockchain technology for Vehicular AdHoc 5G Networks

VANET is a category of MANET that aims to provide wireless communication. It increases the safety of roads and passengers. Millions of people lose their precious lives in accidents yearly, millions are injured, and others incur disability daily. Emergency vehicles need clear roads to reach their destination faster to save lives. Video streaming can be more effective as compared to textual messages and warnings. To address this issue, we proposed a methodology to use visual sensors, cameras, and OBU to record emergency videos. Initially, the frames are detected. After re-recording, the frames detection algorithm detects the specific event from the video frames. Blockchain encrypts an emergency or specific event using hashing algorithms in the second layer of our proposed framework. In the third layer of the proposed methodology, encrypted video is broadcast with the help of 5G wireless technology to the connected nodes in the VANET. The dataset used in this research comprises up to 72 video sequences averaging about 120 seconds per video. All videos have different traffic conditions and vehicles. The ResNet-50 model is used for the feature extraction process of extracted frames. The model is trained using Tensorflow and Keras deep learning models. The Elbow method finds the optimal K number for the K Means model. This data is split into training and testing. 70% is reserved for training the support vector machine (SVM) model and test datasets, while 30%. 98% accuracy is achieved with 98% precision and 99% recall as results for the proposed methodology.

Lightweight ViT with Multiscale Feature Fusion for Driving Risk Rating Warning System

AbstractAddressing the issue of inadequate dynamic object detection accuracy in current road driving warning systems, this study proposes the RepBF‐YOLOv8 detection algorithm aimed at efficient risk identification. The backbone network of YOLOv8n is replaced with the lightweight RepViT architecture, which is more suitable for visual tasks. This replacement simplifies the traditional structure, reduces the complexity of the backbone network, maximizes performance enhancement, and minimizes latency. Additionally, the FPN in the neck section is upgraded to Bi‐FPN, which reduces nodes and span connections and incorporates rapid normalization to achieve fast multi‐scale feature fusion. For risk grading, the algorithm infers distances and collision times, categorizing detected objects into high, medium, and low‐risk levels, and uses different colors to warn the driver. Comparative experimental results show that the optimized algorithm improves Precision by 1.7%, Recall by 2.3%, mAP@0.5 by 1.53%, and mAP@0.5:0.95 by 2.91%. In road tests, the risk warning system achieves a frame detection rate ranging from a minimum of 38.4 fps to a maximum of 59.0 fps. The detection confidence for various objects remains above 0.71, reaching as high as 0.98. Specifically, the “Car” confidence ranges from 0.81 to 0.98, demonstrating the accuracy and robustness of vehicle risk detection.

Maize plant height automatic reading of measurement scale based on improved YOLOv5 lightweight model.

Plant height is a significant indicator of maize phenotypic morphology, and is closely related to crop growth, biomass, and lodging resistance. Obtaining the maize plant height accurately is of great significance for cultivating high-yielding maize varieties. Traditional measurement methods are labor-intensive and not conducive to data recording and storage. Therefore, it is very essential to implement the automated reading of maize plant height from measurement scales using object detection algorithms. This study proposed a lightweight detection model based on the improved YOLOv5. The MobileNetv3 network replaced the YOLOv5 backbone network, and the Normalization-based Attention Module attention mechanism module was introduced into the neck network. The CioU loss function was replaced with the EioU loss function. Finally, a combined algorithm was used to achieve the automatic reading of maize plant height from measurement scales. The improved model achieved an average precision of 98.6%, a computational complexity of 1.2 GFLOPs, and occupied 1.8 MB of memory. The detection frame rate on the computer was 54.1 fps. Through comparisons with models such as YOLOv5s, YOLOv7 and YOLOv8s, it was evident that the comprehensive performance of the improved model in this study was superior. Finally, a comparison between the algorithm's 160 plant height data obtained from the test set and manual readings demonstrated that the relative error between the algorithm's results and manual readings was within 0.2 cm, meeting the requirements of automatic reading of maize height measuring scale.

MediaPipe Frame and Convolutional Neural Networks-Based Fingerspelling Detection in Mexican Sign Language

MediaPipe Frame and Convolutional Neural Networks-Based Fingerspelling Detection in Mexican Sign Language

A Lightweight and High-Precision Passion Fruit YOLO Detection Model for Deployment in Embedded Devices.

In order to shorten detection times and improve average precision in embedded devices, a lightweight and high-accuracy model is proposed to detect passion fruit in complex environments (e.g., with backlighting, occlusion, overlap, sun, cloud, or rain). First, replacing the backbone network of YOLOv5 with a lightweight GhostNet model reduces the number of parameters and computational complexity while improving the detection speed. Second, a new feature branch is added to the backbone network and the feature fusion layer in the neck network is reconstructed to effectively combine the lower- and higher-level features, which improves the accuracy of the model while maintaining its lightweight nature. Finally, a knowledge distillation method is used to transfer knowledge from the more capable teacher model to the less capable student model, significantly improving the detection accuracy. The improved model is denoted as G-YOLO-NK. The average accuracy of the G-YOLO-NK network is 96.00%, which is 1.00% higher than that of the original YOLOv5s model. Furthermore, the model size is 7.14 MB, half that of the original model, and its real-time detection frame rate is 11.25 FPS when implemented on the Jetson Nano. The proposed model is found to outperform state-of-the-art models in terms of average precision and detection performance. The present work provides an effective model for real-time detection of passion fruit in complex orchard scenes, offering valuable technical support for the development of orchard picking robots and greatly improving the intelligence level of orchards.

Video object detection via space–time feature aggregation and result reuse

AbstractWhen detecting the objects in videos, motion always leads to object deterioration, like blurring and occlusion, as well as the strange state of the object's shape and posture. Consequently, the detection of video frames will lead to a decline in accuracy by using the image object detection model. This paper proposes an online video object detection method based on the one‐stage detector YOLOx. First, the module for space–time feature aggregation is given, which uses the space–time information of past frames to enhance the feature quality of the current frame. Then, the module for result reuse is given, which incorporates the detection results of past frames to improve the detection stability of the current frame. By these two modules, the trade‐off between accuracy and speed of video object detection could be achieved. Experimental results on the ImageNet VID show the improvement of speed and accuracy of the proposed method.

Research on Coal Flow Visual Detection and the Energy-Saving Control Method Based on Deep Learning

In this paper, machine vision technology is used to recognize the coal flow on a conveyor belt and control the running speed of a motor according to the coal flow on the conveyor belt to achieve an energy-saving effect and provide technical support for the sustainable development of energy. In order to improve the accuracy of coal flow recognition, this paper proposes the color gain-enhanced multi-scale retina algorithm (AMSRCR) for image preprocessing. Based on the YOLOv8s-cls improved deep learning algorithm YOLO-CFS, the C2f-FasterNet module is designed to realize a lightweight network structure, and the three-dimensional weighted attention module, SimAm, is added to further improve the accuracy of the network without introducing additional parameters. The experimental results show that the recognition accuracy of the improved algorithm YOLO-CFS reaches 93.1%, which is 4.8% higher, and the detection frame rate reaches 32.68 frame/s, which is 5.9% higher. The number of parameters is reduced by 28.4%, and the number of floating-point operations is reduced by 33.3%. These data show that the YOLO-CFS algorithm has significantly improved the accuracy, lightness, and reasoning speed in the coal mine environment. Furthermore, it can satisfy the requirements of coal flow recognition, realize the energy-saving control of coal mine conveyor belts, and achieve the purpose of sustainable development of the coal mining industry.

An efficient novel dual deep network architecture for video forgery detection

The technique of video copy-move forgery (CMF) is commonly employed in various industries; digital videography is regularly used as the foundation for vital graphic evidence that may be modified using the aforementioned method. Recently in the past few decades, forgery in digital images is detected via machine intellect. The second issue includes continuous allocation of parallel frames having relevant backgrounds erroneously results in false implications, detected as CMF regions third include as the CMF is divided into inter-frame or intra-frame forgeries to detect video copy is not possible by most of the existing methods. Thus, this research presents the dual deep network (DDN) for efficient and effective video copy-move forgery detection (VCMFD); DDN comprises two networks; the first detection network (DetNet1) extracts the general deep features and second detection network (DetNet2) extracts the custom deep features; both the network are interconnected as the output of DetNet1 is given to DetNet2. Furthermore, a novel algorithm is introduced for forged frame detection and optimization of the falsely detected frame. DDN is evaluated considering the two benchmark datasets REWIND and video tampering dataset (VTD) considering different metrics; furthermore, evaluation is carried through comparing the recent existing model. DDN outperforms the existing model in terms of various metrics.

Framing is Mightier than the Sword: Detection of Episodic and Thematic Framing in News Media

Many people today watch news videos to get informed. However, news videos can frame information differently and be prone to bias that might lead to miscommunication. Bias is ubiquitous andinherent in interactions between news consumers’ groups, but framing can introduce additional bias in news communications. Citizens that interpret the news have different political orientations and, thus, understand them differently. Experts can be more capable of detecting biased or differently framed information. However, the ever-increasing amount of news videos also make it difficult for experts alone to analyze. While automated methods exist for identifying different types of bias, frame detection approaches, namely episodic and thematic framing, are scarce and focused on texts. In this work, we address the issue of scalable thematic and episodic frame detection in news videos through crowdsourcing and machine learning techniques. We design a crowdsourcing task for annotating thematic and episodic framing in videos with the help of domain experts in political and social sciences. We then use the annotations gathered from experts and crowds to investigate whether machine learning methods can scale up the annotation process by automatically labelling videos on episodic and thematic framing. Our results indicate that framing analysis is a challenging task, both for humans and machines, with high disagreement amongst experts and crowd annotators. Nevertheless, our results suggest that machine learning has potential by combining crowd and expert annotations and building upon them a classifier.