Frame Detection Research Articles

Binocular intelligent following robot based on YOLO-LITE

In order to solve the problem that the deep neural network model is large in scale, the calculation time is too long, and the real-time performance is severely limited when combined with embedded devices, so studied the intelligent follower robot system based on YOLO-LITE algorithm combined with Raspberry Pi 3B+. The system mainly includes camera processing, target detection and other modules. Obtained the internal and external parameters of the camera through calibration, and according to these parameters to correct the binocular camera. Recognized and located the target in each frame of image, calculated the distance from the camera to the target and the center location error, and driven the car to move. The experimental results show that the following car has excellent real-time performance, the average detection frame rate can reach 20Fps, and the average detection accuracy can reach more than 80%.

Measurement and Evaluation of Electric Signal Transmission Through Human Body by Channel Modeling, System Design, and Implementation

Human body communications (HBCs), employing the human body as a signal transmission medium, can provide efficient and intuitive methods to form a network in the body. This article presents a comprehensive study for a highly reliable HBC system, including body channel modeling, transceiver design, and performance evaluation through implementation, in consideration of practical sensor network environments for wearable and implantable devices applicable to healthcare and biosignal acquisition. Body channel characteristics based on capacitive couplings, such as root mean square delay spread and mean path gain (MPG), were explored by measuring the body impulse responses under 12 experimental conditions, determined by the variation in body postures and device locations between the wrist and positions assumed under the scalp through a customized experimental setup with micropig-derived biomembranes (MBMs) used for wrapping the devices to emulate an implantable environment. The proposed transceiver design for digital transmission, including a preamble structure and signal modulation method supporting a maximum data rate of 1 Mb/s, was verified through the performance evaluations conducted for examining the frame detection probability and bit error rate (BER) in the body channel model at multiple operating frequencies of 32, 42, and 64 MHz. The proposed system reliability was demonstrated by the achievement of a BER of below <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${4.7\,\times \, }{10}^{-8}$ </tex-math></inline-formula> through battery-powered implemented devices with dimensions of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${30\times 30 }\,\,{\text {mm}}^{2}$ </tex-math></inline-formula> , comprising a digital signal processing module (DSPM) for signal generation and detection processes, and an analog front-end module (AFEM) for recovering the received signal from signal deterioration by severe path loss and time-dispersive effect in the body channel.

Design of Sanda Action Reconstruction Model Based on 3D Images

In order to improve the training level of Sanda movement, this article uses an image analysis method to reconstruct the detailed characteristics of the movement and apply them to the actual training process. Since the traditional wavelet reconstruction method is affected by the accuracy of the decomposition scale, this paper proposes an improved method of Sanda action based on 3D image reconstruction. First, the method relies on frame adjacent phase compensation and digital image stabilization techniques to perform digital frame operations on the image. Then, scanning and corner detection are used for image reconstruction, where adjacent phase compensation methods are used to match feature points and gray pixels. Image extraction is performed by extracting key feature points of the action 3D image, and a fast frame detection method is used to stabilize the image of the digital image, thereby improving the image quality of image reconstruction. The experimental results show that the method has good image output effect and has a high application value in Sanda guidance and optimization.

Local track to detect for video object detection

The existing methods for video object detection are generally achieved from searching the objects through the entire image. However, they always suffer from large computation consumption as a result of dozens of similar images needing to be operated. To relieve this problem, we propose a Local Track to Detect (LTD) framework to detect video objects by predicting the movements of objects in local areas. LTD can automatically determine key frames and non-key frames, the objects in key frames can be detected by the single frame detector, and the objects in non-key frames can be efficiently detected by the movement prediction module. LTD also has a Siamese module to predict whether objects between the key frame and the non-key frame are the same object to ensure the accuracy of the movement prediction module. Compared with other previous work, our method is more efficient and achieves state-of-the-art performance.

Batalla de frames en la campaña electoral de abril de 2019. Engagement y promoción de mensajes de los partidos políticos en Facebook

Framing processes are one of the preferred objects of study in the area of Communication, despite the frequent criticism regarding their theoretical and methodological strength. This study follows the proposal of Matthes and Kohring (2008) of the detection of frames through statistical analysis of clusters that operationalize the components as defined by Entman (2003) using framing elements. This is applied to the study of the electoral campaign on Facebook of the main Spanish political parties in the election of 28 April 2019. All messages published on this social network during the 15 days of the electoral campaign were captured automatically. The use of four large frames by the parties was detected, corresponding to events and political questions, request for vote, the rival party, and media treatment. The parties used these frames to different degrees: from the institutional and positive campaign of the PSOE, to the campaign aimed at criticizing the ruling party of the main opposition party, the PP, through the inclination towards media treatment of Unidas Podemos, or a focus on their own events and proposals by Ciudadanos and Vox. A negative correlation was detected between the frames most used by the political formations as a whole and the level of user engagement on Facebook, although the use of frames by PSOE and Unidas Podemos showed a greater parallelism with the levels of interaction in this social network. Resumen Los procesos de enmarcado, encuadre o framing son uno de los objetos de estudio preferentes en el área de Comunicación, a pesar de las frecuentes y reputadas críticas sobre su solidez teórica y metodológica. En esta investigación se sigue la propuesta de Matthes y Kohring (2008) para la detección de frames mediante análisis estadístico de conglomerados que operacionaliza en elementos de enmarcado los componentes de la definición de Entman (1993). Se aplica al estudio de la campaña electoral que hicieron los principales partidos políticos españoles a través de Facebook en la convocatoria del 28 de abril de 2019. Se capturaron automatizadamente todos los mensajes publicados en esta red social en los quince días de campaña electoral, detectándose el uso de cuatro grandes frames por parte de los partidos: marco de eventos y cuestiones políticas, marco de petición de voto, marco del partido rival y marco del tratamiento mediático. Los partidos utilizaron en diferente medida estos marcos: desde la campaña institucional y en positivo del PSOE; a la campaña volcada en criticar al partido gobernante del principal partido de la oposición, el PP; pasando por la inclinación hacia el tratamiento mediático de Unidas Podemos, o a centrarse en sus propios actos y propuestas de Ciudadanos y Vox. Se detectó una correlación negativa entre los frames más empleados por las formaciones políticas en su conjunto y el nivel de engagement en Facebook, si bien la utilización concreta de los marcos por parte del PSOE y Unidas Podemos sí mostró cierto paralelismo con los niveles de interacción en esta red social.

Self-Supervised Learning to Detect Key Frames in Videos

Detecting key frames in videos is a common problem in many applications such as video classification, action recognition and video summarization. These tasks can be performed more efficiently using only a handful of key frames rather than the full video. Existing key frame detection approaches are mostly designed for supervised learning and require manual labelling of key frames in a large corpus of training data to train the models. Labelling requires human annotators from different backgrounds to annotate key frames in videos which is not only expensive and time consuming but also prone to subjective errors and inconsistencies between the labelers. To overcome these problems, we propose an automatic self-supervised method for detecting key frames in a video. Our method comprises a two-stream ConvNet and a novel automatic annotation architecture able to reliably annotate key frames in a video for self-supervised learning of the ConvNet. The proposed ConvNet learns deep appearance and motion features to detect frames that are unique. The trained network is then able to detect key frames in test videos. Extensive experiments on UCF101 human action and video summarization VSUMM datasets demonstrates the effectiveness of our proposed method.

Deriving patterns for the vibration-based damage detection in side frames of bogies

Detecting damage in the incipient state is of crucial importance in monitoring the health of engineering structures. Approaching damage detection as an inverse problem is a common method when vibration-based techniques are used. It implies a precise a priori estimation of the modal parameters. We present in this paper a model to predict the frequency changes due to cracks in the side frames of bogies used to detect the welding defects in T-joints. To this aim, we model the structure and perform simulations using the SolidWorks software. The frequencies for the first thirteen modes are derived for the intact structure and the structure with different damage lengths and positions. By these tests, we first desire to find the optimal fixing of the side frame to reduce the number of vibration modes that are not of interest to the damage detection process. After finding the ideal fixing system, we derive patterns, for different defects, in the form of relative frequency shifts. Finally, we conclude about the damages that can be identified using the derived patterns, the conclusions being traced based on the measurability of the frequency changes.

An efficient method to classify GI tract images from WCE using visual words

The digital images made with the Wireless Capsule Endoscopy (WCE) from the patient's gastrointestinal tract are used to forecast abnormalities. The big amount of information from WCE pictures could take 2 hours to review GI tract illnesses per patient to research the digestive system and evaluate them. It is highly time consuming and increases healthcare costs considerably. In order to overcome this problem, the CS-LBP (Center Symmetric Local Binary Pattern) and the ACC (Auto Color Correlogram) were proposed to use a novel method based on a visual bag of features (VBOF). In order to solve this issue, we suggested a Visual Bag of Features(VBOF) method by incorporating Scale Invariant Feature Transform (SIFT), Center-Symmetric Local Binary Pattern (CS-LBP) and Auto Color Correlogram (ACC). This combination of features is able to detect the interest point, texture and color information in an image. Features for each image are calculated to create a descriptor with a large dimension. The proposed feature descriptors are clustered by K- means referred to as visual words, and the Support Vector Machine (SVM) method is used to automatically classify multiple disease abnormalities from the GI tract. Finally, post-processing scheme is applied to deal with final classification results i.e. validated the performance of multi-abnormal disease frame detection.

Joint Local and Global Information Learning With Single Apex Frame Detection for Micro-Expression Recognition.

Micro-expressions (MEs) are rapid and subtle facial movements that are difficult to detect and recognize. Most recent works have attempted to recognize MEs with spatial and temporal information from video clips. According to psychological studies, the apex frame conveys the most emotional information expressed in facial expressions. However, it is not clear how the single apex frame contributes to micro-expression recognition. To alleviate that problem, this paper firstly proposes a new method to detect the apex frame by estimating pixel-level change rates in the frequency domain. With frequency information, it performs more effectively on apex frame spotting than the currently existing apex frame spotting methods based on the spatio-temporal change information. Secondly, with the apex frame, this paper proposes a joint feature learning architecture coupling local and global information to recognize MEs, because not all regions make the same contribution to ME recognition and some regions do not even contain any emotional information. More specifically, the proposed model involves the local information learned from the facial regions contributing major emotion information, and the global information learned from the whole face. Leveraging the local and global information enables our model to learn discriminative ME representations and suppress the negative influence of unrelated regions to MEs. The proposed method is extensively evaluated using CASME, CASME II, SAMM, SMIC, and composite databases. Experimental results demonstrate that our method with the detected apex frame achieves considerably promising ME recognition performance, compared with the state-of-the-art methods employing the whole ME sequence. Moreover, the results indicate that the apex frame can significantly contribute to micro-expression recognition.

Automatic stenosis recognition from coronary angiography using convolutional neural networks

Background and objectiveCoronary artery disease, which is mostly caused by atherosclerotic narrowing of the coronary artery lumen, is a leading cause of death. Coronary angiography is the standard method to estimate the severity of coronary artery stenosis, but is frequently limited by intra- and inter-observer variations. We propose a deep-learning algorithm that automatically recognizes stenosis in coronary angiographic images. MethodsThe proposed method consists of key frame detection, deep learning model training for classification of stenosis on each key frame, and visualization of the possible location of the stenosis. Firstly, we propose an algorithm that automatically extracts key frames essential for diagnosis from 452 right coronary artery angiography movie clips. Our deep learning model is then trained with image-level annotations to classify the areas narrowed by over 50 %. To make the model focus on the salient features, we apply a self-attention mechanism. The stenotic locations are visualized using the activated area of feature maps with gradient-weighted class activation mapping. ResultsThe automatically detected key frame was very close to the manually selected key frame (average distance (1.70 ± 0.12) frame per clip). The model was trained with key frames on internal datasets, and validated with internal and external datasets. Our training method achieved high frame-wise area-under-the-curve of 0.971, frame-wise accuracy of 0.934, and clip-wise accuracy of 0.965 in the average values of cross-validation evaluations. The external validation results showed high performances with the mean frame-wise area-under-the-curve of (0.925 and 0.956) in the single and ensemble model, respectively. Heat map visualization shows the location for different types of stenosis in both internal and external data sets. With the self-attention mechanism, the stenosis could be precisely localized, which helps to accurately classify the stenosis by type. ConclusionsOur automated classification algorithm could recognize and localize coronary artery stenosis highly accurately. Our approach might provide the basis for a screening and assistant tool for the interpretation of coronary angiography.

Vacant Parking Slot Recognition Method for Practical Autonomous Valet Parking System Using around View Image

The parking assist system (PAS) provides information of parking slots around the vehicle. As the demand for an autonomous system is increasing, intelligent PAS has been developed to park the vehicle without the driver’s intervention. To locate parking slots, most existing methods detect slot markings on the ground using an around-view monitoring (AVM) image. There are many types of parking slots of different shapes in the real world. Due to this fact, these methods either limit their target types or use predefined slot information of different types to cover the types. However, the approach using predefined slot information cannot handle more complex cases where the slot markings are connected to other line markings and the angle between slot marking is slightly different from the predefined settings. To overcome this problem, we propose a method to detect parking slots of various shapes without predefined type information. The proposed method is the first to introduce a free junction type feature to represent the structure of parking slot junction. Since the parking slot has a modular or repeated junction pattern at both sides, junction pair consisting of one parking slot can be detected using the free junction type feature. In this process, the geometrically symmetric characteristic of the junction pair is crucial to find each junction pair. The entrance of parking slot is reconstructed according to the structure of junction pair. Then, the vacancy of the parking slot is determined by a support vector machine. The Kalman tracker is applied for each detected parking slot to ensure stability of the detection in consecutive frames. We evaluate the performance of the proposed method by using manually collected datasets, captured in different parking environments. The experimental results show that the proposed method successfully detects various types of parking slots without predefined slot type information in different environments.

Employing an improved cross model cross mode algorithm for damage detection of a steel offshore platform frame using experimental data

In this study, a modified cross model cross mode (CMCM) algorithm is developed to identify and localize the damage based on the changes made in modal physical parameters of a steel platform frame. The research is mostly focused on the first three stages of structural health monitoring due to their close relation to the structural dynamic characteristics and numerical modeling aspects. For this purpose, experimental modal analysis is implemented on a physical model of a 2D jacket platform frame. The finite element analysis is developed based on the recorded data to validate the proposed algorithm. The numerical model is updated employing the improved CMCM method and the results are compared to those of the traditional method. The effect of the noisy data on the performances of the improved method is also investigated. The results indicate the success of the proposed method in identifying the location and the severity of the damage. It is also concluded that the improved iterative methods include advantages such as time saving and higher accuracy in comparison with the traditional one.

Novel approach for automatic mid‐diastole frame detection in 2D echocardiography sequences for performing planimetry of the mitral valve orifice

The mitral valve orifice area is a reliable measure for evaluating mitral valve stenosis (MS) severity, which is obtained by the planimetry of the mid-diastole frame in the echocardiography sequences. Since the manual method for determining this frame is time-consuming and user-dependent, a novel automatic method has been proposed in this study. First, the region of interest (ROI) containing the mitral valve orifice region is detected using circular Hough transform and k-means algorithms. Then, the dimension reduction method is applied to the ROI of each frame to map it into a point in a 2D space. The performance of the local linear embedding (LLE), isometric mapping, kernel principal component analysis (PCA), and linear PCA algorithms has been evaluated in this study. Finally, a distance curve is obtained by calculating the Euclidean distance between consecutive points in 2D space, and the mid-diastole frame is determined by interpreting this curve. The proposed algorithm was validated using 2D echocardiography of the 20 MS patients. Finally, the LLE method showed the best result, and the average frame difference for 20 cases using the proposed method compared with the gold standard (the echo-cardiologist opinion) was 1.40.

Optimized frame detection technique in vehicle accident using deep learning

&nbsp;Video processing becomes one of the most popular and needed steps in machine leering. Todays, Cameras are installed in many places for many reasons including government services. One of the most applications for this concern is traffic police services. One of the main problems of using videos in machine learning application is the duration of the video; which is consuming time, paperwork and space in processing. This leads to increase the computation cost through a high number of frames. This paper proposes an algorithm to optimize videos duration using a Gaussian mixture model (GMM) method for real accident video. The Histogram of Gradient (HoG) has been used to extract the features of the video frames, a scratch CNN has been designed and conducted on two common datasets; Stanford Dogs Dataset (SDD) and Vehicle Make and Model Recognition Dataset (VMMRdb) in addition to a local dataset that created for this research. The experimental work is done in two ways, the first is after applying GMM, the finding revealed that the number of frames in the dataset was decreased by nearly 51%. The second is comparing the accuracy and complexity of these datasets has been done. Whereas the experimental results of accuracy illustrated for the proposed CNN, 85% on the local dataset, 85% on SDD Dataset and 86% on VMMRdb Dataset. However, applying GoogleNet and AlexNet on the same datasets achieved 82%, 79%, 80%, 83%, 81%, 83% respectively.

Video detection of foreign objects on the surface of belt conveyor underground coal mine based on improved SSD

Aiming at the problem of belt conveyor damage caused by the presence of foreign objects on the belt conveyor in coal mines, this paper proposed that video detection of foreign objects on the belt surface was performed based on SSD. Improvements on SSD network are made from following aspects. Firstly, the deep separable convolution method is used to reduce the amount of parameters in the SSD algorithm and improve the speed. Then, GIOU loss function is adopted instead of the position loss function in the original SSD to improve the detection accuracy. Finally, the extracted position of the feature map and the proportion of the default boxes are optimized to improve the detection accuracy. The experiment results show that the improved algorithm proposed in this paper is superior to the original SSD algorithm, the average accuracy rate has been increased from 87.1 to 90.2%, and the detection frame rate has been increased from 32 to 41 FPS.

Multimedia image and video retrieval based on an improved HMM

In today's information age, information is gathered from text and more complex media, such as images, audio, and video. Among these data sources, the rapid growth of video information has led to it to gradually become the main source of information in people's lives. Video information is characterized by many kinds of information, complex forms, and a low degree of structure. Therefore, effectively classifying, managing and retrieving video information has become a difficult problem to solve. In this paper, an improved crow search algorithm is used to process video images, and the information entropy is used to extract the key frames, which reduces the computation burden of each frame feature calculation and feature contrast process, thus shortening the key frame detection time. Then, all the feature sets are extracted and used as input for an HMM according to the observed sequence $$O = O_{1} ,O_{2} ,O_{3} , \cdot \cdot \cdot ,O_{T}$$ of the input image or video data and the initial model parameters $$\lambda = (\pi ,A,B)$$ . According to the training rules, the model parameters are repeatedly adjusted and modified, and the new model $$\overline{\lambda }$$ is constructed step by step to realize the retrieval of multimedia images and videos. The experimental results show that the method has obvious advantages in terms of the retrieval time and retrieval effect and provides new ideas for multimedia image and video retrieval.

A method for single frame detection of infrared dim small target in complex background

In order to solve the problem of high false alarm rate caused by strong disturbance (the ground objects, uncertain strong noise) in complex background, we propose a novel algorithm based on density peaks searching and improved region growing. Initially, density peaks searching is employed to select the candidate targets quickly; then, improved region growing with introducing the relative local contrast and the local gray average is used to extract the features of candidate targets; finally, an adaptive threshold is applied to extract real targets. Experimental results on two real sequences show that the proposed algorithm has a better effectiveness against existing algorithms.

Convolutional neural-based algorithm for port occupancy status detection of optical distribution frames

As optical distribution frames (ODFs) carry increasing amounts of data, issues relating to their port management are becoming crucial. Image backhaul inspection has been widely accepted as an effective technology to monitor the occupancy status of ODF ports. However, the massive amount of on-site images requires a large team of off-site technologists to manually identify the occupancy status of each ODF port, which is time-consuming. We employ the you only look once version 3 (YOLOv3) network to automatically recognize ODF port occupancy. The YOLOv3 is a state-of-the-art convolutional neural network that has been shown to be very efficient for object detection in terms of processing speed and accuracy for the common objects in context. To accommodate ODF images with densely arranged small objects, high resolutions, closely spaced adjacent ports, and occlusion, we modified the original YOLOv3 with four-scale feature fusion, anchor box dimension clustering, and soft nonmaximum suppression filtering. Experiments showed a 7.38% increase in the original YOLOv3 detection accuracy rate of 91.45%. The new method can update the image backhaul inspection to automatically realize port resource management. The number of required port management technologists is considerably reduced, and the accuracy of port resources is increased, resulting in significant network investment savings.

PS-DeVCEM: Pathology-sensitive deep learning model for video capsule endoscopy based on weakly labeled data

We propose a novel pathology-sensitive deep learning model (PS-DeVCEM) for frame-level anomaly detection and multi-label classification of different colon diseases in video capsule endoscopy (VCE) data. Our proposed model is capable of coping with the key challenge of colon apparent heterogeneity caused by several types of diseases. Our model is driven by attention-based deep multiple instance learning and is trained end-to-end on weakly labeled data using video labels instead of detailed frame-by-frame annotation. This makes it a cost-effective approach for the analysis of large capsule video endoscopy repositories. Other advantages of our proposed model include its capability to localize gastrointestinal anomalies in the temporal domain within the video frames, and its generality, in the sense that abnormal frame detection is based on automatically derived image features. The spatial and temporal features are obtained through ResNet50 and residual Long short-term memory (residual LSTM) blocks, respectively. Additionally, the learned temporal attention module provides the importance of each frame to the final label prediction. Moreover, we developed a self-supervision method to maximize the distance between classes of pathologies. We demonstrate through qualitative and quantitative experiments that our proposed weakly supervised learning model gives a superior precision and F1-score reaching, 61.6% and 55.1%, as compared to three state-of-the-art video analysis methods respectively. We also show our model’s ability to temporally localize frames with pathologies, without frame annotation information during training. Furthermore, we collected and annotated the first and largest VCE dataset with only video labels. The dataset contains 455 short video segments with 28,304 frames and 14 classes of colorectal diseases and artifacts. Dataset and code supporting this publication will be made available on our home page.

The 4D Camera – An 87 kHz Frame-rate Detector for Counted 4D-STEM Experiments

Author(s): Ercius, Peter; Johnson, Ian; Brown, Hamish; Pelz, Philipp; Hsu, Shang-Lin; Draney, Brent; Fong, Erin; Goldschmidt, Azriel; Joseph, John; Lee, Jason; Ciston, Jim; Ophus, Colin; Scott, Mary; Selvarajan, Ashwin; Paul, David; Skinner, David; Hanwell, Marcus; Harris, Chris; Avery, Patrick; Stezelberger, Thorsten; Tindall, Craig; Ramesh, Ramamoorthy; Minor, Andrew; Denes, Peter