Video Frame Image Research Articles

A Phlegm Stagnation Monitoring Based on VDS Algorithm.

In the nonmedical sputum monitoring system, a practical solution for phlegm stagnation care of patients was proposed. Through the camera, the video images of patients' laryngeal area were obtained in real time. After processing and analysis on these video frame images, the throat movement area was found out. A three-frame differential method was used to detect the throat moving targets. Anomalies were identified according to the information of moving targets and the proposed algorithm. Warning on the abnormal situation can help nursing personnel to deal with sputum blocking problem more effectively. To monitor the patients' situation in real time, this paper proposed a VDS algorithm, which extracted the speed characteristics of moving objects and combined with the DTW algorithm and SVM algorithm for sequence image classification. Phlegm stagnation symptoms of patients were identified timely for further medical care. In order to evaluate the effectiveness, our method was compared with the DTW, SVM, CTM, and HMM methods. The experimental results showed that this method had a higher recognition rate and was more practical in a nonmedical monitoring system.

Frameworks for Animal and Human Detection using Camera Images to Detect Anomaly

Human and animal interacting events in video image frames will leads to anomalies and it cannot be predicted. These anomalous events was happened due to the interactions of human, animals and birds with each other. Some of the human and animal anamolous interactions will leads to anomalous actions in camera surveillance sites and are to be considered as a serious issue. All anomalies will leads the system or the authorities who dedicated for monitoring to the suspicious events. To detect the anomalies, the animals and human objects must be identified in each image frames first. After object identification events detection phase has to be done. Before going to the event detection phase, noise elimination, shadow detection, object classification can be carried out according to the needs. This paper reveals general detection methodology for animal detection and human crowd. The proposed work mainly concentrated on the detection of animals in human territory based on its palm print images and video images. Human crowd can be treated as smoke- screen for all types of anomalies and so this paper mainly concentrated to detect the crowd well in advance and thereby preventing the crowd before it is happening.

Detection of Fire Regions from a Video Image Frames in YCbCr Color Model

Proposed here is a fire region detection method from a recorded video captured during the occurrence of fire. This method is based only on the chrominance components of the YCbCr color model. To distinguish the fire-region in an image frame of fire video containing the fire region, the difference between Cr and Cb is computed. The difference is enhanced by computing the square of it and then normalize range of difference squared to 0 to 255. It is then binarized at using automatic thresholding method to segment the fire region from the non-fire region. The fire region in the binary is located using the connected component analysis and the region is mapped to the original image frames of the fire video. We have tested the method in the actual fire video and it is found that the method can appropriately locate the fire regions in every image frame of the video. The method is simple and fast and hence can be used to forest fire monitoring using drones.

MSSTResNet-TLD: A robust tracking method based on tracking-learning-detection framework by using multi-scale spatio-temporal residual network feature model

The performance of tracking task is directly dependent on the appearance features of target object, a robust approach for constructing appearance features is crucial for adaptation the appearance change. To construct an accurate and robust appearance model for visual object tracking, we modify original deep residual learning network architecture and name it Multi-Scale Residual Network (MSResNet). The first video frame image and its related information of the current input video sequence are used to learn a multi-scale appearance model of target object and a loss function is minimized over the appearance features. Meanwhile, spatial information of each video frame and temporal information between successive video frames effectively combine with MSResNet. And thus the features are generated by Multi-Scale Spatio-Temporal Residual Network, which is named MSSTResNet feature model, can adapt to scale variation, illumination variation, background clutters, severe deformation of the target object, and so on. We implement a robust tracking method based on tracking-learning-detection framework by using our proposed MSSTResNet feature model and name it MSSTResNet-TLD tracker. Unlike the previous tracking methods, the MSResNet architecture is not offline pre-trained on a large auxiliary datasets but is directly learned end-to-end with a multi-task loss by using the current input video sequence. Furthermore, the multi-task loss function utilizes the classification loss and regression loss that is more accurate for target localization. Our experimental results demonstrate that the proposed tracking method outperforms the current state-of-the-art tracking methods on Visual Object Tracking Benchmark (VOT-2016), Object Tracking Benchmark (OTB-2015), and Unmanned Aerial Vehicles (UAV20L) test datasets. Furthermore, our MSSTResNet-TLD tracker is faster than previous most trackers based on deep Convolutional Neural Network (ConvNet or CNN) and our tracker is extremely robust to the tiny target object. Our source code is available for download athttps://github.com/binger1225/MSSTResNet-TLD-Tracker.

Key pose recognition toward sports scene using deeply-learned model

Key pose recognition (KPR) is widely used in sport analysis, which provides effective tools for coaches, athletes and other professionals to conduct game analysis and auxiliary training. KPR from video stream can be divided into individual-oriented and group-oriented. The former method is based on the segmentation and tracking of each target, and the characteristics of the individual are used to study the events in the group. The latter is to process and sample the global image, obtain the overall information, and then process the collected data to classify the abnormal situation and normal situation. In this paper, we propose key pose recognition method based on deep learning. Specifically, we first train an FCN network for foreground extraction with weightlifting video frame images to remove a large amount of background interference in the weightlifting video image. Further, by fine-tuning the CNN network, a network model suitable for the weight-of-weight video classification of the region of interest is obtained. Finally, according to the classification result, the classification result selection strategy is designed to extract the key pose. In addition, our algorithm can select high image quality key pose frame, which is important for sports training. The experimental results show that the proposed method is very competitive.

Improving Automatic Polyp Detection Using CNN by Exploiting Temporal Dependency in Colonoscopy Video.

Automatic polyp detection has been shown to be difficult due to various polyp-like structures in the colon and high interclass variations in polyp size, color, shape, and texture. An efficient method should not only have a high correct detection rate (high sensitivity) but also a low false detection rate (high precision and specificity). The state-of-the-art detection methods include convolutional neural networks (CNN). However, CNNs have shown to be vulnerable to small perturbations and noise; they sometimes miss the same polyp appearing in neighboring frames and produce a high number of false positives. We aim to tackle this problem and improve the overall performance of the CNN-based object detectors for polyp detection in colonoscopy videos. Our method consists of two stages: a region of interest (RoI) proposal by CNN-based object detector networks and a false positive (FP) reduction unit. The FP reduction unit exploits the temporal dependencies among image frames in video by integrating the bidirectional temporal information obtained by RoIs in a set of consecutive frames. This information is used to make the final decision. The experimental results show that the bidirectional temporal information has been helpful in estimating polyp positions and accurately predict the FPs. This provides an overall performance improvement in terms of sensitivity, precision, and specificity compared to conventional false positive learning method, and thus achieves the state-of-the-art results on the CVC-ClinicVideoDB video data set.

Ship recognition method combined with image segmentation and deep learning feature extraction in video surveillance

To solve the problem of ship recognition in video images, a ship recognition method based on Morphological Watershed image segmentation and Zemike moment is proposed. Firstly, the video frame image is pre-processed by gray algorithm, and then the gray image is filtered by wavelet transform to remove noise. After denoising, the Morphological Watershed algorithm is used to segment the image and extract the ship area in the image. Next, the feature of ship image is extracted based on deep learning convolution neural network (CNN) and Zemike moment method. Finally, the KNN-SVM classifier is trained according to the image features and class labels to realize the automatic recognition of ships. Experimental results show that the method can effectively identify 3 types of ships, with an average detection accuracy of 87%.

Robust Optical Flow Estimation Using Tchebichef Moment Invariant Feature

In this paper, orthogonal moment invariant-based features are used to compute 2D optical flow from sequence of images. The gray level of pixel is described in terms of its local neighborhood using Tchebichef moment invariants rather than its individual intensity value. The description is then normalized in order to make it insensitive to intensity fluctuations due to noise or other perturbations such as varying illumination conditions. The principle of conservation of moment invariance is used to derive overdetermined system of 2D motion constraint equations in local neighborhood of each pixel. The velocity field is then estimated using the least square method. Experimental results are performed on sequential video and thermal image frames under varying environmental conditions. The run time, robustness against noisy and varying illumination conditions and rotation invariance of proposed method are compared with already existing moment and non-moment-based techniques.

Simulation on High Speed Train with Driving Algorithm and Rules for Operation Image Control

Abstract Presently, a high speed train is an emerging system to serve effective mass transportation which is operated on local and national rail networks among neighbor borders. Thailand is recognized as a central both air and land transport containing vehicles and railway. The high speed train projects are planned to connect between Northward to Southward, West to East and North-Eastern. In order to public service, safty and consistency are extreamly significant with high degree of driver’s responsibility. Driver’s license and training certificate is necessary. One of the most effective training system is using the virtual simulator which emulating the reality system and rules. This paper purposes driving simulation of high speed train with imitation emphasizing on driving rules for operation image control. Simulation consists of the major content which is image control interacted with mechanical devices, software and PLC interface. Objects of track, stations and so on are firstly created by using 3D model before assembled in the forms of train route. There are two methods that can be created; video based and animation. As the result of experiment, it is found that the animation with C# programing control is more flexible, less time and effort, better resolution whereas the video frame image processing takes advantage on less complex and easily to produce and usage. This study is concerned only the North-Eastern line of Thailand on the first phase.

Remote Surgeon Hand Motion and Occlusion Removal in Mixed Reality in Breast Surgical Telepresence: Rural and Remote Care

Mixed reality in surgical telepresence has not been practically implemented as yet and research done, until now, is only theoretical. Aiming to implement a proposed new solution by merging virtual and augmented realities and bringing about improvements by removing occlusion and the noise caused by hand motions during surgery. The proposed system consists of an extended Camshift algorithm for the detection of the remote expert surgeon’s hand during surgery and enhancement of an image synthesization algorithm to remove the occlusion by including two Red-Green-Blue-Depth (RGBD) cameras. The experimental results from 10 breast samples reduce the average accuracy error of the video frame image overlay from 1.44 mm to 1.28 mm with average processing time enhanced to 76 sec compared to the state-of-the-art method. The proposed system provides a satisfactory outcome in terms of accuracy and operating time, which allows surgeons in local and remote sites to collaborate more effectively.

Feature detection and pneumonia diagnosis based on clinical lung ultrasound imagery using deep learning

Pneumonia is a common disease with both high morbidity and mortality. The diagnosis of pneumonia remains a clinical challenge, especially in low resource settings where prevalence is high, diagnostic devices are limited, and doctors are scarce. Lung ultrasound has been identified as a useful and low-cost tool for pneumonia diagnosis in many studies. In the present work, we first developed a convolutional neural network (CNN)-based deep learning algorithm to automatically identify four key features linked to lung conditions: pleural line, B-line, consolidation, and pleural effusion. The algorithm was trained using ultrasound data collected from over 150 pediatric and adult patients, with features annotated by expert sonographers. A single shot detection (SSD) framework was developed to detect those features in each video frame image. We then explored the accuracy of diagnosing pneumonia based on one or more lung ultrasound features, using CT as a gold standard. Our results indicate that deep learning algorithms can successfully detect abnormal lung features in ultrasound imagery, and those features can be used to diagnose pneumonia at a high sensitivity level. Computer-assisted ultrasound interpretation has the potential to place point of care, expert-level diagnostic accuracy in the hands of low-resource health care providers for pneumonia diagnosis.

Research on Vehicle Detection Algorithms in Surveillance Video Images

Intelligent traffic video surveillance system, including vehicle inspection, vehicle tracking and license plate recognition module. The video vehicle detection in this paper is divided into four steps: video image pre-processing, background modeling and updating, foreground detection and adhesion vehicle area segmentation. After the collected video sequence is subjected to mean-value down-sampling and gray-scale conversion preprocessing, the background model is built using a mixture of Gaussian models. A sliding window is set on the time axis, and the grayscale video frame image in the sliding window is median-filtered to output a background gradient map. By merging the results of the spatial background difference between the color space and the grayscale gradient, the foreground image detected by the vehicle is obtained. Finally, the background difference is obtained in the gradient domain to obtain the foreground image to achieve vehicle detection.

VisFM: Visual Analysis of Image Feature Matchings

AbstractFeature matching is the most basic and pervasive problem in computer vision and it has become a primary component in big data analytics. Many tools have been developed for extracting and matching features in video streams and image frames. However, one of the most basic tools, that is, a tool for simply visualizing matched features for the comparison and evaluation of computer vision algorithms is not generally available, especially when dealing with a large number of matching lines. We introduce VisFM, an integrated visual analysis system for comprehending and exploring image feature matchings. VisFM presents a matching view with an intuitive line bundling to provide useful insights regarding the quality of matched features. VisFM is capable of showing a summarization of the features and matchings through group view to assist domain experts in observing the feature matching patterns from multiple perspectives. VisFM incorporates a series of interactions for exploring the feature data. We demonstrate the visual efficacy of VisFM by applying it to three scenarios. An informal expert feedback, conducted by our collaborator in computer vision, demonstrates how VisFM can be used for comparing and analysing feature matchings when the goal is to improve an image retrieval algorithm.

An Indoor Scene Recognition-Based 3D Registration Mechanism for Real-Time AR-GIS Visualization in Mobile Applications

Mobile Augmented Reality (MAR) systems are becoming ideal platforms for visualization, permitting users to better comprehend and interact with spatial information. Subsequently, this technological development, in turn, has prompted efforts to enhance mechanisms for registering virtual objects in real world contexts. Most existing AR 3D Registration techniques lack the scene recognition capabilities needed to describe accurately the positioning of virtual objects in scenes representing reality. Moreover, the application of such registration methods in indoor AR-GIS systems is further impeded by the limited capacity of these systems to detect the geometry and semantic information in indoor environments. In this paper, we propose a novel method for fusing virtual objects and indoor scenes, based on indoor scene recognition technology. To accomplish scene fusion in AR-GIS, we first detect key points in reference images. Then, we perform interior layout extraction using a Fully Connected Networks (FCN) algorithm to acquire layout coordinate points for the tracking targets. We detect and recognize the target scene in a video frame image to track targets and estimate the camera pose. In this method, virtual 3D objects are fused precisely to a real scene, according to the camera pose and the previously extracted layout coordinate points. Our results demonstrate that this approach enables accurate fusion of virtual objects with representations of real world indoor environments. Based on this fusion technique, users can better grasp virtual three-dimensional representations on an AR-GIS platform.

Automated multimodal background detection and shadow removal process using robust principal fuzzy gradient partial equation methods in intelligent transportation systems

The continuous developments of the computer technologies in the intelligent transportation systems (ITS) place an important role in the various applications such as vehicle detection, traffic control detection and so on. Hence, in this paper we use the robust principal fuzzy subtraction method and gradient partial equation method (RPFGPE) for investigating the multi-modal background detection-subtraction with illumination changes and cast shadow removal method to effectively recognise the vehicle from the video footage. These methods analyse each and every pixel present in the video frame images while analysing the background as well as cast shadow. The effectiveness of the proposed system is evaluated with the help of the experimental results which is compared with the other methods for background subtraction, cast shadow removal with different illumination changes.

Automated multimodal background detection and shadow removal process using robust principal fuzzy gradient partial equation methods in intelligent transportation systems

The continuous developments of the computer technologies in the intelligent transportation systems (ITS) place an important role in the various applications such as vehicle detection, traffic control detection and so on. Hence, in this paper we use the robust principal fuzzy subtraction method and gradient partial equation method (RPFGPE) for investigating the multi-modal background detection-subtraction with illumination changes and cast shadow removal method to effectively recognise the vehicle from the video footage. These methods analyse each and every pixel present in the video frame images while analysing the background as well as cast shadow. The effectiveness of the proposed system is evaluated with the help of the experimental results which is compared with the other methods for background subtraction, cast shadow removal with different illumination changes.

An Automatic Facial Localization Tracking Identification Biometric System through PCA and Wavelet Distribution in 3D Mesh Environment

Background/Objectives: In this paper, the process of automatically facial region localization and tracking in video frames through 3D mesh model is estimated. Methods/Statistical Analysis: The morphological regions of the face are modeled into 18 geometry based regions based on the various shape and expressions. The feature is estimated based on the covariance matrix in high region space. Then, it undergoes to PCA to estimate facial deformations. Findings: The patterns feature is extracted and it mapped with the multi geometry mapping. Then, the corresponding wavelet transforms extracted region into various dimension for geometry matching for classification. The proposed method achieves robustness based on its accurate transformation through wavelet analysis. The XM2VTSDB multi-modal face database is used to compare the real video sequence images. Application/Improvements: The experimental evaluation shows favorable results and yields 99% detection rate over 15000 video frame images. The frames in the facial tracking data are calculated through various parameters and compared with the ground truth against standard Euler angles of the muti-modal database.

Hybrid human detection and recognition in surveillance

In this paper, we present a hybrid human recognition system for surveillance. A Cascade Head–Shoulder Detector (CHSD) with human body model is proposed to find the face region in a surveillance video frame image. The CHSD is a chain of rejecters which combines the advantages of Haar-like feature and HoG feature to make the detector more efficient and effective. For human recognition, we introduce an Overlapping Local Phase Feature (OLPF) to describe the face region, which can improve the robustness to pose change and blurring. To well model the variations of faces, an Adaptive Gaussian Mixture Model (AGMM) is presented to describe the distributions of the face images. Since AGMM does not need the facial topology, the proposed method is resistant to face detection error caused by imperfect localization or misalignment. Experimental results demonstrate the effectiveness of the proposed method in public dataset as well as real surveillance video.

Extracting Graphics Information for Better Video Compression

Cloud gaming services are heavily dependent on the efficiency of real-time video streaming technology owing to the limited bandwidths of wire or wireless networks through which consecutive frame images are delivered to gamers. Video compression algorithms typically take advantage of similarities among video frame images or in a single video frame image. This paper presents a method for computing and extracting both graphics information and an object's boundary from consecutive frame images of a game application. The method will allow video compression algorithms to determine the positions and sizes of similar image blocks, which in turn, will help achieve better video compression ratios. The proposed method can be easily implemented using function call interception, a programmable graphics pipeline, and off-screen rendering. It is implemented using the most widely used Direct3D API and applied to a well-known sample application to verify its feasibility and analyze its performance. The proposed method computes various kinds of graphics information with minimal overhead.

SU‐E‐J‐195: Evaluation of a Camera‐Based External Patient Position Tracking System

Purpose:To evaluate the accuracy of a camera‐based patient external position monitoring system optimized for localized patient movements.Methods:A prototype software application implements a template‐based tracking algorithm. The application generates templates located at strategic points on the patient's anatomy as seen in an acquired initial video image frame. For subsequent frames, the template pattern is matched in a userdefined region around those points using cross‐correlation, and the match is used to determine the local motion at those points.A motion stage was used to move a fabric sample (representing patient's clothing) to various positions around a 20 by 20 mm space. An optical camera, at 1.5 m distance, monitored the movement of the sample. At each sample position, the software application evaluated the displacement from initial position based on the camera image; results were compared to the known motion stage displacements.A motorized breathing manikin was set up on a treatment machine couch. The same optical camera was positioned so as to monitor both breathing and random motion of the manikin in simulation of a clinical treatment scenario. Setups for both torso tracking (respiratory) and face tracking (stereotactic head) conditions were simulated and measured for precision and accuracy.Results:On the motion stage, calibrated tracking accuracy and precision were both consistently better than 0.5 mm in both vertical and lateral axes. Similar results were achieved for the manikin motion in the clinical simulations.Conclusion:Localized camera‐based tracking is capable of monitoring external changes in patient position to submillimeter level. While not able to track internal tumor motion directly, such a system would nonetheless be able to detect tiny movements of the body, bumps, and respiratory motion, any of which may signal a need to check the position of the tumor.All work funded by Varian Medical Systems Inc.