Monocular Video Camera Research Articles

ANALYSIS OF METHODS FOR DISTANCE ESTIMATION TO AN OBJECT FROM A SINGLE VIDEO CAMERA IMAGE USING NEURAL NETWORKS

This paper discusses about any approaches to determining the distance to an object based on an image generated by a monocular video camera, which use artificial neural networks at various stages of processing. Method based on finding a depth map, detecting an object, and then projecting its coordinates onto the depth map is analyzed. It describes a method that uses the relationship between the real size of an object and its size in the image. It considers a method based on a modification of the YOLO, which allows expanding the resulting descriptor with an additional vector characterizing the distance to the object. Data sets used to train neural networks used in algorithms for calculating the absolute distance to an object based on an image is analyzed. The paper discusses about the effectiveness of the methods considered, their advantages and disadvantages, as well as the prospects for using them for practical solutions.

Passive Visual Odometry of Mobile Transport Robots in Production Areas Using Anchor Points

The article considers the problem of determining the orientation of mobile transport robots, which are now actively used in almost all areas of industry. They make it possible to intensify production, free personnel from performing routine operations, and exclude the influence of the human factor from work. Based on the analysis of various variants of active orientation methods involving the use of binocular video systems installed on the vehicles themselves, it is proposed to use a passive orientation detection system. It includes a single stationary monocular video system, the replacement of onboard video systems on mobile robots with simple radiation sensors, as well as the use of anchor points in the area of their movement. The order of choice of anchor points in solving the problem of orientation of a mobile transport robot using a monocular video system, in particular for a rectangular interior space, is considered. Calculation formulas for determining the coordinates of the sensor from the pixel coordinates of its image obtained from a monocular video camera are determined. The general sequence of actions for determining the orientation of a mobile robotic platform is also considered. Unlike active systems, this method makes it possible to significantly simplify the hardware, significantly facilitate the analysis of the current position of the mobile robot and thereby reduce the computational complexity of calculations due to the fact that there is no need to use a complex mathematical apparatus. It is being replaced by simpler two-dimensional geometric calculations. This approach, due to the unified management, makes it possible to effectively coordinate the actions of a group of mobile transport robots when they are used together, greatly simplifies the solution of a number of tasks for optimizing the intra-shop movement of mobile vehicles.

Algorithms for the visual analysis of an environment by an autonomous mobile robot for area cleanup

Objectives. At present, increasing rates of pollution of vast areas by various types of household waste are becoming an increasingly serious problem. In this connection, the creation of a robotic complex capable of performing autonomous litter collection functions becomes an urgent need. One of the key components of such a complex comprises a vision system for detecting and interacting with target objects. The purpose of this work is to develop the underlying algorithmics for the vision system of robots executing area cleaning functions.Methods. Within the framework ofthe proposed structure ofthe system for visual analysis ofthe external environment, algorithms for detecting and classifying objects of various appearance have been developed using convolutional neural networks. The neural network detector was set up by gradient descent on the open dataset of TACO training samples. To determine the geometric parameters of a surface in the field of view of the robot and estimate the coordinates of objects on the ground, a homography matrix was formed to take into account information about the characteristics and location of the video camera.Results. The developed software and algorithms for a mobile robot equipped with a monocular video camera are capable of implementing the functions of neural network detection and classification of litter objects in the frame, as well as projection of found objects on a terrain map for their subsequent collection.Conclusions. Experimental studies have shown that the developed system of visual analysis of the external environment of an autonomous mobile robot has sufficient efficiency to solve the tasks of detecting litter in the field of view of an autonomous mobile robot.

Vehicle detection systems for intelligent driving using deep convolutional neural networks

In the paper, a vision-based vehicle identification system is proposed for autonomous intelligent car driving. The accurate detection of obstacles (vehicles) during intelligent car driving allows avoiding crashes, preventing accidents, saving people’s lives and reducing harm. The vehicle detection system, which uses low-quality images captured by a monocular video camera mounted at the front of the car, is based on convolutional neural networks (CNN). The CNN can extract global features of the images using convolutional layers and achieves more accurate, and faithful contours of vehicles. The CNN structure proposed in the paper provides high-accuracy detection of vehicle images. The experiments that have been performed using GTI dataset demonstrate that the CNN-based vehicle detection system achieves very accurate results and is more robust to different variations of images.

Comparison of Tracking Approaches Distribution of Attention to Operators with Use of Oculographic Interfaces

The results of a study of two systems for monitoring the attention of the operator, based on a wearable oculointerface using infrared sensors to predict the deviation of the operators pupil, as well as using remote video oculography, which predicts the direction of the operators gaze from the image from a stationary monocular video camera, are presented. It is shown that in the course of the experiments, the wearable oculointerface had a lower average recognition error than the remote oculography interface, however, due to the design features, it is less convenient for the end user.

Feature Based Techniques for a Face Recognition using Supervised Learning Algorithms based on Fixed Monocular Video Camera

Automatic face recognition has ample significance in biometric research. Recent decades have witnessed enormous growth in this research area. Face-based identification is always considered more expedient as compared to other biometric authentications owing to its uniqueness and wide acceptance. The major contribution of this work is twofold; firstly, it comprises an extension of manual thresholding feature-based face recognition approach to an automatic feature-based supervised learning face recognition. Secondly, various new feature sets are proposed and tested on several classifiers for 2, 3, 4, and 5 persons. In addition, the use of slope features of facial components, such as the nose, right eye, left eye, and lips along with other conventional features for face recognition is also a unique contribution of this research. Multiple experiments were performed on the UMT face database. The results demonstrated a comparison of 5 different sets of feature-based approaches on 7 classifiers using the metrics of time efficiency and accuracy. They also depicted that the proposed approaches achieve a percentage accuracy of up to 95.5%.

New Ontology structure for intelligent controlling of traffic signals

This article proposes a novel ontology design for intelligent controlling of traffic signals, considering the investigated factors, crowded factors, road factors, visibility conditions, and emergency situations. Essentially, the proposed method uses video-based knowledge and key feature from a monocular video camera only, capturing footage from either a traffic signals perspective or the top of the road lane. The key factors and entities in the traffic scene are formed into an ontology, which has been evaluated using synthetic datasets to interpret challenging cases. Semantic features related to the key factors in the scene are obtained and fed to the ontology. The experimental results indicate that the proposed method is capable of controlling traffic signals more efficiently than the fixed intervals protocol.

Development of an Automatic Lawnmower with Real-Time Computer Vision for Obstacle Avoidance

This work develops an automatic lawnmower (Auto-Lawnmower) using computer vision technology for obstacle avoidance. Several critical issues have been overcome in this development work, including the development of a simplified convolutional neural network (CNN) for decision making, and sufficiently large datasets needed to train the Auto-Lawnmower. The following key strategies are adapted to ensure necessary functionality and efficacy with minimum cost, considering possible mass production of Auto-Lawnmowers. First, we use at-time avoidance strategy: meaning the Auto-Lawnmower makes turns when it encounters an obstacle. This is possible because of the fact that a lawn mower usually can move at a very low speed (walking speed of a man). Second, we decided to use minimum necessary sensors, so as to make the system of our Auto-Lawnmower as simple as possible and hence can be very practical. A single monocular video camera is, therefore, used as the single sensor for both: (1) to generate real-time movie images of a large number of situations that the Auto-Lawnmower may encounter, which is labelled and builds up the training datasets; (2) to capture the real-time image when the trained Auto-Lawnmower is in action, to decide on either to avoid the obstacle or move forward while cutting the grass. A Law dataset with labels, called LawNet, has been established for training of CNNs. It collects a total of 168,542 labelled images taken from the perspective of our Auto-Lawnmower for lawns at the university campus. A concise CNN is created for high efficiency and trained with our LawNet to drive the Auto-Lawnmower. A prototype of Auto-Lawnmower is finally designed, built and tested for lawn mowing in real situations. It is found that it works well as designed, with minimum sensors (a single camera), and hence it has a good potential for mass production.

Time- and Resource-Efficient Time-to-Collision Forecasting for Indoor Pedestrian Obstacles Avoidance.

As difficult vision-based tasks like object detection and monocular depth estimation are making their way in real-time applications and as more light weighted solutions for autonomous vehicles navigation systems are emerging, obstacle detection and collision prediction are two very challenging tasks for small embedded devices like drones. We propose a novel light weighted and time-efficient vision-based solution to predict Time-to-Collision from a monocular video camera embedded in a smartglasses device as a module of a navigation system for visually impaired pedestrians. It consists of two modules: a static data extractor made of a convolutional neural network to predict the obstacle position and distance and a dynamic data extractor that stacks the obstacle data from multiple frames and predicts the Time-to-Collision with a simple fully connected neural network. This paper focuses on the Time-to-Collision network’s ability to adapt to new sceneries with different types of obstacles with supervised learning.

MulayCap: Multi-Layer Human Performance Capture Using a Monocular Video Camera.

We introduce MulayCap, a novel human performance capture method using a monocular video camera without the need for pre-scanning. The method uses "multi-layer" representations for geometry reconstruction and texture rendering, respectively. For geometry reconstruction, we decompose the clothed human into multiple geometry layers, namely a body mesh layer and a garment piece layer. The key technique behind is a Garment-from-Video (GfV) method for optimizing the garment shape and reconstructing the dynamic cloth to fit the input video sequence, based on a cloth simulation model which is effectively solved with gradient descent. For texture rendering, we decompose each input image frame into a shading layer and an albedo layer, and propose a method for fusing a fixed albedo map and solving for detailed garment geometry using the shading layer. Compared with existing single view human performance capture systems, our "multi-layer" approach bypasses the tedious and time consuming scanning step for obtaining a human specific mesh template. Experimental results demonstrate that MulayCap produces realistic rendering of dynamically changing details that has not been achieved in any previous monocular video camera systems. Benefiting from its fully semantic modeling, MulayCap can be applied to various important editing applications, such as cloth editing, re-targeting, relighting, and AR applications.

Feature Based Techniques for a Driver's Distraction Detection using Supervised Learning Algorithms based on Fixed Monocular Video Camera

Most of the accidents occur due to drowsiness while driving, avoiding road signs and due to driver's distraction. Driver's distraction depends on various factors which include talking with passengers while driving, mood disorder, nervousness, anger, over-excitement, anxiety, loud music, illness, fatigue and different driver’s head rotations due to change in yaw, pitch and roll angle. The contribution of this paper is two-fold. Firstly, a data set is generated for conducting different experiments on driver’s distraction. Secondly, novel approaches are presented that use features based on facial points; especially the features computed using motion vectors and interpolation to detect a special type of driver’s distraction, i.e., driver’s head rotation due to change in yaw angle. These facial points are detected by Active Shape Model (ASM) and Boosted Regression with Markov Networks (BoRMaN). Various types of classifiers are trained and tested on different frames to decide about a driver's distraction. These approaches are also scale invariant. The results show that the approach that uses the novel ideas of motion vectors and interpolation outperforms other approaches in detection of driver’s head rotation. We are able to achieve a percentage accuracy of 98.45 using Neural Network.

Real-time high-fidelity facial performance capture

We present the first real-time high-fidelity facial capture method. The core idea is to enhance a global real-time face tracker, which provides a low-resolution face mesh, with local regressors that add in medium-scale details, such as expression wrinkles. Our main observation is that although wrinkles appear in different scales and at different locations on the face, they are locally very self-similar and their visual appearance is a direct consequence of their local shape. We therefore train local regressors from high-resolution capture data in order to predict the local geometry from local appearance at runtime. We propose an automatic way to detect and align the local patches required to train the regressors and run them efficiently in real-time. Our formulation is particularly designed to enhance the low-resolution global tracker with exactly the missing expression frequencies, avoiding superimposing spatial frequencies in the result. Our system is generic and can be applied to any real-time tracker that uses a global prior, e.g. blend-shapes. Once trained, our online capture approach can be applied to any new user without additional training, resulting in high-fidelity facial performance reconstruction with person-specific wrinkle details from a monocular video camera in real-time.

Ontology-based Framework for Risk Assessment in Road Scenes Using Videos

Recent advances in autonomous vehicle technology pose an important problem of automatic risk assessment in road scenes. This article addresses the problem by proposing a novel ontology tool for assessment of risk in unpredictable road traffic environment, as it does not assume that the road users always obey the traffic rules. A framework for video-based assessment of the risk in a road scene encompassing the above ontology is also presented in the paper. The framework uses as input the video from a monocular video camera only, avoiding the need for additional sometimes expensive sensors. The key entities in the road scene (vehicles, pedestrians, environment objects etc.) are organised into an ontology which encodes their hierarchy, relations and interactions. The ontology tool infers the degree of risk in a given scene using as knowledge video-based features, related to the key entities. The evaluation of the proposed framework focuses on scenarios in which risk results from pedestrian behaviour. A dataset consisting of real-world videos illustrating pedestrian movement is built. Features related to the key entities in the road scene are extracted and fed to the ontology, which evaluates the degree of risk in the scene. The experimental results indicate that the proposed framework is capable of assessing risk resulting from pedestrian behaviour in various road scenes accurately.

3D Head pose estimation and camera mouse implementation using a monocular video camera

In this paper, we present a novel approach to estimate 3D head pose using a monocular video camera for the control of mouse pointer and generating clicking events. Our approach proceeds in four stages. First, the face area is detected using Haar-like features and AdaBoost algorithm. Second, the point features are extracted and tracked over video frames by KLT algorithm. Third, by employing the tracked point features and 2D motion model of the face area, we estimate the 3D rotation matrix and translation vector between web camera and the head position. Finally, the 3D rotation matrix and translation vector are employed to calculate the mouse pointer location on the PC screen and generating clicking events. Furthermore, we propose eye wink detection as an alternative for clicking event implementation.

Eyegaze Detection from Monocular Camera Image for Eyegaze Communication System

An eyegaze interface is one of the key technologies as an input device in the ubiquitous-computing society. In particular, an eyegaze communication system is very important and useful for severely handicapped users such as quadriplegic patients. Most of the conventional eyegaze tracking algorithms require specific light sources, equipment and devices. In this study, a simple eyegaze detection algorithm is proposed using a single monocular video camera. The proposed algorithm works under the condition of fixed head pose, but slight movement of the face is accepted. In our system, we assume that all users have the same eyeball size based on physiological eyeball models. However, we succeed to calibrate the physiologic movement of the eyeball center depending on the gazing direction by approximating it as a change in the eyeball radius. In the gaze detection stage, the iris is extracted from a captured face frame by using the Hough transform. Then, the eyegaze angle is derived by calculating the Euclidean distance of the iris centers between the extracted frame and a reference frame captured in the calibration process. We apply our system to an eyegaze communication interface, and verified the performance through key typing experiments with a visual keyboard on display.

Methods for updating the environment's geometric database in telerobotics

Abstract The paper deals with tools a human operator (HO) can use to complete, on-line, the 3D geometric database of the remote environment, even in case of bad condition of vision. The HO provides the system with the 3D geometric primitive of the object's model and the 2D data from the video image. The sensing system: a monocular video camera and a time of flight IR range finder, provides the system with figures relating to the 3D primitives (location, pose, size). Methods for location and pose determination of known objects and for modelling of unknown ones are presented and assessed. These methods are available for polyhedral and cylinder shaped objects, which represent the bases of most of industrial environments.