Combines Computer Vision Research Articles

Automated Traffic Control System using Big Data and Cognitive Analysis

Today people spend about 4.8 billion hours every year in congestion which could be used productively. The traffic control system currently being used is outdated and heavily dependent on humans. Thus, there is a dire need to upgrade and automate these systems. Using combination of computer vision, big data and machine learning it is possible to design a reliable and scalable system which will help to resolve these traditional problems. This paper provides an insight of how these technologies can be used and the challenges which we will have to face. General Terms Machine learning, big data, Computer Vision, Traffic Control.

From Image to Translation

The lack of computational support has significantly slowed down automatic understanding of endangered languages. In this paper, we take Nyushu (simplified Chinese: 女书; literally: “women’s writing”) as a case study to present the first computational approach that combines Computer Vision and Natural Language Processing techniques to deeply understand an endangered language. We developed an end-to-end system to read a scanned hand-written Nyushu article, segment it into characters, link them to standard characters, and then translate the article into Mandarin Chinese. We propose several novel methods to address the new challenges introduced by noisy input and low resources, including Nyushu-specific feature selection for character segmentation and linking, and character linking lattice based Machine Translation. The end-to-end system performance indicates that the system is a promising approach and can serve as a standard benchmark.

Accuracy of Reconstruction of the Tree Stem Surface Using Terrestrial Close-Range Photogrammetry

Airborne laser scanning (ALS) allows for extensive coverage, but the accuracy of tree detection and form can be limited. Although terrestrial laser scanning (TLS) can improve on ALS accuracy, it is rather expensive and area coverage is limited. Multi-view stereopsis (MVS) techniques combining computer vision and photogrammetry may offer some of the coverage benefits of ALS and the improved accuracy of TLS; MVS combines computer vision research and automatic analysis of digital images from common commercial digital cameras with various algorithms to reconstruct three-dimensional (3D) objects with realistic shape and appearance. Despite the relative accuracy (relative geometrical distortion) of the reconstructions available in the processing software, the absolute accuracy is uncertain and difficult to evaluate. We evaluated the data collected by a common digital camera through the processing software (Agisoft PhotoScan ©) for photogrammetry by comparing those by direct measurement of the 3D magnetic motion tracker. Our analyses indicated that the error is mostly concentrated in the portions of the tree where visibility is lower, i.e., the bottom and upper parts of the stem. For each reference point from the digitizer we determined how many cameras could view this point. With a greater number of cameras we found increasing accuracy of the measured object space point positions (as expected), with a significant positive change in the trend beyond five cameras; when more than five cameras could view this point, the accuracy began to increase more abruptly, but eight cameras or more provided no increases in accuracy. This method allows for the retrieval of larger datasets from the measurements, which could improve the accuracy of estimates of 3D structure of trees at potentially reduced costs.

Accurate and Reliable Detection of Traffic Lights Using Multiclass Learning and Multiobject Tracking

Automatic detection of traffic lights has great importance to road safety. This paper presents a novel approach that combines computer vision and machine learning techniques for accurate detection and classification of different types of traffic lights, including green and red lights both in circular and arrow forms. Initially, color extraction and blob detection are employed to locate the candidates. Subsequently, a pretrained PCA network is used as a multiclass classifier to obtain frame-by-frame results. Furthermore, an online multiobject tracking technique is applied to overcome occasional misses and a forecasting method is used to filter out false positives. Several additional optimization techniques are employed to improve the detector performance and handle the traffic light transitions. When evaluated using the test video sequences, the proposed system can successfully detect the traffic lights on the scene with high accuracy and stable results. Considering hardware acceleration, the proposed technique is ready to be integrated into advanced driver assistance systems or self-driving vehicles. We build our own data set of traffic lights from recorded driving videos, including circular lights and arrow lights in different directions. Our experimental data set is available at http://computing.wpi.edu/Dataset.html.

Application of a computer vision technique to animal-borne video data: extraction of head movement to understand sea turtles’ visual assessment of surroundings

An animal-borne video recording system has recently been developed to study the behavior of free-ranging animals. In contrast to other types of sensor data (i.e., acceleration), video images offer the advantage of directly acquiring information without analysis. However, most previous findings have only been obtained through visual observation of image data. Here, we demonstrate a new method of data analysis for animal-borne videos using a computer vision technique referred to as template matching. As a case study, we tracked the horizontal head movements of green turtles (Chelonia mydas) to investigate how they move their heads to look around the underwater environment. Template matching allowed tracking of head movements with high accuracy (0.34 ± 0.12 % and 0.52 ± 0.29 % of the root-mean-square error on the x- and y-coordinates, respectively), high true (86.2 ± 8.1 %), and low false extraction rates (6.6 ± 8.4 %). However the program sometimes failed because the turtle’s head would move out of range of the video. During cruising swimming, green turtles did not significantly move their heads to one side, moving with a ratio of 50.5:49.5 (left: right). Green turtles moved their heads from side to side more widely and more slowly before (12.0 ± 4.6 point and 0.25 ± 0.03 Hz, respectively) and after taking a breath (27.5 ± 2.9 point and 0.27 ± 0.03 Hz) compared to during cruising swimming (8.4 ± 3.8 point and 0.32 ± 0.01 Hz). Before feeding, turtles moved their heads slowly (0.23 ± 0.03 Hz) and narrowly (9.3 ± 3.6 point). Our combined approach using video and gyro loggers revealed that when making a turn, turtles always turned their heads to the side 1.38 ± 0.77 s before turning their body. Our method enables researchers to quantitatively extract information regarding vision cognition and behavioral responses in green turtles in the wild that could not otherwise be obtained from other sensors used in previous studies. This new method using a combination of computer vision and bio-logging (e.g., gyroscope) can serve as a powerful tool in animal behavior and ecological studies.

Applied-Information Technology in Abnormal Detection for Surveillance Systems

In recent years, video surveillance has become more and more important for enhanced security and it is indispensable technology for fighting against all types of crime with the construction of sky-net in China. Abnormal detection is the focus of intelligent video surveillance and the information of abnormal behavior can be used in the investigation of criminal cases, which combines computer vision and artificial intelligence technology and has wide application prospect in public security work. In this paper, first the current research situation of the intelligent surveillance system is introduced. Then the category of abnormal behavior detection is expounded. Finally the function module of abnormal detection system is designed and the key technology of moving target detection, target tracking and abnormality judgment is discussed in view of the actual situation of surveillance system in criminal cases.

Design of Pneumatical Rubik’s Cube Solver

Rubik’s solver is prime example of science popularizing mechatronic device, combining computer vision techniques, advanced Rubik’s cube solving algorithm and control of industrial pneumatic components. First version of solver constructed by our department was equipped only by 2 lower grippers, resulting in approximately four minutes for single cube solution. Proposed paper describes not only experience gained by development of upgraded 4 gripper version but includes statistical analysis of Thistlethwait’s 45 algorithm required for solving process optimization. Mechanical design, electronics, system overview, performance and limitations of upgraded 4 gripper version are explained in detail as well. Development of vision algorithm for robust cube detection is described in conjoint paper “Robust detection of Rubik’s cube state using Hough transform and advanced clustering functions.”

True-orthophoto generation from UAV images: Implementation of a combined photogrammetric and computer vision approach

Abstract. This paper presents a photogrammetric methodology for true-orthophoto generation with images acquired from UAV platforms. The method is an automated multistep workflow made up of three main parts: (i) image orientation through feature-based matching and collinearity equations / bundle block adjustment, (ii) dense matching with correlation techniques able to manage multiple images, and true-orthophoto mapping for 3D model texturing. It allows automated data processing of sparse blocks of convergent images in order to obtain a final true-orthophoto where problems such as self-occlusions, ghost effects, and multiple texture assignments are taken into consideration. The different algorithms are illustrated and discussed along with a real case study concerning the UAV flight over the Basilica di Santa Maria di Collemaggio in L'Aquila (Italy). The final result is a rigorous true-orthophoto used to inspect the roof of the Basilica, which was seriously damaged by the earthquake in 2009.

Objective definition of rosette shape variation using a combined computer vision and data mining approach.

Computer-vision based measurements of phenotypic variation have implications for crop improvement and food security because they are intrinsically objective. It should be possible therefore to use such approaches to select robust genotypes. However, plants are morphologically complex and identification of meaningful traits from automatically acquired image data is not straightforward. Bespoke algorithms can be designed to capture and/or quantitate specific features but this approach is inflexible and is not generally applicable to a wide range of traits. In this paper, we have used industry-standard computer vision techniques to extract a wide range of features from images of genetically diverse Arabidopsis rosettes growing under non-stimulated conditions, and then used statistical analysis to identify those features that provide good discrimination between ecotypes. This analysis indicates that almost all the observed shape variation can be described by 5 principal components. We describe an easily implemented pipeline including image segmentation, feature extraction and statistical analysis. This pipeline provides a cost-effective and inherently scalable method to parameterise and analyse variation in rosette shape. The acquisition of images does not require any specialised equipment and the computer routines for image processing and data analysis have been implemented using open source software. Source code for data analysis is written using the R package. The equations to calculate image descriptors have been also provided.

A Selection Algorithm of Training Set Based on Similar Classification

License Plate Recognition (LPR) combines computer vision technology andpattern recognition technology and plays an important role in Freeway TollSystem, Urban Road Monitoring System and the Intelligent Parking Lot ManagementSystem. Therefore, it has attracted an ever increasing number of scholars fromhome and abroad. Despite many years of unremitting effort which has resulted inbreakthrough achievements, it remains unsatisfactory in meeting real worldapplication requirements. LPR primarily employs pattern recognition and digitalimage process technology. This paper is focused on the study of patternrecognition. The segmented characters are trained utilizing the BP neuralnetwork. Selecting the ideal training set from the usually large sample set wehave is the first step to train a good network which has a high recognitionrate. At present, training sets are randomly selected, which affects theaccuracy of recognition as well as its speed. Thus, selecting the best trainingsets is of uttermost importance. In this paper, Similarity Comparison Samplingmethod is proposed to improve the training results.

A Fully Automated System for Adherent Cells Microinjection

This paper proposes an automated robotic system to perform cell microinjections to relieve human operators from this highly difficult and tedious manual procedure. The system, which uses commercial equipment currently found on most biomanipulation laboratories, consists of a multitask software framework combining computer vision and robotic control elements. The vision part features an injection pipette tracker and an automatic cell targeting system that is responsible for defining injection points within the contours of adherent cells in culture. The main challenge is the use of bright-field microscopy only, without the need for chemical markers normally employed to highlight the cells. Here, cells are identified and segmented using a threshold-based image processing technique working on defocused images. Fast and precise microinjection pipette positioning over the automatically defined targets is performed by a two-stage robotic system which achieves an average injection rate of 7.6 cells/min with a pipette positioning precision of 0.23μm. The consistency of these microinjections and the performance of the visual targeting framework were experimentally evaluated using two cell lines (CHO-K1 and HEK) and over 500 cells. In these trials, the cells were automatically targeted and injected with a fluorescent marker, resulting in a correct cell detection rate of 87% and a successful marker delivery rate of 67.5%. These results demonstrate that the new system is capable of better performances than expert operators, highlighting its benefits and potential for large-scale application.

Identification of a scaled-model riser dynamics through a combined computer vision and adaptive Kalman filter approach

Aiming at overcoming the difficulties derived from the traditional camera calibration methods to record the underwater environment of a towing tank where experiments of scaled-model risers are carried on, a computer vision method, combining traditional image processing algorithms and a self-calibration technique was implemented. This method was used to identify the coordinates of control-points viewed on a scaled-model riser submitted to a periodic force applied to its fairlead attachment point. To study the observed motion, the riser was represented as a pseudo-rigid body model (PRBM) and the hypotheses of compliant mechanisms theory were assumed in order to cope with its elastic behavior. The derived Lagrangian equations of motion were linearized and expressed as a state-space model in which the state variables include the generalized coordinates and the unknown generalized forces. The state-vector thus assembled is estimated through a Kalman Filter. The estimation procedure allows the determination of both the generalized forces and the tension along the cable, with statistically proven convergence.

Project-Based Learning Example: Controlling an Educational Robotic Arm With Computer Vision

This paper describes a project-based learning case example developed at the University of Lleida, Spain. This proposal was addressed to third-year engineering students and combines knowledge from computer vision and robotic control to complete an automation project task. The learning target was to develop a computer vision system that must detect a small object placed randomly on a target surface and control an educational robotic arm to pick it up and move it to a predefined destination. The educational results of this experience have shown that the development of a practical-learning case based on the combination of computer vision and robotics has increased the motivation of the students and improved the assimilation of theoretical concepts.

Learning visual object detection and localisation using icVision

Building artificial agents and robots that can act in an intelligent way is one of the main research goals in artificial intelligence and robotics. Yet it is still hard to integrate functional cognitive processes into these systems. We present a framework combining computer vision and machine learning for the learning of object recognition in humanoid robots. A biologically inspired, bottom-up architecture is introduced to facilitate visual perception and cognitive robotics research. It aims to mimic processes in the human brain performing visual cognition tasks. A number of experiments with this icVision framework are described. We showcase both detection and identification in the image plane (2D), using machine learning. In addition we show how a biologically inspired attention mechanism allows for fully autonomous learning of visual object representations. Furthermore localising the detected objects in 3D space is presented, which in turn can be used to create a model of the environment.

AN ACCURACY ASSESSMENT OF GEOREFERENCED POINT CLOUDS PRODUCED VIA MULTI-VIEW STEREO TECHNIQUES APPLIED TO IMAGERY ACQUIRED VIA UNMANNED AERIAL VEHICLE

Abstract. Low-cost Unmanned Aerial Vehicles (UAVs) are becoming viable environmental remote sensing tools. Sensor and battery technology is expanding the data capture opportunities. The UAV, as a close range remote sensing platform, can capture high resolution photography on-demand. This imagery can be used to produce dense point clouds using multi-view stereopsis techniques (MVS) combining computer vision and photogrammetry. This study examines point clouds produced using MVS techniques applied to UAV and terrestrial photography. A multi-rotor micro UAV acquired aerial imagery from a altitude of approximately 30–40 m. The point clouds produced are extremely dense (<1–3 cm point spacing) and provide a detailed record of the surface in the study area, a 70 m section of sheltered coastline in southeast Tasmania. Areas with little surface texture were not well captured, similarly, areas with complex geometry such as grass tussocks and woody scrub were not well mapped. The process fails to penetrate vegetation, but extracts very detailed terrain in unvegetated areas. Initially the point clouds are in an arbitrary coordinate system and need to be georeferenced. A Helmert transformation is applied based on matching ground control points (GCPs) identified in the point clouds to GCPs surveying with differential GPS. These point clouds can be used, alongside laser scanning and more traditional techniques, to provide very detailed and precise representations of a range of landscapes at key moments. There are many potential applications for the UAV-MVS technique, including coastal erosion and accretion monitoring, mine surveying and other environmental monitoring applications. For the generated point clouds to be used in spatial applications they need to be converted to surface models that reduce dataset size without loosing too much detail. Triangulated meshes are one option, another is Poisson Surface Reconstruction. This latter option makes use of point normal data and produces a surface representation at greater detail than previously obtainable. This study will visualise and compare the two surface representations by comparing clouds created from terrestrial MVS (T-MVS) and UAV-MVS.

Forensic-case analysis: from 3D imaging to interactive visualization.

An interactive framework prepares raw computed-tomography and magnetic-resonance-imaging scans for courtroom presentations. The framework makes use of combined computer graphics and computer vision techniques to enable a forensic case analysis workflow.

Learning Spatial Object Localization from Vision on a Humanoid Robot

We present a combined machine learning and computer vision approach for robots to localize objects. It allows our iCub humanoid to quickly learn to provide accurate 3D position estimates (in the centimetre range) of objects seen. Biologically inspired approaches, such as Artificial Neural Networks (ANN) and Genetic Programming (GP), are trained to provide these position estimates using the two cameras and the joint encoder readings. No camera calibration or explicit knowledge of the robot's kinematic model is needed. We find that ANN and GP are not just faster and have lower complexity than traditional techniques, but also learn without the need for extensive calibration procedures. In addition, the approach is localizing objects robustly, when placed in the robot's workspace at arbitrary positions, even while the robot is moving its torso, head and eyes.

A semi-interactive panorama based 3D reconstruction framework for indoor scenes

We present a semi-interactive method for 3D reconstruction specialized for indoor scenes which combines computer vision techniques with efficient interaction. We use panoramas, popularly used for visualization of indoor scenes, but clearly not able to show depth, for their great field of view, as the starting point. Exploiting user defined knowledge, in term of a rough sketch of orthogonality and parallelism in scenes, we design smart interaction techniques to semi-automatically reconstruct a scene from coarse to fine level. The framework is flexible and efficient. Users can build a coarse walls-and-floor textured model in five mouse clicks, or a detailed model showing all furniture in a couple of minutes interaction. We show results of reconstruction on four different scenes. The accuracy of the reconstructed models is quite high, around 1% error at full room scale. Thus, our framework is a good choice for applications requiring accuracy as well as application requiring a 3D impression of the scene.

A combined mixed reality and networked home approach to improving user interaction with consumer electronics

User interaction in a modern networked home can be unintuitive when utilizing a smartphone for control. Different layers of menus and the lack of a tangible real-world interaction metaphor complicates the everyday usage of those devices in a smartphone. We propose a solution focusing on mixed reality interaction, by combining computer vision and ubiquitous computing techniques, to allow the user to simply point to the device of interest, to be controlled, via the smartphone camera. This adds value to the consumer electronics ecosystem by improving networked home user interaction experiences.

An inexpensive hand-eye system for undergraduate robotics instruction

Hand-eye systems combine computer vision with kinematics and dynamics calculations to achieve dexterous manipulation. These versatile platforms for teaching robotics principles have not been widely used in undergraduate laboratories due to cost. We describe a new hand-eye system constructed from Robotis Dynamixel servos, a USB interface module, and a webcam, that can be built for under $500 and run by a PC using the Tekkotsu open source software framework. A suggested curriculum is outlined.