Street-level Images Research Articles

Detection, classification, and mapping of U.S. traffic signs using google street view images for roadway inventory management

Abstract Background Maintaining an up-to-date record of the number, type, location, and condition of high-quantity low-cost roadway assets such as traffic signs is critical to transportation inventory management systems. While, databases such as Google Street View contain street-level images of all traffic signs and are updated regularly, their potential for creating an inventory databases has not been fully explored. The key benefit of such databases is that once traffic signs are detected, their geographic coordinates can also be derived and visualized within the same platform. Methods By leveraging Google Street View images, this paper presents a new system for creating inventories of traffic signs. Using computer vision method, traffic signs are detected and classified into four categories of regulatory, warning, stop, and yield signs by processing images extracted from Google Street View API. Considering the discriminative classification scores from all images that see a sign, the most probable location of each traffic sign is derived and shown on the Google Maps using a dynamic heat map. A data card containing information about location and type of each detected traffic sign is also created. Finally, several data mining interfaces are introduced that allow for better management of the traffic sign inventories. Results The experiments conducted on 6.2 miles of I-57 and I-74 interstate highways in the U.S. –with an average accuracy of 94.63 % for sign classification– show the potential of the method to provide quick, inexpensive, and automatic access to asset inventory information. Conclusions Given the reliability in performance shown through experiments and because collecting information from Google Street View imagery is cost-effective, the proposed method has potential to deliver inventory information on traffic signs in a timely fashion and tie into the existing DOT inventory management systems. Such spatio-temporal representations provide DOTs with information on how different types of traffic signs degrade over time and further provides useful condition information necessary for predicting sign replacement plan.

Design and evaluation of a low-cost street-level image capturing vehicle for south-east Asia

Purpose – The purpose of this paper is to present a low-cost, highly mobile system for performing street-level imaging. Street-level imaging and geo-location-based services are rapidly growing in both popularity and coverage. Google Street View and Bing StreetSide are two of the free, online services which allow users to search location-based information on interactive maps. In addition, these services also provide software developers and researchers a rich source of street-level images for different purposes – from identifying traffic routes to augmented reality applications. Currently, coverage for Street View and StreetSide is limited to more affluent Western countries with sparse coverage throughout south-east Asia and Africa. In this paper, we present a low-cost system to perform street-level imaging targeted towards the congested, motorcycle-dominant south-east Asian countries. The proposed system uses a catadioptric imaging system to capture 360-degree panoramic images which are geo-located using an on-board GPS. The system is mounted on the back of a motorcycle to provide maximum mobility and access to narrow roads. An innovative backwards remapping technique for flattening the images is discussed along with some results from the first 150 km which have been captured from Southern Vietnam. Design/methodology/approach – The design was a low-cost prototype design using low-cost off-the-shelf hardware with custom software and assembly to facilitate functionality. Findings – The system was shown to work well as a low-cost omnidirectional mapping solution targeted toward sea-of-motorbike road conditions. Research limitations/implications – Some of the pictures returned by the system were unclear. These could be improved by having artificial lighting (currently only ambient light is used), a gyroscope-stabilised imaging platform and a higher resolution camera. Originality/value – This paper discusses a design which facilitates low-cost, street-level imaging for a sea-of-motorcycle environment. The system uses a catadioptric imaging approach to give a wide field of view without excessive image storage requirements using dozens of cameras.

A CONTENT DISCLOSURE AND TEXT IDENTIFICATION BASED ON ROAD-LEVEL SYMBOLISM USING TCH ALGORITHM

Image enhancement is a process of improving the quality of image by improving its feature. Image contrast enrichment techniques have been largely studied in the past decades. Traffic sign disclosure and realization has been fully prepared for a great past. Traffic board detection and text recognition still leaving a protest in computer view due to its various group and the vast changeability of the data illustrated in them. The essential function can be to make an automated index of the traffic panels placed in a road to holding its care and to aid drivers. Then the figure are defined as a “bag of visual words” and restricted to applying support vector machines. Completely our own text detection and recognition method is tested on those images where a traffic board has been disclosure in system to naturally read and save the data details in the panels. Leading driver aid scheme could also use from text recognition for automated traffic signs and panels description. This visual appearance categorization method is a new approach for traffic panel detection in the state of the art. We propose a language model partly based on a dynamic dictionary for a limited geographical area using a reverse geocoding service. Experimental results on real images from Google Street View prove the efficiency of the proposed method and give way to using street-level images for different applications on ITS.

A CONTENT DISCLOSURE AND TEXT IDENTIFICATION BASED ON ROAD-LEVEL SYMBOLISM USING TCH ALGORITHM

Image enhancement is a process of improving the quality of image by improving its feature. Image contrast enrichment techniques have been largely studied in the past decades. Traffic sign disclosure and realization has been fully prepared for a great past. Traffic board detection and text recognition still leaving a protest in computer view due to its various group and the vast changeability of the data illustrated in them. The essential function can be to make an automated index of the traffic panels placed in a road to holding its care and to aid drivers. Then the figure are defined as a “bag of visual words” and restricted to applying support vector machines. Completely our own text detection and recognition method is tested on those images where a traffic board has been disclosure in system to naturally read and save the data details in the panels. Leading driver aid scheme could also use from text recognition for automated traffic signs and panels description. This visual appearance categorization method is a new approach for traffic panel detection in the state of the art. We propose a language model partly based on a dynamic dictionary for a limited geographical area using a reverse geocoding service. Experimental results on real images from Google Street View prove the efficiency of the proposed method and give way to using street-level images for different applications on ITS.

City Forensics: Using Visual Elements to Predict Non-Visual City Attributes.

We present a method for automatically identifying and validating predictive relationships between the visual appearance of a city and its non-visual attributes (e.g. crime statistics, housing prices, population density etc.). Given a set of street-level images and (location, city-attribute-value) pairs of measurements, we first identify visual elements in the images that are discriminative of the attribute. We then train a predictor by learning a set of weights over these elements using non-linear Support Vector Regression. To perform these operations efficiently, we implement a scalable distributed processing framework that speeds up the main computational bottleneck (extracting visual elements) by an order of magnitude. This speedup allows us to investigate a variety of city attributes across 6 different American cities. We find that indeed there is a predictive relationship between visual elements and a number of city attributes including violent crime rates, theft rates, housing prices, population density, tree presence, graffiti presence, and the perception of danger. We also test human performance for predicting theft based on street-level images and show that our predictor outperforms this baseline with 33% higher accuracy on average. Finally, we present three prototype applications that use our system to (1) define the visual boundary of city neighborhoods, (2) generate walking directions that avoid or seek out exposure to city attributes, and (3) validate user-specified visual elements for prediction.

Exploiting street-level panoramic images for large-scale automated surveying of traffic signs

Accurate and up-to-date inventories of traffic signs contribute to efficient road maintenance and a high road safety. This paper describes a system for the automated surveying of road signs from street-level images. This is an extremely challenging task, as the involved capturings are non-densely sampled, captured under a wide range of weather conditions and signs may be distorted. The described system is designed in a generic and learning-based fashion, which enables the recognition of different sign appearance classes with the same algorithms, based on class-specific training data. The system starts with detection of the signs visible within each image, using a detection cascade. Next, the 3D position of the signs that are detected consequently within consecutive capturings is calculated. Afterwards, each positioned road sign is classified to retrieve its sign type, thereby exploiting all detections used during positioning of the respective sign. The presented system is intended for large-scale application and currently supports 11 sign appearance classes, containing 176 different sign types. Performance evaluations conducted on a large, real-world dataset (68,010 images) show that our approach accurately positions 95.5 % of the 3,385 present signs, where 96.3 % of them are also correctly classified. Furthermore, our system localized 98.5 % of the signs in at least a single image. Our system design allows for appending a limited manual correction stage to attain a very high performance, so that sign inventories can be created cost effectively.

AUTOMATED EXTRACTION OF PHOTOREALISTIC FACADE TEXTURES FROM SINGLE GROUND-LEVEL BUILDING IMAGES

An integrated approach is presented for the automatic extraction of photorealistic facade textures from single street-level building images. The initial facade texture is extracted using Watershed segmentation. The seed pixels (markers) to trigger the segmentation are located automatically both for the foreground (facade) and the background regions, and the segmentation is carried out repetitively until the facade texture is extracted. The extracted facade image is geometrically rectified using a developed automatic technique based on Hough transformation and interest point detection. The occluded areas on facade textures are restored by employing an image matching-based procedure. The approach was tested on two different datasets captured from the residential areas of Ankara, the capital of Turkey. The datasets contain a total of 40 building facade images that were taken from the street-level. The results indicate that the facade textures are extracted adequately. For facade image extraction, an average quantitative accuracy of 83% was achieved. For rectification, 24 out of 40 buildings provided the positional error under 10 pixels at 95% confidence level. The subjective assessment of facade restoration yielded the mean rating value of 2.46 for the datasets used, in which the rating values are ranked between 1 for "Excellent" and 6 for "Unusable".

Text Detection and Recognition on Traffic Panels From Street-Level Imagery Using Visual Appearance

Traffic sign detection and recognition has been thoroughly studied for a long time. However, traffic panel detection and recognition still remains a challenge in computer vision due to its different types and the huge variability of the information depicted in them. This paper presents a method to detect traffic panels in street-level images and to recognize the information contained on them, as an application to intelligent transportation systems (ITS). The main purpose can be to make an automatic inventory of the traffic panels located in a road to support road maintenance and to assist drivers. Our proposal extracts local descriptors at some interest keypoints after applying blue and white color segmentation. Then, images are represented as a “bag of visual words” and classified using Naïve Bayes or support vector machines. This visual appearance categorization method is a new approach for traffic panel detection in the state of the art. Finally, our own text detection and recognition method is applied on those images where a traffic panel has been detected, in order to automatically read and save the information depicted in the panels. We propose a language model partly based on a dynamic dictionary for a limited geographical area using a reverse geocoding service. Experimental results on real images from Google Street View prove the efficiency of the proposed method and give way to using street-level images for different applications on ITS.

Text detection in street level images

Text detection system for natural images is a very challenging task in Computer Vision. Image acquisition introduces distortion in terms of perspective, blurring, illumination, and characters which may have very different shape, size, and color. We introduce in this article a full text detection scheme. Our architecture is based on a new process to combine a hypothesis generation step to get potential boxes of text and a hypothesis validation step to filter false detections. The hypothesis generation process relies on a new efficient segmentation method based on a morphological operator. Regions are then filtered and classified using shape descriptors based on Fourier, Pseudo Zernike moments and an original polar descriptor, which is invariant to rotation. Classification process relies on three SVM classifiers combined in a late fusion scheme. Detected characters are finally grouped to generate our text box hypotheses. Validation step is based on a global SVM classification of the box content using dedicated descriptors adapted from the HOG approach. Results on the well-known ICDAR database are reported showing that our method is competitive. Evaluation protocol and metrics are deeply discussed and results on a very challenging street-level database are also proposed.

Spatial video data collection in a post-disaster landscape: The Tuscaloosa Tornado of April 27th 2011

Fine scale disaster response and recovery data suitable for spatial analysis are still relatively rare. This is unfortunate as insight into spatial patterns of recovery could be invaluable in predicting the reestablishment of homes, streets and neighborhoods. The purpose of this paper is to show how fine scale geographic data can be collected in near real-time for the intermediate phase between response and recovery. These data will initially be used to assess the degree of damage (with relation to the Enhanced F scale) while also establishing a baseline for subsequent recovery monitoring. A spatial video system is used to collect data from the post-disaster landscape of Tuscaloosa which was hit by a large tornado in April 2011. This video, once processed, can be viewed within a Geographic Information System which combines street-level images with exact location. These data can be used to support ongoing recovery efforts, while also archiving a dataset suitable for the spatial analysis of the changing post-disaster landscape.

Taking Online Maps Down to Street Level

Since MapQuest.com's creation in 1996, online mapping systems have rapidly gained worldwide popularity. With Google Maps leading the charge, new sophisticated AJAX (Asynchronous JavaScript and XML) mapping applications began appearing in 2004. A fairly recent offering of the best online mapping systems is aerial imagery. Such imagery takes the so-called GeoWeb to the next level, providing tremendous value to activities such as real estate sales, insurance, environmental management, municipal government, emergency services, and law enforcement, as well as engaging serendipitous Web users. Yet the imagery often is not detailed enough, and viewing buildings and streets from above can be disconcerting. To address this limitation, Google launched the Street View feature of Google Maps in May 2007. The underlying idea is very simple: Provide an interface that can display street-level images in a natural way that enables convenient navigation between images without losing the map context.