Street-level Imagery Research Articles

Street-level: Google Street View’s abstraction by datafication

While aerial photography is associated with vertical objectivity and spatial abstractions, street-level imagery appears less political in its orientation to the particularities of place. I contest this assumption, showing how the aggregation of street-level imagery into “big datasets” allows for the algorithmic sorting of places by their street-level visual qualities. This occurs through an abstraction by “datafication,” inscribing new power geometries onto urban places through algorithmic linkages between visual environmental qualities, geographic information, and valuations of social worth and risk. Though largely missing from media studies of Google Street View, similar issues have been raised in critiques of criminological theories that use place as a proxy for risk. Comparing the Broken Windows theory of criminogenesis with big data applications of street-level imagery informs a critical media studies approach to Google Street View. The final section of this article suggests alternative theoretical orientations for algorithm design that avoid the pitfalls of essentialist equations of place with social character.

Image-Based Mobile Mapping for 3D Urban Data Capture

Ongoing innovations in dense multi-view stereo image matching meanwhile allow for 3D data collection using image sequences captured from mobile mapping platforms even in complex and densely built-up areas. However, the extraction of dense and precise 3D point clouds from such street-level imagery presumes high quality georeferencing as a first processing step. While standard direct georeferencing solves this task in open areas, poor GNSS coverage in densely built-up areas and urban canyons frequently prevents sufficient accuracy and reliability. Thus, we use bundle block adjustment, which additionally integrates tie and control point information for precise georeferencing of our multi-camera mobile mapping system. Subsequently, this allows the adaption of a state-of-the-art dense image matching pipeline to provide a suitable 3D representation of the captured urban structures. In addition to the presentation of different processing steps, this paper also provides an evaluation of the achieved image-based 3D capture in a dense urban environment.

Cities Are Physical Too: Using Computer Vision to Measure the Quality and Impact of Urban Appearance

For social scientists, developing an empirical connection between the physical appearance of a city and the behavior and health of its inhabitants has proved challenging due to a lack of data on urban appearance. Can we use computers to quantify urban appearance from street-level imagery? We describe Streetscore: a computer vision algorithm that measures the perceived safety of streetscapes. Using Streetscore to evaluate 19 American cities, we find that the average perceived safety has a strong positive correlation with population density and household income; and the variation in perceived safety has a strong positive correlation with income inequality.

Real Time Traffic Signboard Detection and Recognition from Street Level Imagery for Smart Vehicle

The text or symbol detection and recognition from traffic panels is a challenging problem. Number of important application areas is dependent on text detection and recognition, including advanced driver assistance systems, road surveying, and autonomous vehicles. In this research project a novel system for the automatic detection and recognition of text and symbol in traffic signs is proposed. Search regions with in the image must be defined. In this particular region locate a large number of candidates, which are then reduced by applying constraints based on temporal and structural information. This problem can be divided in two stages; First stage will be detection of region and second will be character recognition. The detection stage exploits knowledge of the structure of the scene, the size and location of the road in the frame. Once a potential traffic panels has been located, the next stage attempts to recognize text and symbols within the region. For the purpose of text detection MSER is used and for recognition purpose optical Character Recognition method is used. Automatic testing using XML files provide better accuracy.

Visual place recognition with repetitive structures.

Repeated structures such as building facades, fences or road markings often represent a significant challenge for place recognition. Repeated structures are notoriously hard for establishing correspondences using multi-view geometry. They violate the feature independence assumed in the bag-of-visual-words representation which often leads to over-counting evidence and significant degradation of retrieval performance. In this work we show that repeated structures are not a nuisance but, when appropriately represented, they form an important distinguishing feature for many places. We describe a representation of repeated structures suitable for scalable retrieval and geometric verification. The retrieval is based on robust detection of repeated image structures and a suitable modification of weights in the bag-of-visual-word model. We also demonstrate that the explicit detection of repeated patterns is beneficial for robust visual word matching for geometric verification. Place recognition results are shown on datasets of street-level imagery from Pittsburgh and San Francisco demonstrating significant gains in recognition performance compared to the standard bag-of-visual-words baseline as well as the more recently proposed burstiness weighting and Fisher vector encoding.

Cloud-Based Geospatial 3D Image Spaces—A Powerful Urban Model for the Smart City

In this paper, we introduce the concept and an implementation of geospatial 3D image spaces as new type of native urban models. 3D image spaces are based on collections of georeferenced RGB-D imagery. This imagery is typically acquired using multi-view stereo mobile mapping systems capturing dense sequences of street level imagery. Ideally, image depth information is derived using dense image matching. This delivers a very dense depth representation and ensures the spatial and temporal coherence of radiometric and depth data. This results in a high-definition WYSIWYG (“what you see is what you get”) urban model, which is intuitive to interpret and easy to interact with, and which provides powerful augmentation and 3D measuring capabilities. Furthermore, we present a scalable cloud-based framework for generating 3D image spaces of entire cities or states and a client architecture for their web-based exploitation. The model and the framework strongly support the smart city notion of efficiently connecting the urban environment and its processes with experts and citizens alike. In the paper we particularly investigate quality aspects of the urban model, namely the obtainable georeferencing accuracy and the quality of the depth map extraction. We show that our image-based georeferencing approach is capable of improving the original direct georeferencing accuracy by an order of magnitude and that the presented new multi-image matching approach is capable of providing high accuracies along with a significantly improved completeness of the depth maps.

Text Detection and Recognition on Traffic Panels From Street-Level Imagery Using Visual Appearance

Traffic sign detection and recognition has been thoroughly studied for a long time. However, traffic panel detection and recognition still remains a challenge in computer vision due to its different types and the huge variability of the information depicted in them. This paper presents a method to detect traffic panels in street-level images and to recognize the information contained on them, as an application to intelligent transportation systems (ITS). The main purpose can be to make an automatic inventory of the traffic panels located in a road to support road maintenance and to assist drivers. Our proposal extracts local descriptors at some interest keypoints after applying blue and white color segmentation. Then, images are represented as a “bag of visual words” and classified using Naïve Bayes or support vector machines. This visual appearance categorization method is a new approach for traffic panel detection in the state of the art. Finally, our own text detection and recognition method is applied on those images where a traffic panel has been detected, in order to automatically read and save the information depicted in the panels. We propose a language model partly based on a dynamic dictionary for a limited geographical area using a reverse geocoding service. Experimental results on real images from Google Street View prove the efficiency of the proposed method and give way to using street-level images for different applications on ITS.

SLI와 벡터 지도 간 합성을 위한 대응 건물 객체 탐색에 관한 연구

실세계 거리에 대하여 풍부한 시각적 정보를 포함하고 있는 구글 스트리트뷰 등의 위치기반 파노라마 영상(Street-Level Imagery, SLI) 서비스는 타 공간정보 데이터셋과 합성을 통하여 그 활용성이 더욱 향상될 수 있다. 본 연구에서는 이러한 목적을 위해 주요 공간 객체인 건물을 대상으로 보다 정확한 정보 결합을 위하여 가시 건물 단위의 대응을 향상시키는 기법을 제안하였다. 우선, 교차로 매칭을 이용한 위치 조정으로 SLI와 벡터 지도 간에 존재하는 위치 편차를 제거한다. 그리고 건물에 대한 가시 영역을 위치 편차가 제거된 벡터 지도상에서 탐색한 후, 이를 초기 정보로 활용하여 역으로 SLI scene상에서 대응 건물 영역을 탐색하여 최적의 대응 관계를 결정한다. 실험을 통하여 단순히 위치 편차만을 제거한 경우에 비하여 대응 건물 객체들 간에 일치 정확도가 약 8% 향상되는 것을 확인하였다. 따라서 본 연구의 성과는 SLI와 벡터 지도를 연계 활용하는데 있어 유용하게 적용될 수 있을 것이다.

New POI Construction with Street-Level Imagery

New POI Construction with Street-Level Imagery

영역분할에 의한 SLI와 벡터 지도 간의 건물영역 일치도 향상

After the spatial discrepancy between SLI(Street-Level Imagery) and vector map is removed by their conflation, the corresponding building regions can be found based on SLI parameters. The building region correspondence, however, is not perfect even after the conflation. This paper aims to improve the correspondence of building regions by region splitting of an SLI. Regions are initialized by the seed lines, projection of building objects onto SLI scene. First, sky images are generated by filtering, segmentation, and sky region detection. Candidates for split lines are detected by edge detector, and then images are splitted into building regions by optimal split lines based on color difference and sky existence. The experiments demonstrated that the proposed region splitting method had improved the accuracy of building region correspondence from 83.3% to 89.7%. The result can be utilized effectively for enhancement of SLI services.

SLI(Street-level Imagery)와 2D 지도간의 합성을 위한 위치 편차 제거

Recently, web portals have been offering georeferenced SLI(Street-Level Imagery) services, such as Google Streetview. The SLI has a distinctive strength over aerial images or vector maps because it gives us the same view as we see the real world on the street. Based on the characteristic, applicability of the SLI can be increased substantially through conflation with other spatial datasets. However, spatial inconsistency between different datasets is the main reason to decrease the quality of conflation when conflating them. Therefore, this research aims to remove the spatial inconsistency to conflate an SLI with a widely used 2D vector map. The removal of the spatial inconsistency is conducted through three sub-processes of (1) road intersection matching between the SLI trace and the road layer of the vector map for detecting CPPs(Control Point Pairs), (2) inaccurate CPPs filtering by analyzing the trend of the CPPs, and (3) local alignment using accurate CPPs. In addition, we propose an evaluation method suitable for conflation result including an SLI, and verify the effect of the removal of the spatial inconsistency.

Google Street View: Capturing the World at Street Level

Street View serves millions of Google users daily with panoramic imagery captured in hundreds of cities in 20 countries across four continents. A team of Google researchers describes the technical challenges involved in capturing, processing, and serving street-level imagery on a global scale.