View Images Research Articles

How does the visual environment influence pedestrian physiological stress? Evidence from high-density cities using ambulatory technology and spatial machine learning

Revealing the phased impacts of the visual environment on physiological stress in outdoor environments can facilitate accurate and practical planning strategies towards healthy and sustainable living environments. This study adopted ambulatory sensing to monitor high spatiotemporal physiological stress from skin conductance level (SCL) and analyzed the association between the visual environment and the SCL in high-density environments. The multi-level visual environment was quantified from isovist and street view images. The most influential lags for each visual exposure factor were identified. Furthermore, multiple linear regression (MLR) and geographical random forest (GRF) were established to examine the effects of the lagged visual exposure factors on SCL. Particularly, the GRF can explore spatial nonstationary and complex relationships. The MLR (R2 = 0.466) suggests that exposure to trees, sky, and sign symbols can relieve stress. The GRF with 5 visual exposure factors can explain 87.4% of the spatial variance of SCL. This study confirms the effectiveness of monitoring pedestrians’ stress in high-density cities and the phased and non-instantaneous impacts of the visual environment. The results have practical implications in urban design at both district and site levels for a sustainable and psychologically friendly urban environment.

Scientific Visualization for Interrogation of Results in Lighting Simulation System

Lighting modeling software packages often include a tool for processing and investigation of two-dimensional images that are the results of modeling. Functionality of these tools is usually minimal one and often consists of mapping of resultant images into monitor screen only. There are general image viewers and image processing programs. But in practice they either do not contain required functionality, or are so superfluous that result postprocessing becomes inconvenient with them. Therefore we had to implement own viewer that processes high dynamic range images generated by the lighting simulation and realistic computer graphics system as well as visualizes supplementary simulation information needed for engineer analysis. Lighting simulation module transfers to our viewer the generated image in physical values together with supplementary data. These data has per pixel nature and form additional layers of images. Our image processing module provides brightness high dynamic range compression for rendering of images generated with lighting modeling methods including stochastic ones, analysis of the image generation algorithms using additional data layers, applying additional visual effects for improving images appearance, and export of processed images to commonly used formats.

(LC): LiDAR-Camera Loop Constraints for Cross-Modal Place Recognition

Localization has been a challenging task for autonomous navigation. A loop detection algorithm must overcome environmental changes for the place recognition and re-localization of robots. Therefore, deep learning has been extensively studied for the consistent transformation of measurements into localization descriptors. Street view images are easily accessible; however, images are vulnerable to appearance changes. LiDAR can robustly provide precise structural information. However, constructing a point cloud database is expensive, and point clouds exist only in limited places. Different from previous works that train networks to produce shared embedding directly between the 2D image and 3D point cloud, we transform both data into 2.5D depth images for matching. In this work, we propose a novel cross-matching method, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(LC)<inline-formula><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula></i> , for achieving LiDAR localization without a prior point cloud map. To this end, LiDAR measurements are expressed in the form of range images before matching them to reduce the modality discrepancy. Subsequently, the network is trained to extract localization descriptors from disparity and range images. Next, the best matches are employed as a loop factor in a pose graph. Using public datasets that include multiple sessions in significantly different lighting conditions, we demonstrated that LiDAR-based navigation systems could be optimized from image databases and vice versa.

Assessing the Spatial Diversity of Urban Cultural Landscape using Street View Data and Investigating Influential Factors: A Case Study of Beijing

As an open space bearing the urban context, urban cultural landscape is an important place to enhance the connotation of urban space and create the vitality of urban space. Cultural landscape is influenced by multiple factors in the process of formation and evolution, and there are distinct differences in individuation and regionalization. In this paper, with the help of Baidu Street View images, the evaluation system and measurement method are constructed from the five dimensions of architectural style, infrastructure, ecological environment, urban sketch and historical context. Taking Beijing as an example, the man-machine combination method is adopted to explore the factors affecting the spatial heterogeneity of urban cultural landscape based on GIS grid, and effective remediation measures are proposed. It has positive significance to enhance the connotation of urban culture, strengthen the comfort level of citizens and promote the deep development of urban cultural space.

Can good microscale pedestrian streetscapes enhance the benefits of macroscale accessible urban form? An automated audit approach using Google street view images

While pedestrian accessibility measures have been widely used interchangeably with walkability, walkability is not just about accessibility. Microscale streetscape factors can provide qualities that pedestrians seek, including safety from crime, traffic safety, and pleasurability. These qualities can encourage walking through direct contributions or by enhancing the benefits of macroscale accessibility measures, or both. This study examines the main and interaction effects of microscale streetscape factors measured from street view images using computer vision in Atlanta, Georgia, USA. The correlation analysis showed that many microscale factors have moderate to strong correlations with the pedestrian accessibility measure. Binary logistic regressions demonstrated that safety from crime, pleasurability, and composite microscale index have direct contributions to walking mode choice. Regarding moderating effects, traffic safety, safety from crime, and composite microscale index enhanced the benefits of pedestrian accessibility measure. Policies designed to encourage walking can focus on streetscape features for short term efforts. Important caveats for policymakers in using machine-driven data collection are discussed.

Building Façade Color Distribution, Color Harmony and Diversity in Relation to Street Functions: Using Street View Images and Deep Learning

Building façade colors play an important role in influencing urban imageability, attraction and citizens’ experience. However, the relations between street functions and the building façade color distribution, color harmony and color diversity have not been thoroughly examined. We obtained the dominant colors of building façades in Changning District, Shanghai, utilizing Baidu street view images, image semantic segmentation technology and the K-means algorithm. The variations in building façades’ dominant colors, color harmony and diversity across different types of functional streets were examined through logistic regression and ANOVA analyses. The results indicate that, compared to industrial streets, red hues are more common in science education streets, residential streets and mixed functional streets. Business streets are more likely to have hues of green, red and red–purple. Residential streets’ saturation is overall higher than that of industrial streets. In business streets, the medium–high value occurs less frequently than other streets. Moreover, we found that the street building façade colors in industrial streets were more harmonious and less diversified than that in other functional streets. This study has implications for urban color planning practices. Color harmony and color diversity should be well considered in future planning. The role of street functions should also be addressed in building façade color planning, to improve existing planning frameworks as well as related strategies.

Car make and model recognition system using rear-lamp features and convolutional neural networks

Recognizing cars based on their features is a difficult task. We propose a solution that uses a convolutional neural network (CNN) and image binarization method for car make and model classification. Unlike many previous works in this area, we use a feature extraction method combined with a binarization method. In the first stage of the pre-processing part we normalize and change the size of an image. The image is then used to recognize where the rear-lamps are placed on the image. We extract the region and use the image binarization method. The binarized image is used as input to the CNN network that finds the features of a specific car model. We have tested the combinations of three different neural network architectures and eight binarization methods. The convolutional neural network with parameters of the highest quality metrics value is used to find the characteristics of the rear lamps on the binary image. The convolutional network is tested with four different gradient algorithms. We have tested the method on two data sets which differ in the way the images were taken. Each data set consists of three subsets of the same car, but is scaled to different image dimensions. Compared to related works that are based on CNN, we use rear view images in different position and light exposure. The proposed method gives better results compared to most available methods. It is also less complex, and faster to train compared to other methods. The proposed approach achieves an average accuracy of 93,9% on the first data set and 84,5% on the second set.

Multi-scale foreground-background separation for light field depth estimation with deep convolutional networks

There are three viewpoints to divide the depth from light field (DFLF) methods: light field (LF) representation, signal processing, and learning viewpoints. In the signal processing viewpoint in details, the DFLF methods can be divided into three groups: cost-based, foreground-background separation (FBS)-based, and depth model-based methods. Although the deep learning techniques are widely implemented in the cost-based and depth model-based methods for the DFLF, they are not implemented in the FBS-based method, yet. In this paper, we propose a DFLF method based on the FBS with the deep learning techniques. Based on an observation that the angular aliasing artifacts are reduced in the resulting disparity maps obtained from spatially low-scale LF images, multi-scale (MS) FBS score maps are used in the proposed methods. To combine the MS FBS score maps into a single FBS score maps, four directional LF gradients, features from the 4-D LF, and features from the center view images are utilized. Based on the sensitivity and specificity loss, the gradient matching loss between the predicted and ground truth disparity maps is used to improve the performance around the occlusion boundary in the resulting disparity maps. The experimental results show that the proposed method performs reasonably in both the synthetic and real LF datasets. Especially, in the texture-less region in the real dataset, the resulting disparity maps obtained by the proposed method show that the artifacts are reduced compared to those by the existing DFLF methods.

Monitoring and mapping vegetation greenery in residential communities using street view images and a Normalized Vegetation Greenery Index: a case study in Beijing, China

Vegetation greenery is essential for the sensory and psychological wellbeing of residents in residential communities. To enhance the quality of regulations and policies to improve people’s living environments, it is crucial to effectively identify and monitor vegetation greenery from the perspective of the residents using effective images and methods. In this study, Baidu street view (BSV) images and a Normalized Vegetation Greenery Index (NVGI) based method were examined to distinguish vegetation greenery in residential communities of Beijing, China. The magnitude of the vegetation was quantified and graded, and spatial analysis techniques were employed to investigate the spatial characteristics of vegetation greenery. The results demonstrated that (1) the identified vegetation greenery using the proposed NVGI-based method was closely correlated with those of the reference classification (r = 0.993, p = 0.000), surpassing the comparison results from the SVM method, a conventional remote sensing classification means; (2) the vegetation greenery was distributed unevenly in residential communities and can be categorized into four grades, 63.79% of the sampling sites were found with relatively low (Grade II) and moderate (Grade III) vegetation greenery distribution, most of the districts in the study area contained zero-value green view index sites; and (3) there was significant spatial heterogeneity observed in the study area, with low-value clustering (cold spots) predominantly located in the central region and high-value clustering (hot spots) primarily concentrated in the peripheral zone. The findings of this study can be applied in other cities and countries that have street view images available to investigate greenery patterns within residential areas, which can help improve the planning and managing efforts in urban communities.

An integrated 3D printed radioluminescent-based phantom for quality assurance on a robotic-arm linac

Objective. To develop, characterize and improve upon a high-resolution 3D printed radioluminescence-based imaging phantom for quality assurance (QA) of a robotic arm linear accelerator. Approach. A phantom was constructed which consisted of a scintillating sheet, fiducial markers, a low-cost complementary metal-oxide semiconductor camera and a 3D printed light–tight enclosure. The camera, equipped with a 12 mm lens, was angled 45 degrees from the horizontal axis with a direct line of sight of the scintillating sheet. A perspective image transformation with optical distortion correction was employed to obtain beam’s eye view images for different collimators. Beam profiles, Iris™ field size, multileaf collimator leaf positioning and central laser-radiation field coincidence QA tests were performed and compared against data obtained with gafchromic film. The phantom’s short-term stability, sensitivity to changes in output, field size and leaf positioning were also assessed. Main Results. The limiting resolution of the optical system was measured to be ∼0.26 mm. Field size, as measured by the radioluminescence system for Iris apertures, agreed to within 0.2 mm of the values measured using film. The imaging system was sensitive to field size changes well below 0.2 mm and output changes as small as 1 monitor unit (MU). For the optical setup, the mean leaf deviation error for banks X1 and X2 was 0.21 and 0.17 mm at 800 mm source to axis distance, whereas the mean difference for the film dataset was 0.16 mm and 0.22 mm for banks X1 and X2, respectively. The optical system was able to detect leaf positioning errors as small as 0.2 mm. Compared with film data, excellent agreement was seen for relative central axis beam profiles for 10 mm and 5 mm beams. Significance. The phantom presented here is an alternative to film and electronic portal imager devices, due to its low-cost, portability, and high spatial and temporal resolution.

Deep Learning for Medium-Scale Agricultural Crop Detection through Aerial View Images

This research project focuses on utilizing two state-of-the-art YOLOv4-based deep learning models, for large-scale agricultural crop detection using Unmanned Aerial Vehicles (UAVs). The objective is to develop an accurate and efficient crop detection system capable of identifying chili crops, eggplant crops, and empty polybags in agricultural fields. Crops detection is important for the development of a robotic vision in maximize the productivity and efficiency in agriculture associate with the development of concept Industry 4.0. This study seek to explore the comparison between YOLOv4 and YOLOv4 tiny model in term of mean average precision (mAP), precision, recall, F1-score, detection time and memory consumption. A custom dataset with 300 images was collected and annotated into total bounding boxes of 23335 with 6969 chili tree, 15402 eggplant tree and 964 empty polybag. The dataset was separated into train, validation and test set with the ratio of 70:20:10. The dataset was trained into YOLOv4 and YOLOv4 tiny with 2000 iterations. The result has shown that the YOLOv4 has the higher mean AP of 91.49% with 244.2mb memory storage consumption while YOLOv4 tiny achieve lower mean AP of 71.83% with 22.4mb. In summary, this research has significated the implementation of deep learning models to perform large-scale agricultural crop detection and can be further develop into automation industrial 4.0 of local agricultural sector.

Research on Campus Space Features and Visual Quality Based on Street View Images: A Case Study on the Chongshan Campus of Liaoning University

As the university campus is a place for learning, conducting scientific research, and communication, campus street spatial quality has an impact on its users. Therefore, refinement evaluations of campus spatial quality are essential for constructing high-quality campuses. In this study, machine learning was used to conduct semantic segmentation and spatial perception prediction on street view images. The physical features and perception quality of the surrounding areas of the Chongshan campus of Liaoning University were obtained. The study found that the visual beautiful quality (VBQ) of the student living area was the highest, and the VBQ of the teacher living area was the lowest when compared to the research and study area, student living area, sports area, and surrounding area. Greenness and openness had positive influences on VBQ, while enclosure had a negative influence. This study analyzed the influence mechanism operating between spatial physical features and VBQ. The results provide theoretical and technical support for campus space spatial quality construction and improvement.

Measuring changes in neighborhood disorder using Google Street View longitudinal imagery: a feasibility study

ABSTRACT Few studies have used longitudinal imagery of Google Street View (GSV) despite its potential for measuring changes in urban streetscape characteristics relevant to health, such as neighborhood disorder. Neighborhood disorder has been previously associated with health outcomes. We conducted a feasibility study exploring image availability over time in the Philadelphia metropolitan region and describing changes in neighborhood disorder in this region between 2009, 2014, and 2019. Our team audited Street View images from 192 street segments in the Philadelphia Metropolitan Region. On each segment, we measured the number of images available through time, and for locations where imagery from more than one time point was available, we collected eight neighborhood disorder indicators at three different times (up to 2009, up to 2014, and up to 2019). More than 70% of the streets segments had at least one image. Neighborhood disorder increased between 2009 and 2019. Future studies should study the determinants of change of neighborhood disorder using longitudinal GSV imagery.

Forecasting Traffic Speed during Daytime from Google Street View Images using Deep Learning

Traffic forecasting plays an important role in urban planning. Deep learning methods outperform traditional traffic flow forecasting models because of their ability to capture spatiotemporal characteristics of traffic conditions. However, these methods require high-quality historical traffic data, which can be both difficult to acquire and non-comprehensive, making it hard to predict traffic flows at the city scale. To resolve this problem, we implemented a deep learning method, SceneGCN, to forecast traffic speed at the city scale. The model involves two steps: firstly, scene features are extracted from Google Street View (GSV) images for each road segment using pretrained Resnet18 models. Then, the extracted features are entered into a graph convolutional neural network to predict traffic speed at different hours of the day. Our results show that the accuracy of the model can reach up to 86.5% and the Resnet18 model pretrained by Places365 is the best choice to extract scene features for traffic forecasting tasks. Finally, we conclude that the proposed model can predict traffic speed efficiently at the city scale and GSV images have the potential to capture information about human activities.

Left Ventricular Outflow Obstruction in D-Transposition of Great Arteries: A Rare Etiology

A 2-day old neonate [gender: male; weight: 3.2 kg; height: 53cm] presented to the authors institution with an arterial oxygen saturation of 74%. The child was receiving 6 cm H2O continuous positive airway pressure therapy [FIO2: 0.5] and was treated with a prostaglandin E1 infusion [10ng/kg/min]. Transthoracic echocardiography revealed situs solitus, levocardia, normal systemic and pulmonary venous drainage and a 6 mm non-restrictive atrial septal defect with a left to right shunt. The atrioventricular connections were concordant. The aorta was arising from the right ventricle with no right ventricular outflow obstruction [Figure 1]. The pulmonary artery was arising from the left ventricle with confluent good size branch pulmonary arteries with posterior deviation of the conal septum [Figure 2]. A large perimembranous (conoventricular type)1 ventricular septal defect with an inlet [6 mm] and outlet [6 mm] extension and an additional tiny apical defect were demonstrated [Figure 3]. A patent ductus arteriosus [2.7 mm] shunting left to right was also present (not shown). A 2-dimensional echocardiography image in the parasternal long axis view displayed an obstruction to the left ventricular outflow tract [white arrow] and a color Doppler blood flow map demonstrated turbulence in the outflow tract [Figure 4A,B]. In the subcostal view with the probe tilted anteriorly, color flow Doppler mapping showed the ventricular septal defect and also the blood flow acceleration across the left ventricular outflow tract [Figure 5, videolip-1]. At the area of turbulence, the jet width was 0.234 cm[Figure 6A]. A continuous wave Doppler interrogation of the area demonstrated a gradient of ≈ 25 mmHg [Figure 6B]. 2-dimensional transthoracic echocardiography interrogation in the subcostal view identified the etiology of the left ventricular outflow tract obstruction [Figure 7, Videoclip-2]. What is the diagnosis? Figure 2Transthoracic echocardiography image showing the relationship between the pulmonary artery and the left ventricle. View Large Image Figure Viewer Download Hi-res image Figure 3Subcostal 4-chamber transthoracic echocardiography image showing the perimembranous ventricular septal defect. View Large Image Figure Viewer Download Hi-res image Figure 4A&B. Parasternal long axis transthoracic echocardiography 2-dimensional image showing the obstruction in the left ventricular outflow tract ‘white arrow’ [A], and turbulence in the outflow tract by a color Doppler blood flow map ‘white arrow’[B]. View Large Image Figure Viewer Download Hi-res image Figure 5Subcostal transthoracic echocardiography image showing color Doppler flow acceleration over the left ventricular outflow tract and a ventricular septal defect. View Large Image Figure Viewer Download Hi-res image Figure 6A&B. Subcostal transthoracic echocardiography image showing the width of the obstruction [A] and the pressure gradient across the obstruction [B]. View Large Image Figure Viewer Download Hi-res image Figure 7A subcostal transthoracic echocardiography view showing the etiology of the left ventricular outflow tract obstruction in systole. View Large Image Figure Viewer Download Hi-res image

Single-View View Synthesis with Self-rectified Pseudo-Stereo

Synthesizing novel views from a single view image is a highly ill-posed problem. We discover an effective solution to reduce the learning ambiguity by expanding the single-view view synthesis problem to a multi-view setting. Specifically, we leverage the reliable and explicit stereo prior to generate a pseudo-stereo viewpoint, which serves as an auxiliary input to construct the 3D space. In this way, the challenging novel view synthesis process is decoupled into two simpler problems of stereo synthesis and 3D reconstruction. In order to synthesize a structurally correct and detail-preserved stereo image, we propose a self-rectified stereo synthesis to amend erroneous regions in an identify-rectify manner. Hard-to-train and incorrect warping samples are first discovered by two strategies, (1) pruning the network to reveal low-confident predictions; and (2) bidirectionally matching between stereo images to allow the discovery of improper mapping. These regions are then inpainted to form the final pseudo-stereo. With the aid of this extra input, a preferable 3D reconstruction can be easily obtained, and our method can work with arbitrary 3D representations. Extensive experiments show that our method outperforms state-of-the-art single-view view synthesis methods and stereo synthesis methods.

Multivariate two dimensional singular spectrum analysis based fusion method for four view image based object classification

Multivariate two dimensional singular spectrum analysis based fusion method for four view image based object classification

Single-shot extended field of view imaging using point spread function engineering.

We present a single-shot computational imaging system employing pupil phase engineering to extend the field of view (FOV) beyond the physical sensor limit. Our approach uses a point spread function in the form of a multiple-point impulse response (MPIR). Unlike the traditional point-to-point imaging model used by most traditional optical imaging systems, the proposed MPIR model can collect information from within and outside the sensor boundary. The detected raw image despite being scrambled can be decoded via a sparse optimization algorithm to get extended FOV imaging performance. We provide a thorough analysis of MPIR design regarding the number of impulses and their spatial extent. Increasing the number of impulses in MPIR of a given spatial extent leads to better information gathering within the detector region; however, it also reduces contrast in the raw data. Therefore, a trade-off between increasing the information and keeping adequate contrast in the detected data is necessary to achieve high-quality reconstruction. We first illustrate this trade-off with a simulation study and present experimental results on a suitably designed extended FOV imaging system. We demonstrate reconstructed images with a 4× gain in pixels over the native detection area without loss of spatial resolution. The proposed system design considerations are generic and can be applied to various imaging systems for extended FOV performance.

Obscured tree branches segmentation and 3D reconstruction using deep learning and geometrical constraints

The shortage of agricultural labourers worldwide has left many groups unharvested and wasted, which motivates researchers worldwide to research fruit harvesting robots extensively. One of the major problems in fruit harvesting is to selectively avoid hard obstacles such as tree branches so that more optimal picking positions can be found and more fruits throughout the tree can be harvested. However, tree branches are often obscured in unstructured natural orchards and thus necessary branch reconstruction and recovery are required. The current branch reconstruction and recovery methods for harvesting robots focus on planar reconstruction with few occlusions while the existing 3D tree modelling methods are not optimised for harvesting purposes that require low computational cost and high localisation accuracy. This work presented a novel framework that reconstructs and recovers 3D obscured branches from planar images and depth maps captured by an RGB-D camera. The framework comprises three parts: branch segmentation using Unet++, branch reconstruction using Point2Skeleton and branch recovery using a novel obscured branch recovery (OBR) algorithm. Branch segmentation using Unet++ with InceptionV3 encoder shows the best overall result with IoU and F1-score of 0.6249 and 0.7692 respectively. OBR recovery algorithm achieves average reconstruction accuracy of 0.72. The mean error of the reconstructed total surface and obscured surface using OBR is 18.68 mm and 38.11 mm with a standard deviation of 14.3 mm and 32.64 mm. The result shows that this framework can effectively reconstruct spatial information of visible and obscured branches from a single view image which can potentially be utilised in harvesting robots.

Establishing a Reliable Assessment of the Green View Index Based on Image Classification Techniques, Estimation, and a Hypothesis Testing Route

Sustainable development policies and spatial planning for maintaining greenery are crucial for all major cities in the world, and the measurement of green space indicators in planning practice needs to evolve in response to the demands of the times and technological drivers. This study explores an informal urban green space indicator, the green view index (GVI), which uses the visual perception of an observer to measure the quality of urban space by simulating the pedestrian perspective of the road in street-view image data and then calculating the proportion of vegetation in the road landscape. The GVI is different from macro indicators, such as public recreational green space, forest coverage, and green space rate, which are derived from planning data or remote sensing data in traditional urban planning; it starts from the bottom-up perception of individual residents and is more relevant to their subjective demands. At present, most international cities have made outstanding achievements in controlling public recreational green space, forest coverage, green space rates, and other macrolevel indicators of urban spatial quality; however, with the promotion of the concept of “human-oriented” urban planning, the potential restoration of urban spatial quality at the microlevel is gradually being recognized. To ensure the efficiency and reliability of this study, inspired by computer vision techniques and related GVI studies, a research method based on chromaticity was built to identify the proportions of green vegetation in street view images, and the credibility was improved by eliminating unreliable data. By using this method, we could evaluate a city at an overall scale instead of the previous block scale. The final research result showed that Shenzhen is friendly to human visual senses, and the GVI of the streets in developed areas is generally higher than that in developing areas. The geostatistical analysis of the green viewpoint data provides a more intuitive guide for researchers and planners, and it is believed to inform the planning and design of environmentally friendly, smart, and sustainable future cities.