Vanishing point detection using the teaching learning‐based optimisation algorithm

Estimating image vanishing points has many applications in the computer vision field, such as robotic navigation, visual measurement, camera calibration, 3D reconstruction and augmented reality, which requires a balance between accuracy and rate. In this paper, we present an algorithm to accurately and efficiently detect vanishing points and classify lines through the clustering method and binary particle swarm optimization (BPSO). First, lines are clustered according to their slope angles based on an iterative BPSO process, since parallel lines, in a medium-to-long range scene, present similar slopes. The solutions are continuously evaluated using multiple factors, such as the number and length of the line segments and their distance to the related vanishing points. The coefficient of variation is applied to weigh these factors. As a result, all possible non-orthogonal vanishing points in the image are directly detected, irrespective of the camera calibration parameters to avoid mapping segments on the Gaussian sphere. Compared with other algorithms on the York Urban Database, the proposed algorithm exhibits significant performance improvements.

Real time vanishing points detection on smartphones under Manhattan world assumption

Vanishing point (VP) detection is an important task in computer vision with particular importance in video surveillance. However, despite having numerous applications in camera calibration, 3D reconstruction and threat detection, a general method for VP detection has remained elusive. Rather than attempting the infeasible task of VP detection in general scenes, this paper presents a novel method that is fine-tuned to work for railway station environments and is shown to outperform the state-of-the-art for that particular case. The motivation for this is that (a) these environments are particularly susceptible to many types of crime from petty theft to terrorist activity, (b) the number of objects/structures in the scenes are limited, rendering the problem more tractable than the generic case, and (c) they typically have many CCTV cameras already installed. The method presented here commences by extracting edges from the input frame using the Canny edge detector as a pre-processing stage before the standard Hough transform is employed. A novel line clustering algorithm is then applied to determine the parameters of the lines that converge at VPs. This is based on statistics of the detected lines and heuristics about the type of scene. The clustered lines are then used to compute VPs using their intersection points. A voting system is used to optimise detection in an attempt to omit spurious lines. The paper includes a direct comparison to the state-of-the-art and ground truth.

Vanishing point detection for visual surveillance systems in railway platform environments

The analysis of vanishing points on digital images provides strong cues for inferring the 3D structure of the depicted scene and can be exploited in a variety of computer vision applications. In this paper, we propose a method for estimating vanishing points in images of architectural environments that can be used for camera calibration and pose estimation, important tasks in large-scale 3D reconstruction. Our method performs automatic segment clustering in projective space --- a direct transformation from the image space --- instead of the traditional bounded accumulator space. Since it works in projective space, it handles finite and infinite vanishing points, without any special condition or threshold tuning. Experiments on real images show the effectiveness of the proposed method. We identify three orthogonal vanishing points and compute the estimation error based on their relation with the Image of the Absolute Conic (IAC) and based on the computation of the camera focal length.

This paper describes a comparative study of edge operators for the task of detecting dominant vanishing points in the image. Segmentation of line is required in order to detect the vanishing points. Three edge operators such as Sobel, Canny, and LoG (Laplacian of Gaussian) are used in order to compare that edge has influence on detecting the vanishing points. Most of line segments are obtained based on edge detection. The vanishing points are estimated by MSAC (m-estimator sample consensus) based algorithm. First, lines are extracted from edge images produced by edge operators. Second, vanishing points are obtained. The results of line segments and vanishing points detection are compared and discussed. The comparison is carried out based on the result of implementation on images with different buildings.

Vanishing point detection is a fundamental step in computer vision that allows 3D scene understanding and autonomous navigation. Classical techniques have significant challenges when trying to understand scenes that are heavily cluttered and images containing multiple perspective cues, leading to poor or unreliable vanishing point determination. We present a Dual RANSAC with Rescue Midpoint-based Multi-Trend Vanishing Point Detection framework, which targets the simultaneous detection and fine-tuning of multiple, globally consistent vanishing points. The proposed framework introduces a novel Midpoint-based Multi-Trend Random Sample Consensus formulation that operates on line segment midpoints to infer dominant directional groups, thereby eliminating noisy or unstable midpoints and stabilizing subsequent vanishing point inference. The main novelty lies in using line segment midpoints to model the orientation variation as a linear regression in the midpoint-orientation space, which helps reduce sensitivity to endpoint instability. Candidate vanishing points are prioritized through inlier-based confidence ranking and subsequently optimized via an MSAC-based arbiter to resolve hypothesis conflicts and minimize geometric error. We evaluate our work against state-of-the-art techniques such as J-Linkage and Conditional Sample Consensus, over two of the current challenging public datasets that comprise the York Urban Dataset and the Toulouse Vanishing Point Dataset. The results show that the proposed framework achieves a recall of up to 95% and an image success rate of almost 84%, outperforming both J-Linkage and Conditional Sample Consensus, especially under tighter angular thresholds. This demonstrates the ability of the proposed framework to provide enhanced stability and localization accuracy.

Automatic camera calibration is a fundamental technology for 3D traffic parameter extraction. With the popularity of pan-tilt-zoom cameras, this technique demonstrates great potential to enhance traffic safety and efficiency, especially for highways. This paper aims to present a fully automatic calibration method for surveillance cameras in highway scenes. Our system is divided into two stages. In the first stage, a deep convolution neural network was used to estimate a pair of orthogonal vanishing points from multiple vehicles. This process transformed vanishing point detection into an estimation of vehicle direction, which was further determined by introducing the central residual mechanism. In the diamond space, the straight lines formed by these directions accumulated the final positions of the vanishing points. More importantly, we proposed a novel algorithm for estimating the lane width using vehicle trajectories in the second stage. It can be used to calculate the camera height, making the calibration fully automated. We also corrected the distorted lens using vehicle trajectories. Comprehensive experiments were conducted on the proposed data set and the BoxCars116k data set. The results indicate that the composite mechanism (i.e., classification and central residual) significantly improves the accuracy and robustness of the vanishing point estimation. Combined with automatic camera height estimation, our technology is superior to the most representative methods in calibration performance. Since it does not have any constraints on road geometry and camera placement, our approach applies to most highway surveillance systems.

Detecting the vanishing point from a single road image is a challenging problem because there is very limited information in the input image that can help the computer to deduce the genuine location of vanishing point. Besides, the cluttered ambient environment in a real road image sometimes will hinder rather than assist the detection. Learning both the advantages and the limitations of current edge-based and texture-based approaches motivates us to propose a new vanishing point detection method that exploits the intrinsic geometric line structures and color texture properties of general roads. Our approach integrates the efficiency of line segments of edge-based methods, and the orientation coherence concept that is frequently applied in texture-based methods, which can be of great help to improve the accuracy of selecting the right line segments for vanishing point detection. The proposed method has been implemented and tested on over 1000 various road images. These road images exhibit large variations in color, texture, illumination condition, and ambient environment. The experimental results demonstrate that this new method is both efficient and effective in detecting vanishing point when compared to the state-of-the-art edge-based and texture-based methods.

This paper focuses on extracting vehicle trajectory from low-quality, uncalibrated traffic cameras via fully automatic camera calibration and homography transformations. We analyze video streams from 511 traffic surveillance cameras in varying areas and arbitrary relative locations and orientations to roads. We propose a novel automatic calibration methodology that is used to transform the video streams to a top-down perspective from a minute of the video stream. Using this perspective, we can extract both lane changes and vehicle speed. Our automatic camera calibration algorithm is inspired by two vanishing point based automatic camera calibration methods. We implement more accurate and robust methods for the detection of both vanishing points. We show that YOLOv4 combined with DeepSORT is 12.5 times faster than the leading vehicle detection model and achieves a 7.81% higher average precision and 6.04% higher mean average precision in low quality traffic surveillance cameras. We use this model for a more robust estimation of the first vanishing point and more robust object tracking. Furthermore, we propose a fast, novel “guess and check” algorithm for the detection of the second vanishing point to ensure accurate second vanishing point detection. We show that our camera calibration algorithm produces fewer inaccuracies than the state-of-the-art automatic camera calibration methodology in [1] through qualitative results.

Vanishing point detection is a basic work in camera self-calibration, single view reconstruction and series of images matching. Our research is based on line segments clustering method. First, we scan the image with edge detection algorithm for series of line segments. Then, we construct a similar concept space to classify the segments according to the vector distances. At last, we can use each cluster of the line segments to estimate the responsible vanishing point. For the clusters of the line segments indicate the main direction in multiple lines, the detected vanishing points are principal direction points. From the experiments, we approve our algorithm can acquire accurate position of vanishing points in short time.

This paper deals with the detection of orthogonal vanishing points. The aim is to efficiently cope with the clutter edges in real-life images and to determine the camera orientation in the Manhattan world reliably. We are using a modified scheme of the Cascaded Hough Transform where only one Hough space is accumulated – the space of the vanishing points. The parameterization of the vanishing points – the “diamond space” – is based on the PClines line parameterization and it is defined as a mapping of the whole real projective plane to a finite space. Our algorithm for detection of vanishing points operates directly on edgelets detected by an edge detector, skipping the common step of grouping edges into straight lines or line segments. This decreases the number of configuration parameters and reduces the complexity of the algorithm. Evaluated on the York Urban DB, our algorithm yields 98.04 % success rate at 10◦ angular error tolerance, which outperforms comparable existing solutions. Our parameterization of vanishing points is in all aspects linear; it involves no goniometric or other non-linear operations and thus it is suitable for implementation in embedded chips and circuitry. The iterative search scheme allows for finding orthogonal triplets of vanishing points with high accuracy and low computational costs. At the same time, our approach can be used without the orthogonality constraint.

Abstract. Reconstruction of spatial layout of indoor scenes from a single image is inherently an ambiguous problem. However, indoor scenes are usually comprised of orthogonal planes. The regularity of planar configuration (scene layout) is often recognizable, which provides valuable information for understanding the indoor scenes. Most of the current methods define the scene layout as a single cubic primitive. This domain-specific knowledge is often not valid in many indoors where multiple corridors are linked each other. In this paper, we aim to address this problem by hypothesizing-verifying multiple cubic primitives representing the indoor scene layout. This method utilizes middle-level perceptual organization, and relies on finding the ground-wall and ceiling-wall boundaries using detected line segments and the orthogonal vanishing points. A comprehensive interpretation of these edge relations is often hindered due to shadows and occlusions. To handle this problem, the proposed method introduces virtual rays which aid in the creation of a physically valid cubic structure by using orthogonal vanishing points. The straight line segments are extracted from the single image and the orthogonal vanishing points are estimated by employing the RANSAC approach. Many scene layout hypotheses are created through intersecting random line segments and virtual rays of vanishing points. The created hypotheses are evaluated by a geometric reasoning-based objective function to find the best fitting hypothesis to the image. The best model hypothesis offered with the highest score is then converted to a 3D model. The proposed method is fully automatic and no human intervention is necessary to obtain an approximate 3D reconstruction.

When an unmanned surface vehicle (USV) navigates in narrow waterway scenarios, its ability to detect vanishing points accurately and quickly is highly important for safeguarding its navigation safety and realizing automated navigation. We propose a novel approach for detecting vanishing points based on an improved lightweight AlexNet. First, a similarity evaluation calculation method based on image texture features is proposed, by which some scenarios are selected from the filtered Google Street Road Dataset (GSRD). These filtered scenarios, together with the USV Inland Dataset (USVID), compose the training dataset, which is manually labeled according to a non-uniformly distributed grid level. Next, the classical AlexNet was adjusted and optimized by constructing sequential connections of four convolutional layers and four pooling layers and incorporating the Inception A and Inception C structures in the first two convolutional layers. During model training, we formulate vanishing point detection as a classification problem using an output layer with 225 discrete possible vanishing point locations. Finally, we compare and analyze the labeled vanishing point with the detected vanishing point. The experimental results show that the accuracy of our method and the state-of-the-art algorithmic vanishing point detector improves, indicating that our improved lightweight AlexNet can be applied in narrow waterway navigation scenarios and can provide a technical reference for autonomous navigation of USVs.

Camera calibration directly from image sequences of a pedestrian without using any calibration object is a really challenging task and should be well solved in computer vision, especially in visual surveillance. In this paper, we propose a novel camera calibration method based on recovering the three orthogonal vanishing points (TOVPs), just using an image sequence of a pedestrian walking in a straight line, without any assumption of scenes or motions, e.g., control points with known 3D coordinates, parallel or perpendicular lines, non-natural or pre-designed special human motions, as often necessary in previous methods. The traces of shoes of a pedestrian carry more rich and easily detectable metric information than all other body parts in the periodic motion of a pedestrian, but such information is usually overlooked by previous work. In this paper, we employ the images of the toes of the shoes on the ground plane to determine the vanishing point corresponding to the walking direction, and then utilize harmonic conjugate properties in projective geometry to recover the vanishing point corresponding to the perpendicular direction of the walking direction in the horizontal plane and the vanishing point corresponding to the vertical direction. After recovering all of the TOVPs, the intrinsic and extrinsic parameters of the camera can be determined. Experiments on various scenes and viewing angles prove the feasibility and accuracy of the proposed method.