View Monitoring System Research Articles

Semantic Segmentation of Panoramic Images for Real-Time Parking Slot Detection

Autonomous parking is an active field of automatic driving in both industry and academia. Parking slot detection (PSD) based on a panoramic image can effectively improve the perception of a parking space and the surrounding environment, which enhances the convenience and safety of parking. The challenge of PSD implementation is identifying the parking slot in real-time based on images obtained from the around view monitoring (AVM) system, while maintaining high recognition accuracy. This paper proposes a real-time parking slot detection (RPSD) network based on semantic segmentation, which implements real-time parking slot detection on the panoramic surround view (PSV) dataset and avoids the constraint conditions of parking slots. The structural advantages of the proposed network achieve real-time semantic segmentation while effectively improving the detection accuracy of the PSV dataset. The cascade structure reduces the operating parameters of the whole network, ensuring real-time performance, and the fusion of coarse and detailed features extracted from the upper and lower layers improves segmentation accuracy. The experimental results show that the final mIoU of this work is 67.97% and the speed is up to 32.69 fps, which achieves state-of-the-art performance with the PSV dataset.

VAVM: A Flexible Technique for Variable-Angle Around View Monitor System Towards Articulated Engineering Vehicle

The around view monitor system (AVM) has been widely used in passenger vehicles. However, for articulated engineering vehicles (AEV), AVM is difficult to directly apply due to the long body and the articulation angle. In this paper, a flexible technique for variable-angle around view monitor system (VAVM) towards the AEV is proposed, which includes static calibration and dynamic execution. In the static calibration stage, the camera poses in the vehicle coordinate system with the joint of AEV as the coordinate origin can be calibrated using irregularly placed multiple checkerboards without knowing their relative position. This flexible calibration approach makes the variable-angle mapping from the fisheye image to the bird’s eye view image can be directly obtained by updating the camera poses on the tractor according to the articulation angle when the AEV turns. In the dynamic execution stage, an optimization algorithm based on minimizing the photometric error is designed to reduce the cumulative error of the angle sensor. Finally, all the bird’s eye view images are mosaiced and fused to a variable-angle around view image (VAVI). The experimental results show that the VAVM can output globally balanced and seamless VAVIs under different angles and visual environment conditions. Furthermore, the practicability of VAVM is also demonstrated by a real AEV.

Around-View-Monitoring-Based Automatic Parking System Using Parking Line Detection

This paper introduces an automatic parking method using an around view monitoring system. In this method, parking lines are extracted from the camera images, and a route to a targeted parking slot is created. The vehicle then tracks this route to park. The proposed method extracts lines from images using a line filter and a Hough transform, and it uses a convolutional neural network to robustly extract parking lines from the environment. In addition, a parking path consisting of curved and straight sections is created and used to control the vehicle. Perpendicular, angle, and parallel parking paths can be created; however, parking control is applied according to the shape of each parking slot. The results of our experiments confirm that the proposed method has an average offset of 10.3 cm and an average heading angle error of 0.94°.

A Novel AVM Calibration Method Using Unaligned Square Calibration Boards.

An around view monitoring (AVM) system acquires the front, rear, left, and right-side information of a vehicle using four cameras and transforms the four images into one image coordinate system to monitor around the vehicle with one image. Conventional AVM calibration utilizes the maximum likelihood estimation (MLE) to determine the parameters that can transform the captured four images into one AVM image. The MLE requires reference data of the image coordinate system and the world coordinate system to estimate these parameters. In conventional AVM calibration, many aligned calibration boards are placed around the vehicle and are measured to extract the reference sample data. However, accurately placing and measuring the calibration boards around a vehicle is an exhaustive procedure. To remediate this problem, we propose a novel AVM calibration method that requires only four randomly placed calibration boards by estimating the location of each calibration board. First, we define the AVM errors and determine the parameters that minimize the error in estimating the location. We then evaluate the accuracy of the proposed method through experiments using a real-sized vehicle and an electric vehicle for children to show that the proposed method can generate an AVM image similar to the conventional AVM calibration method regardless of a vehicle’s size.

Software Optimization of Vision-Based Around View Monitoring System on Embedded Platform


 
 
 Image processing algorithm requires high computational power. Optimizing the algorithm to be run on an embedded platform is very critical as the platform provides limited computational resources. This research focused on optimizing and implementing a vision-based Around View Monitoring (AVM) system running on two embedded boards of Cortex-A7 quad and Cortex-A15 quad-core, and desktop platform of Intel i7 core. This paper presented a study on several techniques of software optimization that is removing code redundancy and multi-threading. The two methods improve the total processing time of the AVM system by 45% on ARM Cortex-A15 and 47% on ARM Cortex-A7.
 
 

Semantic Segmentation-Based Lane-Level Localization Using Around View Monitoring System

Lane-level localization is a fundamental task for autonomous driving. As front cameras are easily disturbed by dynamic objects in urban environments, this paper presents an accurate lane-level localization approach using the around view monitoring (AVM) system. The paper proposes to detect the road features (i.e., road boundaries and road markings) based on pixel-wise semantic segmentation of raw fisheye images. The method can detect various types of road features and exclude dynamic objects from the localization. To address the problem of AVM-based localization with road features of different characteristics, this paper proposes coarse-scale localization (CSL) and fine-scale localization (FSL) methods for high-accurate localization. The CSL method leverages the road boundaries to provide an initial position; the FSL method estimates a high-accuracy position by matching nearby road markings with the map. The experiments in urban environments demonstrate that the proposed approach achieves centimeter-level localization accuracy with five centimeters in the lateral direction and seventeen centimeters in the longitudinal direction.

Semantic segmentation-based parking space detection with standalone around view monitoring system

An auto-parking system is one of the promising technologies to reduce accidents and enhance driver convenience in parking lots. To accomplish collision-free parking, precise and robust parking space detection is required. However, harsh conditions such as varied illumination in outdoor parking lots and high reflection in indoor parking lots degrade the reliability of parking space detection. In this paper, we propose a unified structure for parking space detection to detect parking slot markings and static obstacles. A fully convolutional network for semantic segmentation can immediately identify free spaces, slot markings, vehicles, and other objects without using a range sensor or 3D reconstruction algorithm. Furthermore, a vertical grid encoding method can simultaneously detect unoccupied slots identified by parking slot markings and empty spaces created by surrounding static objects without sensor fusion. Experimental results show the robustness of the proposed method in various different parking scenarios. Even in challenging conditions such as dark shaded or high-glare areas, the detection performance maintains a precision rate of 96.81% and recall rate of 97.80%.

Automatic Calibration of an Around View Monitor System Exploiting Lane Markings.

This paper proposes a method that automatically calibrates four cameras of an around view monitor (AVM) system in a natural driving situation. The proposed method estimates orientation angles of four cameras composing the AVM system, and assumes that their locations and intrinsic parameters are known in advance. This method utilizes lane markings because they exist in almost all on-road situations and appear across images of adjacent cameras. It starts by detecting lane markings from images captured by four cameras of the AVM system in a cost-effective manner. False lane markings are rejected by analyzing the statistical properties of the detected lane markings. Once the correct lane markings are sufficiently gathered, this method first calibrates the front and rear cameras, and then calibrates the left and right cameras with the help of the calibration results of the front and rear cameras. This two-step approach is essential because side cameras cannot be fully calibrated by themselves, due to insufficient lane marking information. After this initial calibration, this method collects corresponding lane markings appearing across images of adjacent cameras and simultaneously refines the initial calibration results of four cameras to obtain seamless AVM images. In the case of a long image sequence, this method conducts the camera calibration multiple times, and then selects the medoid as the final result to reduce computational resources and dependency on a specific place. In the experiment, the proposed method was quantitatively and qualitatively evaluated in various real driving situations and showed promising results.

Sensor fusion‐based precise obstacle localisation for automatic parking systems

A precise obstacle localisation method for automatic parking systems is proposed. The proposed method improves the obstacle localisation accuracy by fusing only the sensors already installed on mass-produced vehicles. This method finds the initial obstacle location using ultrasonic sensors. On the basis of the initial obstacle location, it detects the obstacle's vertical outlines using images provided by an around view monitor system while the ego-vehicle is moving. The sequentially detected vertical outlines are combined based on the ego-vehicle positions estimated by built-in motion sensors. Finally, the obstacle location is refined by calculating the intersection point of the combined vertical outlines. In experiments, the proposed method was quantitatively evaluated and outperformed the previous method.

어라운드 뷰 모니터링 시스템과 자동 주차 시스템의 통합 구현

어라운드 뷰 모니터링 시스템과 자동 주차 시스템의 통합 구현

Full endoscopic lumbar interbody fusion (FELIF): technical note.

Interbody fusion by open discectomy is the usual treatment for degenerative lumbar disease but requires a relatively long recovery period. Prevalent several interbody fusion methods have shown limitations by its own character. Full endoscopic lumbar interbody fusion (FELIF) has advantages in direct decompression of pathology with minimal invasive technique. We report novel technique of endoscopic fusion method through full guided with endoscopic view monitoring system. The authors provide an introduction of FELIF technique followed by a description of the technical steps with discussion on its indications and advantages. In particular, tips to shorten operation time, simplify technical steps, and avoid complications are also presented. FELIF is a safe and effective interbody fusion option to decompress the lumbar exiting nerve root and ventral side of dura directly with minimal invasive situation. These slides can be retrieved under Electronic Supplementary Material.

Ring Fusion of Fisheye Images Based on Corner Detection Algorithm for Around View Monitoring System of Intelligent Driving

In order to improve the visual effect of the around view monitor (AVM), we propose a novel ring fusion method to reduce the brightness difference among fisheye images and achieve a smooth transition around stitching seam. Firstly, an integrated corner detection is proposed to automatically detect corner points for image registration. Then, we use equalization processing to reduce the brightness among images. And we match the color of images according to the ring fusion method. Finally, we use distance weight to blend images around stitching seam. Through this algorithm, we have made a Matlab toolbox for image blending. 100% of the required corner is accurately and fully automatically detected. The transition around the stitching seam is very smooth, with no obvious stitching trace.

The Lane Recognition Enhancement Algorithms of Around View Monitoring System Based on Automotive Black Boxes

The Lane Recognition Enhancement Algorithms of Around View Monitoring System Based on Automotive Black Boxes

A new calibrator providing easy detection of feature points for calibrating fisheye cameras in vehicle AVM systems

This paper proposes a new planar calibrator, which is suitable for calibrating the fisheye camera in a vehicle around view monitoring system. To facilitate easy feature point detection using the proposed calibrator, we ensure that the shapes of the features and the distances between the adjacent feature points on the calibrator’s image acquired by the fisheye camera are identical squares and equal, respectively, unlike those on the conventional chessboard calibrator. Further, the optimum number and locations of feature points on the proposed calibrator are experimentally determined for ensuring the best camera calibration performance. Compared with the conventional chessboard calibrator and the H-pattern calibrator, the proposed calibrator shows superior performance in terms of feature point detection and camera calibration.

MOST150 네트워크 기반 차량 주변 감시 시스템의 설계 및 구현

Rear view cameras which help to park or to drive backward has been distributed through after-market, but it is inconvenient because they do not provide views of left, right and front sides. Around view monitoring(AVM) systems which can monitor around vehicle at a glance have been developed and equipped by vehicle vendor but systematic studies on these systems is lack. While the AVM system which equipped on Infiniti cars of Nissan is adequate to monitor around vehicle at a glance, it has disadvantages that additional cabling because of using analog cables is required and image quality is lowered due to EMI/EMC intervention. The around view monitoring system implemented in this paper has advantages that there are no EMI/EMC problems because of using optical network and that cabling is simple because of using plug-and-play ways. Additionally, an advantage of MOST150 network is that camera nodes and display node can be easily installed in the form of plug-and-play.

Design and Implementation of an Around View Monitoring System of Selecting Scope of View

Recently certain driver convenience systems including ADAS (Advanced Driver Assistance System) such as navigation and GPS, and driver’s infotainment devices such as DVD players have been produced. AVM systems of these driver convenience systems support safety driving by providing situations around the vehicle to a driver. A driver can't fully monitor the situation around the vehicle at a glance using only rearview mirrors, a room mirror, and front and rear cameras, and using of only these devices causes blind spot which hinders driver’s safety. Blind spot is cause of personal injury and vehicle damage when parking vehicle, passing on a narrow street and turning left or right. Therefore studies on AVM systems have actively been progressing which provides surroundings of vehicles for drivers. Though existing AVM systems combines video images from four cameras and provides a driver with a full vision around the vehicle with no blind spots, the system has limitations that its monitoring area is narrow and fixed. Thus they are suitable for parking but are not suitable for monitoring situations around vehicle in various purposes. Therefore, in this paper, we will design and implement AVM system with adjustable around view to be shown that the proposed system provides multiple scopes of around views.

Sensor Fusion-Based Vacant Parking Slot Detection and Tracking

This paper proposes a vacant parking slot detection and tracking system that fuses the sensors of an Around View Monitor (AVM) system and an ultrasonic sensor-based automatic parking system. The proposed system consists of three stages: parking slot marking detection, parking slot occupancy classification, and parking slot marking tracking. The parking slot marking detection stage recognizes various types of parking slot markings using AVM image sequences. It detects parking slots in individual AVM images by exploiting a hierarchical tree structure of parking slot markings and combines sequential detection results. The parking slot occupancy classification stage identifies vacancies of detected parking slots using ultrasonic sensor data. Parking slot occupancy is probabilistically calculated by treating each parking slot region as a single cell of the occupancy grid. The parking slot marking tracking stage continuously estimates the position of the selected parking slot while the ego-vehicle is moving into it. During tracking, AVM images and motion sensor-based odometry are fused together in the chamfer score level to achieve robustness against inevitable occlusions caused by the ego-vehicle. In the experiments, it is shown that the proposed method can recognize the positions and occupancies of various types of parking slot markings and stably track them under practical situations in a real-time manner. The proposed system is expected to help drivers conveniently select one of the available parking slots and support the parking control system by continuously updating the designated target positions.

Car PC 기반 어라운드뷰 모니터링 시스템의 위치정보 제공 기능

최근 자동차 산업의 활성화로 운전자 보조시스템에 연구가 많이 이루어지고 있으며, 그 중 AVM 시스템(Around View Monitoring System)에 대한 연구 및 개발이 활발해지고 있다. 기존 AVM 시스템은 차량 주변 상황을 실시간으로 제공하기 위해 임베디드 시스템 또는 SoC(System on Chip)의 형태로 개발되었다. 그러나 Car PC가 차량에 장착되어 있으면 AVM 시스템을 추가의 비용이 없이 소프트웨어만으로 시스템을 개발할 수 있다. 본 연구에서는 Car PC 기반의 AVM 시스템 기능에 위치 정보 기능인 위도, 경도, 속도 기능을 추가하여 위치정보를 제공하는 AVM 시스템을 구현하였다. 본 연구에서 구현한 위치 정보를 제공하는 AVM 시스템에 저장 기능을 추가하면 AVM 블랙박스 시스템으로 활용될 수 있다. In recent, the researches on driver assistance systems have been actively performed with development of vehicle industry. AVM(Around View Monitoring) Systems, a part of these systems, have been researching and developing. Existing AVM systems have been developed in the forms of embedded systems or a SoC (System on Chip) to provide view around vehicle in real time. However, if Car PC is equipped with in vehicle, AVM can be developed using only software without additional cost. In this study, we implemented AVM system which provides location information by adding the informations such as latitude, longitude and speed to functions of "Car PC" based AVM system. If storing function is added to the AVM system implemented in this study which provides location information, the system with storing function can be used as AVM black box system.

Development of Moving Object Detection Using Vision System

This paper describes a functionality and test results of moving object detection based on vision system. Moving object detection is realized by just adding image processing software to the vision system with no additional sensor. Moreover, by usage of around view monitor system as vision system, existence of moving objects surrounding the subject vehicle can be informed to support the driver in the parking maneuver. Language: en