Environment Perception System Research Articles

A Multi-Sensor Environmental Perception System for an Automatic Electric Shovel Platform

Electric shovels have been widely used in heavy industrial applications, such as mineral extraction. However, the performance of the electric shovel is often affected by the complicated working environment and the proficiency of the operator, which will affect safety and efficiency. To improve the extraction performance, it is particularly important to study an intelligent electric shovel with autonomous operation technology. An electric shovel experimental platform for intelligent technology research and testing is proposed in this paper. The core of the designed platform is an intelligent environmental sensing/perception system, in which multiple sensors, such as RTK (real-time kinematic), IMU (inertial measurement unit) and LiDAR (light detection and ranging), have been employed. By appreciating the multi-directional loading characteristics of electric shovels, two 2D-LiDARs have been used and their data are synchronized and fused to construct a 3D point cloud. The synchronization is achieved with the assistance of RTK and IMU, which provide pose information of the shovel. In addition, in order to down-sample the LiDAR point clouds to facilitate more efficient data analysis, a new point cloud data processing algorithm including a bilateral-filtering based noise filter and a grid-based data compression method is proposed. The designed platform, together with its sensing system, was tested in different outdoor environment conditions. Compared with the original LiDAR point cloud, the proposed new environment sensing/perception system not only guarantees the characteristic points and effective edges of the measured objects, but also reduces the amount of processing point cloud data and improves system efficiency. By undertaking a large number of experiments, the overall measurement error of the proposed system is within 50 mm, which is well beyond the requirements of electric shovel application. The environment perception system for the automatic electric shovel platform has great research value and engineering significance for the improvement of the service problem of the electric shovel.

Construction of IoT management system for intelligent monitoring of distribution room

In order to further improve the quality and efficiency of distribution network operation and maintenance work, and support the transformation of distribution network technology to digital, lean, and intelligent, this article proposes an IoT management system for intelligent monitoring of power distribution rooms, as well as a design plan for IoT management platforms and edge IoT agents. It also conducts key research on the environmental information monitoring and perception system of the power distribution room, the power protection and security perception system, and establishes an intelligent and visualized intelligent distribution room to realize the digital transformation and intelligent improvement of the operation and maintenance of the power distribution network. The results are applied in the construction of Zhejiang intelligent distribution room, providing important guidance for the standardized construction of intelligent distribution room in China.

Robust Target Recognition and Tracking of Self-Driving Cars With Radar and Camera Information Fusion Under Severe Weather Conditions

Radar and camera information fusion sensing methods are used to solve the inherent shortcomings of the single sensor in severe weather. Our fusion scheme uses radar as the main hardware and camera as the auxiliary hardware framework. At the same time, the Mahalanobis distance is used to match the observed values of the target sequence. Data fusion based on the joint probability function method. Moreover, the algorithm was tested using actual sensor data collected from a vehicle, performing real-time environment perception. The test results show that radar and camera fusion algorithms perform better than single sensor environmental perception in severe weather, which can effectively reduce the missed detection rate of autonomous vehicle environment perception in severe weather. The fusion algorithm improves the robustness of the environment perception system and provides accurate environment perception information for the decision-making system and control system of autonomous vehicles.

Research on Unmanned Surface Vehicles Environment Perception Based on the Fusion of Vision and Lidar

The research of unmanned surface vehicles (usually called USV) is becoming a research hotspot, and real-time and accurate environmental perception is the key technology for achieving autonomous navigation and completing tasks of USV. In this paper, aiming at the technical problems of environmental perception of the USV, from the perspective of sensor multi-information fusion, we build a complete USV perception system (hardware platform and software system). This paper mainly studies objects on the water detection and obstacle avoidance methods based on the information fusion of lidar and vision. In addition, we also studies the method based on morphology to process the reflection of the objects on the water, and the sea-sky detection based on support vector machine (SVM) to assist the objects on the water detection. Through the construction of the USV environment perception system, improving the ability of USV environment perception, and ensuring the safety of autonomous navigation of USV. Our proposed methods have been tested by simulation experiments and actual marine experiment as well as the Hawaii Unmanned Surface Vehicles Challenge, which proves the practicability and stability of our method in the actual environment.

Two Improved Methods of Generating Adversarial Examples against Faster R-CNNs for Tram Environment Perception Systems

Trams have increasingly deployed object detectors to perceive running conditions, and deep learning networks have been widely adopted by those detectors. Growing neural networks have incurred severe attacks such as adversarial example attacks, imposing threats to tram safety. Only if adversarial attacks are studied thoroughly, researchers can come up with better defence methods against them. However, most existing methods of generating adversarial examples have been devoted to classification, and none of them target tram environment perception systems. In this paper, we propose an improved projected gradient descent (PGD) algorithm and an improved Carlini and Wagner (C&W) algorithm to generate adversarial examples against Faster R-CNN object detectors. Experiments verify that both algorithms can successfully conduct nontargeted and targeted white-box digital attacks when trams are running. We also compare the performance of the two methods, including attack effects, similarity to clean images, and the generating time. The results show that both algorithms can generate adversarial examples within 220 seconds, a much shorter time, without decrease of the success rate.

CONCEPT ON LANDMARK DETECTION IN ROAD SCENE IMAGES TAKEN FROM A TOP-VIEW CAMERA SYSTEM

Abstract. In this paper, we demonstrate the inclusion of a top-view camera system mounted on a city bus in an existing sensor setup. A novel sensor setup with five down-facing cameras is mounted on the roof of a MAN Lion’s City 12 city bus to extract landmarks in road scene images. Its positioning is validated by an exemplary detection of lane markings. The concept for further landmark detection with the help of the presented camera system is explained in this paper and sensor data fusion methods are proposed. Based on our previous findings (Albrecht et al., 2019), strengths of the novel sensor system are introduced to improve the current environment perception system. For now, only a qualitative observation of the capability to detect lane markings and other landmarks can be presented. Future work will use the current findings for landmark detection for a vehicle self-localization system.

A real-time traffic environmental perception algorithm fusing stereo vision and deep network1

Object detection and tracking are critical and challenging problems in vehicle environment perception systems, and have received broad attention in recent years. A novel detection and tracking algorithm taking both accuracy and real-time performance into account is proposed in this paper. First, we employ a fusion algorithm based on stereo vision and deep learning in object detection, which achieves high accuracy using two complementary algorithms. Then, a prediction-association algorithm which uses a Kalman filter and Hungarian assignment for multiple object tracking is employed for object tracking. In addition, a detection and tracking framework based on stereo vision improves the robustness of environmental perception system. Experimental results demonstrate that the proposed algorithm has high accuracy and can meet the real-time performance requirement.

Automotive Lidar Modelling Approach Based on Material Properties and Lidar Capabilities.

Development and validation of reliable environment perception systems for automated driving functions requires the extension of conventional physical test drives with simulations in virtual test environments. In such a virtual test environment, a perception sensor is replaced by a sensor model. A major challenge for state-of-the-art sensor models is to represent the large variety of material properties of the surrounding objects in a realistic manner. Since lidar sensors are considered to play an essential role for upcoming automated vehicles, this paper presents a new lidar modelling approach that takes material properties and corresponding lidar capabilities into account. The considered material property is the incidence angle dependent reflectance of the illuminated material in the infrared spectrum and the considered lidar property its capability to detect a material with a certain reflectance up to a certain range. A new material classification for lidar modelling in the automotive context is suggested, distinguishing between 7 material classes and 23 subclasses. To measure angle dependent reflectance in the infrared spectrum, a new measurement device based on a time of flight camera is introduced and calibrated using Lambertian targets with defined reflectance values at , , and . Reflectance measurements of 9 material subclasses are presented and 488 spectra from the NASA ECOSTRESS library are considered to evaluate the new measurement device. The parametrisation of the lidar capabilities is illustrated by presenting a lidar measurement campaign with a new Infineon lidar prototype and relevant data from 12 common lidar types.

Subjective Imaging Effect Assessment for Intelligent Imaging Terminal Design: a Method for Engineering Site

A kind of Subjective Imaging Effect Assessment (SIEA) method and its applications on intelligent imaging terminal design in engineering site are presented. First, some visual assessment indices are used to characterize the imaging effect: the image brightness, the image brightness uniformity, the color image contrast, the image edge blur, the image color difference, the image saturation, the image noise, and the integrated imaging effect index. A linear weighted function is employed to carry out the SIEA computation and the Analytic Hierarchy Process (AHP) technique is used to estimate its weights. Second, a SIEA software is developed. It can play images after the settings of assessment index or assessment reaction time, etc. Third, two cases are used to illustrate the application effects of proposed method: the image enhancement system design for surveillance camera and the imaging environment perception system design for intelligent lighting terminal. A Prior Sequential Stimulus (PSS) experiment is proposed to improve the evaluation stability of SIEA method. Many experiment results have shown the proposed method can realize a stable system design or parameters setting for the intelligent imaging terminal in engineering site.

Research and Application of Information Perception Technology for Mine Safety Helmet

Based on wireless sensor network technology in the Internet of Things, this paper studies and designs a mine environment perception system that is used in coal mines. The sense layer adopts the wireless sensor network based on WIFI technology, takes the miner safety helmet as the carrier, integrates the function module with STM32F107 controller as the core, and becomes the mobile node of the underground wireless sensing network, and realizes the monitoring of the miner’s work environment information and the underground personnel positioning. Through the test, the intelligent mobile terminal can actively perceive the surrounding environment of the underground personnel, realize the real-time monitoring and intelligent warning of the gas concentration, carbon monoxide concentration and temperature in the surrounding area of the miners, and can provide the positioning information for the ground-based biter, which is an important supplement to the safety monitoring of coal mine in China.

基于物联网的云上环境感知系统

The environmental perception system based on Internet of things has got rid of the dilemma of data collection in previous environmental monitoring systems. It can collect data in real time and upload it at high speed, and conduct data dynamic analysis in real time. It has greatly saved the cost of the previous handwritten data, and the speed of data collection and analysis has improved by leaps and leaps. Besides, on the basis of detecting water quality and soil pH, a variety of information data that modern people pay more attention to, such as PM2.5 and CO2 concentration, have been added, which solves the problem of single data in the old environmental perception system, and can better detect environmental conditions to meet the needs of modern people. Environmental data is stored in the database as a kind of data resource, which can be used to analyze the trend of environmental information and displayed on mobile phone and Internet pages. This system considers the cost problem to complete the stronger performance with lower cost, has satisfied the modern people to live the environment the demanding request, causes the people to live in the surroundings several meters air, the water quality, the soil real-time monitoring environment.

A General Approach for Simulating Rain Effects on Sensor Data in Real and Virtual Environments

Driving automation systems typically use surround sensors to perceive their local environment. Accidents with automated vehicles have shown that errors in sensor data measurement and interpretation can lead to fatal injuries. It is thus necessary to test the reliability of environmental perception systems before market introduction. Since critical weather conditions are random, rare, and change quickly, relevant data sets are biased towards clear conditions. Consequently, detection algorithms based on these data suffer from limited performance. This article focuses on a two-step approach based on a) an indoor rain facility to test under reproducible, realistic conditions and b) physical-based models to enrich sensor data from clear conditions with virtual rain effects in a post-processing step. We concentrate on data from camera, lidar, and radar sensors. Experimental results show that both approaches simulate critical effects on raw sensor data and, therefore, enable replicable testing and validation at every stage of development.

INSPEX: Optimize Range Sensors for Environment Perception as a Portable System.

Environment perception is crucial for the safe navigation of vehicles and robots to detect obstacles in their surroundings. It is also of paramount interest for navigation of human beings in reduced visibility conditions. Obstacle avoidance systems typically combine multiple sensing technologies (i.e., LiDAR, radar, ultrasound and visual) to detect various types of obstacles under different lighting and weather conditions, with the drawbacks of a given technology being offset by others. These systems require powerful computational capability to fuse the mass of data, which limits their use to high-end vehicles and robots. INSPEX delivers a low-power, small-size and lightweight environment perception system that is compatible with portable and/or wearable applications. This requires miniaturizing and optimizing existing range sensors of different technologies to meet the user’s requirements in terms of obstacle detection capabilities. These sensors consist of a LiDAR, a time-of-flight sensor, an ultrasound and an ultra-wideband radar with measurement ranges respectively of 10 m, 4 m, 2 m and 10 m. Integration of a data fusion technique is also required to build a model of the user’s surroundings and provide feedback about the localization of harmful obstacles. As primary demonstrator, the INSPEX device will be fixed on a white cane.

An object detection and tracking algorithm based on LiDAR and camera information fusion

As an important part of intelligent vehicle, environmental perception system mainly refers to the detection of the surrounding environment of the vehicle by the sensors attached on the vehicle. In order to ensure the accuracy and stability of the intelligent vehicle environmental perception system, it is necessary to use intelligent vehicle sensors to detect and track objects in the passable area. In this paper, an object detection and tracking algorithm based on the LiDAR and camera information fusion is proposed. The algorithm uses the point cloud data clustering method of LiDAR to detect the objects in the passable area and project them onto the image to determine the tracking objects. After the objects are determined, the algorithm uses color information to track objects in the image sequence. Since the object tracking algorithm based on image is easily affected by light, shadow and background interference, the algorithm uses LiDAR point cloud to modify the tracking results. This paper uses KITTI data set to verify and test this algorithm and experiments show that the target area detection overlap of the proposed target detection and tracking algorithm is 83.10% on average and the tracking success rate is 80.57%. Compared with particle filtering algorithm, the average region overlap increased by 29.47% and the tracking success rate increased by 19.96%.

MWNet: object detection network applicable for different weather conditions

The existing vehicle environment perception systems remain limited with regard to the ability to detect objects under complex weather conditions. This study proposes a novel network architecture named multi-weather network (MWNet), which can improve the performance of the on-board object detection system under extreme weather conditions. It consists of an encoder and a decoder. The encoder is comprised of shared convolutional layers used to extract features, while the decoder consists of three subnets, namely weather classification subnet, bad weather detection subnet, and fair weather detection subnet. Moreover, the results are satisfactory even for images photographed under different weather and illumination conditions.

A Systematic Review of Perception System and Simulators for Autonomous Vehicles Research.

This paper presents a systematic review of the perception systems and simulators for autonomous vehicles (AV). This work has been divided into three parts. In the first part, perception systems are categorized as environment perception systems and positioning estimation systems. The paper presents the physical fundamentals, principle functioning, and electromagnetic spectrum used to operate the most common sensors used in perception systems (ultrasonic, RADAR, LiDAR, cameras, IMU, GNSS, RTK, etc.). Furthermore, their strengths and weaknesses are shown, and the quantification of their features using spider charts will allow proper selection of different sensors depending on 11 features. In the second part, the main elements to be taken into account in the simulation of a perception system of an AV are presented. For this purpose, the paper describes simulators for model-based development, the main game engines that can be used for simulation, simulators from the robotics field, and lastly simulators used specifically for AV. Finally, the current state of regulations that are being applied in different countries around the world on issues concerning the implementation of autonomous vehicles is presented.

Object tracking method based on data computing

The data computing process is utilized in various areas such as autonomous driving. Autonomous vehicles are intended to detect and track nearby moving objects avoiding collisions and to navigate in complex situations, such as heavy traffic and dense pedestrian areas. Therefore, object tracking is the core technology in the environment perception systems of autonomous vehicles and requires the monitoring of surrounding objects and the prediction of the moving states of objects in real time. In this paper, a multiple object tracking method based on light detection and ranging (LiDAR) data is proposed by using a Kalman filter and data computing process. We suppose that the movements of the tracking objects are captured consecutively as frames; thus, model-based detection and tracking of dynamic objects are possible. A Kalman filter is applied for predicting posterior state of tracking object based on anterior state of the tracking object. State denotes the positions, shapes, and sizes of objects. By computing the likelihood probability between predicted tracking objects and clusters which registered from tracking objects, the data association process of the tracking objects can be generated. Experimental results showed enhanced object tracking performance in a dynamic environment. The average matching probability of the tracking object was greater than 92.9%.

Security in the Internet of Things Supported by Mobile Edge Computing

While applications of the heterogeneous Internet of Things (IoT) proliferate, their performance requirements in terms of latency and jitter become more stringent and difficult to be met by the traditional cloud computing paradigm. As a result, a new paradigm called multi-access edge computing has emerged in recent years. It complements the centralized cloud platform by providing additional resources at the edge of radio access networks, and provides better support of IoT applications. One of the challenges of supporting IoT applications in mobile edge computing (MEC) is security. In this article, we will scrutinize the security issues of two IoT applications that can exploit the benefits of MEC. The first one is an environment perception system based on an industrial IoT network. The second one, which is gaining momentum fast, is mobile IoT based on a network of unmanned aerial vehicles.

Path Tracking for Autonomous Race Car Based on G-G Diagram

Currently, the autonomous driving technique is attracting increasing research focus from all over the world. Generally, the control systems of an autonomous vehicle include environment perception, path planning and path tracking control systems. In this paper, the path tracking control issue of the autonomous vehicle will be focused on. Few of the previous proposed path tracking controllers consider the vehicle driving and handling limit, which degenerates the potential of the autonomous vehicle to finish the desired path as quick as possible. To this end, this paper proposes a path tracking controller for autonomous vehicle, which aims at pushing it to the driving and handling limit. The limit dynamic performance of the autonomous vehicle is represented by the G-G diagram, which indicates the acceleration capability of the autonomous vehicle. The G-G diagram is obtained by phase portrait method and it is validated by a modified FSAE racing car. Finally, a path tracking controller based on the G-G diagram is proposed. The simulation validation results demonstrate the effectiveness of the proposed controller.

Autonomous road detection and modeling for UGVs using vision-laser data fusion

Autonomous road detection and modeling play a key role for UGVs navigating in complex outdoor environments. This paper investigates road detection and description for UGVs in various outdoor scenes under different weather conditions. A novel environment perception system that includes two cameras and multiple laser range finders is introduced firstly. Taking classification accuracy and time-cost into account, 8-dimensional features are selected from a 91-dimensional candidate feature set using Adaboost algorithm. To adapt to the diversity of road scenes under different weather conditions in different seasons, an online classifier based on SVM is proposed to replace the fixed one. Moreover, a road region adjusting algorithm is present to eliminate misclassified regions especially when the roads have fuzzy boundaries or obstacles. Finally, a RANSAC spline fitting algorithm is adopted to provide an accurate road border model for UGVs’ autonomous path planning and navigation. A series of experiments are conducted by using a self-built UGV platform and experimental results show the validity and practicality of the proposed approaches.