Self-driving Vehicles Research Articles

Object Recognition in AI

Object Recognition Using AI is a pioneering concept at the crossroads of artificial intelligence and computer vision, devoted to automating the identification and localization of objects within images and video streams. This paradigm harnesses machine learning algorithms to emulate human visual cognition, empowering systems to discern and precisely pinpoint diverse objects, transcending traditional image analysis. The essence of Object Recognition Using AI lies in its transformative impact across a multitude of applications. In retail, it revolutionizes inventory management by autonomously recognizing products on shelves. Within the security domain, it enhances surveillance systems, enabling real-time tracking of individuals and objects. In the automotive landscape, it empowers self-driving vehicles to interpret their surroundings, safeguarding passengers and pedestrians alike. Our research endeavors to advance the field of object recognition through the implementation of Mask R-CNN, a state-of-the-art deep learning architecture. We aim to achieve highly accurate object detection and segmentation by leveraging the power of convolutional neural networks. In pursuit of this, we conducted comprehensive experiments and evaluations. Our results showcase the effectiveness of Mask R-CNN in various real-world scenarios, with impressive precision and recall rates. In conclusion, this study contributes valuable insights and tools to the realm of computer vision, enabling enhanced object recognition applications. Keywords:Object Recognition, Mask R-CNN, Deep Learning, Convolutional Neural Networks, Computer Vision.

The reasonable robot – introduction: Artificial Intelligence and the law

Artificial intelligence (AI) and people do not compete on a level playing field. Self-driving vehicles may be safer than human drivers, but laws often penalize such technology. People can provide superior customer service, but companies are automating to reduce their taxes. AI can innovate more effectively, but an antiquated legal framework restricts inventive AI. In The Reasonable Robot, Ryan Abbott argues that the law should not discriminate between AI and human behavior and proposes a new legal principle that will ultimately improve human welfare. This work should be read by anyone interested in the rapidly evolving relationship between AI and the law.

Demystifying the black box: an overview of explainability methods in machine learning

Deep learning algorithms have achieved remarkable accuracy in various domains such as image classification, face recognition, sentiment analysis, text classification, healthcare, and self-driving vehicles. However, their complex and opaque structures often hinder their adoption in mission-critical applications. The lack of interpretability in these models raises concerns about their reliability and trustworthiness. To address this challenge, Explainable Artificial Intelligence (XAI) methods have emerged to provide human-comprehensible interpretations of AI outcomes. In this paper, we delve into the latest advancements in XAI techniques, focusing on their methodologies, algorithms, and the scope of interpretations they offer. Our study revolves around evaluating these algorithms based on the quality of explanations they generate, their limitations, and their practical applications. By critically assessing their strengths and weaknesses, we aim to shed light on the potential of XAI methods to bridge the gap between high-performance accuracy and interpretability in deep learning models. Through this comprehensive analysis, we aim to provide a deeper understanding of the state-of-the-art XAI techniques, empowering researchers and practitioners to make informed decisions regarding the integration of explainability into their AI systems.

3D LiDAR Multi-Object Tracking with Short-Term and Long-Term Multi-Level Associations

LiDAR-based Multi-Object Tracking (MOT) is a critical technology employed in various autonomous systems, including self-driving vehicles and autonomous delivery robots. In this paper, a novel LiDAR-based 3D MOT approach is introduced. The proposed method was built upon the Tracking-by-Detection (TbD) paradigm and incorporated multi-level associations that exploit an object’s short-term and long-term relationships with the existing tracks. Specifically, the short-term association leverages the fact that objects do not move much between consecutive frames. In contrast, the long-term association assesses the degree to which a long-term trajectory aligns with current detections. The evaluation of the matching between the current detection and the maintained trajectory was performed using a Graph Convolutional Network (GCN). Furthermore, an inactive track was maintained to address the issue of incorrect ID switching for objects that have been occluded for an extended period. The proposed method was evaluated on the KITTI benchmark MOT tracking dataset and achieved a Higher-Order Tracking Accuracy (HOTA) of 75.65%, marking a 5.66% improvement over the benchmark method AB3DMOT, while also accomplishing the number of ID switches of 39, 74 less than AB3DMOT. These results confirmed the effectiveness of the proposed approach in diverse road environments.

Trajectory Prediction with Attention-Based Spatial–Temporal Graph Convolutional Networks for Autonomous Driving

Accurate and reliable trajectory prediction is crucial for autonomous vehicles to achieve safe and efficient operation. Vehicles perceive the historical trajectories of moving objects and make predictions of behavioral intentions for a future period of time. With the predicted trajectories of moving objects such as obstacle vehicles, pedestrians, and non-motorized vehicles as inputs, self-driving vehicles can make more rational driving decisions and plan more reasonable and safe vehicle motion behaviors. However, due to traffic environments such as intersection scenes with highly interdependent and dynamic attributes, the task of motion anticipation becomes challenging. Existing works focus on the mutual relationships among vehicles while ignoring other potential essential interactions such as vehicle–traffic rules. These studies have not yet deeply explored the intensive learning of interactions between multi-agents, which may result in evaluation deviations. Aiming to meet these issues, we have designed a novel framework, namely trajectory prediction with attention-based spatial–temporal graph convolutional networks (TPASTGCN). In our proposal, the multi-agent interaction mechanisms, including vehicle–vehicle and vehicle–traffic rules, are meticulously highlighted and integrated into one homogeneous graph by transferring the time-series data of traffic lights into the spatial–temporal domains. Through integrating the attention mechanism into the adjacency matrix, we effectively learn the different strengths of interactive association and improve the model’s ability to capture critical features. Simultaneously, we construct a hierarchical structure employing the spatial GCN and temporal GCN to extract the spatial dependencies of traffic networks. Profiting from the gated recurrent unit (GRU), the scene context in temporal dimensions is further attained and enhanced with the encoder. In such a way, the GCN and GRU networks are fused as a features extractor module in the proposed framework. Finally, the future potential trajectories generation tasks are performed by another GRU network. Experiments on real-world datasets demonstrate the superior performance of the scheme compared with several baselines.

Detect Lane Line for Self-Driving Car Using Hue Saturation Lightness and Hue Saturation Value Color Transformation

Self-driving vehicles require the ability to perceive and understand their surroundings, just like human drivers. It entails navigating efficiently on roads, obeying traffic signs and signals, and avoiding collisions with other vehicles and pedestrians. To address the challenges associated with object detection in self-driving cars, an effort was made to demonstrate lane detection using the OpenCV library. To achieve this goal, the well-established probabilistic Hough transform technique is used for line detection. Before applying Hough transforms, several pre-processing techniques are used, including converting the image to grayscale, camera calibration, and implementing a masking filter. In addition, edge detection is performed using the edge detection method. The study also indicates a preference for the use of HSL (Hue, Saturation, and Lightness) and HSV (Hue, Saturation, Value) color spaces. When HSL is applied, white lines appear purer and brighter, resulting in superior performance compared to using HSV specifically to detect white. This algorithm proved particularly effective in detecting straight lanes, which achieved an accuracy ratio of 96.06%. By incorporating these methodologies, the lane detection algorithm implemented with the OpenCV library addresses the challenges of self-driving vehicles, providing them with improved perception capabilities similar to human drivers.

Energy-Saving Speed Planning for Electric Vehicles Based on RHRL in Car following Scenarios

Eco-driving is a driving vehicle strategy aimed at minimizing energy consumption; that is, it is a method to improve vehicle efficiency by optimizing driving behavior without making any hardware changes, especially for autonomous vehicles. To enhance energy efficiency across various driving scenarios, including road slopes, car following scenarios, and traffic signal interactions, this research introduces an energy-conserving speed planning approach for self-driving electric vehicles employing reinforcement learning. This strategy leverages vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication to acquire real-time data regarding traffic signal timing, leading vehicle speeds, and other pertinent driving conditions. In the framework of rolling horizon reinforcement learning (RHRL), predictions are made in each window using a rolling time domain approach. In the evaluation stage, Q-learning is used to obtain the optimal evaluation value, so that the vehicle can reach a reasonable speed. In conclusion, the algorithm’s efficacy is confirmed through vehicle simulation, with the results demonstrating that reinforcement learning adeptly modulates vehicle speed to minimize energy consumption, all while taking into account factors like road grade and maintaining a secure following distance from the preceding vehicle. Compared with the results of traditional adaptive cruise control (ACC), the algorithm can save 11.66% and 30.67% of energy under two working conditions.

Guided Direct Time-of-Flight Lidar Using Stereo Cameras for Enhanced Laser Power Efficiency

Self-driving vehicles demand efficient and reliable depth-sensing technologies. Lidar, with its capability for long-distance, high-precision measurement, is a crucial component in this pursuit. However, conventional mechanical scanning implementations suffer from reliability, cost, and frame rate limitations. Solid-state lidar solutions have emerged as a promising alternative, but the vast amount of photon data processed and stored using conventional direct time-of-flight (dToF) prevents long-distance sensing unless power-intensive partial histogram approaches are used. In this paper, we introduce a groundbreaking ‘guided’ dToF approach, harnessing external guidance from other onboard sensors to narrow down the depth search space for a power and data-efficient solution. This approach centers around a dToF sensor in which the exposed time window of independent pixels can be dynamically adjusted. We utilize a 64-by-32 macropixel dToF sensor and a pair of vision cameras to provide the guiding depth estimates. Our demonstrator captures a dynamic outdoor scene at 3 fps with distances up to 75 m. Compared to a conventional full histogram approach, on-chip data is reduced by over twenty times, while the total laser cycles in each frame are reduced by at least six times compared to any partial histogram approach. The capability of guided dToF to mitigate multipath reflections is also demonstrated. For self-driving vehicles where a wealth of sensor data is already available, guided dToF opens new possibilities for efficient solid-state lidar.

Dynamic path planning in autonomous driving

With the development of emerging technologies such as the Internet of Things (IoT), Artificial Intelligence (AI) and fifth-generation cellular network technology (5G), good conditions are provided for the level of automation, providing the necessary assistance to coordinate the environment, planning decisions and path optimisation for self-driving vehicles. This paper briefly outlines and analyses the key technologies required to realize autonomous driving and summarizes the corresponding research results for each part; by considering the operation process of self-driving vehicles, this paper chooses path planning as the focus of research and presents it in the form of a methodology; finally, it concludes with a summary of personal opinions and a discussion of possible future research directions.

Review Paper on Enhanced Image Captioning with Deep Learning: Encoder-Decoder and Attention Mechanism

Image captioning involves the generation of textual descriptions that describe the content within an image. This process finds extensive utility in diverse applications, including the analysis of large, unlabelled image datasets, uncovering concealed patterns to facilitate machine learning applications, guiding self-driving vehicles, and developing software solutions to aid visually impaired individuals. The implementation of image captioning relies heavily on deep learning models, a technological frontier that has simplified the task of generating captions for images. This paper focuses on the utilisation of encoder-decoder model with attention mechanism for image captioning. In classic image captioning model, the words usually describe only a part of the image, however with attention mechanism special attention is given to the low level and high level features of the image. Object detection using attention mechanism has shown to have increased the CIDEr score by 15%. With the use of stable dataset of MSCOCO through keras datasets, it is possible to score more on caption generation and accurate description of image.

Development and research of deep neural network fusion computer vision technology

Abstract Deep learning (DL) has revolutionized advanced digital picture processing, enabling significant advancements in computer vision (CV). However, it is important to note that older CV techniques, developed prior to the emergence of DL, still hold value and relevance. Particularly in the realm of more complex, three-dimensional (3D) data such as video and 3D models, CV and multimedia retrieval remain at the forefront of technological advancements. We provide critical insights into the progress made in developing higher-dimensional qualities through the application of DL, and also discuss the advantages and strategies employed in DL. With the widespread use of 3D sensor data and 3D modeling, the analysis and representation of the world in three dimensions have become commonplace. This progress has been facilitated by the development of additional sensors, driven by advancements in areas such as 3D gaming and self-driving vehicles. These advancements have enabled researchers to create feature description models that surpass traditional two-dimensional approaches. This study reveals the current state of advanced digital picture processing, highlighting the role of DL in pushing the boundaries of CV and multimedia retrieval in handling complex, 3D data.

Object detection: review and improvement based on deep learning

In recent years, with the quantum leap in deep learning, self-driving vehicle as one of its applications has been gaining tremendously increasing popularity as well as making a multitude of achievements. Object detection, which has made significant contribution to driver-less vehicle, has had been applied to a tremendously wide range of fields. However, reports relevant to automatic vehicle stating that accidents are caused by Automatic driving technology, present problems pointing out that existing target detection algorithms, which are already fairly well reliable, can probably be interfered by adverse conditions such as high temperature, raise dust and transmission loss, and be not capable of providing precise output. This paper recaps on these previous classic algorithms, and their large-scale application domain. Meanwhile, this paper presents improvements focusing on enhancing the robustness of these algorithms to overcome these problems caused by adverse conditions and improve the accuracy. Thus these improvements could augment the security of these driver-less vehicles, and eventually reduce traffic accident mortality relative to self-driving vehicles and safeguard road safety, and may potentially benefit to further research.

Risks to the Public

Where to begin? Things seem to be getting wildly out of control. I make a comment in the arti cial intelligence subsection here, relating to risks of autonomous robotic airplane pilots (Pibot, Wingman). That comment is also applicable to self-driving vehicles, hospital AI, and other life-critical or mission-critical automated and semi-automated systems: We desperately need evidence-based assurance rather than over-hyped assertions that we should simply trust the developers and operating managers. Oth- erwise, the integrity and credibility of our rampantly increasing overdependence on untrustworthy technol- ogy may cause a collapse of trust in our technology. Undoubtedly, avoiding such a systemic failure may require radical changes in how computer science and system engineering should be taught and practiced, along with corresponding oversight, and serious penalties for failures. I once again invoke the Einstein Principle that Everything should be made as simple as possible, but no simpler. It is just one of many important principles that are needed to make our system development into a solid discipline. However, beware of simplistic legislation, charlatans, frauds, and other recurring risks.

Pothole Detection System for Automobiles using Machine Learning

Abstract: In recent years, rapid economic growth and technological progress have had a profound impact on traditional transportation systems. One notable concern in terms of traffic safety is the discomfort and risk posed by rough roads, particularly those marred by potholes. This research project seeks to address these issues by not only improving road safety but also enhancing the level of automation in driving. With the proliferation of self-driving vehicles equipped with advanced autopilot systems, there arises a pressing need for real-time sensing of road conditions. To achieve this goal, the project employs machine learning techniques and image processing to detect potholes in real-time and relay this information to other vehicles via an internet connection. The research phase involved a thorough evaluation of various software algorithms and hardware tools, with an emphasis on criteria such as efficiency, cost-effectiveness, accuracy, and practicality for assembly. In addition to these considerations, deep learning and machine learning methodologies were integrated with hardware tools to formulate a refined set of requirements for the pothole detection system.

Enhancing Autonomous Vehicle Lateral Control: A Linear Complementarity Model-Predictive Control Approach

Model-predictive control (MPC) offers significant advantages in addressing constraint-related challenges and plays a pivotal role in self-driving car technology. Its primary goal is to achieve precise trajectory tracking while prioritizing vehicle stability and safety. However, real-time operations often face challenges related to computational demands and low computational efficiency. To address these challenges, this paper introduces a novel lateral control algorithm for self-driving vehicles, which utilizes the linear complementarity problem (LCP) instead of the conventional quadratic programming (QP) method as the MPC optimization solution. This innovative approach incorporates the electric steering system into the vehicle dynamics model, allowing for precise torque regulation of the steering motor and enhancing control accuracy. The MPC algorithm adopts the LCP solution method to calculate control signals based on the vehicle’s state, ensuring both rapid and stable vehicle control. Simulation results demonstrate that the proposed MPC algorithm, utilizing the LCP solution method, effectively addresses efficiency issues in the lateral motion of self-driving cars. This leads to improvements in both driving stability and real-time performance. Overall, this innovative approach lays a solid foundation for the practical implementation of self-driving cars.

Applying ethical theories to the decision-making of self-driving vehicles: A systematic review and integration of the literature

Applying ethical theories to the decision-making of self-driving vehicles: A systematic review and integration of the literature

Modelling Weather Precipitation Intensity on Surfaces in Motion with Application to Autonomous Vehicles.

With advances in the development of autonomous vehicles (AVs), more attention has been paid to the effects caused by adverse weather conditions. It is well known that the performance of self-driving vehicles is reduced when they are exposed to stressors that impair visibility or cause water or snow accumulation on sensor surfaces. This paper proposes a model to quantify weather precipitation, such as rain and snow, perceived by moving vehicles based on outdoor data. The modeling covers a wide range of parameters, such as varying the wind direction and realistic particle size distributions. The model allows the calculation of precipitation intensity on inclined surfaces of different orientations and on a circular driving path. The modeling results were partially validated against direct measurements carried out using a test vehicle. The model outputs showed a strong correlation with the experimental data for both rain and snow. Mitigation strategies for heavy precipitation on vehicles can be developed, and correlations between precipitation rate and accumulation level can be traced using the presented analytical model. A dimensional analysis of the problem highlighted the critical parameters that can help the design of future experiments. The obtained results highlight the importance of the angle of the sensing surface for the perceived precipitation level. The proposed model was used to analyze optimal orientations for minimization of the precipitation flux, which can help to determine the positioning of sensors on the surface of autonomous vehicles.

Quantum-Tuned Cascade Multijunction Infrared Photodetector.

Emerging applications such as augmented reality, self-driving vehicles, and quantum information technology require optoelectronic devices capable of sensing a low number of photons with high sensitivity (including gain) and high speed and that could operate in the infrared at telecom windows beyond silicon's bandgap. State-of-the-art semiconductors achieve some of these functions through costly and not easily scalable doping and epitaxial growing methods. Colloidal quantum dots (QDs), on the other hand, could be easily tuned and are compatible with consumer electronics manufacturing. However, the development of a QD infrared photodetector with high gain and high response speed remains a challenge. Herein, we present a QD monolithic multijunction cascade photodetector that advances in the speed-sensitivity-gain space through precise control over doping and bandgap. We achieved this by implementing a QD stack in which each layer is tailored via bandgap tuning and electrostatic surface manipulation. The resulting junctions sustain enhanced local electric fields, which, upon illumination, facilitate charge tunneling, recirculation, and gain, but retain low dark currents in the absence of light. Using this platform, we demonstrate an infrared photodetector sensitive up to 1500 nm, with a specific detectivity of ∼3.7 × 1012 Jones, a 3 dB bandwidth of 300 kHz (0.05 cm2 device), and a gain of ∼70× at 1300 nm, leading to an overall gain-bandwidth product over 20 MHz, in comparison with 3 kHz of standard photodiode devices of similar areas.

Toward personalized decision making for autonomous vehicles: A constrained multi-objective reinforcement learning technique

Reinforcement learning promises to provide a state-of-the-art solution to the decision making problem of autonomous driving. Nonetheless, numerous real-world decision making problems involve balancing multiple conflicting or competing objectives. In addition, passengers may typically prefer to explore diversified driving modes through their specific preferences (i.e., relative importance of different objectives). Taking into account these demands, traditional reinforcement learning algorithms with applications in personalized self-driving vehicles remain challenging. Consequently, here we present a novel constrained multi-objective reinforcement learning technique for personalized decision making in autonomous driving, with the goal of learning a single model for Pareto optimal policies across the space of all possible user preferences. Specifically, a nonlinear constraint incorporating a user-specified preference and a vectorized action–value function is introduced to ensure both diversity in learned decision behaviors and efficient alignment between the user-specified preference and the corresponding optimal policy. Additionally, a constrained multi-objective actor–critic approach is advanced to approximate the Pareto optimal policies for any user-specified preferences while adhering to the nonlinear constraint. Finally, the proposed personalized decision making scheme for autonomous driving is assessed in a highway on-ramp merging scenario with dynamic traffic flows. The results demonstrate the effectiveness of our method by comparing it with classical and state-of-the-art baselines.

Applying a Novel Image Recognition Curve-Fitting Control Strategy Combined with a Cloud Monitoring Technique into an Electric Self-Driving Vehicle (ESDV) to Improve Its Operation Efficiency

This study aims to develop an image recognition curve-fitting (IRCF) control strategy integrated with a cloud monitoring technique for application in electric self-driving vehicles (ESDVs) to improve their operation efficiency. The study focuses on an electric vehicle designed to reduce the carbon emissions and promote sustainability. The main camera, combined with the IRCF control strategy, was used to control the ESDV to enhance its operational efficiency. The proposed ESDV employs a pair of cameras to capture images and transmit them to the cloud-based web monitoring platform in real time. This allows the researchers to adjust the control parameters and promptly remove the road obstacles. The ESDV is equipped with a horn, two ultrasonic sensors, and an LED display, which can instantly detect the obstacles ahead of and behind the vehicle. When there are obstacles on the road, the vehicle will automatically stop, and the LED display will provide a visual representation of the obstacles, accompanied by the sounding of the horn as a warning signal. Meanwhile, the secondary camera detects the signal mark and feeds it back to the LED display, thereby informing passengers and other road users about the prevailing driving conditions. The proposed IRCF control strategy was compared with the traditional Hough line detection method on a 110 m ring road. The results revealed that the proposed control strategy outperformed the traditional Hough line detection method in terms of speed, efficiency, and running dexterity. Therefore, integrating the proposed control strategy into the automatic assistance driving system can improve the ESDV’s operation efficiency. Furthermore, the combination of the obstacle detection and signal sign detection functions for the ESDV used in this study can better fulfill the actual ESDV operation requirements on the road.