학술논문

Generating Autonomous Driving Hazard Test Scenarios Using Multi-Agent Proximal Policy Optimization and Enhanced Artificial Potential Field Method
Document Type
Conference
Source
2024 43rd Chinese Control Conference (CCC) Chinese Control Conference (CCC), 2024 43rd. :5445-5452 Jul, 2024
Subject
Computing and Processing
Robotics and Control Systems
Signal Processing and Analysis
Transportation
Simulation
Reinforcement learning
Hazards
Behavioral sciences
Trajectory
Autonomous vehicles
Optimization
Autonomous Driving Test
Reinforcement Learning
Hazardous Scenario Generation
Driving Hazard Field
Language
ISSN
1934-1768
Abstract
The discovery of hazardous scenarios is crucial for the testing and optimization of autonomous driving strategies. However, effective testing of driving strategies faces two key challenges. Firstly, when testing a well-trained autonomous driving strategy, the probability of encountering hazardous scenarios in natural environments is low. Secondly, vehicles in hazardous scenarios should exhibit behavior as close as possible to that of human drivers, thus generating more realistic hazardous scenarios. To address the challenges identified, this study enhances the artificial potential field method, resulting in the Driving Hazard Field (DHF). Furthermore, by integrating the theory of the Driving Hazard Field with the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm, a hazardous scenario generation framework called DHF-MAPPO is developed. The vehicles surrounding the test vehicle are treated as agents, and the multi-agents trained within this framework not only possess the strong exploration characteristics of reinforcement learning but also, guided by the Driving Hazard Field, exhibit behaviors more akin to human drivers and follow more reasonable driving trajectories. The agents learn to interact with the test vehicle during the driving process, enabling the tested autonomous vehicle to face more complex hazardous scenarios, accelerating autonomous driving testing. The effectiveness of this approach is validated in a high-speed road scenario. Experimental results indicate a significant reduction in the autonomous driving performance of the test vehicle in the hazardous scenarios generated by the framework, with a notable increase in the risk coefficient during the driving process.