학술논문

Multi-Agent Deep Reinforcement Learning for Spectrum Management in V2X with Social Roles
Document Type
Conference
Source
GLOBECOM 2023 - 2023 IEEE Global Communications Conference Global Communications Conference, GLOBECOM 2023 - 2023 IEEE. :2293-2298 Dec, 2023
Subject
Communication, Networking and Broadcast Technologies
Components, Circuits, Devices and Systems
Engineering Profession
General Topics for Engineers
Power, Energy and Industry Applications
Signal Processing and Analysis
Simulation
Deep reinforcement learning
Behavioral sciences
Global communication
Vehicle-to-everything
Radio spectrum management
V2x
resource allocation
multi-agent reinforcement learning
social roles
Language
ISSN
2576-6813
Abstract
In a vehicle-to-everything (V2X) communication system involving multiple vehicle types, there is a more challenging and practical problem compared to a single-type scenario. Each vehicle type acts autonomously with distinct communication policies. While prior knowledge can establish behavior for each agent type, it may reduce the adaptability and versatility of the system. This paper proposes a role-oriented actor-critic (ROAC) approach, where vehicles of similar types share similar policies in a satellite-assisted V2X network for more precise and effective spectrum management. The vehicles are trained to optimize system utility by selecting transmission modes, power levels, and sub-channels. The social role properties enable each agent to make better decisions based on the environment and its type. The ROAC model provides 8-10% higher normalized system utility over other advanced methods, even with vehicle-role extension, in situations with heavier traffic.