학술논문

QMIX-Based Multi-Agent Reinforcement Learning for Electric Vehicle-Facilitated Peak Shaving
Document Type
Conference
Source
GLOBECOM 2023 - 2023 IEEE Global Communications Conference Global Communications Conference, GLOBECOM 2023 - 2023 IEEE. :1693-1698 Dec, 2023
Subject
Communication, Networking and Broadcast Technologies
Components, Circuits, Devices and Systems
Engineering Profession
General Topics for Engineers
Power, Energy and Industry Applications
Signal Processing and Analysis
Costs
Simulation
Power system dynamics
Reinforcement learning
Electric vehicles
Numerical models
Vehicle dynamics
Peak shaving
electric vehicle
multi-agent rein-forcement learning
QMIX
Language
ISSN
2576-6813
Abstract
Given the energy storage capacity and rapid response capabilities, electric vehicles (EVs) hold the potential to offer auxiliary services, including peak shaving and frequency regulation, to the power grid during emergencies. Nevertheless, effectively coordinating EVs presents a complex challenge for multiple charging stations (CSs), which must integrate dynamic traffic fluctuations into a schedule while ensuring sufficient power resources for these auxiliary services. To tackle these challenges, this paper proposes an across-realm decision-making framework to dispatch EVs to CSs for peak shaving considering dynamic traffic conditions and power constraints. We develop a cooperative multi-agent reinforcement learning (MARL) strategy, which capitalizes on the collaboration among CSs by formulating the EV dispatching problem as a Markov Game. The CSs are regarded as learning agents and a QMIX network with bidding mechanism is applied to train the joint action towards maximizing long-term team rewards. The proposed method has been tested and compared to the existing reinforcement learning and non-reinforcement learning methods, and the simulation results have demonstrated the efficiency of the proposed approach considering real-world scenarios.