학술논문

Reinforcement Contract Design for Vehicular-Edge Computing Scheduling and Energy Trading via Deep Q-Network With Hybrid Action Space
Document Type
Periodical
Source
IEEE Transactions on Mobile Computing IEEE Trans. on Mobile Comput. Mobile Computing, IEEE Transactions on. 23(6):6770-6784 Jun, 2024
Subject
Computing and Processing
Communication, Networking and Broadcast Technologies
Signal Processing and Analysis
Task analysis
Contracts
Processor scheduling
Computational modeling
Vehicle dynamics
Indexes
Dynamic scheduling
Charging scheduling
contract design
parameterized deep q-network
smart charging network
social welfare
vehicular edge computing
Language
ISSN
1536-1233
1558-0660
2161-9875
Abstract
The advancements in information and communication technology have led to the emergence of innovative edge computing models that incorporate the computing power of vehicles into the energy sector. Electric vehicles (EVs), functioning as edge computing nodes, offer flexible computing offloading services for charging stations (CS). However, coordinating EV computing and charging should consider the interdependence with CS's specific computing requirements due to information asymmetry. Additionally, it is crucial to consider EV's charging demands and their social distance to computing tasks. In this context, it is natural to view EVs and CSs as self-interested prosumers who prioritize their individual utilities. To address the integration of strategic EV-CS interactions and uncertainties into the joint computing scheduling and energy trading, this paper proposes a parameterized deep Q-network-based reinforcement contract design framework, which employs a hybrid action space to design contracts that facilitate CSs in pairing computing tasks and charging resources with EVs. The objective is to incentivize EV participation and maximize long-term social welfare by incorporating incentive compatibility, individual rationality constraints, and capacity constraints into the contract design. Experimental results demonstrate that the proposed framework surpasses parameterized deep deterministic policy gradient-based and greedy-based contract designs, and achieves near-optimal solutions by solving deterministic optimizations.