학술논문

ESN Reinforcement Learning for Spectrum and Flight Control in THz-Enabled Drone Networks
Document Type
Periodical
Source
IEEE/ACM Transactions on Networking IEEE/ACM Trans. Networking Networking, IEEE/ACM Transactions on. 30(2):782-795 Apr, 2022
Subject
Communication, Networking and Broadcast Technologies
Computing and Processing
Signal Processing and Analysis
Drones
Reinforcement learning
Wireless sensor networks
Microwave communication
Cellular networks
Autonomous aerial vehicles
Wireless networks
Terahertz band
Mmwave band
mircowave band
drone networks
echo state network
reinforcement learning
Language
ISSN
1063-6692
1558-2566
Abstract
Terahertz (THz)-band communications have been envisioned as a key technology to support ultra-high-data-rate applications in 5G-beyond (or 6G) wireless networks. Compared to the microwave and mmWave bands, the main challenges with the THz band are in its i) large path loss hence limited network coverage and ii) visible-light-like propagation characteristics hence poor support of mobility in blockage-rich environments. This paper studies quantitatively the applicability of THz-band communications in blockage-rich mobile environments, focusing on a new network scenario called FlyTera . In FlyTera , a set of hotspots mounted on flying drones collaboratively provide data streaming services to ground users, in the microwave, mmWave and THz bands. We first provide a mathematical formulation of the FlyTera control problem, where the objective is to maximize the network spectral efficiency by jointly controlling the flight of the drone hotspots, their association to the ground users, and the spectrum bands used by the users. To solve the resulting problem, which is shown to be a mixed integer nonlinear nonconvex programming (MINLP) problem, we design distributed solution algorithms based on a combination of echo state learning and reinforcement learning. An extensive simulation campaign is then conducted with SimBAG, a newly developed Simulator of Broadband Aerial-Ground wireless networks. It is shown that no single spectrum band can meet the requirements of high data rate and wide coverage simultaneously. Moreover, from the network-level point of view, THz-band communications can significantly benefit from the mobility of the flying drones, and on average $4 - 6$ times higher (rather than lower) throughput can be achieved in mobile than in static environments.