학술논문
분산 강화학습 기반 무선랜 네트워크 최적화 / Optimization of wireless LAN networks based on distributed reinforcement learning
Document Type
Dissertation/ Thesis
Author
Source
Subject
Language
Korean
Abstract
인공지능 (Artificial Intelligence: AI)와 사물 인터넷 (Internet of Things: IoT)는 일상에서 사용자들에게 이전보다 높은 편의성과 접근성을 제공하고 있다. IoT 환경은 막대한 양의 데이터 교환이 다양한 기기 간의 통신으로 이뤄진다. 5G, 6G와 같은 고속통신망의 등장은 기존의 유·무선 통신망이 가졌던 통신속도의 한계를 개선함으로써 IoT 서비스를 더욱 발전시켰다. 다양한 무선기기들의 IoT 서비스를 위해 Wi-Fi를 사용한 무선랜 네트워크 시스템이 주로 사용되고 있으며, 이를 위해 다수의 액세스 포인트 (Access Point: AP)가 필요하다. 그러나 단순히 실내에 설치하는 AP를 늘릴 경우, AP 간의 신호 간섭이 증가함으로써 신호 대 잡음간섭비 (Signal to Interference plus Noise Ratio: SINR)은 낮아지게 된다. 증가한 AP의 신호 간섭과 낮은 SINR은 스테이션 (Station: STA)이 연결되기 원하는 AP에 연결되지 않는 상황이 발생하기 때문에, 간섭 전력 대비 높은 신호가 요구된다. 이런 상황을 해결하기 위해 AP의 송신전력 최적화 연구가 진행됐으나, 데이터 처리량 최대화가 목적이었으며 STA의 공정성을 동시에 고려한 연구는 아직 미비하다. 또한, 실내에서 AP를 사용하는 대상은 IoT 기기들뿐만 아니라, 사용자들도 존재하기 때문에 AP 링크의 수는 같은 장소여도 시간별로 달라질 수 있다. 제한된 자원을 사용하는 무선통신에서 다수의 링크가 존재할 경우, 링크 당 사용 가능한 자원은 감소하게 되며, 같은 시간대에 신호들이 충돌하는 횟수는 증가하여 데이터 교환이 중요한 환경에서 문제로 언급된다.해당 연구에서는 간섭의 증가, 불균형적인 링크의 분포를 해결하기 위한 접근으로 송신전력과 IEEE 802.11 물리계층 기능 중 하나인 가용 채널 평가 임계치 (Clear Channel Assessment threshold: CCA threshold)를 조절하여 STA의 데이터 처리량과 공정성을 강화학습으로 계산했다. 그러나 전통적인 강화학습은 계산이 단순한 환경에선 효율적이나, 상태가 늘어나는 복잡한 환경에서는 그 한계가 존재한다. 고려할 에이전트의 수가 늘어날수록 계산의 복잡성과 크기가 증가하기 때문에 전통적인 강화학습 대신, 각 에이전트가 학습을 이어가는 분산형 강화학습을 통한 접근이 필요하다. 연구에서는 AP 2개부터 AP 4개의 통신환경을 고려하여 진행했다. 그 결과, 전통적인 강화학습과 유사한 성능을 지니는 분산형 강화학습이 좀 더 빠른 속도로 학습이 가능한 것을 확인했다. 또한, 송신전력과 CCA threshold의 조절로 전체 무선 LAN의 성능을 최대화가 가능함을 확인했다. 해당 결과를 통해 실내 AP의 효율적인 제어는 송신전력과 CCA threshold를 사용하는 것이 효율적임을 결론지었다.
Artificial Intelligence (AI) and Internet of Things (IoT) provide users withgreater convenience and accessibility in everyday life. IoT environments involve avast amount of data exchange between various devices through wirelesscommunication networks. The emergence of high speed communication networkssuch as 5G and 6G has further developed IoT services by improving thecommunication speed limitations of existing wired and wireless networks. WirelessLAN network systems that use Wi-Fi are mainly used for IoT services for variouswireless devices with multiple access points (APs). However, simply increasing thenumber of APs installed indoors leads to increased signal interference between APs,resulting in a decrease in the signal to interference plus noise ratio (SINR).Increased AP signal interference and low SINR lead to situations where the station(STA) cannot connect to the desired AP, requiring a higher SINR. To address thisissue, research has been conducted on optimizing the transmission power of APs,but this was mainly focused on maximizing throughput, and research consideringthe fairness of STAs is still lacking. Additionally, as indoor AP users include notonly IoT devices but also users, the number of AP links can vary by time, evenin the same location. In wireless communication, where limited resources are used,the number of links decreases per available resource, and the number of collisionsbetween signals increases, causing problems in environments where data exchange iscritical.In this study, the approach to solving the problem of increased interference anduneven distribution of links was to adjust the transmission power and ClearChannel Assessment threshold (CCA threshold), one of the IEEE 802.11 physicallayer functions, to enhance the throughput and fairness of STAs throughreinforcement learning. However traditional reinforcement learning has its limits incomplex environments with increasing states because the complexity and size ofcomputation increase with the number of agents to consider. Therefore a distributedreinforcement learning approach where each agent continues its learning process isnecessary instead of traditional reinforcement learning. The study consideredwireless LAN networks including 2 to 4 APs. As a result, it was confirmed thatdistributed reinforcement learning has similar performance to traditional, i.e.,centralized, reinforcement learning and the distributed approach can learn at a fasterspeed than the centralized one. Furthermore, it was confirmed that adjusting thetransmission power and CCA threshold can maximize the performance of the entirewireless LAN networks in terms of throughput and fairness. Based on these results,it was concluded that efficient control of indoor APs is achieved by using thetransmission power and CCA threshold control mechanism
Artificial Intelligence (AI) and Internet of Things (IoT) provide users withgreater convenience and accessibility in everyday life. IoT environments involve avast amount of data exchange between various devices through wirelesscommunication networks. The emergence of high speed communication networkssuch as 5G and 6G has further developed IoT services by improving thecommunication speed limitations of existing wired and wireless networks. WirelessLAN network systems that use Wi-Fi are mainly used for IoT services for variouswireless devices with multiple access points (APs). However, simply increasing thenumber of APs installed indoors leads to increased signal interference between APs,resulting in a decrease in the signal to interference plus noise ratio (SINR).Increased AP signal interference and low SINR lead to situations where the station(STA) cannot connect to the desired AP, requiring a higher SINR. To address thisissue, research has been conducted on optimizing the transmission power of APs,but this was mainly focused on maximizing throughput, and research consideringthe fairness of STAs is still lacking. Additionally, as indoor AP users include notonly IoT devices but also users, the number of AP links can vary by time, evenin the same location. In wireless communication, where limited resources are used,the number of links decreases per available resource, and the number of collisionsbetween signals increases, causing problems in environments where data exchange iscritical.In this study, the approach to solving the problem of increased interference anduneven distribution of links was to adjust the transmission power and ClearChannel Assessment threshold (CCA threshold), one of the IEEE 802.11 physicallayer functions, to enhance the throughput and fairness of STAs throughreinforcement learning. However traditional reinforcement learning has its limits incomplex environments with increasing states because the complexity and size ofcomputation increase with the number of agents to consider. Therefore a distributedreinforcement learning approach where each agent continues its learning process isnecessary instead of traditional reinforcement learning. The study consideredwireless LAN networks including 2 to 4 APs. As a result, it was confirmed thatdistributed reinforcement learning has similar performance to traditional, i.e.,centralized, reinforcement learning and the distributed approach can learn at a fasterspeed than the centralized one. Furthermore, it was confirmed that adjusting thetransmission power and CCA threshold can maximize the performance of the entirewireless LAN networks in terms of throughput and fairness. Based on these results,it was concluded that efficient control of indoor APs is achieved by using thetransmission power and CCA threshold control mechanism