Title | UAV-to-Device Underlay Communications: Age of Information Minimization by Multi-Agent Deep Reinforcement Learning |
Authors | Wu, Fanyi Zhang, Hongliang Wu, Jianjun Han, Zhu Poor, H. Vincent Song, Lingyang |
Affiliation | Peking Univ, Dept Elect Engn, Beijing 100871, Peoples R China Princeton Univ, Dept Elect & Comp Engn, Princeton, NJ 08544 USA Univ Houston, Dept Elect & Comp Engn, Houston, TX 77004 USA Kyung Hee Univ, Dept Comp Sci & Engn, Seoul 446701, South Korea |
Keywords | RESOURCE-ALLOCATION DESIGN OPTIMIZATION 5G |
Issue Date | Jul-2021 |
Publisher | IEEE TRANSACTIONS ON COMMUNICATIONS |
Abstract | In recent years, unmanned aerial vehicles (UAVs) have unlocked numerous sensing applications, which are expected to add billions of dollars to the world economy in the next decade. To further improve the Quality-of-Service in these applications, the 3rd Generation Partnership Project has considered the use of terrestrial cellular networks to support UAV sensing services, also known as the cellular Internet of UAVs. In this paper, we consider a cellular Internet of UAVs, where the sensory data can be transmitted either to the base station via cellular links, or to the mobile devices by underlay UAV-to-Device (U2D) communications. To evaluate the freshness of the sensory data, the concept of age of information (AoI) is adopted, in which a lower AoI implies fresher data. Since UAVs' AoIs are determined by their trajectories during sensing and transmission, we investigate the AoI minimization problem for UAVs by designing their trajectories. This problem is a Markov decision problem with an infinite state-action space, and thus we utilize multi-agent deep reinforcement learning to approximate the state-action space. Then, we propose a multi-UAV trajectory design algorithm to solve this problem. Simulation results show that our proposed algorithm can achieve a lower AoI than a greedy algorithm, policy gradient algorithm, and overlay U2D scheme. |
URI | http://hdl.handle.net/20.500.11897/619057 |
ISSN | 0090-6778 |
DOI | 10.1109/TCOMM.2021.3065135 |
Indexed | SCI(E) |
Appears in Collections: | 工学院 |