Intelligent Network Selection Algorithm for Multiservice Users in 5G Heterogeneous Network System: Nash Q-Learning Method

The 5G heterogeneous network architecture integrates different radio access technologies (RATs), which will support the large-scale communication connection of massive Internet-of-Things (IoT) devices. However, as the rapid growth of IoT connections, personalized requirements of services requested and heterogeneity deepening of the network system, how to design an intelligent network selection scheme for user devices (UDs) is becoming a crucial challenge in the 5G heterogeneous network system. Most of the existing network selection methods only optimize the selection strategies from the user side or network side, which results in heavy network congestion, poor user experience, and system performance degradation. Accordingly, we propose a multiagent $Q$ -learning network selection (MAQNS) algorithm based on Nash $Q$ -learning, which can learn a joint optimal selection strategy to improve system throughput and reduce user blocking on the premise of ensuring the requirements of IoT services. In particular, we apply the discrete-time Markov chains to model the network selection, and the analytic hierarchy process (AHP) and gray relation analysis (GRA) are jointly utilized to obtain user preferences for each network. Finally, performance evaluation demonstrates that comparing to the existing schemes, MAQNS proposed cannot only improve system throughput and reduce user blocking but also promote user experience on average energy efficiency and delay.