Energy-efficient power allocation and user association in heterogeneous networks with deep reinforcement learning

Chi Kai Hsieh, Kun Lin Chan, Feng Tsun Chien*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


This paper studies the problem of joint power allocation and user association in wireless heterogeneous networks (HetNets) with a deep reinforcement learning (DRL)-based approach. This is a challenging problem since the action space is hybrid, consisting of continuous actions (power allocation) and discrete actions (device association). Instead of quantizing the continuous space (i.e., possible values of powers) into a set of discrete alternatives and applying traditional deep reinforcement approaches such as deep Q learning, we propose working on the hybrid space directly by using the novel parameterized deep Q-network (P-DQN) to update the learning policy and maximize the average cumulative reward. Furthermore, we incorporate the constraints of limited wireless backhaul capacity and the quality-of-service (QoS) of each user equipment (UE) into the learning process. Simulation results show that the proposed P-DQN outperforms the traditional approaches, such as the DQN and distance-based association, in terms of energy efficiency while satisfying the QoS and backhaul capacity constraints. The improvement in the energy efficiency of the proposed P-DQN on average may reach 77.6% and 140.6% over the traditional DQN and distance-based association approaches, respectively, in a HetNet with three SBS and five UEs.

Original languageEnglish
Article number4135
JournalApplied Sciences (Switzerland)
Issue number9
StatePublished - 1 May 2021


  • Energy efficiency
  • Power allocation
  • Reinforcement learning
  • User clustering

Fingerprint Dive into the research topics of 'Energy-efficient power allocation and user association in heterogeneous networks with deep reinforcement learning'. Together they form a unique fingerprint.

Cite this