Smooth Deep Reinforcement Learning for Power Control for Spectrum Sharing in Cognitive Radios

Lujuan Dang, Wanli Wang, Chi K. Tse, Francis C.M. Lau, Shiyuan Wang

Research output: Journal article publicationJournal articleAcademic researchpeer-review

Abstract

Spectrum sharing in a cognitive radio system involves a secondary user updating transmit power for sharing spectrum with a primary user. The deep <italic>Q</italic>-network in the framework of deep reinforcement learning achieves transmit power control by a deep neural network for learning a nonlinear mapping from states to <italic>Q</italic>-values. Since a deep neural network is confronted with noise susceptibility, the deep <italic>Q</italic>-network produces deteriorative network parameters and volatile <italic>Q</italic>-values in the presence of contaminated states. In view of the positive effect of kernel least mean square (KLMS) for signal smoothing, we combine KLMS with the deep <italic>Q</italic>-network for smoothing network-generated outputs. Since an inappropriate step size of KLMS causes under-smoothing or over-smoothing issues, a weighting procedure using past <italic>Q</italic>-values is proposed for cooperating with KLMS. We assess the incremental ratio of the success rate of the smooth deep <italic>Q</italic>-network to that of the deep <italic>Q</italic>-network (RSR) in cognitive radios. Simulations show that RSR has averaged almost over 30% at the early stage of power control. In particular, the maximum RSR reaches almost over 80% or 180% at different scenarios of power control for the primary user. In addition, the smooth deep <italic>Q</italic>-network achieves an improved success rate in comparison with other algorithms.

Original languageEnglish
Article number9810823
Pages (from-to)1-13
Number of pages13
JournalIEEE Transactions on Wireless Communications
DOIs
Publication statusPublished - Jun 2022

Keywords

  • cognitive radio system
  • deep reinforcement learning
  • Interference
  • Kernel
  • kernel least mean square
  • Neural networks
  • Power control
  • Quality of service
  • Receivers
  • Signal to noise ratio
  • Spectrum sharing
  • weighting procedure

ASJC Scopus subject areas

  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Smooth Deep Reinforcement Learning for Power Control for Spectrum Sharing in Cognitive Radios'. Together they form a unique fingerprint.

Cite this