A Q -Learning Based Framework for Congested Link Identification

Shengli Pan, Peng Li, Deze Zeng, Song Guo, Guangmin Hu

Research output: Journal article publicationJournal articleAcademic researchpeer-review

4 Citations (Scopus)

Abstract

Network congestion will result in significant performance degradation or even failures of many bandwidth-hungry Internet of Things (IoT) applications. Accurate and efficient congested link identification has become a foundational issue to IoT applications like self-driving cars, digital health, smart city, and so on. However, directly monitoring the massive number of interior links often introduces high operation cost or even is infeasible in practice, giving rise to indirect monitoring techniques like network Boolean tomography. Nevertheless, in many networks, the number of their interior links is larger than their end-to-end paths, making it very challenging for network Boolean tomography to find a determined solution. To resolve this issue, most of current methods try to utilize some prerequisites, such as the link congestion probabilities. While these probabilities might be hard or even unable to be obtained accurately in dynamical networks, limiting the practical deployment. In this paper, we are motivated to design a framework of congested link identification without any prerequisite or assumption. We first novelly model the congested link identification procedures as a Markov decision processes (MDPs), and then employ a reinforcement learning technology, i.e., Q -learning, to solve this MDP. The simulation results show that our proposed scheme can autonomously and efficiently explore the unknown network environment, and is able to achieve better adaptivity and correctness, without any prior knowledge comparing to existing methods.

Original languageEnglish
Article number8769959
Pages (from-to)9668-9678
Number of pages11
JournalIEEE Internet of Things Journal
Volume6
Issue number6
DOIs
Publication statusPublished - Dec 2019

Keywords

  • Congested link identification
  • end-to-end
  • network management
  • network tomography
  • reinforcement learning

ASJC Scopus subject areas

  • Signal Processing
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications

Cite this