Intelligent VNF Orchestration and Flow Scheduling via Model-Assisted Deep Reinforcement Learning

Lin Gu, Deze Zeng, Wei Li, Song Guo, Albert Y. Zomaya, Hai Jin

Research output: Journal article publicationJournal articleAcademic researchpeer-review

23 Citations (Scopus)


Hosting virtualized network functions (VNF) has been regarded as an effective way to realize network function virtualization (NFV). Considering the cost diversity in cloud computing, from the perspective of service providers, it is significant to orchestrate the VNFs and schedule the traffic flows for network utility maximization (NUM) as it implies maximal revenue. However, traditional heuristic solutions based on optimization models usually follow some assumptions, limiting their applicability. Recent studies have shown that deep reinforcement learning (DRL) is a promising way to tackle such limitations. However, DRL agent training also suffers from slow convergence problem, especially with complex control problems. We notice that optimization models actually can be applied to accelerate the DRL training. Therefore, we are motivated to design a model-assisted DRL framework for VNF orchestration in this paper. Other than letting the agent blindly explore actions, the heuristic solutions are used to guide the training process. Based on such principle, the DRL framework is also redesigned accordingly. Experiment results validate the high efficiency of our model-assisted DRL framework as it not only converges $23\times$ faster than traditional DRL algorithm, but also with higher performance at the same time.

Original languageEnglish
Article number8931775
Pages (from-to)279-291
Number of pages13
JournalIEEE Journal on Selected Areas in Communications
Issue number2
Publication statusPublished - Feb 2020
Externally publishedYes


  • deep reinforcement learning
  • flow scheduling
  • network utility maximization
  • VNF orchestration

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Cite this