Deep reinforcement learning based VNF management in geo-distributed edge computing

Lin Gu, Deze Zeng, Wei Li, Song Guo, Albert Zomaya, Hai Jin

Research output: Unpublished conference presentation (presented paper, abstract, poster)Conference presentation (not published in journal/proceeding/book)Academic researchpeer-review

10 Citations (Scopus)


Edge computing is an effective approach for resource provisioning at the network edge to host virtualized network functions (VNF). Considering the cost diversity in edge computing, from the perspective of service providers, it is significant to orchestrate the VNFs and schedule the traffic flows for network utility maximization (NUM) as it implies maximal revenue. However, traditional model-based optimization methods usually follow some assumptions and impose certain limitations. In this paper, inspired by the success of deep reinforcement learning in solving complicated control problems, we propose a deep deterministic policy gradients (DDPG) based algorithm. We first formulate the NUM problem with the consideration of end-to-end delays and various operation costs into a non-convex optimization problem and prove it to be NP-hard. We then redesign the exploration method and invent a dual replay buffer structure to customize the DDPG. Meanwhile, we also apply our formulation to guide our replay buffer update. Through extensive trace-driven experiments, we show the high efficiency of our customized DDPG based algorithm as it significantly outperforms both model-based methods and traditional non-customized DDPG based algorithm.

Original languageEnglish
Number of pages10
Publication statusPublished - Jul 2019
Event39th IEEE International Conference on Distributed Computing Systems, ICDCS 2019 - Richardson, United States
Duration: 7 Jul 20199 Jul 2019


Conference39th IEEE International Conference on Distributed Computing Systems, ICDCS 2019
Country/TerritoryUnited States


  • Deep Reinforcement Learning
  • Flow Scheduling
  • VNF Orchestration

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

Cite this