# Intelligent VNF Orchestration and Flow Scheduling via Model-Assisted Deep Reinforcement Learning

Lin Gu, Deze Zeng, Wei Li, Song Guo, Albert Y. Zomaya, Hai Jin

Research output: Journal article publicationJournal articleAcademic researchpeer-review

17 Citations (Scopus)

## Abstract

Hosting virtualized network functions (VNF) has been regarded as an effective way to realize network function virtualization (NFV). Considering the cost diversity in cloud computing, from the perspective of service providers, it is significant to orchestrate the VNFs and schedule the traffic flows for network utility maximization (NUM) as it implies maximal revenue. However, traditional heuristic solutions based on optimization models usually follow some assumptions, limiting their applicability. Recent studies have shown that deep reinforcement learning (DRL) is a promising way to tackle such limitations. However, DRL agent training also suffers from slow convergence problem, especially with complex control problems. We notice that optimization models actually can be applied to accelerate the DRL training. Therefore, we are motivated to design a model-assisted DRL framework for VNF orchestration in this paper. Other than letting the agent blindly explore actions, the heuristic solutions are used to guide the training process. Based on such principle, the DRL framework is also redesigned accordingly. Experiment results validate the high efficiency of our model-assisted DRL framework as it not only converges $23\times$ faster than traditional DRL algorithm, but also with higher performance at the same time.

Original language English 8931775 279-291 13 IEEE Journal on Selected Areas in Communications 38 2 https://doi.org/10.1109/JSAC.2019.2959182 Published - Feb 2020 Yes

## Keywords

• deep reinforcement learning
• flow scheduling
• network utility maximization
• VNF orchestration

## ASJC Scopus subject areas

• Computer Networks and Communications
• Electrical and Electronic Engineering