Guided probabilistic reinforcement learning for sampling-efficient maintenance scheduling of multi-component system

Yiming Zhang, Dingyang Zhang, Xiaoge Zhang, Lemiao Qiu, Felix T.S. Chan, Zili Wang, Shuyou Zhang

Research output: Journal article publicationJournal articleAcademic researchpeer-review

5 Citations (Scopus)

Abstract

In recent years, multi-agent deep reinforcement learning has progressed rapidly as reflected by its increasing adoptions in industrial applications. This paper proposes a Guided Probabilistic Reinforcement Learning (Guided-PRL) model to tackle maintenance scheduling of multi-component systems in the presence of uncertainty with the goal of minimizing the overall life-cycle cost. The proposed Guided-PRL is deeply rooted in the Actor-Critic (AC) scheme. Since traditional AC falls short in sampling efficiency and suffers from getting stuck in local minima in the context of multi-agent reinforcement learning, it is thus challenging for the actor network to converge to a solution of desirable quality even when the critic network is properly configured. To address these issues, we develop a generic framework to facilitate effective training of the actor network, and the framework consists of environmental reward modeling, degradation formulation, state representation, and policy optimization. The convergence speed of the actor network is significantly improved with a guided sampling scheme for environment exploration by exploiting rules-based domain expert policies. To handle data scarcity, the environmental modeling and policy optimization are approximated with Bayesian models for effective uncertainty quantification. The Guided-PRL model is evaluated using the simulations of a 12-component system as well as GE90 and CFM56 engines. Compared with four alternative deep reinforcement learning schemes, the Guided-PRL lowers life-cycle cost by 34.92% to 88.07%. In comparison with rules-based expert policies, the Guided-PRL decreases the life-cycle cost by 23.26% to 51.36%.

Original languageEnglish
Pages (from-to)677-697
Number of pages21
JournalApplied Mathematical Modelling
Volume119
DOIs
Publication statusPublished - Jul 2023

Keywords

  • Deep Reinforcement Learning
  • Maintenance Scheduling
  • Multi-component System
  • Probabilistic Machine Learning
  • Sampling-Efficient Learning

ASJC Scopus subject areas

  • Modelling and Simulation
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Guided probabilistic reinforcement learning for sampling-efficient maintenance scheduling of multi-component system'. Together they form a unique fingerprint.

Cite this