ℓ-Step-Ahead Active Learning-Based Dual Control for Exploration and Exploitation in Auto-Optimization

In this paper, a ℓ-step-ahead active learning-based dual control for exploration and exploitation (LAL-DCEE) is proposed to address the challenges of auto-optimization amidst unknown references and environments. The algorithm of LALDCEE features a dual-loop structure, employing future gradients by looking ahead ℓ-step to guide the next control command. This approach comprises an inner loop for fast gradient updates and an outer loop for slow gradient updates. Specifically, gradients for future ℓ-step are iteratively refined within the inner loop using ensemble-based active learning, enabling rapid calculation of cost function gradients (i.e. estimated reference trajectory). These refined gradients then inform the outer loop, where a dual controller tailored for a system characterized by a general linear form steers the controlled system. The stability analysis of LAL-DCEE has been rigorously established. Additionally, numerical examples are employed to illustrate the effectiveness of the proposed method.