Modeling and energy dynamic control for a ZEH via hybrid model-based deep reinforcement learning

Efficient and flexible energy management strategy can play an important role in energy conservation in building sector. The model-free reinforcement learning control of building energy systems generally requires an enormous amount of training data and low learning efficiency creates an obstacle to practice. This work proposes a hybrid model-based reinforcement learning framework to optimize the indoor thermal comfort and energy cost-saving performances of a ZEH (zero energy house) space heating system using relatively short-period monitored data. The reward function is designed regarding energy cost, PV self-consumption and thermal discomfort, proposed agents can interact with the reduced-order thermodynamic model and an uncertain environment, and makes optimal control policies through the learning process. Simulation results demonstrate that proposed agents achieve efficient convergence, D3QN presents a superiority of convergence performance. To evaluate the performances of proposed algorithms, the trained agents are tested using monitored data. With learned policies, the self-learning agents could balance the needs of thermal comfort, energy cost saving and increasing on-site PV consumption compared with the baselines. The comparative analysis shows that D3QN achieved over 30% cost savings compared with measurement results. D3QN outperforms DQN and Double DQN agents in test scenarios maintaining more stable temperatures under various outside conditions.