Sustainable Life-Cycle Maintenance Policymaking for Network-Level Deteriorating Bridges with a Convolutional Autoencoder-Structured Reinforcement Learning Agent

Bridges play a significant role in urban areas, and their performance and safety are highly related to the carbon emissions of infrastructure systems. Previous studies have mainly offered maintenance policies that balance structural safety with overall costs. Considering the goal of achieving near-zero global carbon emissions by 2050, this study proposes a policymaking agent based on a convolutional autoencoder-structured deep-Q network (ConvAE-DQN) for managing deteriorating bridges at the network level while considering sustainability performance. This agent considers environmental, economic, and safety metrics, including spatially correlated structural failure probability, traffic volume, bridge size, and others, which are transformed into a multiattribute utility model to form the reward function. Reinforcement learning is employed to optimize the life-cycle maintenance planning to minimize the total carbon emissions and economic costs while maximizing regional safety performance. The proposed method is substantiated by developing sustainable life-cycle maintenance policies for an existing bridge network in Northern China. It is found that the proposed ConvAE-DQN policymaking agent could output efficient and sustainable life-cycle maintenance policies, which are annually stable and easy to schedule. The utility-based reward function enhances the stability and convergence efficiency of the algorithm. This study also assesses the impact of budget levels on network-level bridge safety and carbon footprint.