Forecasting backdraft with multimodal method: Fusion of fire image and sensor data

Experienced firefighters can fuse the flame image, smoke pattern, and varying temperature, sound, and odour in complex and fast-changing fire scenes to foresee flashover and explosion. This study mimics firefighters and proposes a novel transformer algorithm for the fusion of fire images and temperature sensor data to forecast the backdraft explosion in a building fire. The model of backdraft forecast is demonstrated with full-scale fire tests. After training 2674 fire scenarios with various fire intensities and images from various view angles, the Fusion-Transformer model can forecast the risk of backdraft with an overall accuracy of 84%. Moreover, the occurrence time and explosion scale of backdraft can be predicted with the Mean Absolute Error (MAE) of 1.6 s and 0.14 m, respectively. Compared with the single modal model, the fusion of fire images and temperature sensor data improves the accuracy of backdraft forecast by over 50%. This work demonstrates the use of a transformer algorithm in forecasting fire evolution and critical events. It also bridges the gap between data fusion methods and fire forecast, which inspires future universal AI-driven smart firefighting practices.