Recovering traffic data from the corrupted noise: A doubly physics-regularized denoising diffusion model

Zhenjie Zheng, Zhengli Wang, Zijian Hu, Zihan Wan, Wei Ma

Research output: Journal article publicationJournal articleAcademic researchpeer-review

Abstract

Noise is inevitable in the collection of traffic data, which may cause accuracy and stability issues in smart mobility applications. In the literature, most of the existing studies on traffic data denoising assume that noises follow specific distributions (e.g., Gaussian) or structures (e.g., sparsity). However, various noises may coexist in the traffic data and their distributions or structures are usually unknown in the real world. This leads to the corrupted noise and brings a huge challenge to the traffic data recovery due to the irregular characteristics of such noise. In this research, we develop a doubly physics-regularized denoising diffusion model to address this issue. The proposed denoising diffusion model gradually adds noise to the data and then learns to recover the true value from noises. Each layer of the model can be perceived as a filter that removes noises with different scales, which helps to disentangle the coupling of noises. To make up for the insufficient information of noise distributions or structures, we incorporate prior traffic domain knowledge of the fundamental diagram into the denoising diffusion model using. Under suitable assumptions, we prove that our model can output an unbiased estimation of the true value. We validate our model using two real datasets collected in Japan and China. Results demonstrate that our model outperforms other denoising methods, such as filter-based methods, compressed sensing, robust principal component analysis, and machine learning methods.

Original languageEnglish
Article number104513
JournalTransportation Research Part C: Emerging Technologies
Volume160
DOIs
Publication statusPublished - Mar 2024

Keywords

  • Corrupted noise
  • Data recovery
  • Denoising diffusion model
  • Fundamental diagram

ASJC Scopus subject areas

  • Civil and Structural Engineering
  • Automotive Engineering
  • Transportation
  • Management Science and Operations Research

Fingerprint

Dive into the research topics of 'Recovering traffic data from the corrupted noise: A doubly physics-regularized denoising diffusion model'. Together they form a unique fingerprint.

Cite this