Multimodal Spatio-Temporal Prediction with Stochastic Adversarial Networks

Research output: Journal article publicationJournal articleAcademic researchpeer-review

1 Citation (Scopus)


Spatio-temporal (ST) data is a collection of multiple time series data with different spatial locations and is inherently stochastic and unpredictable. An accurate prediction over such data is an important building block for several urban applications, such as taxi demand prediction, traffic flow prediction, and so on. Existing deep learning based approaches assume that outcome is deterministic and there is only one plausible future; therefore, cannot capture the multimodal nature of future contents and dynamics. In addition, existing approaches learn spatial and temporal data separately as they assume weak correlation between them. To handle these issues, in this article, we propose a stochastic spatio-temporal generative model (named D-GAN) which adopts Generative Adversarial Networks (GANs)-based structure for more accurate ST prediction in multiple time steps. D-GAN consists of two components: (1) spatio-temporal correlation network which models spatio-temporal joint distribution of pixels and supports a stochastic sampling of latent variables for multiple plausible futures; (2) a stochastic adversarial network to jointly learn generation and variational inference of data through implicit distribution modeling. D-GAN also supports fusion of external factors through explicit objective to improve the model learning. Extensive experiments performed on two real-world datasets show that D-GAN achieves significant improvements and outperforms baseline models.

Original languageEnglish
Article number18
Pages (from-to)1-22
JournalACM Transactions on Intelligent Systems and Technology
Issue number2
Publication statusPublished - Apr 2022


  • deep learning
  • Generative adversarial networks
  • spatio-temporal prediction

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Artificial Intelligence


Dive into the research topics of 'Multimodal Spatio-Temporal Prediction with Stochastic Adversarial Networks'. Together they form a unique fingerprint.

Cite this