Robust Online Matrix Factorization for Dynamic Background Subtraction

Hongwei Yong, Deyu Meng, Wangmeng Zuo, Lei Zhang

Research output: Journal article publicationJournal articleAcademic researchpeer-review

81 Citations (Scopus)


We propose an effective online background subtraction method, which can be robustly applied to practical videos that have variations in both foreground and background. Different from previous methods which often model the foreground as Gaussian or Laplacian distributions, we model the foreground for each frame with a specific mixture of Gaussians (MoG) distribution, which is updated online frame by frame. Particularly, our MoG model in each frame is regularized by the learned foreground/background knowledge in previous frames. This makes our online MoG model highly robust, stable and adaptive to practical foreground and background variations. The proposed model can be formulated as a concise probabilistic MAP model, which can be readily solved by EM algorithm. We further embed an affine transformation operator into the proposed model, which can be automatically adjusted to fit a wide range of video background transformations and make the method more robust to camera movements. With using the sub-sampling technique, the proposed method can be accelerated to execute more than 250 frames per second on average, meeting the requirement of real-time background subtraction for practical video processing tasks. The superiority of the proposed method is substantiated by extensive experiments implemented on synthetic and real videos, as compared with state-of-the-art online and offline background subtraction methods.

Original languageEnglish
Pages (from-to)1726-1740
Number of pages15
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number7
Publication statusPublished - 1 Jul 2018


  • Backgroun0d subtraction
  • low-rank matrix factorization
  • mixture of Gaussians
  • online learning
  • subspace learning

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Cite this