Learning Gated Non-Local Residual for Single-Image Rain Streak Removal

Lei Zhu, Zijun Deng, Xiaowei Hu, Haoran Xie, Xuemiao Xu, Jing Qin, Pheng Ann Heng

Research output: Journal article publicationJournal articleAcademic researchpeer-review

36 Citations (Scopus)


This work presents a gated non-local deep residual learning framework for image deraining. It can avoid the over-deraining or under-deraining caused by the global residual learning in existing deraining networks, since the learned soft gate in our method adaptively adjusts the amount of global residual to be passed for generating the final derained result. To generate feature maps for global residual prediction, we develop a non-local guided attention module (NLAM), which first obtains non-local features by exploiting spatial inter-dependencies among all the feature positions of local features produced by convolutional neural network (CNN), and then leverages the attention mechanism to merge the local and non-local features based on their complementary relation. Moreover, we develop a channel-wise gated prediction module to learn a soft gate on the global residual by explicitly modelling channel inter-dependencies of the feature maps obtained from NLAM. Experiments on four deraining benchmark datasets and real-world rainy images show that our network has a quantitative and qualitative improvement over state-of-the-arts.

Original languageEnglish
Article number9187841
Pages (from-to)2147-2159
Number of pages13
JournalIEEE Transactions on Circuits and Systems for Video Technology
Issue number6
Publication statusPublished - Jun 2021


  • gated learning
  • non-local residual learning
  • Single-image rain streak removal

ASJC Scopus subject areas

  • Media Technology
  • Electrical and Electronic Engineering


Dive into the research topics of 'Learning Gated Non-Local Residual for Single-Image Rain Streak Removal'. Together they form a unique fingerprint.

Cite this