TY - JOUR
T1 - Fast User-Guided Single Image Reflection Removal via Edge-Aware Cascaded Networks
AU - Zhang, Huaidong
AU - Xu, Xuemiao
AU - He, Hai
AU - He, Shengfeng
AU - Han, Guoqiang
AU - Qin, Jing
AU - Wu, Dapeng
N1 - Funding Information:
Manuscript received June 16, 2019; revised September 21, 2019; accepted October 25, 2019. Date of publication November 4, 2019; date of current version July 24, 2020. The work was supported in part by National Natural Science Foundation of China under Grants 61772206, U1611461, and 61472145, in part by the Guangdong R&D Key Project of China under Grant 2018B010107003, in part by the Guangdong High-Level Personnel Program under Grant 2016TQ03X319, in part by the Guangdong National Science Foundation under Grant 2017A030311027, in part by the Guangzhou Key Project in Industrial Technology under Grant 201802010027, and in part by the CCF-Tencent Open Research Fund under Grant CCF-Tencent RAGR20190112. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. S. Rahardja. (Corresponding author: Xuemiao Xu.) H. Zhang, H. He, S. He, and G. Han are with the School of Computer Science and Engineering, South China University of Technology, Guangdong 510640, China (e-mail: [email protected]; [email protected]; [email protected]; [email protected]).
Publisher Copyright:
© 1999-2012 IEEE.
PY - 2020/8
Y1 - 2020/8
N2 - Taking photos through a glass window leads to glare or reflection, which might distract the viewer from the scene behind the window. In this paper, we involve user interaction to tackle the ill-posedness of the reflection removal problem. Users are allowed to draw strokes or lassos to indicate the background and reflection layers. Instead of designing hand-crafted features, we propose the edge-aware cascaded networks for reflection removal. The proposed network is a two-stage pipeline. The first stage takes the edge hints converted from user guidance and the image with reflection as input, and then separates the input image into the background and reflection layers. The second stage involves a refinement network to recover the missing details of the background layers. We simulate different types of user guidance, and the networks are trained on simulated data. The cascaded networks are end-to-end and perform with a single feed-forward pass, enabling fast editing. Extensive experimental evaluations demonstrate that the proposed used-guided reflection removal network yields better performance than the state-of-the-art methods on real-world scenarios. Furthermore, we show that novice users can easily generate reflection-free images, and large improvements in reflection removal quality can be obtained in just one minute.
AB - Taking photos through a glass window leads to glare or reflection, which might distract the viewer from the scene behind the window. In this paper, we involve user interaction to tackle the ill-posedness of the reflection removal problem. Users are allowed to draw strokes or lassos to indicate the background and reflection layers. Instead of designing hand-crafted features, we propose the edge-aware cascaded networks for reflection removal. The proposed network is a two-stage pipeline. The first stage takes the edge hints converted from user guidance and the image with reflection as input, and then separates the input image into the background and reflection layers. The second stage involves a refinement network to recover the missing details of the background layers. We simulate different types of user guidance, and the networks are trained on simulated data. The cascaded networks are end-to-end and perform with a single feed-forward pass, enabling fast editing. Extensive experimental evaluations demonstrate that the proposed used-guided reflection removal network yields better performance than the state-of-the-art methods on real-world scenarios. Furthermore, we show that novice users can easily generate reflection-free images, and large improvements in reflection removal quality can be obtained in just one minute.
KW - convolutional neural network
KW - image refinement
KW - Reflection removal
KW - user interaction
UR - http://www.scopus.com/inward/record.url?scp=85089341551&partnerID=8YFLogxK
U2 - 10.1109/TMM.2019.2951461
DO - 10.1109/TMM.2019.2951461
M3 - Journal article
AN - SCOPUS:85089341551
SN - 1520-9210
VL - 22
SP - 2012
EP - 2023
JO - IEEE Transactions on Multimedia
JF - IEEE Transactions on Multimedia
IS - 8
M1 - 8890835
ER -