TY - GEN
T1 - A multi-task network for joint specular highlight detection and removal
AU - Fu, Gang
AU - Zhang, Qing
AU - Zhu, Lei
AU - Li, Ping
AU - Xiao, Chunxia
N1 - Funding Information:
We would like to thank Wenxi Gan and Siyuan Yang for their hard work in building the dataset. This work was partly supported by the National Key Research and Development Program of China (2017YFB1002600), the Key Technological Innovation Projects of Hubei Province (2018AAA062), NSFC (NO. 61972298), and CAAI-Huawei MindSpore Open Fund.
Publisher Copyright:
© 2021 IEEE
PY - 2021/6
Y1 - 2021/6
N2 - Specular highlight detection and removal are fundamental and challenging tasks. Although recent methods have achieved promising results on the two tasks by training on synthetic training data in a supervised manner, they are typically solely designed for highlight detection or removal, and their performance usually deteriorates significantly on real-world images. In this paper, we present a novel network that aims to detect and remove highlights from natural images. To remove the domain gap between synthetic training samples and real test images, and support the investigation of learning-based approaches, we first introduce a dataset with about 16K real images, each of which has the corresponding ground truths of highlight detection and removal. Using the presented dataset, we develop a multi-task network for joint highlight detection and removal, based on a new specular highlight image formation model. Experiments on the benchmark datasets and our new dataset show that our approach clearly outperforms state-of-the-art methods for both highlight detection and removal.
AB - Specular highlight detection and removal are fundamental and challenging tasks. Although recent methods have achieved promising results on the two tasks by training on synthetic training data in a supervised manner, they are typically solely designed for highlight detection or removal, and their performance usually deteriorates significantly on real-world images. In this paper, we present a novel network that aims to detect and remove highlights from natural images. To remove the domain gap between synthetic training samples and real test images, and support the investigation of learning-based approaches, we first introduce a dataset with about 16K real images, each of which has the corresponding ground truths of highlight detection and removal. Using the presented dataset, we develop a multi-task network for joint highlight detection and removal, based on a new specular highlight image formation model. Experiments on the benchmark datasets and our new dataset show that our approach clearly outperforms state-of-the-art methods for both highlight detection and removal.
UR - http://www.scopus.com/inward/record.url?scp=85118214208&partnerID=8YFLogxK
U2 - 10.1109/CVPR46437.2021.00766
DO - 10.1109/CVPR46437.2021.00766
M3 - Conference article published in proceeding or book
AN - SCOPUS:85118214208
T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
SP - 7748
EP - 7757
BT - Proceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021
PB - IEEE Computer Society
T2 - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021
Y2 - 19 June 2021 through 25 June 2021
ER -