TY - JOUR
T1 - NormAttention-PSN: A High-frequency Region Enhanced Photometric Stereo Network with Normalized Attention
AU - Ju, Yakun
AU - Shi, Boxin
AU - Jian, Muwei
AU - Qi, Lin
AU - Dong, Junyu
AU - Lam, Kin Man
N1 - Funding Information:
The work was supported by the Key-Area Research and Development Program of Guangdong Province (2020B090928001), the Project of Strategic Importance Fund from The Hong Kong Polytechnic University (No. ZE1X), the National Key R &D Program of China under Grant (2018AAA0100602), the National Key Scientific Instrument and Equipment Development Projects of China (41927805), and the National Natural Science Foundation of China (61872012, 62136001, 61976123, 61601427), the Key Development Program for Basic Research of Shandong Province (ZR2020ZD44), and the Taishan Young Scholars Program of Shandong Province.
Publisher Copyright:
© 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.
PY - 2022/12
Y1 - 2022/12
N2 - Photometric stereo aims to recover the surface normals of a 3D object from various shading cues, establishing the relationship between two-dimensional images and the object geometry. Traditional methods usually adopt simplified reflectance models to approximate the non-Lambertian surface properties, while recently, photometric stereo based on deep learning has been widely used to deal with non-Lambertian surfaces. However, previous studies are limited in dealing with high-frequency surface regions, i.e., regions with rapid shape variations, such as crinkles, edges, etc., resulted in blurry reconstructions. To alleviate this problem, we present a normalized attention-weighted photometric stereo network, namely NormAttention-PSN, to improve surface orientation prediction, especially for those complicated structures. In order to address these challenges, in this paper, we (1) present an attention-weighted loss to produce better surface reconstructions, which applies a higher weight to the detail-preserving gradient loss in high-frequency areas, (2) adopt a double-gate normalization method for non-Lambertian surfaces, to explicitly distinguish whether the high-frequency representation is stimulated by surface structure or spatially varying reflectance, and (3) adopt a parallel high-resolution structure to generate deep features that can maintain the high-resolution details of surface normals. Extensive experiments on public benchmark data sets show that the proposed NormAttention-PSN significantly outperforms traditional calibrated photometric stereo algorithms and state-of-the-art deep learning-based methods.
AB - Photometric stereo aims to recover the surface normals of a 3D object from various shading cues, establishing the relationship between two-dimensional images and the object geometry. Traditional methods usually adopt simplified reflectance models to approximate the non-Lambertian surface properties, while recently, photometric stereo based on deep learning has been widely used to deal with non-Lambertian surfaces. However, previous studies are limited in dealing with high-frequency surface regions, i.e., regions with rapid shape variations, such as crinkles, edges, etc., resulted in blurry reconstructions. To alleviate this problem, we present a normalized attention-weighted photometric stereo network, namely NormAttention-PSN, to improve surface orientation prediction, especially for those complicated structures. In order to address these challenges, in this paper, we (1) present an attention-weighted loss to produce better surface reconstructions, which applies a higher weight to the detail-preserving gradient loss in high-frequency areas, (2) adopt a double-gate normalization method for non-Lambertian surfaces, to explicitly distinguish whether the high-frequency representation is stimulated by surface structure or spatially varying reflectance, and (3) adopt a parallel high-resolution structure to generate deep features that can maintain the high-resolution details of surface normals. Extensive experiments on public benchmark data sets show that the proposed NormAttention-PSN significantly outperforms traditional calibrated photometric stereo algorithms and state-of-the-art deep learning-based methods.
KW - Deep neural network
KW - High-frequency surface normals
KW - Non-Lambertian
KW - Photometric stereo
UR - http://www.scopus.com/inward/record.url?scp=85138822569&partnerID=8YFLogxK
U2 - 10.1007/s11263-022-01684-8
DO - 10.1007/s11263-022-01684-8
M3 - Journal article
AN - SCOPUS:85138822569
SN - 0920-5691
VL - 130
SP - 3014
EP - 3034
JO - International Journal of Computer Vision
JF - International Journal of Computer Vision
IS - 12
ER -