TY - GEN
T1 - Learning Photometric Stereo via Manifold-based Mapping
AU - Ju, Yakun
AU - Jian, Muwei
AU - Dong, Junyu
AU - Lam, Kin Man
N1 - Funding Information:
This paper proposes a deep learning method for photometric stereo. The proposed method can take an arbitrary number of images as input and regresses a fine surface normal. We apply the ISoMap method for mapping the combined highdimensional features from the extractor network to a lowdimensional manifold, rather than using the max pooling fusion method. Experiments on a public benchmark show that our method outperforms existing approaches on objects with complicated structures and strongly non-Lambertian surfaces. We have further evaluated our method on the utilization rate, by measuring how well our method can extract useful information from input images, when the number is increasing. Based on the experiment results, we can conclude that our method can achieve state-of-the-art performance, when the number of input images increases to a certain level. Acknow ledgm ent The work was supported by the National Key Scientific Instrument and Equipment Development Projects of China (41927805), the National Natural Science Foundation of China (61501417, 61976123, 41906177), and the Taishan Young Scholars Program of Shandong Province.
Publisher Copyright:
© 2020 IEEE.
PY - 2020/12/1
Y1 - 2020/12/1
N2 - Three-dimensional reconstruction technologies are fundamental problems in computer vision. Photometric stereo recovers the surface normals of a 3D object from varying shading cues, prevailing in its capability for generating fine surface normal. In recent years, deep learning-based photometric stereo methods are capable of improving the surface-normal estimation under general non-Lambertian surfaces, due to its powerful fitting ability on the non-Lambertian surface. These state-of-the-art methods however usually regress the surface normal directly from the high-dimensional features, without exploring the embedded structural information. This results in the underutilization of the information available in the features. Therefore, in this paper, we propose an efficient manifold-based framework for learning-based photometric stereo, which can better map combined high-dimensional feature spaces to low-dimensional manifolds. Extensive experiments show that our method, learning with the low-dimensional manifolds, achieves more accurate surface-normal estimation, outperforming other state-of-the-art methods on the challenging DiLiGenT benchmark dataset.
AB - Three-dimensional reconstruction technologies are fundamental problems in computer vision. Photometric stereo recovers the surface normals of a 3D object from varying shading cues, prevailing in its capability for generating fine surface normal. In recent years, deep learning-based photometric stereo methods are capable of improving the surface-normal estimation under general non-Lambertian surfaces, due to its powerful fitting ability on the non-Lambertian surface. These state-of-the-art methods however usually regress the surface normal directly from the high-dimensional features, without exploring the embedded structural information. This results in the underutilization of the information available in the features. Therefore, in this paper, we propose an efficient manifold-based framework for learning-based photometric stereo, which can better map combined high-dimensional feature spaces to low-dimensional manifolds. Extensive experiments show that our method, learning with the low-dimensional manifolds, achieves more accurate surface-normal estimation, outperforming other state-of-the-art methods on the challenging DiLiGenT benchmark dataset.
KW - 3D reconstruction
KW - deep learning
KW - manifold-based mapping
KW - photometric stereo
UR - http://www.scopus.com/inward/record.url?scp=85099475959&partnerID=8YFLogxK
U2 - 10.1109/VCIP49819.2020.9301860
DO - 10.1109/VCIP49819.2020.9301860
M3 - Conference article published in proceeding or book
AN - SCOPUS:85099475959
T3 - 2020 IEEE International Conference on Visual Communications and Image Processing, VCIP 2020
SP - 411
EP - 414
BT - 2020 IEEE International Conference on Visual Communications and Image Processing, VCIP 2020
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2020 IEEE International Conference on Visual Communications and Image Processing, VCIP 2020
Y2 - 1 December 2020 through 4 December 2020
ER -