Learning Deep Photometric Stereo Network with Reflectance Priors

Yakun Ju, Cong Zhang, Songsong Huang, Yuan Rao, Kin Man Lam

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

3 Citations (Scopus)


Photometric stereo recovers the surface normals of an object from images with varying shading cues. Conventional photometric stereo methods attempt to use handcrafted reflectance models to approximate surface normals, while deep learning-based networks have shown a much more powerful ability to handle non-Lambertian objects. However, none of the existing deep learning methods explores how prior reflectance information can be used to optimize surface-normal prediction. In this paper, we first present the introduction of reflectance prior to deep photometric stereo models. Our explorations include how the reflectance prior can simplify the optimization of deep networks by reparametrizing the weights, and (2) eliminate the impacts of surfaces with spatially varying reflectance for all-pixel input photometric stereo methods. To achieve these goals, we propose a residual fusion module (RFM) in our method, which explicitly extracts features useful for surface-normal recovery and removes those features influenced by reflectance. Additionally, we design a shading extractor with multi-scale and global-local feature fusion operations, which can fuse features with different receptive fields and better utilize the non-maximum features missing in the max-pooling operation. Experiments and ablation studies verify the accuracy and effectiveness of the proposed reflectance prior network on a widely used benchmark.

Original languageEnglish
Title of host publicationProceedings - 2023 IEEE International Conference on Multimedia and Expo, ICME 2023
PublisherIEEE Computer Society
Number of pages6
ISBN (Electronic)9781665468916
Publication statusPublished - Aug 2023
Event2023 IEEE International Conference on Multimedia and Expo, ICME 2023 - Brisbane, Australia
Duration: 10 Jul 202314 Jul 2023

Publication series

NameProceedings - IEEE International Conference on Multimedia and Expo
ISSN (Print)1945-7871
ISSN (Electronic)1945-788X


Conference2023 IEEE International Conference on Multimedia and Expo, ICME 2023


  • deep neural networks
  • Photometric stereo
  • prior reflectance
  • surface normal recovery

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications


Dive into the research topics of 'Learning Deep Photometric Stereo Network with Reflectance Priors'. Together they form a unique fingerprint.

Cite this