NormAttention-PSN: A High-frequency Region Enhanced Photometric Stereo Network with Normalized Attention

Yakun Ju, Boxin Shi, Muwei Jian, Lin Qi, Junyu Dong, Kin Man Lam

Research output: Journal article publicationJournal articleAcademic researchpeer-review

36 Citations (Scopus)

Abstract

Photometric stereo aims to recover the surface normals of a 3D object from various shading cues, establishing the relationship between two-dimensional images and the object geometry. Traditional methods usually adopt simplified reflectance models to approximate the non-Lambertian surface properties, while recently, photometric stereo based on deep learning has been widely used to deal with non-Lambertian surfaces. However, previous studies are limited in dealing with high-frequency surface regions, i.e., regions with rapid shape variations, such as crinkles, edges, etc., resulted in blurry reconstructions. To alleviate this problem, we present a normalized attention-weighted photometric stereo network, namely NormAttention-PSN, to improve surface orientation prediction, especially for those complicated structures. In order to address these challenges, in this paper, we (1) present an attention-weighted loss to produce better surface reconstructions, which applies a higher weight to the detail-preserving gradient loss in high-frequency areas, (2) adopt a double-gate normalization method for non-Lambertian surfaces, to explicitly distinguish whether the high-frequency representation is stimulated by surface structure or spatially varying reflectance, and (3) adopt a parallel high-resolution structure to generate deep features that can maintain the high-resolution details of surface normals. Extensive experiments on public benchmark data sets show that the proposed NormAttention-PSN significantly outperforms traditional calibrated photometric stereo algorithms and state-of-the-art deep learning-based methods.

Original languageEnglish
Pages (from-to)3014-3034
Number of pages21
JournalInternational Journal of Computer Vision
Volume130
Issue number12
DOIs
Publication statusPublished - Dec 2022

Keywords

  • Deep neural network
  • High-frequency surface normals
  • Non-Lambertian
  • Photometric stereo

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'NormAttention-PSN: A High-frequency Region Enhanced Photometric Stereo Network with Normalized Attention'. Together they form a unique fingerprint.

Cite this