MAENet: Boost image-guided point cloud completion more accurate and even

  • Moyun Liu
  • , Ziheng Yang
  • , Bing Chen
  • , Youping Chen
  • , Jingming Xie
  • , Lei Yao
  • , Lap Pui Chau
  • , Jiawei Du
  • , Joey Tianyi Zhou

Research output: Journal article publicationJournal articleAcademic researchpeer-review

Abstract

Point clouds offer a powerful representation for capturing real-world 3D structures, making them indispensable in applications such as robotics, autonomous driving, and embodied AI. However, due to sensor limitations and occlusions, the captured point clouds are often incomplete, posing significant challenges for accurate 3D perception. Leveraging color images to guide point cloud completion has shown promise. In this paper, we propose a novel image-guided framework to advance this domain. Our approach generates two coarse point clouds: one reconstructed from the image and the other completed from the partial input. Unlike conventional unidirectional optimization methods that use images solely to enhance point clouds, we introduce a bi-directional pyramid optimization that fuses holistic shape cues from image-based reconstruction and stereo-rich details from point-based completion. This combination of complementary features significantly enhances completion accuracy. More importantly, we analyze the most advanced decoder used in recent image-guided point cloud completion methods and identify a key limitation: uneven point density. Uneven density can degrade geometric fidelity, lead to over-sampling in certain regions, and cause under-representation in others. To address this, we propose a novel propagation and density refinement process that balances point density, further refining the output. Extensive experiments show that our proposed More Accurate and Even Network (MAENet) not only surpasses all state-of-the-art methods in terms of accuracy, but also generates more uniform point clouds. Furthermore, MAENet shows strong generalization performance on unseen categories. The code will be made available at https://github.com/lmomoy/MAENet.

Original languageEnglish
Article number103179
Pages (from-to)1-12
JournalInformation Fusion
Volume122
DOIs
Publication statusPublished - Apr 2025

Keywords

  • Bi-directional optimization
  • Multi-modal fusion
  • Point cloud completion
  • Point density balancing

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Information Systems
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'MAENet: Boost image-guided point cloud completion more accurate and even'. Together they form a unique fingerprint.

Cite this