Deep learning framework with Local Sparse Transformer for construction worker detection in 3D with LiDAR

Mingyu Zhang, Lei Wang, Shuai Han, Shuyuan Wang, Heng Li

Research output: Journal article publicationJournal articleAcademic researchpeer-review


Autonomous equipment is playing an increasingly important role in construction tasks. It is essential to equip autonomous equipment with powerful 3D detection capability to avoid accidents and inefficiency. However, there is limited research within the construction field that has extended detection to 3D. To this end, this study develops a light detection and ranging (LiDAR)-based deep-learning model for the 3D detection of workers on construction sites. The proposed model adopts a voxel-based anchor-free 3D object detection paradigm. To enhance the feature extraction capability for tough detection tasks, a novel Transformer-based block is proposed, where the multi-head self-attention is applied in local grid regions. The detection model integrates the Transformer blocks with 3D sparse convolution to extract wide and local features while pruning redundant features in modified downsampling layers. To train and test the proposed model, a LiDAR point cloud dataset was created, which includes workers in construction sites with 3D box annotations. The experiment results indicate that the proposed model outperforms the baseline models with higher mean average precision and smaller regression errors. The method in the study is promising to provide worker detection with rich and accurate 3D information required by construction automation.

Original languageEnglish
JournalComputer-Aided Civil and Infrastructure Engineering
Publication statusAccepted/In press - May 2024

ASJC Scopus subject areas

  • Civil and Structural Engineering
  • Computer Science Applications
  • Computer Graphics and Computer-Aided Design
  • Computational Theory and Mathematics


Dive into the research topics of 'Deep learning framework with Local Sparse Transformer for construction worker detection in 3D with LiDAR'. Together they form a unique fingerprint.

Cite this