Recognizing Diverse Construction Activities in Site Images via Relevance Networks of Construction-Related Objects Detected by Convolutional Neural Networks

Xiaochun Luo, Heng Li, Dongping Cao, Fei Dai, Joonoh Seo, Sanghyun Lee

Research output: Journal article publicationJournal articleAcademic researchpeer-review

69 Citations (Scopus)


Timely and overall knowledge of the states and resource allocation of diverse activities on construction sites is critical to resource leveling, progress tracking, and productivity analysis. Despite its importance, this task is still performed manually. Previous studies have taken a significant step forward in introducing computer vision technologies, although they have been oriented toward limited classes of objects or limited types of activities. Furthermore, they especially focus on single activity recognition, where an image contains only the execution of an activity by one or a few objects. This paper introduces a two-step method for recognizing diverse construction activities in still site images. It detects 22 classes of construction-related objects using convolutional neural networks. With objects detected, semantic relevance representing the likelihood of the cooperation or coexistence between two objects in a construction activity, spatial relevance representing the two-dimensional pixel proximity in the image coordinates, and activity patterns are defined to recognize 17 types of construction activities. The advantage of the proposed method is its potential to recognize diverse concurrent construction activities in a fully automatic way. Therefore, it is possible to save managers' valuable time in manual data collection and concentrate their attention on solving problems that necessarily demand their expertise.
Original languageEnglish
Article number04018012
JournalJournal of Computing in Civil Engineering
Issue number3
Publication statusPublished - 1 May 2018


  • Construction activity recognition
  • Convolutional neural networks
  • Relevance networks
  • Semantic relevance
  • Spatial relevance

ASJC Scopus subject areas

  • Civil and Structural Engineering
  • Computer Science Applications

Cite this