Combining computer vision with semantic reasoning for on-site safety management in construction

Haitao Wu, Botao Zhong, Heng Li, Peter Love, Xing Pan, Neng Zhao

Research output: Journal article publicationJournal articleAcademic researchpeer-review

42 Citations (Scopus)


Computer vision has been utilized to extract safety-related information from images with the advancement of video monitoring systems and deep learning algorithms. However, construction safety management is a knowledge-intensive task; for instance, safety managers rely on safety regulations and their prior knowledge during a jobsite safety inspection. This paper presents a conceptual framework that combines computer vision and ontology techniques to facilitate the management of safety by semantically reasoning hazards and corresponding mitigations. Specifically, computer vision is used to detect visual information from on-site photos while the safety regulatory knowledge is formally represented by ontology and semantic web rule language (SWRL) rules. Hazards and corresponding mitigations can be inferred by comparing extracted visual information from construction images with pre-defined SWRL rules. Finally, the example of falls from height is selected to validate the theoretical and technical feasibility of the developed conceptual framework. Results show that the proposed framework operates similar to the thinking model of safety managers and can facilitate on-site hazard identification and prevention by semantically reasoning hazards from images and listing corresponding mitigations.

Original languageEnglish
Article number103036
JournalJournal of Building Engineering
Publication statusPublished - Oct 2021


  • Computer vision
  • Construction safety management
  • Hazard identification
  • Ontology
  • Semantic reasoning

ASJC Scopus subject areas

  • Civil and Structural Engineering
  • Architecture
  • Building and Construction
  • Safety, Risk, Reliability and Quality
  • Mechanics of Materials


Dive into the research topics of 'Combining computer vision with semantic reasoning for on-site safety management in construction'. Together they form a unique fingerprint.

Cite this