A transformer-based deep learning model for recognizing communication-oriented entities from patents of ICT in construction

Hengqin Wu, Geoffrey Qiping Shen, Xue Lin, Minglei Li, Clyde Zhengdao Li

Research output: Journal article publicationJournal articleAcademic researchpeer-review

1 Citation (Scopus)


The patents of information and communication technology (ICT) in construction are valuable sources of technological solutions to communication problems in the construction practice. However, it is often difficult for practitioners and stakeholders to identify the key communication functionalities from complicated expressions in the patent documents. Addressing such challenges, this study develops a deep learning model to enable automatic recognition of communication-oriented entities (CEs) from patent documents. The proposed model is structured based on the Transformer, consisting of feed-forward and self-attention neural networks to better recognize ambiguous and unknown entities by utilizing contextual information. The validation results showed that the proposed model has superior performance in CE recognition than traditional recurrent neural networks (RNN)-based models, especially in recognizing ambiguous and unknown entities. Moreover, experimental results on some research literature and a real-life project report showed satisfactory performance of the model in CE recognition across different document types.

Original languageEnglish
Article number103608
JournalAutomation in Construction
Publication statusPublished - May 2021


  • Construction industry
  • Contextual information
  • Deep learning
  • Entity recognition
  • Information and communications technology (ICT)
  • Transformer

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Civil and Structural Engineering
  • Building and Construction

Cite this