Development of a Fatigue Detection and Early Warning System for Crane Operators: A Preliminary Study

Pengkun Liu, Hung Lin Chi, Xiao Li, Dongsheng Li

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

4 Citations (Scopus)


Fatigue of operators due to intensive workloads and long working time is one of the significant constraints lead to inefficient crane operations and safety issues. It can be potentially prevented through early warnings of fatigue for further appropriate work shift arrangements. Recently, many deep neural networks have been developed for the fatigue detection of vehicle drivers, through training and processing the facial image or video data of the drivers from available datasets. However, these datasets are difficult to be directly used for the fatigue detections under crane operation scenarios due to the variations of facial features and movement patterns between crane operators and vehicle drivers. Furthermore, there is no representative and public dataset with the facial information of crane operators under construction scenarios. Therefore, this study aims to explore and analyze the features of available datasets and the corresponding data acquisition methods suitable for crane operators' fatigue detection, further providing collection guidelines on crane operators dataset for early warning system development. A hybrid learning architecture is proposed by combining convolutional neural networks (CNN) and long short-term memory (LSTM) for fatigue detection. In order to establish a unified evaluation criterion, the effort of the study includes relabeling three available vehicle drivers datasets, NTHU-DDD, UTA-RLDD, and YawnDD, with human-verified labels at the minute segment level, and to train three hybrid fatigue detection models separately. Then the three trained models are used to evaluate the fatigue status on facial videos of licensed crane operators under simulated crane operation scenarios. The results show that the average test losses are 0.78458, 0.32191, and 0.20294 individually. One of the three datasets with the pretending facial fatigue features is comparatively more accurate in detecting operators' status than the rest of those with subtle facial fatigue features. Further comparisons in terms of labeling interval, environment, and accuracy are discussed for future dataset collections.

Original languageEnglish
Title of host publicationConstruction Research Congress 2020
Subtitle of host publicationComputer Applications - Selected Papers from the Construction Research Congress 2020
EditorsPingbo Tang, David Grau, Mounir El Asmar
PublisherAmerican Society of Civil Engineers (ASCE)
Number of pages10
ISBN (Electronic)9780784482865
Publication statusPublished - 2020
EventConstruction Research Congress 2020: Computer Applications - Tempe, United States
Duration: 8 Mar 202010 Mar 2020

Publication series

NameConstruction Research Congress 2020: Computer Applications - Selected Papers from the Construction Research Congress 2020


ConferenceConstruction Research Congress 2020: Computer Applications
Country/TerritoryUnited States

ASJC Scopus subject areas

  • Civil and Structural Engineering
  • Building and Construction


Dive into the research topics of 'Development of a Fatigue Detection and Early Warning System for Crane Operators: A Preliminary Study'. Together they form a unique fingerprint.

Cite this