HCF: A Hybrid CNN Framework for Behavior Detection of Distracted Drivers

Chen Huang, Chen Huang, Xiaochen Wang, Jiannong Cao, Shihui Wang, Shihui Wang, Yan Zhang, Yan Zhang

Research output: Journal article publicationJournal articleAcademic researchpeer-review

68 Citations (Scopus)


Distracted driving causes a large number of traffic accident fatalities and is becoming an increasingly important issue in recent research on traffic safety. Gesture patterns are less distinguishable in vehicles due to in-vehicle physical constraints and body occlusions from the drivers. However, by capitalizing on modern camera technology, convolutional neural network (CNN) can be used for visual analysis. In this paper, we present a hybrid CNN framework (HCF) to detect the behaviors of distracted drivers by using deep learning to process image features. To improve the accuracy of the driving activity detection system, we first apply a cooperative pretrained model that combines ResNet50, Inception V3 and Xception to extract driver behavior features based on transfer learning. Second, because the features extracted by pretrained models are independent, we concatenate the extracted features to obtain comprehensive information. Finally, we train the fully connected layers of the HCF to filter out anomalies and hand movements associated with non-distracted driving. We apply an improved dropout algorithm to prevent the proposed HCF from overfitting to the training data. During the evaluation, we apply the class activation mapping (CAM) technique to highlight the feature area involving ten tested classes of typical distracted driving behaviors. The experimental results show that the proposed HCF achieves the classification accuracy of 96.74% when detecting distracted driving behaviors, demonstrating that it can potentially help drivers maintain safe driving habits.

Original languageEnglish
Article number9113267
Pages (from-to)109335-109349
Number of pages15
JournalIEEE Access
Publication statusPublished - 2020


  • convolutional neural network
  • Distracted drivers
  • fusion model
  • transfer learning

ASJC Scopus subject areas

  • General Computer Science
  • General Materials Science
  • General Engineering


Dive into the research topics of 'HCF: A Hybrid CNN Framework for Behavior Detection of Distracted Drivers'. Together they form a unique fingerprint.

Cite this