Depth video-based two-stream convolutional neural networks for driver fatigue detection

Xiaoxi Ma; Lap Pui Chau; Kim Hui Yap

doi:10.1109/ICOT.2017.8336111

Depth video-based two-stream convolutional neural networks for driver fatigue detection

Xiaoxi Ma
, Lap Pui Chau
, Kim Hui Yap

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

34 Citations (Scopus)

Abstract

Recently, much research efforts have been dedicated to the development of computer-vision-based driver fatigue detection systems. Most of them utilize the RGB data, and focus on driver status detection during the day. However, drivers are more likely to be tired and drowsy during night time. In this paper, we present a driver fatigue detection system based on CNN using depth video sequences, which helps to provide alerts properly to fatigue drivers during the night time. Specifically, the two-stream CNN architecture incorporates spatial information of current depth frame and temporal information of neighboring depth frames which is represented by motion vectors. Besides, we propose a background removal system for depth video sequence of driving. Our method is trained and evaluated on our driver behavior dataset. Experiments show that the accuracy of the proposed method achieves 91.57%, which outperforms the baseline system within the recent state-of-the-art.

Original language	English
Title of host publication	Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017
Editors	Lei Wang, Minghui Dong, Yanfeng Lu, Haizhou Li
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	155-158
Number of pages	4
ISBN (Electronic)	9781538632758
DOIs	https://doi.org/10.1109/ICOT.2017.8336111
Publication status	Published - 10 Apr 2018
Externally published	Yes
Event	5th International Conference on Orange Technologies, ICOT 2017 - Singapore, Singapore Duration: 8 Dec 2017 → 10 Dec 2017

Publication series

Name	Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017
Volume	2018-January

Conference

Conference	5th International Conference on Orange Technologies, ICOT 2017
Country/Territory	Singapore
City	Singapore
Period	8/12/17 → 10/12/17

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Keywords

Action recognition
Depth videos
Driver fatigue detection
Two-stream CNN

ASJC Scopus subject areas

Health Informatics
Instrumentation
Computer Networks and Communications
Computer Science Applications
Human-Computer Interaction
Information Systems
Health(social science)

More information

10.1109/ICOT.2017.8336111

Cite this

Ma, X., Chau, L. P., & Yap, K. H. (2018). Depth video-based two-stream convolutional neural networks for driver fatigue detection. In L. Wang, M. Dong, Y. Lu, & H. Li (Eds.), Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017 (pp. 155-158). (Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017; Vol. 2018-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICOT.2017.8336111

Ma, Xiaoxi ; Chau, Lap Pui ; Yap, Kim Hui. / Depth video-based two-stream convolutional neural networks for driver fatigue detection. Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017. editor / Lei Wang ; Minghui Dong ; Yanfeng Lu ; Haizhou Li. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 155-158 (Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017).

@inproceedings{563fa9c6fb654464b35035f96751a852,

title = "Depth video-based two-stream convolutional neural networks for driver fatigue detection",

abstract = "Recently, much research efforts have been dedicated to the development of computer-vision-based driver fatigue detection systems. Most of them utilize the RGB data, and focus on driver status detection during the day. However, drivers are more likely to be tired and drowsy during night time. In this paper, we present a driver fatigue detection system based on CNN using depth video sequences, which helps to provide alerts properly to fatigue drivers during the night time. Specifically, the two-stream CNN architecture incorporates spatial information of current depth frame and temporal information of neighboring depth frames which is represented by motion vectors. Besides, we propose a background removal system for depth video sequence of driving. Our method is trained and evaluated on our driver behavior dataset. Experiments show that the accuracy of the proposed method achieves 91.57\%, which outperforms the baseline system within the recent state-of-the-art.",

keywords = "Action recognition, Depth videos, Driver fatigue detection, Two-stream CNN",

author = "Xiaoxi Ma and Chau, \{Lap Pui\} and Yap, \{Kim Hui\}",

note = "Publisher Copyright: {\textcopyright} 2017 IEEE.; 5th International Conference on Orange Technologies, ICOT 2017 ; Conference date: 08-12-2017 Through 10-12-2017",

year = "2018",

month = apr,

day = "10",

doi = "10.1109/ICOT.2017.8336111",

language = "English",

series = "Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "155--158",

editor = "Lei Wang and Minghui Dong and Yanfeng Lu and Haizhou Li",

booktitle = "Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017",

}

Ma, X, Chau, LP & Yap, KH 2018, Depth video-based two-stream convolutional neural networks for driver fatigue detection. in L Wang, M Dong, Y Lu & H Li (eds), Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017. Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017, vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 155-158, 5th International Conference on Orange Technologies, ICOT 2017, Singapore, Singapore, 8/12/17. https://doi.org/10.1109/ICOT.2017.8336111

Depth video-based two-stream convolutional neural networks for driver fatigue detection. / Ma, Xiaoxi; Chau, Lap Pui; Yap, Kim Hui.
Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017. ed. / Lei Wang; Minghui Dong; Yanfeng Lu; Haizhou Li. Institute of Electrical and Electronics Engineers Inc., 2018. p. 155-158 (Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017; Vol. 2018-January).

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

TY - GEN

T1 - Depth video-based two-stream convolutional neural networks for driver fatigue detection

AU - Ma, Xiaoxi

AU - Chau, Lap Pui

AU - Yap, Kim Hui

PY - 2018/4/10

Y1 - 2018/4/10

N2 - Recently, much research efforts have been dedicated to the development of computer-vision-based driver fatigue detection systems. Most of them utilize the RGB data, and focus on driver status detection during the day. However, drivers are more likely to be tired and drowsy during night time. In this paper, we present a driver fatigue detection system based on CNN using depth video sequences, which helps to provide alerts properly to fatigue drivers during the night time. Specifically, the two-stream CNN architecture incorporates spatial information of current depth frame and temporal information of neighboring depth frames which is represented by motion vectors. Besides, we propose a background removal system for depth video sequence of driving. Our method is trained and evaluated on our driver behavior dataset. Experiments show that the accuracy of the proposed method achieves 91.57%, which outperforms the baseline system within the recent state-of-the-art.

AB - Recently, much research efforts have been dedicated to the development of computer-vision-based driver fatigue detection systems. Most of them utilize the RGB data, and focus on driver status detection during the day. However, drivers are more likely to be tired and drowsy during night time. In this paper, we present a driver fatigue detection system based on CNN using depth video sequences, which helps to provide alerts properly to fatigue drivers during the night time. Specifically, the two-stream CNN architecture incorporates spatial information of current depth frame and temporal information of neighboring depth frames which is represented by motion vectors. Besides, we propose a background removal system for depth video sequence of driving. Our method is trained and evaluated on our driver behavior dataset. Experiments show that the accuracy of the proposed method achieves 91.57%, which outperforms the baseline system within the recent state-of-the-art.

KW - Action recognition

KW - Depth videos

KW - Driver fatigue detection

KW - Two-stream CNN

UR - https://www.scopus.com/pages/publications/85048583168

U2 - 10.1109/ICOT.2017.8336111

DO - 10.1109/ICOT.2017.8336111

M3 - Conference article published in proceeding or book

AN - SCOPUS:85048583168

T3 - Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017

SP - 155

EP - 158

BT - Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017

A2 - Wang, Lei

A2 - Dong, Minghui

A2 - Lu, Yanfeng

A2 - Li, Haizhou

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 5th International Conference on Orange Technologies, ICOT 2017

Y2 - 8 December 2017 through 10 December 2017

ER -

Ma X, Chau LP, Yap KH. Depth video-based two-stream convolutional neural networks for driver fatigue detection. In Wang L, Dong M, Lu Y, Li H, editors, Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017. Institute of Electrical and Electronics Engineers Inc. 2018. p. 155-158. (Proceedings of the 2017 International Conference on Orange Technologies, ICOT 2017). doi: 10.1109/ICOT.2017.8336111

Depth video-based two-stream convolutional neural networks for driver fatigue detection

Abstract

Publication series

Conference

UN SDGs

Keywords

ASJC Scopus subject areas

More information

Other files and links

Fingerprint

Cite this