Accurate estimation of pedestrian orientation from on-board camera and inertial sensors

Yanlei Gu, Li Ta Hsu, Lijia Xie, Shunsuke Kamijo

Research output: Journal article publicationJournal articleAcademic researchpeer-review

7 Citations (Scopus)


Autonomous driving is not only required to detect pedestrians around vehicles, but also expected to understand the behaviors of pedestrians. Pedestrian body orientation and head orientation are the relevant indicators of the pedestrian intention. This paper proposes an accurate estimation system to recognize the pedestrian body orientation and the pedestrian head orientation from on-board camera and inertial sensors. The proposed system discretizes the body orientation and the head orientation into 16 directions. In order to achieve the accurate orientation estimation, a novel training database is established, which includes strongly labeled data and weakly labeled data. Semi-Supervised Learning method is employed to annotate the weakly labeled data, and to generate the accurate classifier based on the proposed training database. In addition, the temporal constraint and the human physical model constraint are considered in orientation estimation, which are beneficial to the reasonable and stable result of orientation estimation for the pedestrian in image sequences. This estimated result is the orientation in camera space. The comprehension of the pedestrian behavior needs to be conducted in the real world space. Therefore, this paper proposes to model the motion of the host vehicle using inertial sensor, then transforms the estimated orientation from camera space to the real world space by considering the vehicle and pedestrian motion. The represented orientation indicates the behavior of the pedestrian more directly. Finally, a series of experiments demonstrate the effectiveness of the proposed pedestrian orientation system.

Original languageEnglish
Pages (from-to)271-281
Number of pages11
JournalIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
Issue number1
Publication statusPublished - 1 Jan 2016
Externally publishedYes


  • On-board camera
  • Orientation estimation
  • Pedestrian
  • Vehicle motion

ASJC Scopus subject areas

  • Signal Processing
  • Computer Graphics and Computer-Aided Design
  • Electrical and Electronic Engineering
  • Applied Mathematics


Dive into the research topics of 'Accurate estimation of pedestrian orientation from on-board camera and inertial sensors'. Together they form a unique fingerprint.

Cite this