DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction

Changhao Chen, Chris Xiaoxuan Lu, Bing Wang, Niki Trigoni, Andrew Markham

Research output: Journal article publicationJournal articleAcademic researchpeer-review

24 Citations (Scopus)


Dynamical models estimate and predict the temporal evolution of physical systems. State-space models (SSMs) in particular represent the system dynamics with many desirable properties, such as being able to model uncertainty in both the model and measurements, and optimal (in the Bayesian sense) recursive formulations, e.g., the Kalman filter. However, they require significant domain knowledge to derive the parametric form and considerable hand tuning to correctly set all the parameters. Data-driven techniques, e.g., recurrent neural networks, have emerged as compelling alternatives to SSMs with wide success across a number of challenging tasks, in part due to their impressive capability to extract relevant features from rich inputs. They, however, lack interpretability and robustness to unseen conditions. Thus, data-driven models are hard to be applied in safety-critical applications, such as self-driving vehicles. In this work, we present DynaNet, a hybrid deep learning and time-varying SSM, which can be trained end-to-end. Our neural Kalman dynamical model allows us to exploit the relative merits of both SSM and deep neural networks. We demonstrate its effectiveness in the estimation and prediction on a number of physically challenging tasks, including visual odometry, sensor fusion for visual-inertial navigation, and motion prediction. In addition, we show how DynaNet can indicate failures through investigation of properties, such as the rate of innovation (Kalman gain).

Original languageEnglish
Pages (from-to)5479-5491
Number of pages13
JournalIEEE Transactions on Neural Networks and Learning Systems
Issue number12
Publication statusPublished - 1 Dec 2021
Externally publishedYes


  • Deep neural network (DNN)
  • dynamical model
  • motion estimation
  • state-space model (SSM)

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence


Dive into the research topics of 'DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction'. Together they form a unique fingerprint.

Cite this