Automatic hyoid bone tracking in real-time ultrasound swallowing videos using deep learning based and correlation filter based trackers

Shurui Feng, Queenie Tsung Kwan Shea, Kwok Yan Ng, Cheuk Ning Tang, Elaine Kwong, Yongping Zheng

Research output: Journal article publicationJournal articleAcademic researchpeer-review


(1) Background: Ultrasound provides a radiation-free and portable method for assessing swallowing. Hyoid bone locations and displacements are often used as important indicators for the evaluation of swallowing disorders. However, this requires clinicians to spend a great deal of time reviewing the ultrasound images. (2) Methods: In this study, we applied tracking algorithms based on deep learning and correlation filters to detect hyoid locations in ultrasound videos collected during swallowing. Fifty videos were collected from 10 young, healthy subjects for training, evaluation, and testing of the trackers. (3) Results: The best performing deep learning algorithm, Fully-Convo-lutional Siamese Networks (SiamFC), proved to have reliable performance in getting accurate hyoid bone locations from each frame of the swallowing ultrasound videos. While having a real-time frame rate (175 fps) when running on an RTX 2060, SiamFC also achieved a precision of 98.9% at the threshold of 10 pixels (3.25 mm) and 80.5% at the threshold of 5 pixels (1.63 mm). The tracker’s root-mean-square error and average error were 3.9 pixels (1.27 mm) and 3.3 pixels (1.07 mm), re-spectively. (4) Conclusions: Our results pave the way for real-time automatic tracking of the hyoid bone in ultrasound videos for swallowing assessment.

Original languageEnglish
Article number3712
Issue number11
Publication statusPublished - 1 Jun 2021


  • Correlation filters
  • Deep learning
  • Dysphagia
  • Hyoid bone
  • Real-time
  • SiamFC
  • Swallowing
  • Tracking
  • Ultrasound videos

ASJC Scopus subject areas

  • Analytical Chemistry
  • Information Systems
  • Atomic and Molecular Physics, and Optics
  • Biochemistry
  • Instrumentation
  • Electrical and Electronic Engineering

Cite this