Body surface feature-based multi-modal Learning for Diabetes Mellitus detection

Jinxing Li, Bob Zhang, Guangming Lu, Jane You, David Zhang

Research output: Journal article publicationJournal articleAcademic researchpeer-review

17 Citations (Scopus)


In recent year, the number of people who are suffering from the Diabetes Mellitus (DM) has increased remarkably and the detection of DM disease has attracted much attention. Different from some existing methods which are invasive, Traditional Chinese Medicine (TCM) provides a non-invasive strategy for DM diagnosis by exploiting some features in the body surface, including the tongue, face, sublingual vein, pulse and odor. Since a combination of these modalities would contribute to improving detection performance, a novel multi-modal learning method is proposed to learn a shared latent variable among the tongue, face, sublingual, pulse and odor information, which efficiently exploits the correlation. In detail, the raw images or signals of five modalities are first captured through our non-invasive devices. Their corresponding features are then extracted, respectively. Finally, a shared auto-encoder gaussian process latent variable model (SAGP) is introduced to learn a latent variable for various modalities in a non-linear and generative way. An efficient algorithm is designed to optimize the proposed model. The DM detection experiments are conducted on a dataset composed of 548 Healthy and 356 DM samples collected by us and the results substantiate the superiority of the proposed method.

Original languageEnglish
Pages (from-to)1-14
Number of pages14
JournalInformation Sciences
Publication statusPublished - Jan 2019


  • Body surface feature
  • Diabetes mellitus
  • Gaussian process
  • Multi-modal

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Theoretical Computer Science
  • Computer Science Applications
  • Information Systems and Management
  • Artificial Intelligence

Cite this