Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information

Qiangguo Jin; Changjiang Zou; Hui Cui; Changming Sun; Shu Wei Huang; Yi Jie Kuo; Ping Xuan; Leilei Cao; Ran Su; Leyi Wei; Henry B.L. Duh; Yu Pin Chen

doi:10.1007/978-3-031-43987-2_9

Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information

Qiangguo Jin
, Changjiang Zou
, Hui Cui
, Changming Sun
, Shu Wei Huang
, Yi Jie Kuo
, Ping Xuan
, Leilei Cao
, Ran Su
, Leyi Wei
, Henry B.L. Duh
, Yu Pin Chen

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

7 Citations (Scopus)

Abstract

Sarcopenia is a condition of age-associated muscle degeneration that shortens the life expectancy in those it affects, compared to individuals with normal muscle strength. Accurate screening for sarcopenia is a key process of clinical diagnosis and therapy. In this work, we propose a novel multi-modality contrastive learning (MM-CL) based method that combines hip X-ray images and clinical parameters for sarcopenia screening. Our method captures the long-range information with Non-local CAM Enhancement, explores the correlations in visual-text features via Visual-text Feature Fusion, and improves the model’s feature representation ability through Auxiliary contrastive representation. Furthermore, we establish a large in-house dataset with 1,176 patients to validate the effectiveness of multi-modality based methods. Significant performances with an AUC of 84.64%, ACC of 79.93%, F1 of 74.88%, SEN of 72.06%, SPC of 86.06%, and PRE of 78.44%, show that our method outperforms other single-modality and multi-modality based methods.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings
Editors	Hayit Greenspan, Hayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, Russell Taylor
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	85-94
Number of pages	10
ISBN (Print)	9783031439865
DOIs	https://doi.org/10.1007/978-3-031-43987-2_9
Publication status	Published - 1 Oct 2023
Event	26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023 - Vancouver, Canada Duration: 8 Oct 2023 → 12 Oct 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14225 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023
Country/Territory	Canada
City	Vancouver
Period	8/10/23 → 12/10/23

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Keywords

Contrastive learning
Multi-modality feature fusion
Sarcopenia screening

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

More information

10.1007/978-3-031-43987-2_9

Cite this

Jin, Q., Zou, C., Cui, H., Sun, C., Huang, S. W., Kuo, Y. J., Xuan, P., Cao, L., Su, R., Wei, L., Duh, H. B. L., & Chen, Y. P. (2023). Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information. In H. Greenspan, H. Greenspan, A. Madabhushi, P. Mousavi, S. Salcudean, J. Duncan, T. Syeda-Mahmood, & R. Taylor (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings (pp. 85-94). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14225 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-43987-2_9

Jin, Qiangguo ; Zou, Changjiang ; Cui, Hui et al. / Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. editor / Hayit Greenspan ; Hayit Greenspan ; Anant Madabhushi ; Parvin Mousavi ; Septimiu Salcudean ; James Duncan ; Tanveer Syeda-Mahmood ; Russell Taylor. Springer Science and Business Media Deutschland GmbH, 2023. pp. 85-94 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{aa38c4cafff9475ea2ccf4d9079b57de,

title = "Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information",

abstract = "Sarcopenia is a condition of age-associated muscle degeneration that shortens the life expectancy in those it affects, compared to individuals with normal muscle strength. Accurate screening for sarcopenia is a key process of clinical diagnosis and therapy. In this work, we propose a novel multi-modality contrastive learning (MM-CL) based method that combines hip X-ray images and clinical parameters for sarcopenia screening. Our method captures the long-range information with Non-local CAM Enhancement, explores the correlations in visual-text features via Visual-text Feature Fusion, and improves the model{\textquoteright}s feature representation ability through Auxiliary contrastive representation. Furthermore, we establish a large in-house dataset with 1,176 patients to validate the effectiveness of multi-modality based methods. Significant performances with an AUC of 84.64\%, ACC of 79.93\%, F1 of 74.88\%, SEN of 72.06\%, SPC of 86.06\%, and PRE of 78.44\%, show that our method outperforms other single-modality and multi-modality based methods.",

keywords = "Contrastive learning, Multi-modality feature fusion, Sarcopenia screening",

author = "Qiangguo Jin and Changjiang Zou and Hui Cui and Changming Sun and Huang, \{Shu Wei\} and Kuo, \{Yi Jie\} and Ping Xuan and Leilei Cao and Ran Su and Leyi Wei and Duh, \{Henry B.L.\} and Chen, \{Yu Pin\}",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.; 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023 ; Conference date: 08-10-2023 Through 12-10-2023",

year = "2023",

month = oct,

day = "1",

doi = "10.1007/978-3-031-43987-2\_9",

language = "English",

isbn = "9783031439865",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "85--94",

editor = "Hayit Greenspan and Hayit Greenspan and Anant Madabhushi and Parvin Mousavi and Septimiu Salcudean and James Duncan and Tanveer Syeda-Mahmood and Russell Taylor",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings",

address = "Germany",

}

Jin, Q, Zou, C, Cui, H, Sun, C, Huang, SW, Kuo, YJ, Xuan, P, Cao, L, Su, R, Wei, L, Duh, HBL & Chen, YP 2023, Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information. in H Greenspan, H Greenspan, A Madabhushi, P Mousavi, S Salcudean, J Duncan, T Syeda-Mahmood & R Taylor (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14225 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 85-94, 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023, Vancouver, Canada, 8/10/23. https://doi.org/10.1007/978-3-031-43987-2_9

Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information. / Jin, Qiangguo; Zou, Changjiang; Cui, Hui et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. ed. / Hayit Greenspan; Hayit Greenspan; Anant Madabhushi; Parvin Mousavi; Septimiu Salcudean; James Duncan; Tanveer Syeda-Mahmood; Russell Taylor. Springer Science and Business Media Deutschland GmbH, 2023. p. 85-94 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14225 LNCS).

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

TY - GEN

T1 - Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information

AU - Jin, Qiangguo

AU - Zou, Changjiang

AU - Cui, Hui

AU - Sun, Changming

AU - Huang, Shu Wei

AU - Kuo, Yi Jie

AU - Xuan, Ping

AU - Cao, Leilei

AU - Su, Ran

AU - Wei, Leyi

AU - Duh, Henry B.L.

AU - Chen, Yu Pin

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.

PY - 2023/10/1

Y1 - 2023/10/1

N2 - Sarcopenia is a condition of age-associated muscle degeneration that shortens the life expectancy in those it affects, compared to individuals with normal muscle strength. Accurate screening for sarcopenia is a key process of clinical diagnosis and therapy. In this work, we propose a novel multi-modality contrastive learning (MM-CL) based method that combines hip X-ray images and clinical parameters for sarcopenia screening. Our method captures the long-range information with Non-local CAM Enhancement, explores the correlations in visual-text features via Visual-text Feature Fusion, and improves the model’s feature representation ability through Auxiliary contrastive representation. Furthermore, we establish a large in-house dataset with 1,176 patients to validate the effectiveness of multi-modality based methods. Significant performances with an AUC of 84.64%, ACC of 79.93%, F1 of 74.88%, SEN of 72.06%, SPC of 86.06%, and PRE of 78.44%, show that our method outperforms other single-modality and multi-modality based methods.

AB - Sarcopenia is a condition of age-associated muscle degeneration that shortens the life expectancy in those it affects, compared to individuals with normal muscle strength. Accurate screening for sarcopenia is a key process of clinical diagnosis and therapy. In this work, we propose a novel multi-modality contrastive learning (MM-CL) based method that combines hip X-ray images and clinical parameters for sarcopenia screening. Our method captures the long-range information with Non-local CAM Enhancement, explores the correlations in visual-text features via Visual-text Feature Fusion, and improves the model’s feature representation ability through Auxiliary contrastive representation. Furthermore, we establish a large in-house dataset with 1,176 patients to validate the effectiveness of multi-modality based methods. Significant performances with an AUC of 84.64%, ACC of 79.93%, F1 of 74.88%, SEN of 72.06%, SPC of 86.06%, and PRE of 78.44%, show that our method outperforms other single-modality and multi-modality based methods.

KW - Contrastive learning

KW - Multi-modality feature fusion

KW - Sarcopenia screening

UR - https://www.scopus.com/pages/publications/85174743637

U2 - 10.1007/978-3-031-43987-2_9

DO - 10.1007/978-3-031-43987-2_9

M3 - Conference article published in proceeding or book

AN - SCOPUS:85174743637

SN - 9783031439865

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 85

EP - 94

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings

A2 - Greenspan, Hayit

A2 - Madabhushi, Anant

A2 - Mousavi, Parvin

A2 - Salcudean, Septimiu

A2 - Duncan, James

A2 - Syeda-Mahmood, Tanveer

A2 - Taylor, Russell

PB - Springer Science and Business Media Deutschland GmbH

T2 - 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023

Y2 - 8 October 2023 through 12 October 2023

ER -

Jin Q, Zou C, Cui H, Sun C, Huang SW, Kuo YJ et al. Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information. In Greenspan H, Greenspan H, Madabhushi A, Mousavi P, Salcudean S, Duncan J, Syeda-Mahmood T, Taylor R, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 85-94. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-43987-2_9