TY - JOUR
T1 - Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation
AU - Chen, Cheng
AU - Dou, Qi
AU - Chen, Hao
AU - Qin, Jing
AU - Heng, Pheng Ann
N1 - Funding Information:
Manuscript received December 26, 2019; revised February 1, 2020; accepted February 3, 2020. Date of publication February 10, 2020; date of current version June 30, 2020. This work was supported by the HK RGC TRS Project under Grant T42-409/18-R, in part by the Hong Kong Innovation and Technology Fund under Project ITS/426/17FP and Project ITS/311/18FP, in part by the CUHK T Stone Robotics Institute, and in part by the Hong Kong Research Grants Council under Grant PolyU 152035/17E. (Corresponding author: Qi Dou.) Cheng Chen and Qi Dou are with the Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong (e-mail: [email protected]; [email protected]).
Publisher Copyright:
© 1982-2012 IEEE.
PY - 2020/7
Y1 - 2020/7
N2 - Unsupervised domain adaptation has increasingly gained interest in medical image computing, aiming to tackle the performance degradation of deep neural networks when being deployed to unseen data with heterogeneous characteristics. In this work, we present a novel unsupervised domain adaptation framework, named as Synergistic Image and Feature Alignment (SIFA), to effectively adapt a segmentation network to an unlabeled target domain. Our proposed SIFA conducts synergistic alignment of domains from both image and feature perspectives. In particular, we simultaneously transform the appearance of images across domains and enhance domain-invariance of the extracted features by leveraging adversarial learning in multiple aspects and with a deeply supervised mechanism. The feature encoder is shared between both adaptive perspectives to leverage their mutual benefits via end-to-end learning. We have extensively evaluated our method with cardiac substructure segmentation and abdominal multi-organ segmentation for bidirectional cross-modality adaptation between MRI and CT images. Experimental results on two different tasks demonstrate that our SIFA method is effective in improving segmentation performance on unlabeled target images, and outperforms the state-of-the-art domain adaptation approaches by a large margin.
AB - Unsupervised domain adaptation has increasingly gained interest in medical image computing, aiming to tackle the performance degradation of deep neural networks when being deployed to unseen data with heterogeneous characteristics. In this work, we present a novel unsupervised domain adaptation framework, named as Synergistic Image and Feature Alignment (SIFA), to effectively adapt a segmentation network to an unlabeled target domain. Our proposed SIFA conducts synergistic alignment of domains from both image and feature perspectives. In particular, we simultaneously transform the appearance of images across domains and enhance domain-invariance of the extracted features by leveraging adversarial learning in multiple aspects and with a deeply supervised mechanism. The feature encoder is shared between both adaptive perspectives to leverage their mutual benefits via end-to-end learning. We have extensively evaluated our method with cardiac substructure segmentation and abdominal multi-organ segmentation for bidirectional cross-modality adaptation between MRI and CT images. Experimental results on two different tasks demonstrate that our SIFA method is effective in improving segmentation performance on unlabeled target images, and outperforms the state-of-the-art domain adaptation approaches by a large margin.
KW - adversarial learning
KW - cross-modality learning
KW - image segmentation
KW - Unsupervised domain adaptation
UR - http://www.scopus.com/inward/record.url?scp=85087468237&partnerID=8YFLogxK
U2 - 10.1109/TMI.2020.2972701
DO - 10.1109/TMI.2020.2972701
M3 - Journal article
C2 - 32054572
AN - SCOPUS:85087468237
SN - 0278-0062
VL - 39
SP - 2494
EP - 2505
JO - IEEE Transactions on Medical Imaging
JF - IEEE Transactions on Medical Imaging
IS - 7
M1 - 8988158
ER -