3D Multi-Attention Guided Multi-Task Learning Network for Automatic Gastric Tumor Segmentation and Lymph Node Classification

Yongtao Zhang, Haimei Li, Jie Du, Jing Qin, Tianfu Wang, Yue Chen, Bing Liu, Wenwen Gao, Guolin Ma, Baiying Lei

Research output: Journal article publicationJournal articleAcademic researchpeer-review

47 Citations (Scopus)


Automatic gastric tumor segmentation and lymph node (LN) classification not only can assist radiologists in reading images, but also provide image-guided clinical diagnosis and improve diagnosis accuracy. However, due to the inhomogeneous intensity distribution of gastric tumor and LN in CT scans, the ambiguous/missing boundaries, and highly variable shapes of gastric tumor, it is quite challenging to develop an automatic solution. To comprehensively address these challenges, we propose a novel 3D multi-attention guided multi-task learning network for simultaneous gastric tumor segmentation and LN classification, which makes full use of the complementary information extracted from different dimensions, scales, and tasks. Specifically, we tackle task correlation and heterogeneity with the convolutional neural network consisting of scale-aware attention-guided shared feature learning for refined and universal multi-scale features, and task-aware attention-guided feature learning for task-specific discriminative features. This shared feature learning is equipped with two types of scale-aware attention (visual attention and adaptive spatial attention) and two stage-wise deep supervision paths. The task-aware attention-guided feature learning comprises a segmentation-aware attention module and a classification-aware attention module. The proposed 3D multi-task learning network can balance all tasks by combining segmentation and classification loss functions with weight uncertainty. We evaluate our model on an in-house CT images dataset collected from three medical centers. Experimental results demonstrate that our method outperforms the state-of-the-art algorithms, and obtains promising performance for tumor segmentation and LN classification. Moreover, to explore the generalization for other segmentation tasks, we also extend the proposed network to liver tumor segmentation in CT images of the MICCAI 2017 Liver Tumor Segmentation Challenge. Our implementation is released at https://github.com/infinite-tao/MA-MTLN.

Original languageEnglish
Article number9366506
Pages (from-to)1618-1631
Number of pages14
JournalIEEE Transactions on Medical Imaging
Issue number6
Publication statusPublished - Jun 2021


  • CT scans
  • Gastric tumor segmentation
  • lymph node classification
  • multi-attention
  • multi-task learning

ASJC Scopus subject areas

  • Software
  • Radiological and Ultrasound Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of '3D Multi-Attention Guided Multi-Task Learning Network for Automatic Gastric Tumor Segmentation and Lymph Node Classification'. Together they form a unique fingerprint.

Cite this