VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images

Hao Chen, Qi Dou, Lequan Yu, Jing Qin, Pheng Ann Heng

Research output: Journal article publicationReview articleAcademic researchpeer-review

522 Citations (Scopus)


Segmentation of key brain tissues from 3D medical images is of great significance for brain disease diagnosis, progression assessment and monitoring of neurologic conditions. While manual segmentation is time-consuming, laborious, and subjective, automated segmentation is quite challenging due to the complicated anatomical environment of brain and the large variations of brain tissues. We propose a novel voxelwise residual network (VoxResNet) with a set of effective training schemes to cope with this challenging problem. The main merit of residual learning is that it can alleviate the degradation problem when training a deep network so that the performance gains achieved by increasing the network depth can be fully leveraged. With this technique, our VoxResNet is built with 25 layers, and hence can generate more representative features to deal with the large variations of brain tissues than its rivals using hand-crafted features or shallower networks. In order to effectively train such a deep network with limited training data for brain segmentation, we seamlessly integrate multi-modality and multi-level contextual information into our network, so that the complementary information of different modalities can be harnessed and features of different scales can be exploited. Furthermore, an auto-context version of the VoxResNet is proposed by combining the low-level image appearance features, implicit shape information, and high-level context together for further improving the segmentation performance. Extensive experiments on the well-known benchmark (i.e., MRBrainS) of brain segmentation from 3D magnetic resonance (MR) images corroborated the efficacy of the proposed VoxResNet. Our method achieved the first place in the challenge out of 37 competitors including several state-of-the-art brain segmentation methods. Our method is inherently general and can be readily applied as a powerful tool to many brain-related studies, where accurate segmentation of brain structures is critical.
Original languageEnglish
Pages (from-to)446-455
Number of pages10
Publication statusPublished - 15 Apr 2018


  • 3D deep learning
  • Auto-context
  • Brain segmentation
  • Convolutional neural network
  • Multi-level contextual information
  • Multi-modality
  • Residual learning

ASJC Scopus subject areas

  • Neurology
  • Cognitive Neuroscience


Dive into the research topics of 'VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images'. Together they form a unique fingerprint.

Cite this