Graph based multi-scale neighboring topology deep learning for kidney and tumor segmentation

Objective. Effective learning and modelling of spatial and semantic relations between image regions in various ranges are critical yet challenging in image segmentation tasks. Approach. We propose a novel deep graph reasoning model to learn from multi-order neighborhood topologies for volumetric image segmentation. A graph is first constructed with nodes representing image regions and graph topology to derive spatial dependencies and semantic connections across image regions. We propose a new node attribute embedding mechanism to formulate topological attributes for each image region node by performing multi-order random walks (RW) on the graph and updating neighboring topologies at different neighborhood ranges. Afterwards, multi-scale graph convolutional autoencoders are developed to extract deep multi-scale topological representations of nodes and propagate learnt knowledge along graph edges during the convolutional and optimization process. We also propose a scale-level attention module to learn the adaptive weights of topological representations at multiple scales for enhanced fusion. Finally, the enhanced topological representation and knowledge from graph reasoning are integrated with content features before feeding into the segmentation decoder. Main results. The evaluation results over public kidney and tumor CT segmentation dataset show that our model outperforms other state-of-the-art segmentation methods. Ablation studies and experiments using different convolutional neural networks backbones show the contributions of major technical innovations and generalization ability. Significance. We propose for the first time an RW-driven MCG with scale-level attention to extract semantic connections and spatial dependencies between a diverse range of regions for accurate kidney and tumor segmentation in CT volumes.