SCN: Switchable context network for semantic segmentation of RGB-D images

Di Lin, Ruimao Zhang, Yuanfeng Ji, Ping Li, Hui Huang

Research output: Journal article publicationJournal articleAcademic researchpeer-review

76 Citations (Scopus)

Abstract

Context representations have been widely used to profit semantic image segmentation. The emergence of depth data provides additional information to construct more discriminating context representations. Depth data preserves the geometric relationship of objects in a scene, which is generally hard to be inferred from RGB images. While deep convolutional neural networks (CNNs) have been successful in solving semantic segmentation, we encounter the problem of optimizing CNN training for the informative context using depth data to enhance the segmentation accuracy. In this paper, we present a novel switchable context network (SCN) to facilitate semantic segmentation of RGB-D images. Depth data is used to identify objects existing in multiple image regions. The network analyzes the information in the image regions to identify different characteristics, which are then used selectively through switching network branches. With the content extracted from the inherent image structure, we are able to generate effective context representations that are aware of both image structures and object relationships, leading to a more coherent learning of semantic segmentation network. We demonstrate that our SCN outperforms state-of-the-art methods on two public datasets.

Original languageEnglish
Pages (from-to)1120-1131
Number of pages12
JournalIEEE Transactions on Cybernetics
Volume50
Issue number3
DOIs
Publication statusPublished - Mar 2020
Externally publishedYes

Keywords

  • Computer architecture
  • Context representation
  • convolutional neural network (CNN)
  • Cybernetics
  • Image color analysis
  • Image resolution
  • Image segmentation
  • RGB-D images
  • semantic segmentation
  • Semantics
  • Switches

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'SCN: Switchable context network for semantic segmentation of RGB-D images'. Together they form a unique fingerprint.

Cite this