Severity-Aware Semantic Segmentation with Reinforced Wasserstein Training

Xiaofeng Liu, Wenxuan Ji, Jia You, Georges El Fakhri, Jonghye Woo

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

6 Citations (Scopus)


Semantic segmentation is a class of methods to classify each pixel in an image into semantic classes, which is critical for autonomous vehicles and surgery systems. Crossentropy (CE) loss-based deep neural networks (DNN) achieved great success w.r.t. the accuracy-based metrics, e.g., mean Intersection-over Union. However, the CE loss has a limitation in that it ignores varying degrees of severity of pair-wise misclassified results. For instance, classifying
a car into the road is much more terrible than recognizing it as a bus. To sidestep this, in this work, we propose to incorporate the severity-aware inter-class correlation into our Wasserstein training framework by configuring its ground distance matrix. In addition, our method can adaptively learn the ground metric in a high-fidelity simulator, following a reinforcement alternative optimization scheme. We evaluate our method using the CARLA simulator with the Deeplab backbone, demonstraing that our method significantly improves the survival time in the CARLA simulator. In addition, our method can be readily applied to existing DNN architectures and algorithms while yielding superior performance. We report results from experiments carried out with the CamVid and Cityscapes datasets.
Original languageEnglish
Title of host publicationProc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR’2020), Virtual, 14-19 June, 2020
PublisherIEEE Computer Society
Number of pages10
Publication statusPublished - Aug 2020
EventIEEE Conference on Computer Vision and Pattern Recognition 2020 -
Duration: 14 Jun 202019 Jun 2020

Publication series

NameProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
PublisherIEEE Computer Society
ISSN (Print)1063-6919


ConferenceIEEE Conference on Computer Vision and Pattern Recognition 2020
Abbreviated titleIEEE CVPR 2020
Internet address

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Cite this