Adapting semantic segmentation of urban scenes via mask-aware gated discriminator

Yong Xiang Lin, Daniel Stanley Tan, Wen-Huang Cheng, Kai Lung Hua

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

Training a deep neural network for semantic segmentation relies on pixel-level ground truth labels for supervision. However, collecting large datasets with pixel-level annotations is very expensive and time consuming. One workaround is to utilize synthetic data where we can generate potentially unlimited data with their corresponding ground truth labels. Unfortunately, networks trained on synthetic data perform poorly on real images due to the domain shift problem. Domain adaptation techniques have shown potential in transferring the knowledge learned from synthetic data to real world data. Prior works have mostly leveraged on adversarial training to perform a global aligning of features. However, we observed that background objects have lesser variations across different domains as opposed to foreground objects. Using this insight, we propose a method for domain adaptation that models and adapts foreground objects and background objects separately. Our approach starts with a fast style transfer to match the appearance of the inputs. This is followed by a foreground adaptation module that learns a foreground mask that is used by our gated discriminator in order to adapt the foreground and background objects separately. We demonstrate in our experiments that our model outperforms several state-of-the-art baselines in terms of mean intersection over union (mIoU).

Original languageEnglish
Title of host publicationProceedings - 2019 IEEE International Conference on Multimedia and Expo, ICME 2019
PublisherIEEE Computer Society
Pages218-223
Number of pages6
ISBN (Electronic)9781538695524
DOIs
StatePublished - 1 Jul 2019
Event2019 IEEE International Conference on Multimedia and Expo, ICME 2019 - Shanghai, China
Duration: 8 Jul 201912 Jul 2019

Publication series

NameProceedings - IEEE International Conference on Multimedia and Expo
Volume2019-July
ISSN (Print)1945-7871
ISSN (Electronic)1945-788X

Conference

Conference2019 IEEE International Conference on Multimedia and Expo, ICME 2019
CountryChina
CityShanghai
Period8/07/1912/07/19

Keywords

  • Domain adaptation
  • Gated-convolution
  • Semantic segmentation

Fingerprint Dive into the research topics of 'Adapting semantic segmentation of urban scenes via mask-aware gated discriminator'. Together they form a unique fingerprint.

Cite this