CS-WSCDNet: Class Activation Mapping and Segment Anything Model-Based Framework for Weakly Supervised Change Detection

Lukang Wang, Min Zhang, Wenzhong Shi

Research output: Journal article publicationJournal articleAcademic researchpeer-review

1 Citation (Scopus)

Abstract

Change detection (CD) using deep learning techniques is a trending topic in the field of remote sensing; however, most existing networks require pixel-level labels for supervised learning, which is difficult and time-consuming to label all changed pixels from multitemporal images. To address this challenge, we propose a novel framework for weakly supervised CD (WSCD), namely CS-WSCDNet, which can achieve pixel-level results by training on samples with image-level labels. Specifically, the framework is built upon the localization capability of class activation mapping (CAM) and the powerful zero-shot segmentation ability of the foundation model, i.e., the segment anything model (SAM). After training an image-level classifier to identify whether changes have occurred in the image pair, CAM is used to roughly localize the regions of change in the image pairs. Subsequently, SAM is employed to optimize these rough regions and generate pixel-level pseudo labels for changed objects. These pseudo labels are then used to train a CD model at the pixel level. To evaluate the effectiveness of CS-WSCDNet, experiments are conducted on two high-resolution remote sensing datasets. It shows that the proposed framework not only achieves state-of-the-art (SOTA) performance in WSCD tasks but also demonstrates the potential of weakly supervised learning in the field of CD. The demo codes are available at https://github.com/WangLukang/CS-WSCDNet.

Original languageEnglish
Article number5624812
Pages (from-to)1-12
Number of pages12
JournalIEEE Transactions on Geoscience and Remote Sensing
Volume61
DOIs
Publication statusPublished - 2023

Keywords

  • Change detection (CD)
  • class activation mapping (CAM)
  • deep learning
  • high-resolution images
  • remote sensing
  • segment anything model (SAM)
  • weakly supervised learning

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • General Earth and Planetary Sciences

Fingerprint

Dive into the research topics of 'CS-WSCDNet: Class Activation Mapping and Segment Anything Model-Based Framework for Weakly Supervised Change Detection'. Together they form a unique fingerprint.

Cite this