S2GAE: Self-Supervised Graph Autoencoders are Generalizable Learners with Graph Masking

Qiaoyu Tan, Ninghao Liu, Xiao Huang, Soo Hyun Choi, Li Li, Rui Chen, Xia Hu

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

21 Citations (Scopus)

Abstract

Self-supervised learning (SSL) has been demonstrated to be effective in pre-training models that can be generalized to various downstream tasks. Graph Autoencoder (GAE), an increasingly popular SSL approach on graphs, has been widely explored to learn node representations without ground-truth labels. However, recent studies show that existing GAE methods could only perform well on link prediction tasks, while their performance on classification tasks is rather limited. This limitation casts doubt on the generalizability and adoption of GAE. In this paper, for the first time, we show that GAE can generalize well to both link prediction and classification scenarios, including node-level and graph-level tasks, by redesigning its critical building blocks from the graph masking perspective. Our proposal is called Self-Supervised Graph Autoencoder - S2GAE, which unleashes the power of GAEs with minimal yet nontrivial efforts. Specifically, instead of reconstructing the whole input structure, we randomly mask a portion of edges and learn to reconstruct these missing edges with an effective masking strategy and an expressive decoder network. Moreover, we theoretically prove that S2GAE could be regarded as an edge-level contrastive learning framework, providing insights into why it generalizes well. Empirically, we conduct extensive experiments on 21 benchmark datasets across link prediction and node & graph classification tasks. The results validate the superiority of S2GAE against state-of-the-art generative and contrastive methods. This study demonstrates the potential of GAE as a universal representation learner on graphs. Our code is publicly available at https://github.com/qiaoyu-tan/S2GAE.

Original languageEnglish
Title of host publicationWSDM 2023 - Proceedings of the 16th ACM International Conference on Web Search and Data Mining
PublisherAssociation for Computing Machinery, Inc
Pages787-795
Number of pages9
ISBN (Electronic)9781450394079
DOIs
Publication statusPublished - 27 Feb 2023
Event16th ACM International Conference on Web Search and Data Mining, WSDM 2023 - Singapore, Singapore
Duration: 27 Feb 20233 Mar 2023

Publication series

NameWSDM 2023 - Proceedings of the 16th ACM International Conference on Web Search and Data Mining

Conference

Conference16th ACM International Conference on Web Search and Data Mining, WSDM 2023
Country/TerritorySingapore
CitySingapore
Period27/02/233/03/23

Keywords

  • masked autoencoders
  • masked graph autoencoder
  • self-supervised learning

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Software

Fingerprint

Dive into the research topics of 'S2GAE: Self-Supervised Graph Autoencoders are Generalizable Learners with Graph Masking'. Together they form a unique fingerprint.

Cite this