DWSSA: Alleviating over-smoothness for deep Graph Neural Networks

Qirong Zhang, Jin Li, Qingqing Ye, Yuxi Lin, Xinlong Chen, Yang Geng Fu

Research output: Journal article publicationJournal articleAcademic researchpeer-review

Abstract

Graph Neural Networks (GNNs) have demonstrated great potential in achieving outstanding performance in various graph-related tasks, e.g., graph classification and link prediction. However, most of them suffer from the following issue: shallow networks capture very limited knowledge. Prior works design deep GNNs with more layers to solve the issue, which however introduces a new challenge, i.e., the infamous over-smoothness. Graph representation over emphasizes node features but only considers the static graph structure with a uniform weight are the key reasons for the over-smoothness issue. To alleviate the issue, this paper proposes a Dynamic Weighting Strategy (DWS) for addressing over-smoothness. We first employ Fuzzy C-Means (FCM) to cluster all nodes into several groups and get each node's fuzzy assignment, based on which a novel metric function is devised for dynamically adjusting the aggregation weights. This dynamic weighting strategy not only enables the intra-cluster interactions, but also inter-cluster aggregations, which well addresses undifferentiated aggregation caused by uniform weights. Based on DWS, we further design a Structure Augmentation (SA) step for addressing the issue of underutilizing the graph structure, where some potentially meaningful connections (i.e., edges) are added to the original graph structure via a parallelable KNN algorithm. In general, the optimized Dynamic Weighting Strategy with Structure Augmentation (DWSSA) alleviates over-smoothness by reducing noisy aggregations and utilizing topological knowledge. Extensive experiments on eleven homophilous or heterophilous graph benchmarks demonstrate the effectiveness of our proposed method DWSSA in alleviating over-smoothness and enhancing deep GNNs performance.

Original languageEnglish
Article number106228
JournalNeural Networks
Volume174
DOIs
Publication statusPublished - Jun 2024

Keywords

  • Clustering
  • Deep graph neural networks
  • Node classification
  • Over-smoothness
  • Structure augmentation

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'DWSSA: Alleviating over-smoothness for deep Graph Neural Networks'. Together they form a unique fingerprint.

Cite this