On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks - Equipe Multimédia
Communication Dans Un Congrès Année : 2023

On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks

Résumé

Graph Neural Networks (GNNs) have succeeded in various computer science applications, yet deep GNNs underperform their shallow counterparts despite deep learning's success in other domains. Over-smoothing and over-squashing are key challenges when stacking graph convolutional layers, hindering deep representation learning and information propagation from distant nodes. Our work reveals that over-smoothing and over-squashing are intrinsically related to the spectral gap of the graph Laplacian, resulting in an inevitable trade-off between these two issues, as they cannot be alleviated simultaneously. To achieve a suitable compromise, we propose adding and removing edges as a viable approach. We introduce the Stochastic Jost and Liu Curvature Rewiring (SJLR) algorithm, which is computationally efficient and preserves fundamental properties compared to previous curvature-based methods. Unlike existing approaches, SJLR performs edge addition and removal during GNN training while maintaining the graph unchanged during testing. Comprehensive comparisons demonstrate SJLR's competitive performance in addressing over-smoothing and over-squashing. CCS CONCEPTS • Computing methodologies → Machine learning algorithms; • Computer systems organization → Neural networks.
Fichier principal
Vignette du fichier
CIKM_2023_Camera_Ready.pdf (878.09 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04263891 , version 1 (29-10-2023)

Identifiants

Citer

Jhony H Giraldo, Konstantinos Skianis, Thierry Bouwmans, Fragkiskos D. Malliaros. On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks. CIKM 2023 - The 32nd ACM International Conference on Information and Knowledge Management, Oct 2023, Birmingham United Kingdom, United Kingdom. pp.566-576, ⟨10.1145/3583780.3614997⟩. ⟨hal-04263891⟩
152 Consultations
276 Téléchargements

Altmetric

Partager

More