Abstract.

The scattering transform is a multilayered wavelet-based architecture that acts as a model of convolutional neural networks. Recently, several works have generalized the scattering transform to graph-structured data. Our work builds on these constructions by introducing windowed and nonwindowed geometric scattering transforms for graphs based on two very general classes wavelets, which are in most cases based on asymmetric matrices. We show that these transforms have many of the same theoretical guarantees as their symmetric counterparts. As a result, the proposed construction unifies and extends known theoretical results for many of the existing graph scattering architectures. Therefore, it helps bridge the gap between geometric scattering and other graph neural networks by introducing a large family of networks with provable stability and invariance guarantees. These results lay the groundwork for future deep learning architectures for graph-structured data that have learned filters and also provably have desirable theoretical properties.

Keywords

  1. graph neural networks
  2. geometric deep learning
  3. wavelets
  4. scattering transform

MSC codes

  1. 05C62
  2. 05C81
  3. 42C15
  4. 68R10
  5. 68T07

Get full access to this article

View all available purchase options and get full access to this article.

Supplementary Materials

Index of Supplementary Materials
Title of paper: Understanding Graph Neural Networks with Asymmetric Geometric Scattering Transforms
Authors: Michael Perlmutter, Alexander Tong, Feng Gao, Guy Wolf, and Matthew Hirn
File: supplement.pdf
Type: PDF
Contents: This supplemental material contains proofs of theoretical results presented in the main paper.

References

1.
J. Andén and S. Mallat, Multiscale scattering for audio classification, in 12th International Society for Music Information Retrieval Conference, 2011, pp. 657–662.
2.
M. Belkin and P. Niyogi, Convergence of Laplacian eigenmaps, in Advances in Neural Information Processing Systems, 2007, pp. 129–136.
3.
D. Burago, S. Ivanov, and Y. Kurylev, A graph discretization of the Laplace-Beltrami operator, J. Spectr. Theory, 4 (2013), pp. 675–714, https://doi.org/10.4171/JST/83.
4.
E. Castro, A. Benz, A. Tong, G. Wolf, and S. Krishnaswamy, Uncovering the folding landscape of RNA secondary structure using deep graph embeddings, in 2020 IEEE International Conference on Big Data, 2020, pp. 4519–4528, https://doi.org/10.1109/BigData50022.2020.9378305.
5.
X. Chen, X. Cheng, and S. Mallat, Unsupervised deep Haar scattering on graphs, in Conference on Neural Information Processing Systems, Vol. 27, 2014, pp. 1709–1717.
6.
X. Cheng, X. Chen, and S. Mallat, Deep Haar scattering networks, Inf. Inference, 5 (2016), pp. 105–133.
7.
V. Chudacek, R. Talmon, J. Anden, S. Mallat, R. R. Coifman, P. Abry, and M. Doret, Low dimensional manifold embedding for scattering coefficients of intrapartum fetale heart rate variability, in 2014 International IEEE Conference in Medicine and Biology, 2014, pp. 6373–6376.
8.
R. R. Coifman and S. Lafon, Diffusion maps, Appl. Comput. Harmon. Anal., 21 (2006), pp. 5–30.
9.
M. Defferrard, X. Bresson, and P. Vandergheynst, Convolutional neural networks on graphs with fast localized spectral filtering, in Advances in Neural Information Processing Systems, Vol. 29, 2016, pp. 3844–3852.
10.
K. Fujiwara, Eigenvalues of Laplacians on a closed Riemannian manifold and its nets, Proc. Amer. Math. Soc., 123 (1995), pp. 2585–2594.
11.
F. Gama, J. Bruna, and A. Ribeiro, Stability of graph scattering transforms, in Advances in Neural Information Processing Systems, Vol. 33, 2019, pp. 8038–8048.
12.
F. Gama, A. Ribeiro, and J. Bruna, Diffusion scattering transforms on graphs, in International Conference on Learning Representations, 2019.
13.
F. Gao, G. Wolf, and M. Hirn, Geometric scattering for graph data analysis, in Proceedings of the 36th International Conference on Machine Learning, Vol. 97, 2019, pp. 2122–2131.
14.
M. Hirn, S. Mallat, and N. Poilvert, Wavelet scattering regression of quantum chemical energies, Multiscale Model. Simul., 15 (2017), pp. 827–863.
15.
T. Kipf and M. Welling, Semi-supervised classification with graph convolutional networks, in Proceedings of the International Conference on Learning Representations, 2016.
16.
R. Levie, M. Bronstein, W. Huang, L. Bucci, and G. Kutyniok, Transferability of Spectral Graph Convolutional Neural Networks, preprint, arXiv:1907.12972v2, 2020.
17.
R. Levie, F. Monti, X. Bresson, and M. M. Bronstein, Cayleynets: Graph convolutional neural networks with complex rational spectral filters, IEEE Trans. Signal Process., 67 (2018), pp. 97–109.
18.
S. Mallat, Group invariant scattering, Comm. Pure Appl. Math., 65 (2012), pp. 1331–1398.
19.
Y. Min, F. Wenkel, M. Perlmutter, and G. Wolf, Can Hybrid Geometric Scattering Networks Help Solve the Maximal Clique Problem?, preprint, https://arxiv.org/abs/2206.01506v2, 2022.
20.
Y. Min, F. Wenkel, and G. Wolf, Scattering GCN: Overcoming oversmoothness in graph convolutional networks, in Advances in Neural Information Processing Systems, Vol. 33, 2020, pp. 14498–14508.
21.
Y. Min, F. Wenkel, and G. Wolf, Geometric scattering attention networks, in Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing, 2021, pp. 8518–8522.
22.
B. Nadler, S. Lafon, R. R. Coifman, and I. G. Kevrekidis, Diffusion maps, spectral clustering and reaction coordinates of dynamical systems, Appl. Comput. Harmon. Anal., 21 (2006), pp. 113–127.
23.
E. Oyallon and S. Mallat, Deep roto-translation scattering for object classification, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 2865–2873.
24.
M. Perlmutter, F. Gao, G. Wolf, and M. Hirn, Geometric scattering networks on compact Riemannian manifolds, in Mathematical and Scientific Machine Learning Conference, 2020, pp. 570–604.
25.
Z. Shi, Convergence of Laplacian Spectra from Random Samples, preprint, arXiv:1507.00151, 2015.
26.
A. Singer, From graph to manifold Laplacian: The convergence rate, Appl. Comput. Harmon. Anal., 21 (2006), pp. 128–134.
27.
A. Tong, F. Wenkel, K. Macdonald, S. Krishnaswamy, and G. Wolf, Data-driven learning of geometric scattering modules for GNNs, in 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing, 2021, pp. 1–6, https://doi.org/10.1109/MLSP52302.2021.9596169.
28.
N. Trillos, M. Gerlach, M. Hein, and D. Slepcev, Error estimates for spectral convergence of the graph Laplacian on random geometric graphs toward the Laplace-Beltrami operator, Found. Comput. Math., 20 (2018), pp. 827–887, https://doi.org/10.1007/s10208-019-09436-w.
29.
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, Graph attention networks, in International Conference on Learning Representations, 2018.
30.
K. Xu, W. Hu, J. Leskovec, and S. Jegelka, How powerful are graph neural networks?, in International Conference on Learning Representations, 2019.
31.
D. Zou and G. Lerman, Encoding robust representation for graph generation, in International Joint Conference on Neural Networks, 2019, pp. 1–9.
32.
D. Zou and G. Lerman, Graph convolutional neural networks via scattering, Appl. Comput. Harmon. Anal., 49 (2019), pp. 1046–1074.

Information & Authors

Information

Published In

cover image SIAM Journal on Mathematics of Data Science
SIAM Journal on Mathematics of Data Science
Pages: 873 - 898
ISSN (online): 2577-0187

History

Submitted: 4 January 2022
Accepted: 6 June 2023
Published online: 25 October 2023

Keywords

  1. graph neural networks
  2. geometric deep learning
  3. wavelets
  4. scattering transform

MSC codes

  1. 05C62
  2. 05C81
  3. 42C15
  4. 68R10
  5. 68T07

Authors

Affiliations

Michael Perlmutter
Boise State University, Department of Mathematics, 1910 University Drive, Boise, ID 83706 USA.
Alexander Tong
Department of Computer Science and Operations Research, Université de Montréal, Montréal, QC H3T 1J4, Canada, and Mila - Quebec AI Institute, Montréal, QC H2S 3H1, Canada.
Feng Gao
Department of Environmental Health Sciences, Mailman School of Public Health, Columbia University, New York, NY 10032 USA.
Department of Mathematics and Statistics, Université de Montréal, Montréal, QC H3T 1J4, Canada, and Mila - Quebec AI Institute, Montréal, QC H2S 3H1, Canada.
Department of Computational Mathematics, Science, and Engineering and Department of Mathematics, Michigan State University, East Lansing, MI 48824 USA.

Funding Information

IVADO: PRF-2019-3583139727
Canada CIFAR AI Chair
Funding: This research was partially funded by IVADO (Institut de valorisation des données), grant PRF-2019-3583139727, and FRQNT (Fonds de recherche du Québec, Nature et technologies), grant 299376, Canada CIFAR AI Chair (fourth author); NIH (National Institutes of Health), NIGMS grant R01GM135929 (fifth and fourth authors); and NSF (National Science Foundation), DMS grant 1845856 (fifth author). The content provided here is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies.

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

View Options

View options

PDF

View PDF

Full Text

View Full Text

Figures

Tables

Media

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media