#gnn
Graph Network as Arbitrary Inductive Bias
src.
The architecture of a neural network imposes some kind of structure that lends itself to particular types of problem (CNN, RNN). Thus, you can think of this as some form of inductive bias. An interesting view of [[graph-neural-networks]] is that essentially these provide arbitrary inductive bias, since the goal is to learn the architecture?
| Component | Entities | Relations | Inductive Bias | Invariance |
|---|---|---|---|---|
| FC | Units | All-to-all | Weak | - |
| Conv. | Grid elements | Local | Locality | Spatial transl. |
| Recurrent | Time | Sequential | Sequentially | Time transl. |
| Graph | Nodes | Edges | Arbitrary | V,E permute |
From (Battaglia et al. 2018Battaglia, Peter W, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, et al. 2018. “Relational inductive biases, deep learning, and graph networks.” arXiv.org, June. http://arxiv.org/abs/1806.01261v3.)
(Though, is that really what GNNs really do?)