Title: On Symmetries and Feature Learning in Simple Neural Networks
For all their mathematical mysteries, two important features of neural networks are their ability to encode symmetries into their architectures, and their ability to ‘discover’ hidden low-dimensional structures within high-dimensional data.
