Why Deep Learning Works: Perspectives from Theoretical Chemistry, Charles Martin

We present new ideas which attempt to explain why Deep Learning works, taking lessons from Theoretical Chemistry, and integrating ideas from Protein Folding, Renormalization Group, and Quantum Chemistry.
We address the idea that spin glasses make good models for Deep Learning, and discuss both the p-spherical spin glass models used by LeCun, and the spin-glass-of-minimal frustration, proposed by Wolynes for protein folding some 20 years ago.
We argue that Deep Learning energy models resemble the energy models developed for protein folding, and, in contrast to the p-spin spherical models, suggest the energy landscape of a deep learning model should be ruggedly convex. We compare and contrast this to hypothesis to current suggestions as to why Deep Learning works.
We show the relationship between RBMs and Variational Renormalization Group, and explain the importance in modeling neuro-dynamics. We then discuss how the RG transform can be used as a path to construct an Effective Hamiltonian for Deep Learning that would help illuminate why these models work so well.