News

Power Laws in Deep Learning 2: Universality - Machine Learning

Power Law Distributions in Deep Learning In a previous post, we saw that the Fully Connected (FC) layers of the most common pre-trained Deep Learning display power law behavior.  Specifically, for each FC weight matrix $latex \mathbf{W}&bg=ffffff $, we compute the eigenvalues $latex...
Want to leave a comment?