Go to index

SELU

Reading time : ~1 min

by Subhaditya Mukherjee

• [23]SELU
• Notes

Paper notes for the paper

[23]SELU

• Klambauer, G., Unterthiner, T., Mayr, A., & Hochreiter, S. (2017). Self-normalizing neural networks. In Advances in neural information processing systems (pp. 971-980). Paper

Notes

• train deep networks with many layers
• employ strong regularization schemes
• which induce self-normalizing properties like variance stabilization which in turn avoids exploding and vanishing gradients.

Related posts:  FP16  AI Superpowers Kai Fu Lee  Digital Minimalism Cal Newport  More Deep Learning, Less Crying - A guide  Super resolution  Federated Learning  Taking Batchnorm For Granted  A murder mystery and Adversarial attack  Thank you and a rain check  Pruning