Dropout (Just notes)
 Dropout (Just notes)
Srivastava, Nitish, et al. "Dropout: a simple way to prevent neural networks from overfitting." Journal of Machine Learning Research 15.1 (2014): 1929-1958.
- Reduces overfitting
- Thinned networks -> dropout samples from
- Approximately combine different NN architectures simultaneously
- Optimal prob of retention : 1>x>.5
- Can be added with Restricted Boltzmann machines too