I am just simply using KLqp, and my neural network model is very simple so it would be using the reparameteric gradient, which is the ADVI algorithm in the ADVI paper.
I have found that
- using Adam optimizer is really helpful especially in training neural networks
- using batch training is helpful
- number of samples doesn’t matter a lot
Please comment below if you find other things helpful!
Can I ask how you got that working? I’m a tensorflow beginner and is trying to figure out how to implement a neural network for CIFAR10 with KLqp. However, my model currently won’t train properly and I’m unsure how to debug it.
I think try to fully understand this tutorial is the way to go:
Hope this helps!
I found initialization with MAP solution is extremely helpful for dealing with very large neural network models.
I also found the get_parents and get_variable method in RandomVariable class is very helpful in debugging process.
老铁，你有evaluate(‘binary_accuracy’) 或者 'categoriate_accuracy’吗？可否看下你这个的代码？如果可以的话，麻烦大侠 发给我的邮箱可否？(邮箱：email@example.com)