# Practical experiences using ADVI training Bayesian neural network

**URL:** https://discourse.edwardlib.org/t/practical-experiences-using-advi-training-bayesian-neural-network/836
**Category:** General
**Created:** [June 29, 2018, 9:39pm UTC](https://discourse.edwardlib.org/t/practical-experiences-using-advi-training-bayesian-neural-network/836 "2018-06-29T21:39:36Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![pswpswpsw](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/pswpswpsw/32/332_2.png) [@pswpswpsw](https://discourse.edwardlib.org/u/pswpswpsw)
#### Post date: [June 29, 2018, 9:39pm UTC](https://discourse.edwardlib.org/t/practical-experiences-using-advi-training-bayesian-neural-network/836/1 "2018-06-29T21:39:36Z")

</div>

Hello!

I am just simply using KLqp, and my neural network model is very simple so it would be using the reparameteric gradient, which is the ADVI algorithm in the ADVI paper.

I have found that

- using **Adam optimizer** is really helpful especially in training neural networks
- using batch training is helpful
- number of samples doesn’t matter a lot

Please comment below if you find other things helpful!

---

<div class="post-metadata">

### Author: ![jmzeng](https://avatars.discourse-cdn.com/v4/letter/j/ee59a6/32.png) [@jmzeng](https://discourse.edwardlib.org/u/jmzeng)
#### Post date: [July 13, 2018, 9:39pm UTC](https://discourse.edwardlib.org/t/practical-experiences-using-advi-training-bayesian-neural-network/836/2 "2018-07-13T21:39:29Z")

</div>

Hi,

Can I ask how you got that working? I’m a tensorflow beginner and is trying to figure out how to implement a neural network for CIFAR10 with KLqp. However, my model currently won’t train properly and I’m unsure how to debug it.

Thanks!!

---

<div class="post-metadata">

### Author: ![pswpswpsw](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/pswpswpsw/32/332_2.png) [@pswpswpsw](https://discourse.edwardlib.org/u/pswpswpsw)
#### Post date: [July 19, 2018, 4:32am UTC](https://discourse.edwardlib.org/t/practical-experiences-using-advi-training-bayesian-neural-network/836/3 "2018-07-19T04:32:04Z")

</div>

Hi,

I think try to **fully** understand this tutorial is the way to go:

[http://edwardlib.org/tutorials/bayesian-neural-network](http://edwardlib.org/tutorials/bayesian-neural-network)

Hope this helps!

---

<div class="post-metadata">

### Author: ![pswpswpsw](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/pswpswpsw/32/332_2.png) [@pswpswpsw](https://discourse.edwardlib.org/u/pswpswpsw)
#### Post date: [August 19, 2018, 11:14pm UTC](https://discourse.edwardlib.org/t/practical-experiences-using-advi-training-bayesian-neural-network/836/4 "2018-08-19T23:14:44Z")

</div>

I found initialization with MAP solution is extremely helpful for dealing with very large neural network models.

---

<div class="post-metadata">

### Author: ![pswpswpsw](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/pswpswpsw/32/332_2.png) [@pswpswpsw](https://discourse.edwardlib.org/u/pswpswpsw)
#### Post date: [August 19, 2018, 11:15pm UTC](https://discourse.edwardlib.org/t/practical-experiences-using-advi-training-bayesian-neural-network/836/5 "2018-08-19T23:15:30Z")

</div>

I also found the get\_parents and get\_variable method in RandomVariable class is very helpful in debugging process.

---

<div class="post-metadata">

### Author: ![freedom521jin](https://avatars.discourse-cdn.com/v4/letter/f/d26b3c/32.png) [@freedom521jin](https://discourse.edwardlib.org/u/freedom521jin)
#### Post date: [May 9, 2019, 9:00am UTC](https://discourse.edwardlib.org/t/practical-experiences-using-advi-training-bayesian-neural-network/836/6 "2019-05-09T09:00:05Z")

</div>

老铁，你有evaluate(‘binary\_accuracy’) 或者 'categoriate\_accuracy’吗？可否看下你这个的代码？如果可以的话，麻烦大侠 发给我的邮箱可否？(邮箱：stonejack@foxmail.com)  
感谢大侠！