Hi, I’m trying to implement a very simple Variational Bayes regression model, specifically the one from chapter 10.3 of Bishop’s Pattern Recognition and Machine Learning . The code below that uses Edward runs succesfuly, but the fit is very poor. Using the method described in the book, the fit is p…

Did you check that the algorithm converged? My guess is it hasn’t. One useful tool is to use TensorBoard to track the loss function (see the tutorial on the website).

Indeed, the algorithm did not converge. But even setting the iteration count to 5k or 50k doe not help as the loss oscillates around some value. Is there maybe something else I am doing wrong or is this problem not suitable for solving with edward (which is otherwise great!) ? [image]

ed.KLqp has noisy gradients if you use a Gamma distribution—it relies on score function gradients, which tend to be unstable for Gamma’s. You can try increasing n_samples in inference.initialize(); this will help reduce variance. (See the KLqp tutorial for the mathematical details.) Alternatively, y…

Bayesian regression

VitSkvara June 26, 2017, 12:21pm 5

Thanks for the answer. Using more samples was somewhat helpful, as well as using the lognormal for alpha. Best results were obtained using ed.KLpq.

Topic		Replies	Views
Unable to inference the variance in a simple bayesian linear regression	0	917	January 7, 2018
Poisson-gamma regression	6	2033	March 19, 2017
Biased variance estimates when using KLqp for Bayesian Linear Regression	7	1414	February 9, 2018
Understanding Edward KLqp algorithm	2	1532	July 23, 2018
A toy normal model failed (klqp) and why?	2	1704	July 25, 2017

Bayesian regression

Related topics