In certain models, we are expected to evaluate the variance of the posterior (e.g. Linear regression with ARD) using a hyper-prior. Usually, I have seen that Half-Cauchy hyper-prior is used in such occasions, due the certain proprieties of the variance (because variance is always > 0 ).
However, Edward does not implement Half-Cauchy distribution. Therefore, what should be a suitable distributions to consider as the distribution of the variance when performing the KLqp?
Thanks
How about using the TransformedDistribution model to create the necessary probability distribution?
http://edwardlib.org/api/ed/models/TransformedDistribution
We can create, for example, log-normal distribution using this model.
Similarly, can’t we create Half-Cauchy distribution?
Yeah, I suppose we could do that.
I know how to implement log-normal using TransformedDistribution yet I’m not sure how to extend that for Half-Cauchy implementation.
Me, too. I do not know how to extend it for half-Cauchy implementation.
By the way, in Pyro, half-Cauthy distribution is implemented.
http://docs.pyro.ai/en/0.2.1-release/_modules/pyro/distributions/half_cauchy.html
Will not it be helpful for something?
Have you been introduced to the following resources before?
Gelman’s paper on priors for variance parameters in hierarchical models (where the half cauchy idea comes from I think):
http://www.stat.columbia.edu/~gelman/research/published/taumain.pdf
The regularized horseshoe (the horseshoe and horseshoe+ are covered, so you can refer to them if you want):
Prior choice recommendations (not just for variance) from the STAN team:
Then this resource has some alternative parameterizations of the half cauchy:
https://betanalpha.github.io/assets/case_studies/fitting_the_cauchy.html
I hope this helps!