Suitable Distribution for variance as a Hyperprior


#1

In certain models, we are expected to evaluate the variance of the posterior (e.g. Linear regression with ARD) using a hyper-prior. Usually, I have seen that Half-Cauchy hyper-prior is used in such occasions, due the certain proprieties of the variance (because variance is always > 0 ).

However, Edward does not implement Half-Cauchy distribution. Therefore, what should be a suitable distributions to consider as the distribution of the variance when performing the KLqp?

Thanks


#2

How about using the TransformedDistribution model to create the necessary probability distribution?
http://edwardlib.org/api/ed/models/TransformedDistribution
We can create, for example, log-normal distribution using this model.
Similarly, can’t we create Half-Cauchy distribution?


#3

Yeah, I suppose we could do that.

I know how to implement log-normal using TransformedDistribution yet I’m not sure how to extend that for Half-Cauchy implementation.


#4

Me, too. I do not know how to extend it for half-Cauchy implementation.
By the way, in Pyro, half-Cauthy distribution is implemented.
http://docs.pyro.ai/en/0.2.1-release/_modules/pyro/distributions/half_cauchy.html
Will not it be helpful for something?


#5

Have you been introduced to the following resources before?

Gelman’s paper on priors for variance parameters in hierarchical models (where the half cauchy idea comes from I think):
http://www.stat.columbia.edu/~gelman/research/published/taumain.pdf

The regularized horseshoe (the horseshoe and horseshoe+ are covered, so you can refer to them if you want):

Prior choice recommendations (not just for variance) from the STAN team:

Then this resource has some alternative parameterizations of the half cauchy:
https://betanalpha.github.io/assets/case_studies/fitting_the_cauchy.html

I hope this helps!