Confusion about Empirical distributions in Edward

1

Three quick questions I couldn’t work out from the docs:

  1. Is it correct to use a.params (for a = Empirical(…)) to get the tensor of samples of an Empirical distribution after inference? If so, what order are they in? I was going to implement burn in by dropping, say, the first quarter of the samples, but are these at the top or bottom of the array of samples?

  2. I tried using a mixture variable (a categorical distribution over gaussians) in my model - but as the categorical distribution got large (e.g. 1000 elements in the support), it got way too slow to run. It seems that it was making the tensorflow graph way too large, so I’ve rewritten the mixture explicitly, in a vectorized form. Is this my best option?

  3. HMC supports joint inference, right? I’m trying to do joint inference to calculate a posterior on two Gaussian priors, but am not having too much success. Should I lower the step size of HMC?

Thanks again - Edward has been pretty invaluable for this project!

2

  1. MonteCarlo writes samples to parameters in Empirical distributions sequentially (see “Notes” in MonteCarlo's API. To do burn-in, you can drop the first half, e.g., kept_samples = x.params[500:].

  2. Could you provide some code? It depends on how you write the model. Mixture collapses the categorical variable, which means it would be slow if the number of labels is very large as in your case. You’d have to do inference jointly and not collapsed.

  3. Yes. HMC is not very automatic. I recommend tuning both step_size and n_steps.