Hi. I’m beginer of Edward and English.
I tried Mixture of Linear Regression (I think we can find this model in Bishop’s Book) with Edward.
My code is here.
I tried two approach but both did not work.
My first approach is with
I did not use
Mixture because I want to get posterior distribution of
Cat (Categorical Distribution). I don’t know why it do not work.
My second approach is coding mixture model directly.
I think that it did not work because
OneHotCategorical do not have
I have two questions.
- Why my first approach do not work? Are there any way works?
- Is there any reason that
OneHotCategorical don’t have
Categorical Distribution can be used as index.
But with Edward, slice or
tf.gather do not work for
On the other hand, if I use
ed.MetropolisHastings, it works. (I reached this method thanks to this post)
But because acceptance rate is very small (less than 0.1), it looks not converge.
The model you’re trying to fit is:
p(y | z = 1) = Normal(mu_1 + x beta_1, sigma_1)
p(y | z = 0) = Normal(mu_0 + x beta_0, sigma_0)
ParamMixture is not appropriate for this model because you need to infer which component each data point came from, not just the mixture proportions.
Assuming only few classes, you could marginalize over
z, but that would require a custom
RandomVariable in Edward.
You could also do this using expectation maximization, but that is probabily easier expressed in numpy/scipy rather than Edward.
Thank you for your reply!
Since I want to avoid marginalizing over
z, it sounds difficult to infer with Edward.
I will try to implement Gibbs sampler or Mean Field Approximation with numpy/scipy.