Mixture of Linear Regression: How?


Hi. I’m beginer of Edward and English.

I tried Mixture of Linear Regression (I think we can find this model in Bishop’s Book) with Edward.
My code is here.

I tried two approach but both did not work.

My first approach is with ParamMixture.
I did not use Mixture because I want to get posterior distribution of Cat (Categorical Distribution). I don’t know why it do not work.

My second approach is coding mixture model directly.
I think that it did not work because OneHotCategorical do not have conjugate_log_prob method.

I have two questions.

  1. Why my first approach do not work? Are there any way works?
  2. Is there any reason that OneHotCategorical don’t have conjugate_log_prob?


With PyMC3, Categorical Distribution can be used as index.
(see https://stats.stackexchange.com/questions/185812/regression-mixture-in-pymc3)

But with Edward, slice or tf.gather do not work for ed.Gibbs.

On the other hand, if I use ed.MetropolisHastings, it works. (I reached this method thanks to this post)
But because acceptance rate is very small (less than 0.1), it looks not converge.