Hi. I’m beginer of Edward and English.

I tried Mixture of Linear Regression (I think we can find this model in Bishop’s Book) with Edward.

My code is **here**.

I tried two approach but both did not work.

My first approach is with `ParamMixture`

.

I did not use `Mixture`

because I want to get posterior distribution of `Cat`

(Categorical Distribution). I don’t know why it do not work.

My second approach is coding mixture model directly.

I think that it did not work because `OneHotCategorical`

do not have `conjugate_log_prob`

method.

I have two questions.

- Why my first approach do not work? Are there any way works?
- Is there any reason that
`OneHotCategorical`

don’t have `conjugate_log_prob`

?

With PyMC3, `Categorical`

Distribution can be used as index.

(see https://stats.stackexchange.com/questions/185812/regression-mixture-in-pymc3)

But with Edward, slice or `tf.gather`

do not work for `ed.Gibbs`

.

On the other hand, if I use `ed.MetropolisHastings`

, it works. (I reached this method thanks to this post)

But because acceptance rate is very small (less than 0.1), it looks not converge.

The model you’re trying to fit is:

```
p(y | z = 1) = Normal(mu_1 + x beta_1, sigma_1)
p(y | z = 0) = Normal(mu_0 + x beta_0, sigma_0)
```

`ParamMixture`

is not appropriate for this model because you need to infer which component each data point came from, not just the mixture proportions.

Assuming only few classes, you could marginalize over `z`

, but that would require a custom `RandomVariable`

in Edward.

You could also do this using expectation maximization, but that is probabily easier expressed in numpy/scipy rather than Edward.

@aksarkar

Thank you for your reply!

Since I want to avoid marginalizing over `z`

, it sounds difficult to infer with Edward.

I will try to implement Gibbs sampler or Mean Field Approximation with numpy/scipy.