# Mixture of Linear Regression: How?

Hi. I’m beginer of Edward and English.

I tried Mixture of Linear Regression (I think we can find this model in Bishop’s Book) with Edward.
My code is here.

I tried two approach but both did not work.

My first approach is with `ParamMixture`.
I did not use `Mixture` because I want to get posterior distribution of `Cat` (Categorical Distribution). I don’t know why it do not work.

My second approach is coding mixture model directly.
I think that it did not work because `OneHotCategorical` do not have `conjugate_log_prob` method.

I have two questions.

1. Why my first approach do not work? Are there any way works?
2. Is there any reason that `OneHotCategorical` don’t have `conjugate_log_prob`?

With PyMC3, `Categorical` Distribution can be used as index.
(see https://stats.stackexchange.com/questions/185812/regression-mixture-in-pymc3)

But with Edward, slice or `tf.gather` do not work for `ed.Gibbs`.

On the other hand, if I use `ed.MetropolisHastings`, it works. (I reached this method thanks to this post)
But because acceptance rate is very small (less than 0.1), it looks not converge.

The model you’re trying to fit is:

``````p(y | z = 1) = Normal(mu_1 + x beta_1, sigma_1)
p(y | z = 0) = Normal(mu_0 + x beta_0, sigma_0)
``````

`ParamMixture` is not appropriate for this model because you need to infer which component each data point came from, not just the mixture proportions.

Assuming only few classes, you could marginalize over `z`, but that would require a custom `RandomVariable` in Edward.

You could also do this using expectation maximization, but that is probabily easier expressed in numpy/scipy rather than Edward.

@aksarkar
Since I want to avoid marginalizing over `z`, it sounds difficult to infer with Edward.