# Confused by error message from inference.run() for LDA with KLqp

**URL:** <https://discourse.edwardlib.org/t/confused-by-error-message-from-inference-run-for-lda-with-klqp/119>\
**Category:** General\
**Created:** [May 5, 2017, 10:18pm UTC](https://discourse.edwardlib.org/t/confused-by-error-message-from-inference-run-for-lda-with-klqp/119 "2017-05-05T22:18:48Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![kpf](https://avatars.discourse-cdn.com/v4/letter/k/ed8c4c/32.png) [@kpf](https://discourse.edwardlib.org/u/kpf)\
**Post date:** [May 5, 2017, 10:18pm UTC](https://discourse.edwardlib.org/t/confused-by-error-message-from-inference-run-for-lda-with-klqp/119/1 "2017-05-05T22:18:48Z")

</div>

I’m just getting started setting up Bayesian models in Edward so this may be a dumb question. I’ve been working through the tutorials with no problems and now I’m trying to set up Latent Dirichlet Allocation in Edward. I’m following the data structure used in the Stan manual example for LDA, which uses two single, long vectors listing token ids and the associated document number rather than the typical sparse matrix representation.  
Everything seems to go fine until I try to run the inference step, at which point I get an error message that I don’t know how to interpret.

Here’s my attempt to code that model:

```
doc = tf.placeholder(tf.int32, N)

theta = Dirichlet(tf.zeros([M,K]) + 1/K)
phi = Dirichlet(tf.zeros([K,V]) + 1/V)

z = Categorical(tf.gather(theta, tf.gather(doc, tf.range(N))))
w = Categorical(tf.gather(phi, tf.gather(z, tf.range(N))))

```

Here’s the variational model and the KLqp inference:

```
qz = Empirical(params=tf.Variable(tf.zeros([1, N], dtype=tf.int32)))
qw = Empirical(params=tf.Variable(tf.zeros([1, N], dtype=tf.int32)))
qtheta = Empirical(params=tf.Variable(tf.zeros([1, M, K], dtype=tf.float32) + 1/K))
qphi = Empirical(params=tf.Variable(tf.zeros([1, K, V], dtype=tf.float32) + 1/V))

its = 500
inference = ed.KLqp({z: qz, w: qw, theta: qtheta, phi: qphi}, data={w: w_, doc: doc_})
inference.run(n_iter=its, n_print=100, n_samples=10)

```

I seem to be missing something basic here. Here’s the error message I get from inference.run():

```
>>> inference.run(n_iter=its, n_print=100, n_samples=10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/kpf/venv/edward/src/edward/edward/inferences/inference.py", line 116, in run
    self.initialize(*args, **kwargs)
  File "/home/kpf/venv/edward/src/edward/edward/inferences/klqp.py", line 86, in initialize
    return super(KLqp, self).initialize(*args, **kwargs)
  File "/home/kpf/venv/edward/src/edward/edward/inferences/variational_inference.py", line 73, in initialize
    self.loss, grads_and_vars = self.build_loss_and_gradients(var_list)
  File "/home/kpf/venv/edward/src/edward/edward/inferences/klqp.py", line 127, in build_loss_and_gradients
    return build_reparam_loss_and_gradients(self, var_list)
  File "/home/kpf/venv/edward/src/edward/edward/inferences/klqp.py", line 375, in build_reparam_loss_and_gradients
    qz_copy = copy(qz, scope=scope)
  File "/home/kpf/venv/edward/src/edward/edward/util/random_variables.py", line 232, in copy
    new_rv = type(rv)(*args, **kwargs)
TypeError: __init__ () missing 1 required positional argument: 'params'

```

Any idea what might be going on here? I’m also getting errors from other inference methods so my guess is I’ve just set something up wrong in the model.

---

<div class="post-metadata">

**Author:** ![suderoy](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/suderoy/32/239_2.png) [@suderoy](https://discourse.edwardlib.org/u/suderoy)\
**Post date:** [May 12, 2017, 2:44pm UTC](https://discourse.edwardlib.org/t/confused-by-error-message-from-inference-run-for-lda-with-klqp/119/2 "2017-05-12T14:44:27Z")

</div>

As far I understand from [http://edwardlib.org/api/inference-data-subsampling](http://edwardlib.org/api/inference-data-subsampling)

One should use Empirical with SGLD (stochastic gradient Langevin dynamics) not for inference using KL divergence. You have to explicitly specify the variational parameters there similar to they have done in the data-subsampling example.

---

<div class="post-metadata">

**Author:** ![dustin](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/dustin/32/134_2.png) [@dustin](https://discourse.edwardlib.org/u/dustin)\
**Post date:** [May 29, 2017, 4:54am UTC](https://discourse.edwardlib.org/t/confused-by-error-message-from-inference-run-for-lda-with-klqp/119/3 "2017-05-29T04:54:59Z")

</div>

I agree with @suderoy. You (typically) have to use a parametric distribution for variational inference.

That specific error occurs when trying to copy an Empirical random variable. That’s unexpected behavior which I’m fixing in [a pull request (#658)](https://github.com/blei-lab/edward/pull/658).

---

<div class="post-metadata">

**Author:** ![kpf](https://avatars.discourse-cdn.com/v4/letter/k/ed8c4c/32.png) [@kpf](https://discourse.edwardlib.org/u/kpf)\
**Post date:** [May 29, 2017, 5:01pm UTC](https://discourse.edwardlib.org/t/confused-by-error-message-from-inference-run-for-lda-with-klqp/119/4 "2017-05-29T17:01:37Z")

</div>

Thanks @suderoy and @dustin. That explains it!

---

<div class="post-metadata">

**Author:** ![dustin](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/dustin/32/134_2.png) [@dustin](https://discourse.edwardlib.org/u/dustin)\
**Post date:** [July 3, 2017, 10:24am UTC](https://discourse.edwardlib.org/t/confused-by-error-message-from-inference-run-for-lda-with-klqp/119/5 "2017-07-03T10:24:27Z")

</div>

Related Github issues:

[https://github.com/blei-lab/edward/issues/423](https://github.com/blei-lab/edward/issues/423)

[https://github.com/blei-lab/edward/issues/463](https://github.com/blei-lab/edward/issues/463)

[https://github.com/blei-lab/edward/issues/473](https://github.com/blei-lab/edward/issues/473)

[https://github.com/blei-lab/edward/issues/476](https://github.com/blei-lab/edward/issues/476)

---

<div class="post-metadata">

**Author:** ![lir](https://avatars.discourse-cdn.com/v4/letter/l/edb3f5/32.png) [@lir](https://discourse.edwardlib.org/u/lir)\
**Post date:** [April 10, 2018, 5:28pm UTC](https://discourse.edwardlib.org/t/confused-by-error-message-from-inference-run-for-lda-with-klqp/119/6 "2018-04-10T17:28:08Z")

</div>

Did you solve it?  
Why is it not possible to estimate Categorical distribution?