# Iterative estimators ("bayes filters") in Edward?

**URL:** <https://discourse.edwardlib.org/t/iterative-estimators-bayes-filters-in-edward/104>\
**Category:** General\
**Created:** [April 28, 2017, 3:29pm UTC](https://discourse.edwardlib.org/t/iterative-estimators-bayes-filters-in-edward/104 "2017-04-28T15:29:59Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![xperroni](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/xperroni/32/30_2.png) [@xperroni](https://discourse.edwardlib.org/u/xperroni)\
**Post date:** [April 28, 2017, 3:29pm UTC](https://discourse.edwardlib.org/t/iterative-estimators-bayes-filters-in-edward/104/1 "2017-04-28T15:29:59Z")

</div>

Hello all,

I’m a post-doc researcher working on self-driving vehicles. I’ve recently learned of probabilistic programming languages, and I am looking into how they could be used to solve problems in autonomous driving. I’ve got particularly hooked on Edward and how it enables mixing together deep learning networks and probabilistic techniques.

I plan to give a presentation on Edward to my lab next Thursday. I would like to present a simple demo showing how it could be used to do online estimation of a robot’s position and speed from an input stream of noisy position sensor readings. The problem is that I can’t wrap my mind around how a predict-update loop would be implemented in Edward. I’ve seen a couple examples using `tf.scan` but I’m not sure that’s what I need.

Could anyone give me a hand?

---

<div class="post-metadata">

**Author:** ![dustin](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/dustin/32/134_2.png) [@dustin](https://discourse.edwardlib.org/u/dustin)\
**Post date:** [April 28, 2017, 5:04pm UTC](https://discourse.edwardlib.org/t/iterative-estimators-bayes-filters-in-edward/104/2 "2017-04-28T17:04:53Z")

</div>

To handle streaming data, the easiest place to get started might be with the [Bayesian linear regression](https://github.com/blei-lab/edward/blob/master/examples/bayesian_linear_regression_ppc.py) example. Instead of calling `inference.run()`, you can manually write the training loop:

```python
# placeholders on data, where we will pass in values during training
X_ph = tf.placeholder(tf.float32, [None, D])
y_ph = tf.placeholder(tf.float32, [None])

inference = ed.KLqp({w: qw, b: qb}, data={X: X_ph, y: y_ph})
inference.initialize()
tf.global_variables_initializer().run()

for _ in range(inference.n_iter):
  X_batch, y_batch = next_batch()
  info_dict = inference.update({X_ph: X_batch, y_ph: y_batch})
  inference.print_progress(info_dict)

```

Here, `next_batch()` produces the next batch of data available to us. For example:

```python
def next_batch():
  M = 10 # batch size
  w = np.ones(D) * 5 # true parameters
  x = np.random.randn(M, D)
  y = np.dot(x, w) + np.random.normal(0, 0.1, size=M)
  return x, y

```

This performs one update of inference for each new batch of data. You might also run `update()` multiple times for each new batch.

Note that some mathematical consideration needs to made depending on the choice of inference. Not all enable online estimation. For example with `ed.KLqp`, you need to properly scale the data’s computation, or otherwise the prior will overwhelm each minibatch’s likelihood. To do this, add the `scale` argument to inference,

```python
inference.initialize(scale={y: 100.0})

```

This scales all computation on the minibatch by 100. If we were doing subsampling of a fixed data set of size N and always subsampling a size of M, then the proper scale factor would be N / M. For infinite data, the scale factor is a little more arbitrary and relates to the population posterior [(McInerney et al., 2015)](https://arxiv.org/abs/1507.05253).

An alternative for the most vanilla Bayes filter is to do recursive posterior inferences: first perform inference on one data set to get a posterior, `qw`; then build a new model with `qw` as the prior and perform inference on another data set to get a new posterior, `qw2`; repeat. One disadvantage is that the graph is very large if you aim to repeat this process many times. But if you know how many times you’ll do this process, then the graph is finite and you won’t need dynamic things like `tf.scan`.

---

<div class="post-metadata">

**Author:** ![janekberger](https://avatars.discourse-cdn.com/v4/letter/j/848f3c/32.png) [@janekberger](https://discourse.edwardlib.org/u/janekberger)\
**Post date:** [April 30, 2017, 10:13am UTC](https://discourse.edwardlib.org/t/iterative-estimators-bayes-filters-in-edward/104/3 "2017-04-30T10:13:50Z")

</div>

Hi Dustin,

thanks for the feeding example. Quick question: in the following line, what objects would X and y be and what shape would they have?

`inference = ed.KLqp({w: qw, b: qb}, data={X: X_ph, y: y_ph})`

---

<div class="post-metadata">

**Author:** ![dustin](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/dustin/32/134_2.png) [@dustin](https://discourse.edwardlib.org/u/dustin)\
**Post date:** [April 30, 2017, 10:26am UTC](https://discourse.edwardlib.org/t/iterative-estimators-bayes-filters-in-edward/104/4 "2017-04-30T10:26:43Z")

</div>

The assumption is that `X` and `y` are random variables with shape compatible with the values they’re binded to. For linear models, I guess you don’t actually need `X`. More concretely it would be

```python
X = tf.placeholder(tf.float32, [None, D])
y_ph = tf.placeholder(tf.float32, [None])

w = Normal(loc=tf.zeros(D), scale=tf.ones(D))
b = Normal(loc=tf.zeros(1), scale=tf.ones(1))
y = Normal(loc=ed.dot(X, w) + b, scale=1.0)

inference = ed.KLqp({w: qw, b: qb}, data={y: y_ph})

```

If you know the batch size you can set the `None`s in the placeholders at some fixed value.

Then you would feed in

```python
inference.update({X: X_batch, y_ph: y_batch})

```

---

<div class="post-metadata">

**Author:** ![janekberger](https://avatars.discourse-cdn.com/v4/letter/j/848f3c/32.png) [@janekberger](https://discourse.edwardlib.org/u/janekberger)\
**Post date:** [April 30, 2017, 10:38am UTC](https://discourse.edwardlib.org/t/iterative-estimators-bayes-filters-in-edward/104/5 "2017-04-30T10:38:42Z")

</div>

Cool, thank you! This might be worth to be added to the notebook collection as a general data feed example.

Sorry for highjacking the thread!

---

<div class="post-metadata">

**Author:** ![dustin](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/dustin/32/134_2.png) [@dustin](https://discourse.edwardlib.org/u/dustin)\
**Post date:** [April 30, 2017, 10:52am UTC](https://discourse.edwardlib.org/t/iterative-estimators-bayes-filters-in-edward/104/6 "2017-04-30T10:52:09Z")

</div>

re:notebook. That makes sense. Contributions welcome. Note this is actually a special case of the [data subsampling guide](http://edwardlib.org/api/inference-data-subsampling) without local variables. But it’s significantly easier to work with because there’s no local inferences, so it’s worth writing standalone.

(The topic is relevant; no highjacking crime is committed.)