Renyi divergence variational inference

[jbr]

Thanks for the very detailed reply.

@dustin

• In klqp.py, we don’t place the build_loss_and_gradients function as a method inside KLqp because we use it across many KLqp algorithms. Since your function is only used in one class, it’s recommended you write it as a method (c.f., klpq.py).

Yes that make sense. I’ll change that.

@dustin

• What’s justification for a default alpha=0.2?

None. It was just to avoid being by default in a special case for testing in case I was not using the correct way to feed the parameter alpha. If I was to release the code I would set a default alpha=1. to get VI by default.

@dustin

• What does a ‘min’ backward pass correspond to? I’m not sure if I recall a VR-min; does it correspond to alpha → \infty? I haven’t done the math.

I need to do the math to but yes that my intuition on this. alpha → \infty correspond to a zero-enforcing behavior. In that case it would make sense to learn from the sample with the smallest loss (hand waving here — I’ll do the math).

@dustin

• Is the logF = tf.reshape(logF, [inference.n_samples, 1]) reshape necessary? Seems like you could just do logF = tf.stack(logF)

True. I’ll remove that

@dustin

• Since you only clip on the LHS, you can change logF = tf.log(tf.clip_by_value(tf.reduce_mean(tf.exp(logF - logF_max), 0), 1e-9, np.inf)) to use tf.maximum(1e-9, *).

Again true.

@dustin

Would you be interested in submitting a PR? The algorithm would be a nice addition to Edward’s arsenal.

Sure why not. Let me clean the code a bit, do the math for VR-min and I’ll open a request.

@dustin

@jbr

So I think that should be trick 2 and 3 covered. Are I’m completely wrong?

You’re correct. But it also covers trick 1: if the distributions have the property reparameterization_type == tf.contrib.distributions.FULLY_REPARAMETERIZED, then gradients with respect to distribution parameters backpropagate through the sampling. See also discussion in Gradient is incorrect for log pdf of Normal distribution · Issue #7236 · tensorflow/tensorflow · GitHub.

Informative pointer thanks.