Tutorial for multiple variational methods using Poisson regression?

1

I’m an independent researcher working on a pet project. The general idea is to use a Gaussian Process nonlinear autoregressive model to do forecasting.

I hope to work up to the larger model by building (and understanding) simpler ones. So, I thought I would start by writing a tutorial for variations on Poisson GP regression.

Perhaps such a tutorial would be useful to other new Edward users, beside myself. If anyone is interested in assisting, let me know. I would make a Jupyter notebook, and might use both AutoGrad and Edward (TensorFlow) to highlight differences and similarities. I want to make sure I understand what is happening under the hood.

For inference, I’d like to use a Gaussian mean field for q(), a normalizing flow over Gaussian mean fields, and a likelihood-free variational approach.

I have zero experience with Edward, so I’d be grateful for assistance.

John

2

Can you elaborate on what you mean by a “Poisson GP regression” model? There are examples online of linear models and logistic regression. It’s easy to write a Poisson likelihood and use a log link function instead of Bernoulli-logit.

A piece of advice: Edward models are tightly integrated into the TensorFlow graph. This means instead of autograd, you can write down the model to leverage tf.gradients over, say, its log joint density. “Mak[ing] sure that [you] understand what is happening” could still be done in Edward. You just code up the model in Edward and not rely on one of the built-in inference engines.

3

Thanks Dustin. I think I need to dig into the (impressive) Edward code a bit more. But so far I am not seeing a normalizing flow inference engine. I do see the ImplicitKLqp and the others I was interested in.

I’m curious about differences/strengths/weaknesses between Autograd and TensorFlow, and am more familiar with the syntax of the former. I will try to work out a few problems for side-by-side comparisons.

John

1 Like
4

Just to reiterate: A tutorial explaining how to build a GP regression model, and how to handle all sorts of variations you might need for forecasting, would be amazing. Let me know if I can help.

5

Thanks much Dustin. I’ve started work on a tutorial, but it will take me some time. Once its in shape, or if I have questions, I’ll drop you a note.
Best,
John