Implementation of ZIP model in Edwards

1

Hello,

I have implemented a zero-inflated Poisson model. However, I am getting trivial posterior estimates for both the mean and the Bernoulli probability. I am not sure what is wrong with the model specification. My goal is to build a ZIP Factor Model using the Metropolis-Hasting sampler. I am using edward==1.3.1 and tensorflow==1.1.0. I have taken snippets from my jupyter notebook and posted them below:

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import edward as ed
import numpy as np
import tensorflow as tf

# Generate Data

np.random.seed(56789)

N = 1000
a  = 0.5
b  = 2
p_0 = 0.2
lam_true = np.random.gamma(shape=a, scale=b)
z_true = np.random.binomial(n=1, p=p_0, size=N)

from edward.models import Gamma, Poisson, Bernoulli, Beta, Empirical


print('lambda={}'.format(lam_true))
print('p={}'.format(p_0))

# Prior definition
alpha = tf.Variable(1.8, trainable=False)
beta = tf.Variable(1.0, trainable=False)

a = tf.Variable(1.0, trainable=False)
b = tf.Variable(4.0, trainable=False)

# Posterior inference

# Probabilistic model

lam = Gamma(alpha, beta)
p = Beta(a, b)
z = Bernoulli(tf.ones(N) * p)
y = Poisson(rate=tf.scalar_mul(lam, tf.to_float(1 - z, name='ToFloat')))

# Inference
T = 10000                        # Number of samples.
qlam = Empirical(params=tf.Variable(tf.zeros(T) + 0.5))
qp = Empirical(params=tf.Variable(tf.zeros(T) + 0.5))
qz = Empirical(params=tf.Variable(tf.zeros([T, N], dtype=tf.int32)))

glam = Gamma(1.8, 1.0)
gp = Beta(1.0, 4.0)
gz = Bernoulli(tf.zeros(N))

inference = ed.MetropolisHastings(latent_vars={lam: qlam, p: qp, z: qz},
                                  proposal_vars={lam: glam, p: gp, z: gz},
                                  data={y: y_obs})

inference.initialize()

sess = ed.get_session()
tf.global_variables_initializer().run()

for _ in range(inference.n_iter):
    info_dict = inference.update()
    inference.print_progress(info_dict)

t = info_dict['t']
if t == 1 or t % inference.n_print == 0:
    qlam_mean, qp_mean = sess.run([qlam.mean(), qp.mean()])
    print("")
    print("Inferred lambda:")
    print(qlam_mean)
    print("Inferred probability:")
    print(qp_mean)
y_obs = np.random.poisson(lam=lam_true*(1 - z_true))

Best,
Mark

2

I ran the code. (I had to put y_obs somewhere higher before instantiating inference for it to run.)

it shows that the acceptance probability for MH is zero. My guess is that MH fails because it’s making a proposal in a (N + 1 + 1=1002)-dimensional space. This proposal will almost always be rejected. Have you tried an alternative inference alg?

3

Hi Dustin,

I have been able to implement zero-inflated Poisson model in Edward using ed.MAP and the result are good. However, I have not had much success with other inference approaches in Edward.

I have been able to implement a zero-inflated Poisson factor model with an IBP prior in JAGS from a small slice of training data with excellent results. By the way, I’m a postdoc in the Engelhardt lab. I would like some help in implementing this model and making it scalable in Edward.

Best,
Mark

4

It’s hard to say as it’s a model with discrete latent variables (and a Gamma prior). I’m not sure if any current methods would do well on that model—my suggestion is to try coding up a model-specific inference algorithm.

Alternatively, change the model to use differentiably reparameterizable distributions such as a relaxed Bernoulli over Bernoulli and log-Normal over Gamma. Then my guess is ed.KLqp will “just work”.

5

Hi Mark,

Could you share your code for ed.MAP inference on the ZIP model?

Thanks

6

Hi Pedro,

I do have a Jupyter notebook that has an implementation of ZIP model in Edwards. Can you give me your email address so that I can send you the notebook? I hope it will be instructive.

Best,
Mark

1 Like
7

Could you please send me the file as well?

Thanks.

8

@mac449 I sent you a private message. Thanks!

1 Like
9

Hi @mac449

Is it possible to get a hold of the jupyter notebook’s as well for the ZIP models you’ve developed Edward - would be awesome to look through.