Hello,
I am trying to do inference on a hierarchical model. Each observation h_ij
is assumed to follow a Bernoulli distribution parametrized by exp(-x_ij ** 2)
, where each x_ij
is assumed to have come from a Normal centered at A.dot(z)
where A is a parameter that maps samples z
from a K-dimensional space to a D-dimensional space. These latent z
s are assumed to follow a Normal distribution. My goal is to infer x
, z
and A
. For this, I am using Variational Inference, but my losses quickly (after the first iteration typically) go to nan
or inf
.
What could be the cause of this problem? Below is the code I am using.
A = Normal(loc=tf.zeros([D, K]), scale=2.0 * tf.ones([D, K]))
z = Normal(loc=tf.zeros([N, K]), scale=tf.ones([N, K]))
x = Normal(loc=tf.matmul(A, z, transpose_b=True), scale=tf.ones([D, N]))
h = Bernoulli(probs=tf.exp(-x**2))
sess = ed.get_session()
x_train, h_train, z_train, A_true = sess.run([x, h, z, A])
qA = Normal(loc=tf.Variable(tf.random_normal([D, K])),
scale=tf.nn.softplus(tf.Variable(tf.random_normal([D, K]))))
qz = Normal(loc=tf.Variable(tf.random_normal([N, K])),
scale=tf.nn.softplus(tf.Variable(tf.random_normal([N, K]))))
qx = Normal(loc=tf.Variable(tf.random_normal([D, N])),
scale=tf.nn.softplus(tf.Variable(tf.random_normal([D, N]))))
inference = ed.KLqp({A: qA, z: qz, x: qx}, data={h: h_train})
inference.run(n_iter=500, n_print=100, n_samples=10)