How to handle missing values in Gaussian Matrix Factorization

2

In the literature, this is known as implicit feedback. From my (limited) understanding, there are a few ways to handle the zeroes:

  1. Treat the zeros as part of the data, as you mention. For Gaussian MF, you then have to downweight the zeros somehow during inference via large penalizations. Poisson MF naturally solves this by defining a sparse generative process.
  2. Treat the zeroes as missing values (latent variables), and marginalize them out. This can be tough in most cases. To do this in Edward, include a tf.placeholder for the indicators. Here’s an example.
I = tf.placeholder(tf.int32)
mu = tf.matmul(U, V, transpose_b=True)
sigma = tf.ones([M, N])
Y_obs = Normal(mu=tf.gather(mu, I), sigma=tf.gather(sigma, I))
Y_mis = Normal(mu=tf.gather(mu, 1 - I), sigma=tf.gather(sigma, 1 - I))

qY_mis = Normal(
    mu=tf.Variable(tf.random_uniform(Y_mis.shape)),
    sigma=tf.Variable(tf.nn.softplus(tf.random_uniform(Y_mis.shape)))
)

inference = ed.KLqp({U: qU, V: qV, Y_mis: qY_mis}, data={Y_obs: y_train, I: I_train})

Your mileage will vary depending on how you structure the indicators and nonzero values in the matrix.