# Nonnegative Matrix Factorization

**URL:** <https://discourse.edwardlib.org/t/nonnegative-matrix-factorization/341>\
**Category:** General\
**Created:** [September 5, 2017, 6:31pm UTC](https://discourse.edwardlib.org/t/nonnegative-matrix-factorization/341 "2017-09-05T18:31:24Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jacobcvt12](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/jacobcvt12/32/111_2.png) [@jacobcvt12](https://discourse.edwardlib.org/u/jacobcvt12)\
**Post date:** [September 5, 2017, 6:31pm UTC](https://discourse.edwardlib.org/t/nonnegative-matrix-factorization/341/1 "2017-09-05T18:31:24Z")

</div>

I am trying to use Edward for non-negative matrix factorization (NMF). I’ve written two toy examples, one of which has no constraints and works. However, the NMF yields all `nan` values for the decomposed matrices.

The first method is mostly borrowed from one of your examples on github

```python
import edward as ed
import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf

from edward.models import Normal, Exponential, Gamma
from numpy.random import normal, exponential

def build_toy_dataset(U, V, N, M, noise_std=0.1):
  R = np.dot(np.transpose(U), V) + normal(0, noise_std, size=(N, M))
  return R

N = 50 # number of users
M = 60 # number of movies
D = 3 # number of latent factors

# true latent factors
U_true = normal(size=(D, N))
V_true = normal(size=(D, M))

# DATA
R_true = build_toy_dataset(U_true, V_true, N, M)

# MODEL
U = Normal(loc=tf.zeros([D, N]), scale=tf.ones([D, N]))
V = Normal(loc=tf.zeros([D, M]), scale=tf.ones([D, M]))
R = Normal(loc=tf.matmul(tf.transpose(U), V), scale=tf.ones([N, M]))

# INFERENCE
qU = Normal(loc=tf.Variable(tf.random_normal([D, N])),
            scale=tf.nn.softplus(tf.Variable(tf.random_normal([D, N]))))
qV = Normal(loc=tf.Variable(tf.random_normal([D, M])),
            scale=tf.nn.softplus(tf.Variable(tf.random_normal([D, M]))))

optimizer = tf.train.AdagradOptimizer(learning_rate=0.01)
inference.initialize(optimizer=optimizer)
inference = ed.KLqp({U: qU, V: qV}, data={R: R_true})
inference.run(n_iter=1000)

```

The model is fit to the data with no problems. However, if I simulate `U` and `V` from exponentials, use exponential priors, and exponential variational distributions, the decomposed matrices consist of `nan`s.

```python
def build_toy_dataset(U, V, N, M, noise_std=0.1):
  R = np.dot(np.transpose(U), V) + normal(0, noise_std, size=(N, M))
  return R

N = 50 # number of users
M = 60 # number of movies
D = 3 # number of latent factors

# true latent factors
U_true = exponential(size=(D, N))
V_true = exponential(size=(D, M))

# DATA
R_true = build_toy_dataset(U_true, V_true, N, M)

# MODEL
U = Exponential(rate=tf.ones([D, N]))
V = Exponential(rate=tf.ones([D, M]))
R = Normal(loc=tf.matmul(tf.transpose(U), V), scale=tf.ones([N, M]))

# INFERENCE
qU = Exponential(rate=tf.nn.softplus(tf.Variable(tf.random_normal([D, N]))))
qV = Exponential(rate=tf.nn.softplus(tf.Variable(tf.random_normal([D, M]))))

optimizer = tf.train.AdagradOptimizer(learning_rate=0.01)
inference.initialize(optimizer=optimizer)
inference = ed.KLqp({U: qU, V: qV}, data={R: R_true})
inference.run(n_iter=1000)

```

Is there something wrong with the inference method being chosen by `KLqp`? Or is there a problem with using exponentials as variational densities? (certainly something like a gamma would be better, but I wanted to start simple)

Thanks!  
Jacob

---

<div class="post-metadata">

**Author:** ![jhmarcus](https://avatars.discourse-cdn.com/v4/letter/j/ce7236/32.png) [@jhmarcus](https://discourse.edwardlib.org/u/jhmarcus)\
**Post date:** [September 5, 2017, 7:17pm UTC](https://discourse.edwardlib.org/t/nonnegative-matrix-factorization/341/2 "2017-09-05T19:17:48Z")

</div>

Hey @jacobcvt12,

This is post is relevant to you …

> [@Loss increasing in inference of Poisson Factorization](https://discourse.edwardlib.org/t/loss-increasing-in-inference-of-poisson-factorization/221):
>
> I am try to implement a simple Poisson Factorization theta ~ Gamma(a, b) beta ~ Gamma(c, d) x ~ Poisson( theta . beta ) My codes: from edward.models import Poisson, Gamma import edward as ed # Toy dataset M = (np.random.random((300,200))\>0.9).astype(int) # Model theta = Gamma(1.\*tf.ones([300, 10]), 1.) beta = Gamma(1.\*tf.ones([200, 10]), 1.) X = Poisson(tf.matmul(theta, tf.transpose(beta))) # Inference q\_theta = Gamma(tf.nn.softplus(tf.Variable(tf.random\_normal([300, 10]))), …

maybe you could also try using a lognormal variational distributions for U and V?

---

<div class="post-metadata">

**Author:** ![dustin](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/dustin/32/134_2.png) [@dustin](https://discourse.edwardlib.org/u/dustin)\
**Post date:** [September 5, 2017, 7:41pm UTC](https://discourse.edwardlib.org/t/nonnegative-matrix-factorization/341/3 "2017-09-05T19:41:45Z")

</div>

+1 to @jhmarcus’ answer. For lognormal variational approximations against Gamma priors, I also recommend looking at [`examples/deep_exponential_family.py`](https://github.com/blei-lab/edward/blob/master/examples/deep_exponential_family.py).
