# Unsuccesful Cholesky decomposition - Gaussian Process Regression

**URL:** <https://discourse.edwardlib.org/t/unsuccesful-cholesky-decomposition-gaussian-process-regression/649>\
**Category:** General\
**Created:** [February 26, 2018, 7:47pm UTC](https://discourse.edwardlib.org/t/unsuccesful-cholesky-decomposition-gaussian-process-regression/649 "2018-02-26T19:47:15Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![bkazi](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/bkazi/32/383_2.png) [@bkazi](https://discourse.edwardlib.org/u/bkazi)\
**Post date:** [February 26, 2018, 7:47pm UTC](https://discourse.edwardlib.org/t/unsuccesful-cholesky-decomposition-gaussian-process-regression/649/1 "2018-02-26T19:47:15Z")

</div>

I’m trying to implement a Gaussian Process regressor using Edward

I’ve used the following code to build the model

```auto
X = tf.placeholder(tf.float32, [N, 1])
f = MultivariateNormalFullCovariance(
        loc=tf.zeros([N]),
        covariance_matrix=rbf(X)
    )
y = MultivariateNormalDiag(
        loc=f,
        scale_diag=tf.ones([N]) * 0.3)

```

and then the proposal distribution and inference as follows

```auto
qf = MultivariateNormalFullCovariance(loc=tf.Variable(tf.random_normal([N])),
            covariance_matrix=tf.Variable(tf.random_normal([N, N])))
inference = ed.KLqp({f: qf}, data={X: x_train, y: y_train})
inference.run(n_iter=5000)

```

I feel like I might have got something wrong. The inference throws an error saying that the Cholesky decomposition was unsuccessful.

Are there any examples that could help?

---

<div class="post-metadata">

**Author:** ![dustin](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/dustin/32/134_2.png) [@dustin](https://discourse.edwardlib.org/u/dustin)\
**Post date:** [February 26, 2018, 8:32pm UTC](https://discourse.edwardlib.org/t/unsuccesful-cholesky-decomposition-gaussian-process-regression/649/2 "2018-02-26T20:32:57Z")

</div>

Your variational approximation has a N x N matrix of free parameters. It is not guaranteed to be positive-semi definite.

Have you looked at the GP classification tutorial ([http://edwardlib.org/tutorials/supervised-classification](http://edwardlib.org/tutorials/supervised-classification))? Another helpful reference is [`examples/cox_process.py`](https://github.com/blei-lab/edward/blob/master/examples/cox_process.py) which adds an epsilon to the diagonal of the covariance matrix to improve stability.

---

<div class="post-metadata">

**Author:** ![bkazi](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/bkazi/32/383_2.png) [@bkazi](https://discourse.edwardlib.org/u/bkazi)\
**Post date:** [February 27, 2018, 1:53pm UTC](https://discourse.edwardlib.org/t/unsuccesful-cholesky-decomposition-gaussian-process-regression/649/3 "2018-02-27T13:53:55Z")

</div>

The epsilon was really useful for the rbf, thanks!

My reasoning for the entire N x N matrix was to capture the covariances between the data in the GP, because I’m having issues making predictions.

I make predictions by sampling the predictive posterior and then averaging.

```auto
post = ed.copy(y, {f: qf})
samples = [sess.run(post, feed_dict={X: x_test}) for _ in range(1000)]

```

The issue is that this produces the same mean as the posterior (conditioned on `x_train`) and doesn’t reflect the `x_test` data. I thought having the covariances would help.

Is there something wrong with what I’m doing or is there a better approach?

---

<div class="post-metadata">

**Author:** ![wawaph](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/wawaph/32/272_2.png) [@wawaph](https://discourse.edwardlib.org/u/wawaph)\
**Post date:** [March 16, 2018, 11:19pm UTC](https://discourse.edwardlib.org/t/unsuccesful-cholesky-decomposition-gaussian-process-regression/649/4 "2018-03-16T23:19:01Z")

</div>

Did you ever get your GP to extrapolate to test points that weren’t in the training set? And could you get the test set to be a different size than the training set? I think I can get comparable GP results as I get from sklearn GaussianProcessRegressor but, so far, I can’t get predictions for other points.

---

<div class="post-metadata">

**Author:** ![charleshamesse](https://avatars.discourse-cdn.com/v4/letter/c/c6cbf5/32.png) [@charleshamesse](https://discourse.edwardlib.org/u/charleshamesse)\
**Post date:** [April 3, 2018, 10:10pm UTC](https://discourse.edwardlib.org/t/unsuccesful-cholesky-decomposition-gaussian-process-regression/649/5 "2018-04-03T22:10:15Z")

</div>

Any luck getting the GP to extrapolate to unobserved test points? I can’t seem to find how to do this.