Cholesky decomposition issue when learning covariance matrices

1

I’m trying to infer a set of covariance matrixes for latent factors. My model looks something like the following for a single latent factor.

mu ~ Normal
sigma ~ Wishart
for (n=1...N)
    x ~ MultivariateNormal(mu, tf.matrix_inverse(sigma))
...

I’m using ed.KLqp and as long as I’m trying to infer sigma, I observe that inference runs successfully for a some number of iterations, and then I hit:

InvalidArgumentError (see above for traceback): Cholesky decomposition was not successful. The input might not be valid.

I assume this means that the covariance matrix input to the multivariate normal isn’t positive definite (or otherwise acceptable?). I’ve tried both MultivariateNormalTriL and MultivariateNormalFullCovariance for x and WishartCholesky and WishartFull for sigma. I’ve also tried using tf.cholesky and omitting tf.matrix_inverse, just for kicks; the results are always the same. I’m just not sure how to go about debugging this; any help would be appreciated.

2

Hello, you can take a look at this.
https://github.com/ecosang/misc/blob/master/multivariate_normal.ipynb
I guess the model requires positivie def. matrix (even in MultivariatenormalFullCovariance.

3 Likes
3

Thanks so much! :smile:

4

Thanks for providing the example. I am having a similar issue as ajbc.
What is it about your solution that makes it work?
I tried the exact same but with WishartFull and MVNFull, instead of cholesky version, and it does not work.

I get the same error as ajbc, or sometimes a python has stopped working message

    2018-02-13 15:52:45.086096: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: Quadro M2000M, pci bus id: 0000:01:00.0, compute capability: 5.0)
2018-02-13 15:52:56.338238: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\kernels\cuda_solvers.cc:159] Creating CudaSolver handles for stream 000002296C2B6F20
2018-02-13 15:52:56.546476: E C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\stream_executor\cuda\cuda_event.cc:49] Error polling for event status: failed to query event: CUDA_ERROR_LAUNCH_FAILED
2018-02-13 15:52:56.546584: F C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\common_runtime\gpu\gpu_event_mgr.cc:203] Unexpected Event status: 1
5

Line 5:
sigma = edm.WishartCholesky(df=nu0, scale=psi0,cholesky_input_output_matrices=True)

I notice you do not take the invese of the sample from the wishart, i.e. you don’t have conjugacy anymore. Is this why this works without numerical issues?