I am having issues applying SGHMC inference on a convolutional neural network, such as LeNet. We believe our LeNet architecture is valid, but after 50000 iterations, the network does not learn anything.
We tested on applying MNIST dataset on a single layer NN with SGHMC, and we got 90% after 10000 iteration with batch size of 500.
Does Edward/SGHMC work on when using convolution operation, or are we doing something wrong with our model.
Here is our notebook of training LeNet with SGHMC with MNIST dataset.
Can anyone advise/help with why SGHMC is performing? My teammate and I would really appreciate it!