Hi all. Im new to Edward and I am trying to build up a LSTM network to do some electric load forecasting work. I am using SGHMC (Chen 2014) and right now I have met some very weird issues .
Firstly no matter how I change the step_size and friction parameters, the Acceptance Rate seems to be always larger or equal to zero.
Second, I would like to use mini-batch training technique and it seems not working. I tried to imitate the way in the batch training tutorial and here is what I did.
Does SGHMC support batch training? From my test I found that n_iter is fixed to the first dimension of empirical variable and the scale setting scale={y: M/batch} must equal to 1 in order to get some results(though still rubbish), which seems to imply that I can only use 1 batch of data, i.e. the whole data set as one big batch.
Could anyone please give some comments? I am a beginner to Edward so appreciate anything.
Thanks