# Re-using a saved Bayesian neural network

**URL:** <https://discourse.edwardlib.org/t/re-using-a-saved-bayesian-neural-network/997>\
**Category:** General\
**Created:** [December 10, 2018, 12:45am UTC](https://discourse.edwardlib.org/t/re-using-a-saved-bayesian-neural-network/997 "2018-12-10T00:45:00Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![RossChapman](https://avatars.discourse-cdn.com/v4/letter/r/c67d28/32.png) [@RossChapman](https://discourse.edwardlib.org/u/RossChapman)\
**Post date:** [December 10, 2018, 12:45am UTC](https://discourse.edwardlib.org/t/re-using-a-saved-bayesian-neural-network/997/1 "2018-12-10T00:45:00Z")

</div>

Hi all,

I have a question about how to re-use Bayesian neural networks built with Edward.

I have built and trained a simple Bayesian neural network.

I imagine that I might want to re-use this model for analyses some time in the future. For example, I might want to save the model that I have built today, but re-use it in several months time to make predictions from a new data set that has become available.

I have saved the parameter set as follows:

saver = tf.train.Saver()  
saver.save(sess, “path/fileForBayesianNeuralNetwork”)

I have re-loaded the parameter set in a new python script using the following code:

sess =tf.Session()

fname=“path/fileForBayesianNeuralNetwork”  
loader = tf.train.import\_meta\_graph(fname+’.meta’)  
loader.restore(sess,fname)  
graph = tf.get\_default\_graph()  
x\_data = graph.get\_tensor\_by\_name(“x\_data:0”)  
print(‘X:’,x\_data)

q\_W1\_sample = graph.get\_tensor\_by\_name(“q\_W1/sample/Reshape:0”)

q\_W1 = Normal(loc=graph.get\_tensor\_by\_name(“q\_W1/loc:0”),  
scale=graph.get\_tensor\_by\_name(“q\_W1/scale:0”),  
value=q\_W1\_sample)  
q\_B1\_sample = graph.get\_tensor\_by\_name(“q\_B1/sample/Reshape:0”)

q\_B1 = Normal(loc=graph.get\_tensor\_by\_name(“q\_B1/loc:0”),  
scale=graph.get\_tensor\_by\_name(“q\_B1/scale:0”),  
value=q\_B1\_sample)  
q\_W2\_sample = graph.get\_tensor\_by\_name(“q\_W2/sample/Reshape:0”)

q\_W2 = Normal(loc=graph.get\_tensor\_by\_name(“q\_W2/loc:0”),  
scale=graph.get\_tensor\_by\_name(“q\_W2/scale:0”),  
value=q\_W2\_sample)  
q\_B2\_sample = graph.get\_tensor\_by\_name(“q\_B2/sample/Reshape:0”)

q\_B2 = Normal(loc=graph.get\_tensor\_by\_name(“q\_B2/loc:0”),  
scale=graph.get\_tensor\_by\_name(“q\_B2/scale:0”),  
value=q\_B2\_sample)

I am not sure how I proceed from here to use the model for new predictions. Do I need to re-run the inference process with the re-loaded parameters, and will that involve creating a new and distinct model, or can I somehow re-used the saved model to make new predictions?

Many thanks for your assistance.

---

<div class="post-metadata">

**Author:** ![freedom521jin](https://avatars.discourse-cdn.com/v4/letter/f/d26b3c/32.png) [@freedom521jin](https://discourse.edwardlib.org/u/freedom521jin)\
**Post date:** [May 9, 2019, 8:41am UTC](https://discourse.edwardlib.org/t/re-using-a-saved-bayesian-neural-network/997/2 "2019-05-09T08:41:04Z")

</div>

I have a question: I cannot save the model and cannot evaluate test\_dataset to get ‘accuracy’.  
Would you send your good ways to [stonejack@foxmail.com](mailto:stonejack@foxmail.com)(my email)?  
Appreiciate it!

---

<div class="post-metadata">

**Author:** ![RossChapman](https://avatars.discourse-cdn.com/v4/letter/r/c67d28/32.png) [@RossChapman](https://discourse.edwardlib.org/u/RossChapman)\
**Post date:** [August 7, 2019, 12:58am UTC](https://discourse.edwardlib.org/t/re-using-a-saved-bayesian-neural-network/997/3 "2019-08-07T00:58:05Z")

</div>

Thanks for your comment [freedom521jin](https://discourse.edwardlib.org/u/freedom521jin).

I will describe my solution and would welcome any comments that might correct or improve my code.

I define some global variables for a Bayesian Neural Network with 14 hidden layers:

#create session  
sess = ed.get\_session()

#define the number of samples for training and prediction  
nSamp = 10000

#define the nuber of hidden layers  
hL = 14

I then build and train the BNN using data:

tf.app.flags.DEFINE\_string(‘f’, ‘’, ‘kernel’)

D=inputVars.shape[1] #11 #this is the No. of features  
N=inputVars.shape[0]

tf.flags.DEFINE\_integer(“N”, default=N, help=“Number of data points.”)  
tf.flags.DEFINE\_integer(“D”, default=D, help=“Number of features.”)

FLAGS = tf.flags.FLAGS

def neural\_network(x):  
h = tf.tanh(tf.matmul(x, W0) + B0)  
h = tf.matmul(h, W1) + B1  
return tf.reshape(h, [-1])

w0=hL  
w1=1  
b0=w0  
b1=w1

W0 = Normal(loc=tf.zeros([D, w0]), scale=tf.ones([D, w0]))  
W1 = Normal(loc=tf.zeros([w0, w1]), scale=tf.ones([w0, w1]))  
B0 = Normal(loc=tf.zeros(b0), scale=tf.ones(b0))  
B1 = Normal(loc=tf.zeros(b1), scale=tf.ones(b1))

x\_data = tf.placeholder(shape=[None, D], dtype=tf.float32, name=“x\_data”)  
y = Normal(loc=neural\_network(x\_data), scale=0.1 \* tf.ones(FLAGS.N), name=“y”)

output = Normal(loc=neural\_network(x\_data), scale=0.1 \* tf.ones(1)) #w0))

q\_W1 = Normal(loc=tf.Variable(tf.zeros([w0, w1])), name=(“q\_W1”),  
scale=tf.nn.softplus(tf.Variable(tf.zeros([w0, w1]))))  
q\_b1 = Normal(loc=tf.Variable(tf.zeros(b1)), name=(“q\_b1”),  
scale=tf.nn.softplus(tf.Variable(tf.zeros(b1))))  
q\_W0 = Normal(loc=tf.Variable(tf.zeros([D, w0])), name=(“q\_W0”),  
scale=tf.nn.softplus(tf.Variable(tf.zeros([D, w0]))))  
q\_b0 = Normal(loc=tf.Variable(tf.zeros(b0)), name=(“q\_b0”),  
scale=tf.nn.softplus(tf.Variable(tf.zeros(b0))))

inference = ed.KLqp({W0: q\_W0, B0: q\_b0,  
W1: q\_W1, B1: q\_b1}, data={x\_data: inputVars, y: yld})  
inference.run(n\_iter=nSamp)#,logdir=logDir)

I then print the weights and biases to be able to check that any later reloading has occurred correctly.

vars = tf.trainable\_variables()  
print(vars)  
for vr in vars:  
print(vr.name)  
print(sess.run([v for v in tf.trainable\_variables() if v.name == vr.name]))

I then saved the BNN using the saver method:

saver = tf.train.Saver()  
save\_path = saver.save(sess, ‘pathway to …/fileName.ckpt’)

To reload in a new file, I begin by creating thj global variables:

#create session  
sess = ed.get\_session()

#define the number of hidden layers  
hL = 14

#define the number of samples for prediction  
nSamp=10000

I can then define a new BNN and load the saved parameters into it:

w1x = tf.placeholder(“float”, name=“w1x”)  
w2x = tf.placeholder(“float”, name=“w2x”)  
b1x = tf.placeholder(“float”, name=“b1x”)  
b2x = tf.placeholder(“float”, name=“b2x”)

fname=“pathway/filename.ckpt”  
loader = tf.train.import\_meta\_graph(fname+’.meta’)  
loader.restore(sess,fname)  
graph = tf.get\_default\_graph()

#create empty BNN

w0=hL  
w1=1  
b0=w0  
b1=w1

D=inputVars.shape[1]

#a place holder for input data …  
x\_data = tf.placeholder(shape=[None, D], dtype=tf.float32, name=“x\_data”)  
#x\_data\_2 = tf.placeholder(shape=[None, D], dtype=tf.float32, name=“x\_data\_2”)

W0 = Normal(loc=tf.zeros([D, w0]), scale=tf.ones([D, w0]))  
W1 = Normal(loc=tf.zeros([w0, w1]), scale=tf.ones([w0, w1]))  
B0 = Normal(loc=tf.zeros(b0), scale=tf.ones(b0))  
B1 = Normal(loc=tf.zeros(b1), scale=tf.ones(b1))

def neural\_network(x):  
h = tf.tanh(tf.matmul(x, W0) + B0)  
h = tf.matmul(h, W1) + B1  
return tf.reshape(h, [-1])

Next I populate and prepare the BNN:

output = Normal(loc=neural\_network(x\_data), scale=0.1 \* tf.ones(1))

#populate with reloaded data

q\_W1=(Normal(loc=sess.run(‘q\_W1/loc:0’),  
scale=sess.run(‘q\_W1/scale:0’)))  
q\_b1 = Normal(loc=graph.get\_tensor\_by\_name(“q\_b1/loc:0”),  
scale=graph.get\_tensor\_by\_name(“q\_b1/scale:0”))  
q\_W0=(Normal(loc=sess.run(‘q\_W0/loc:0’),  
scale=sess.run(‘q\_W0/scale:0’)))  
q\_b0=(Normal(loc=sess.run(‘q\_b0/loc:0’),  
scale=sess.run(‘q\_b0/scale:0’)))

#create a copy for use  
y\_post=ed.copy(output, {W0: q\_W0, B0: q\_b0, W1: q\_W1, B1: q\_b1 })

I can re-use the trainable\_variables method to check that the weights and biases are correct:

vars = tf.trainable\_variables()  
print(vars)  
for vr in vars:  
print(vr.name)  
print(sess.run([v for v in tf.trainable\_variables() if v.name == vr.name]))

I hope that this description is useful and would welcome any improvements.