# Softmax logit start to take on value nan after first iteratioin

**URL:** <https://discourse.edwardlib.org/t/softmax-logit-start-to-take-on-value-nan-after-first-iteratioin/757>\
**Category:** General\
**Created:** [April 22, 2018, 6:36pm UTC](https://discourse.edwardlib.org/t/softmax-logit-start-to-take-on-value-nan-after-first-iteratioin/757 "2018-04-22T18:36:18Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![bhomass](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/bhomass/32/264_2.png) [@bhomass](https://discourse.edwardlib.org/u/bhomass)\
**Post date:** [April 22, 2018, 6:36pm UTC](https://discourse.edwardlib.org/t/softmax-logit-start-to-take-on-value-nan-after-first-iteratioin/757/1 "2018-04-22T18:36:18Z")

</div>

I have a model which worked well for two previous datasets. It involved feeding a multivariate logit into a softmax. When I started the training, the first logProb came out as very small numbers, but did not go out of range.

returned combined\_prob[[3.7668025322495853e-81 2.2536902189765332e-83][5.1639261639860367e-81 3.0895938365906408e-83][5.6208860329797729e-81 3.3629944139766141e-83][6.1139865878104518e-81 3.6580180813654438e-83][6.4315076637965463e-81]…]  
returning log\_final\_prob = [[-189.70496 -194.82378][-189.38948 -194.50832][-189.30469 -194.42352][-189.2206 -194.33943][-189.16997]…]

Then when the second iterations started, the logit became all nan  
x\_logits=[0][[[[nan nan][nan nan]][[nan nan][nan nan]][

When I looked at the results from the prev data sets, the logProb is also small but bigger than this  
returned combined\_prob[[1.3671217001712804e-64 1.0386060698097945e-65][5.1544762073173032e-65 3.9154561505711869e-66][7.343496514211793e-67 5.5782213630433148e-68][1.54969592578499e-67 1.1771704259268e-68][1.3523815348803295e-64]…]  
returning log\_final\_prob = [[-151.43477 -154.01218][-152.41019 -154.98772][-156.66141 -159.23894][-158.21716 -160.79469][-151.4456]…]

Is the difference in logProb the cause of the problem? that this dataset finally pushed the computation out of range? is there a way to adjust the model to stay within range?

I tried inference.run(optimizer=tf.train.AdamOptimizer(learning\_rate=4e-4)), made no difference