# Hierarchical Dirichlet Model

**URL:** <https://discourse.edwardlib.org/t/hierarchical-dirichlet-model/849>\
**Category:** General\
**Created:** [July 11, 2018, 9:34pm UTC](https://discourse.edwardlib.org/t/hierarchical-dirichlet-model/849 "2018-07-11T21:34:57Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![pindapuj](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/pindapuj/32/338_2.png) [@pindapuj](https://discourse.edwardlib.org/u/pindapuj)\
**Post date:** [July 11, 2018, 9:34pm UTC](https://discourse.edwardlib.org/t/hierarchical-dirichlet-model/849/1 "2018-07-11T21:34:57Z")

</div>

Hello Edwardians and @dustin !

I am trying to implement a simplified Hierarchical Dirichlet Process Model from (Teh et. al 2006, [https://www.stat.berkeley.edu/~aldous/206-Exch/Papers/hierarchical\_dirichlet.pdf](https://www.stat.berkeley.edu/~aldous/206-Exch/Papers/hierarchical_dirichlet.pdf)). I’m generally a PyMC3 user but was excited when I saw the DirichletProcess as a random variable. The model described in the paper is below:

![42%20PM](https://canada1.discourse-cdn.com/flex035/uploads/edwardlib/original/1X/1b2bfc9803f25b8509e8503e0e0729aa9ddf27ea.jpg)

My naive implementation abusing the DirichletProcess RV is below: `H = Dirichlet([0.5])  
gamma = Gamma(5.0,1.0)

alpha\_0 = Gamma(5.0,1.0)  
G\_0dp = DirichletProcess(gamma, H)

alpha\_1 = Gamma(.1,.1)  
G\_1dp = DirichletProcess(alpha\_0,G\_0dp.base)

G\_jdp = DirichletProcess(alpha\_1,G\_1dp.base,sample\_shape=1000)  
T = 1000

qH = Dirichlet(tf.Variable([0.5]))  
qGamma = Gamma(tf.Variable(5.0),tf.Variable(1.0))  
qalpha\_0 = Gamma(tf.Variable(5.0),tf.Variable(1.0))  
qG\_0dp = DirichletProcess(qGamma,qH)  
qalpha\_1 = Gamma(tf.Variable(.1),tf.Variable(.1))  
qG\_1dp = DirichletProcess(qalpha\_0,qG\_0dp.base)  
#qG\_jdp = DirichletProcess(qalpha\_1,qG\_1dp.base,sample\_shape=1000)

#xH = Empirical(tf.Variable())

latent\_vars = {H: qH, gamma: qGamma,alpha\_0: qalpha\_0,G\_0dp: qG\_0dp, alpha\_1: qalpha\_1,G\_1dp: qG\_1dp}  
data = {G\_jdp: np.random.normal(6.0,1.5,[1000,1])}

inference = ed.KLqp(latent\_vars,data=data)  
inference.run(n\_samples=5, n\_iter=2500)`

where I get an error that ‘prob is not implemented.’ I know people were interested in the topic before (#232), but that issue was never resolved.

I’d really appreciate any pointers in designing such a solution! I also think other Edwardians would appreciate an HDP as well.

Thank you!  
pindapuj

---

<div class="post-metadata">

**Author:** ![pindapuj](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/pindapuj/32/338_2.png) [@pindapuj](https://discourse.edwardlib.org/u/pindapuj)\
**Post date:** [July 11, 2018, 9:36pm UTC](https://discourse.edwardlib.org/t/hierarchical-dirichlet-model/849/2 "2018-07-11T21:36:30Z")

</div>

Full Error

* * *

NotImplementedError Traceback (most recent call last)  
/Users/Puja/anaconda/lib/python3.6/site-packages/tensorflow/python/ops/distributions/distribution.py in \_call\_log\_prob(self, value, name, \*\*kwargs)  
693 try:  
–\> 694 return math\_ops.log(self.\_prob(value, \*\*kwargs))  
695 except NotImplementedError:

/Users/Puja/anaconda/lib/python3.6/site-packages/tensorflow/python/ops/distributions/distribution.py in \_prob(self, value)  
711 def \_prob(self, value):  
–\> 712 raise NotImplementedError(“prob is not implemented”)  
713

NotImplementedError: prob is not implemented

During handling of the above exception, another exception occurred:

NotImplementedError Traceback (most recent call last)  
 in ()  
16  
17 inference = ed.KLqp(latent\_vars,data=data)  
—\> 18 inference.run(n\_samples=5, n\_iter=2500)

/Users/Puja/anaconda/lib/python3.6/site-packages/edward/inferences/inference.py in run(self, variables, use\_coordinator, \*args, \*\*kwargs)  
123 Passed into `initialize`.  
124 “”"  
–\> 125 self.initialize(\*args, \*\*kwargs)  
126  
127 if variables is None:

/Users/Puja/anaconda/lib/python3.6/site-packages/edward/inferences/klqp.py in initialize(self, n\_samples, kl\_scaling, \*args, \*\*kwargs)  
108 self.n\_samples = n\_samples  
109 self.kl\_scaling = kl\_scaling  
–\> 110 return super(KLqp, self).initialize(\*args, \*\*kwargs)  
111  
112 def build\_loss\_and\_gradients(self, var\_list):

/Users/Puja/anaconda/lib/python3.6/site-packages/edward/inferences/variational\_inference.py in initialize(self, optimizer, var\_list, use\_prettytensor, global\_step, \*args, \*\*kwargs)  
66 var\_list = list(var\_list)  
67  
—\> 68 self.loss, grads\_and\_vars = self.build\_loss\_and\_gradients(var\_list)  
69  
70 if self.logging:

/Users/Puja/anaconda/lib/python3.6/site-packages/edward/inferences/klqp.py in build\_loss\_and\_gradients(self, var\_list)  
158 # return build\_score\_entropy\_loss\_and\_gradients(self, var\_list)  
159 # else:  
–\> 160 return build\_score\_rb\_loss\_and\_gradients(self, var\_list)  
161  
162

/Users/Puja/anaconda/lib/python3.6/site-packages/edward/inferences/klqp.py in build\_score\_rb\_loss\_and\_gradients(inference, var\_list)  
1065 q\_log\_probs[s][qz] = tf.reduce\_sum(  
1066 inference.scale.get(z, 1.0) \*  
-\> 1067 qz\_copy.log\_prob(tf.stop\_gradient(dict\_swap[z])))  
1068  
1069 for z in six.iterkeys(inference.latent\_vars):

/Users/Puja/anaconda/lib/python3.6/site-packages/tensorflow/python/ops/distributions/distribution.py in log\_prob(self, value, name)  
707 values of type `self.dtype`.  
708 “”"  
–\> 709 return self.\_call\_log\_prob(value, name)  
710  
711 def \_prob(self, value):

/Users/Puja/anaconda/lib/python3.6/site-packages/tensorflow/python/ops/distributions/distribution.py in \_call\_log\_prob(self, value, name, \*\*kwargs)  
694 return math\_ops.log(self.\_prob(value, \*\*kwargs))  
695 except NotImplementedError:  
–\> 696 raise original\_exception  
697  
698 def log\_prob(self, value, name=“log\_prob”):

/Users/Puja/anaconda/lib/python3.6/site-packages/tensorflow/python/ops/distributions/distribution.py in \_call\_log\_prob(self, value, name, \*\*kwargs)  
689 value = ops.convert\_to\_tensor(value, name=“value”)  
690 try:  
–\> 691 return self.\_log\_prob(value, \*\*kwargs)  
692 except NotImplementedError as original\_exception:  
693 try:

/Users/Puja/anaconda/lib/python3.6/site-packages/tensorflow/python/ops/distributions/distribution.py in \_log\_prob(self, value)  
683  
684 def \_log\_prob(self, value):  
–\> 685 raise NotImplementedError(“log\_prob is not implemented”)  
686  
687 def \_call\_log\_prob(self, value, name, \*\*kwargs):

NotImplementedError: log\_prob is not implemented