I’m a bit surprised to find I can’t specify a PointMass q over some RVs when using KLqp. Is that correct?
I realize I can do ed.MAP but it would be nice to be able to do variational EM without having to explicitly setup the E and M steps separately.
To define KL(Q || P) requires Q to be absolutely continuous wrt P so you can’t really use a Dirac mass as Q.
The KL isn’t defined sure because of the entropy term blowing up but everything still works. Just view the dirac as a Gaussian with a tiny fixed variance though and this is fine. The entropy doesn’t depend on the mean so you can ignore it, and \int q log p ~= log p(theta), so you recover MAP inference over the variables with dirac q. This approach has been used for some variables in various VB papers.