# Implementing Linear regression with Automatic Relevance Determination

**URL:** https://discourse.edwardlib.org/t/implementing-linear-regression-with-automatic-relevance-determination/827
**Category:** General
**Created:** [June 26, 2018, 5:53am UTC](https://discourse.edwardlib.org/t/implementing-linear-regression-with-automatic-relevance-determination/827 "2018-06-26T05:53:30Z")
**Posts on this page:** 1
**Page:** 1

<div class="post-metadata">

### Author: ![nadheesh](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.edwardlib.org/nadheesh/32/323_2.png) [@nadheesh](https://discourse.edwardlib.org/u/nadheesh)
#### Post date: [June 26, 2018, 5:53am UTC](https://discourse.edwardlib.org/t/implementing-linear-regression-with-automatic-relevance-determination/827/1 "2018-06-26T05:53:30Z")

</div>

I’m new to Edward. I want implement Linear regression with ARD. This is my current implementation

```
# initiate placeholders
X = tf.placeholder(tf.float32, [None, d])

# initiate the noise
sigma = ed.models.TransformedDistribution(
    distribution=ed.models.Normal(loc=0.0, scale=0.25),
    bijector=bijector.Exp())

# initiating the hyperprior
alpha = ed.models.TransformedDistribution(
    distribution=ed.models.Normal(loc=0.0, scale=1.0),
    bijector=bijector.Exp())

# initiating the priors
w = Normal(loc=tf.zeros(d), scale=tf.ones(d) * alpha)
b = Normal(loc=tf.zeros(1), scale=tf.ones(1))

# initiate the likelihood
y = Normal(loc=ed.dot(X, w) + b, scale=sigma * tf.ones(1))

# initiate the posteriors
qw = Normal(loc=tf.get_variable("qw/loc", [d]),
                 scale=tf.nn.softplus(tf.get_variable("qw/scale", [d])))

qb = Normal(loc=tf.get_variable("qb/loc", [1]),
                 scale=tf.nn.softplus(tf.get_variable("qb/scale", [1])))

qsigma = ed.models.TransformedDistribution(
    distribution=ed.models.Normal(loc=0.0, scale=0.25),
    bijector=bijector.Exp())

qalpha = ed.models.TransformedDistribution(
    distribution=ed.models.Normal(loc=0.0, scale=0.25),
    bijector=bijector.Exp())

# inference
inference = ed.KLqp({w: qw, b: qb, sigma: qsigma, self.alpha : self.qalpha },
                                 data={X: train_X, y: train_y})
inference.run(n_iter=500)

```

However this implementation have a very high error than the linear regression without ARD. It seems that I’m doing something wrong.

I implemented this model by looking at various sources [[1](https://github.com/blei-lab/edward/issues/498)], [[2](https://discourse.edwardlib.org/t/unable-to-inference-the-variance-in-a-simple-bayesian-linear-regression/573)].

- I have seen that TransformedDistribution is used to define the noise of the likelihood and the hyperpriors. What is the purpose of transformed distribution?

- Can’t we define those without transformed distribution (similar to w and b)?

- What am I doing wrong here? Can someone please help me to fix this model?