# How do you find the y-intercept of the least squares regression line for the data set (1,8) (2,7) (3, 5)?

May 11, 2018

$\hat{y} = - \frac{3}{2} x + \frac{29}{3}$

That's a $\beta = - \frac{3}{2}$ and an $\alpha$ aka y-intercept of $\frac{29}{3}$.

#### Explanation:

Let's derive least squares regression because I'm rusty.

Our model for the data is a linear equation with two parameters, $\alpha \mathmr{and} \beta$.

$\hat{y} = \alpha x + \beta$

Our total error is the sum of the squared residuals for each data point.

$E = {\sum}_{i = 1}^{n} {\left({y}_{i} - \hat{{y}_{i}}\right)}^{2} = \sum {\left({y}_{i} - \alpha {x}_{i} - \beta\right)}^{2}$

To control clutter I'll just write $\sum$ for ${\sum}_{i = 1}^{n} .$

We minimize $E$ by setting the partials to zero:

$0 = \frac{\partial E}{\partial \alpha} = \sum - {x}_{i} \left({y}_{i} - \alpha {x}_{i} - \beta\right)$

$\sum {x}_{i} {y}_{i} = \alpha \sum {x}_{i}^{2} + \beta \sum {x}_{i}$

$0 = \frac{\partial E}{\partial \beta} = \sum - \left({y}_{i} - \alpha {x}_{i} - \beta\right)$

$\sum {y}_{i} = \alpha \sum {x}_{i} + n \beta$

That last one comes from ${\sum}_{i = 1}^{n} \beta = \beta .$

We have two equations in two unknowns. I remember $M A = S$ has solutions $A = {M}^{- 1} S$ and for a two by two matrix $M = \left(a , b , \quad \quad c , d\right)$ and $S = {\left(s , t\right)}^{T}$ we get

${M}^{- 1} S = \frac{1}{a d - b c} \left(\mathrm{ds} - b t , - c s + a t\right)$

Back to the problem. Let's declutter even more and write

$\sum {x}_{i} = n \overline{x} , \sum {x}_{i} {y}_{i} = n \overline{x y} , \quad \sum {x}_{i}^{2} = n \overline{{x}^{2}} ,$ etc. We rewrite our system, cancelling the $n$s:

$\overline{x y} = \alpha \overline{{x}^{2}} + \beta \overline{x}$

$\overline{y} = \alpha \overline{x} + \beta$

Applying our solution, we substitute into our solution

$a = \overline{{x}^{2}} , b = c = \overline{x} , d = 1 , s = \overline{x y} , t = \overline{y}$

giving

 alpha = { bar{xy} - bar{x} \ bar{y}}/{ bar{x^2} - bar{x}^2 }

beta = { - bar{x} \ bar{xy} + bar{x^2} bar{y}}/{ bar{x^2} - bar{x}^2 }

I don't know if that's right, but it's giving me flashbacks. Let's try our numbers.
$x \setminus y \setminus \setminus {x}^{2} \setminus \setminus {y}^{2} \setminus \setminus x y$
$1 \setminus 8 \setminus \setminus \setminus 1 \setminus \setminus \setminus \setminus 64 \setminus \setminus \setminus 8$
$2 \setminus 7 \setminus \setminus \setminus 4 \setminus \setminus \setminus \setminus 49 \setminus 14$
$3 \setminus 5 \setminus \setminus \setminus 9 \setminus \setminus \setminus \setminus 25 \setminus 15$
$6 \setminus 20 \setminus 14 \setminus 138 \setminus 37$ TOTALS

$\alpha = \frac{\frac{37}{3} - \left(\frac{6}{3}\right) \left(\frac{20}{3}\right)}{\left(\frac{14}{3}\right) - {\left(\frac{6}{3}\right)}^{2}} = - \frac{3}{2}$

$\beta = \frac{- \left(\frac{6}{3}\right) \left(\frac{37}{3}\right) + \left(\frac{14}{3}\right) \left(\frac{20}{3}\right)}{\left(\frac{14}{3}\right) - {\left(\frac{6}{3}\right)}^{2}} = \frac{29}{3}$

Model:

$\hat{y} = - \frac{3}{2} x + \frac{29}{3}$ Check:

Let's calculate the squared error:

${\left(8 - \left(- \frac{3}{2} \left(1\right) + \left(\frac{29}{3}\right)\right)\right)}^{2} + {\left(7 - \left(- \frac{3}{2} \left(2\right) + \left(\frac{29}{3}\right)\right)\right)}^{2} + {\left(5 - \left(- \frac{3}{2} \left(3\right) + \left(\frac{29}{3}\right)\right)\right)}^{2} = \frac{1}{6}$

The theory says if we change $a$ or $b$ by a little bit (or a lot) we'll always get a bigger error. Let's pop a few into the computer:

${\left(8 - \left(- 1.501 \left(1\right) + \left(\frac{29}{3}\right)\right)\right)}^{2} + {\left(7 - \left(- 1.501 \left(2\right) + \left(\frac{29}{3}\right)\right)\right)}^{2} + {\left(5 - \left(- 1.501 \left(3\right) + \left(\frac{29}{3}\right)\right)\right)}^{2} \approx 0.16668067$

${\left(8 - \left(- \frac{3}{2} \left(1\right) + 10\right)\right)}^{2} + {\left(7 - \left(- \frac{3}{2} \left(2\right) + 10\right)\right)}^{2} + {\left(5 - \left(- \frac{3}{2} \left(3\right) + 10\right)\right)}^{2} = \frac{1}{2}$

Let's call that checked.