Relating regression to projection?

Question

I recently learned that one can think of regression as a projection of a vector in a high dimension space onto the other vector. I tried implementing this and got it to work:

C=(dot(A,B)/norm(B)^2)*B; %projecting A onto B (A is the independent variable)
coef=regress(A,b);
D=B*coef; trying to obtain the same result as C by doing regression

So I found C and D to be equivalent. I have a two part question.

1) How do I do the equivalent of a multiple regression by projection? Am I supposed to project onto a plane? How does that work if I have 2 or 3 dependent variables?

2) Using the projection method, how would you calculate the residuals? Is there an advantage to calculating the residuals one way or the other, especially if I project onto a plane instead of another vector vs doing a regular multiple regression?

I can't find online resources about this either, so I'd appreciate any help.

Answer 1

I suggest watching Ted Shrifin's lectures from Math 3500 at University of Georgia - Atlanta:

Inconsistency and Normal Equations

Then

Projections and Max/Min