About PCA I read that we were looking for the direction $w_1 \in \mathbb{R^p}$ with $||w||=1$ such that the variance of our data projected onto this direction is maximal. The word "projection" makes me confused
For example imagine we have a matrix $M_{4,3}(\mathbb{R})$ like $X = \begin{bmatrix} a_{1,1} & a_{1,2}\\ a_{2,1} & a_{2,2}\\ a_{3,1} & a_{3,2}\\ a_{4,1} & a_{4,2} \end{bmatrix}$ (variables in columns)
Once the PCA is done, I take the eigenvector with the largest eigenvalue, say $w_1 \in \mathbb{R^{2}}$ and if I want to project for example $ v=[a_{1,1}, a_{1,2}]$ on the base $w_1$ I have to do the dot product between $v$ and $w_1$ right?
But for me intuitively if I want to project on the vector $w_1$ I have to do $(v \cdot w_1)*w_1$ (multiply the result of the dot product by $w_1$. I know it's false, but it makes me confused about the projections.
If take a simpler example the canonical $u = \begin{bmatrix} 1 \\ 0 \end{bmatrix}$ and for example $v = \begin{bmatrix} 2, 3\\ \end{bmatrix}$. What is the exact and intuitive difference between: $v\cdot u$ and $(v\cdot u)*u$ (except that one is length and the other the length multiply by the vector)
Thanks for your help