Here are the description about how PCA (Principal component analysis) is calculated. At the beginning, it is mentioned projection onto a one dimensional space. And it is using dot product method. My confusion is, what is the exact geometrical meaning of dot product of two vector is? And how it translate into projection (does it only works if B is a unit vector)? And also wondering if dot product could be used for projection not only to a one dimensional space, but to a 2- or higher dimensional space?
BTW, this is my understanding how dot product of two vector is calculated.
$A \circ B = $ ||A|| * ||B|| * cos$(\theta)$
The projection is a scaled projection. This is evident in your equation $a\cdot b=|a||b|\cos(\phi)$. It is equally true to say $a$ is projected onto $b$ or $b$ is projected onto $a$, and the equation shows that the magnitude of $a$ is scaled by $b$ after it is projected, or conversely the magnitude of $b$ is scaled by $a$ after it is projected. If you are projecting $a$ onto $b$ and $b$ is a unit vector, then it is an unscaled projection and you can think of the result being the length of the shadow of $a$ cast in the direction of $b$; otherwise, both magnitudes come into play. For two and three dimensions you use the equation by calculating the length of each vector (e.g., $|a|$) and the angle between. Computing angles between vectors in 4+ dimensions is hard to imagine; however, the dot product can also be calculated as $a\cdot b=\sum a_{i}b_{i}$ where the subscript indicates the coordinates of the vectors. It can be shown that the two approaches to computing the dot product are equivalent. If you are wondering where the dot product came from and why it works, the answer has to do with quaternion theory developed by Hamilton for multiplying complex numbers. Maxwell later split up the quaternion product into the dot product (i.e., the real part of the product) and the cross product (i.e., the complex part of the product).