Principal components analysis (dimensionality reduction)

Question

For $x_1, ..., x_n$ with $x_i$: $d$-dimensional vector, we want to project all $x_i$ onto a linear subspace of the original space where the points lie. The trick is to find a line that maximizes the variance of the points, then find a line perpendicular to the line found, and so on. Then, after having $d-1$ lines, project all the points to the coordinate system formed by all lines. My first confusion is how to find a line that maximizes the variance: in the picture, it seems that the first line ( the horizontal one ) can rotate around that white point ( and thus we can find a best position of that line ), but how do we find the white point over which to rotate? Also, at which point the $j$-th line should be perpendicular to $j-1$-th line? Thanks.

Answer 1

PCA rotates the axis around the origin, i.e. $(0,0)$ in your example.

And all the lines/axes are crossing the origin.