In PCA, why for every $x \in \mathbb{R}^n$, $x=\sum_{k=1}^n u^T_k x \space u_k$?

Question

In PCA, why for every $x \in \mathbb{R}^n$, $x=\sum_{k=1}^n (u^T_k x) \space u_k$?

Where $\{u_1,...,u_n\}$ is orthonormal basis and $||u||^2=u^T_i u_i=1 \forall i$.

Is this some standard vector projection?

Answer 1

If $(u_1,\dots,u_n)$ is an orthonormal basis of any vector space $V$ equipped with an inner product $\langle \dot{}, \dot{} \rangle$ then

$$\forall x \in V,\; x =\sum_{k=1}^n \langle u_k, x\rangle u_k.$$

Indeed, if $a_1,\dots,a_n$ are the coordinates of $x$ in this basis, then $\displaystyle x=\sum_{j=1}^n a_j u_j$ and $\forall k\in \{1,\dots,n\},\;\langle x,u_k\rangle = \sum_{j=1}^n a_j \langle u_j,u_k\rangle = a_k$ since $\langle u_j,u_k\rangle = 1$ if $i=j$ and $0$ otherwise.

To conclude, note that on $\Bbb R^n$, the map defined by: $$\forall x\in\Bbb R^n,\;\forall y\in\Bbb R^n,\;\langle x,y\rangle := x^Ty$$ Is an inner product on $\Bbb R^n$.