Recently we studied PCA.
As input, we have vectors:
$v_1,...,v_n \in R^d$ and $k$, where $k$ is the desirable dimension.
Then, we introduce matrix $A:$ $$A = \frac{1}{n} \sum_{i=1}^{n} v_iv_i^{T},$$ where $A$ is $d \times d$.
I have very naïve question: why is $A$ actually a matrix? Is $v_iv_i^{T}$ is a scalar, or I am wrong?
$$v_i=\left(\begin{array}{c}v_{i1} \\ \vdots \\ v_{id}\end{array} \right)$$ so $v_i^T v_i =\sum_{j=1}^dv_{ij}^2$ and $\sum_{i=1}^n v_i^T v_i =\sum_{i=1}^n\sum_{j=1}^dv_{ij}^2$ are scalars but $$v_i v_i ^T=\left(\begin{array}{ccc}v_{i1}^2 & \cdots & v_{i1}v_{id}\\ \vdots & \ddots & \vdots \\ v_{id}v_{i1} & \cdots & v_{id}^2 \end{array} \right)$$ and $$\sum_{i=1}^n v_i v_i ^T=\left(\begin{array}{ccc} \sum_{i=1}^nv_{i1}^2 & \cdots & \sum_{i=1}^nv_{i1}v_{id}\\ \vdots & \ddots & \vdots \\ \sum_{i=1}^n v_{id}v_{i1} & \cdots & \sum_{i=1}^n v_{id}^2 \end{array} \right)$$