Suppose $A=\sum_{i=1}^n X_iX_i'$, is there a relationship between the minimum eigenvalue of $A$ and the minimum eigenvalue for each $X_iX_i'$?

Question

Suppose we have a $l\times l$ matrix $A=\sum_{i=1}^n X_iX_i'$, where each $X_i$ is a $l\times 1$ vector. What's the relationship between the minimum eigenvalue of $A$ and the minimum eigenvalue for each $X_iX_i'$? To me, it seems that $\lambda_{min}(A)\geq\sum_{i=1}^n\lambda_{min}(X_iX_i')$ looks plausible. Because $\lambda_{min}(X_iX_i')=0$ for each $i$, but at the same time we usually have $\lambda_{min}(A)>0$ (as in the case of least squares problem). Also, is there an upper bound for $\lambda_{min}(A)$ constructed using $X_iX_i'$?

Answer 1

$X_iX'_i$ has rank $1$ or $0$, therefore its smallest eigenvalue is $0$ if $\ell>1.$ In this case the other eigenvalue is the number $X'_iX_i=\|X_i\|^2\geq 0.$ If $X_1,\ldots,X_n$ do not generate $\mathbb{R}^{\ell}$ but only a subspace of dimension $r$ the rank of $A$ is $r$ and the smallest eigenvalue of $A$ is $0.$ This is in particular the case when $n<\ell.$ The only interesting case is $r\geq \ell.$ In this case $A$ is positive definite and can be written $UDU'$ where $U$ is an orthogonal matrix and where $D=\mathrm{diag}(\lambda_1,\ldots,\lambda_{\ell})$ with $0<\lambda_1\leq \ldots\leq \lambda_{\ell}.$ Denoting $Y_i=U'X_i$ we have $ \|Y_i\|^2=\|X_i\|^2$ and the smallest eigenvalue of $A$ is $$\lambda_1=\sum_{i=1}^nY_{i1}^2\leq \sum_{i=1}^n\|X_i\|^2.$$