Matrix inequality and inner product

Question

When I read a note, I came across the following derivation which I cannot understand:

Let $\hat{v}$ is the eigenvector of $\hat{\Sigma}$ and $v$ is the eigenvector of $\Sigma$, then

$$\hat{v}'(\hat{\Sigma}-\Sigma)\hat{v}-v'(\hat{\Sigma}-\Sigma)v=\langle\hat{\Sigma}-\Sigma,\hat{v}\hat{v}'-vv'\rangle\le||\hat{\Sigma}-\Sigma||_{op}||\hat{v}\hat{v}'-vv'||_1$$

where $||\cdot||_{op}$ is the operator norm of a matrix and $||\cdot||_{1}$ is 1-norm, i.e. the largest absolute column sum. I cannot understand either of the equality(what is a inner product of a matrix) or the inequality (maybe a Holder?).

Answer 1

The inner product and the estimate: $$\langle A,B\rangle= \mbox{tr} \; A^T B= \sum_{ij} A_{ij}B_{ij} \leq \sum_{j} \sup_m |A_{mj}|\ \sum_i |B_{ij}|$$ $$\leq \left( \sum_{j} \sup_m |A_{mj}|\right)\ \left(\sup_k \sum_i |B_{ik}|\right)=\|A^T\|_{\rm op} \|B\|_1$$

Answer 2

$$\begin{array}{rl} \hat{\mathrm v}^T (\hat{\Sigma} - \Sigma) \hat{\mathrm v} - \mathrm v^T (\hat{\Sigma} - \Sigma) \mathrm v &= \mbox{tr} (\hat{\mathrm v}^T (\hat{\Sigma} - \Sigma) \hat{\mathrm v}) - \mbox{tr} (\mathrm v^T (\hat{\Sigma} - \Sigma) \mathrm v)\\ &= \mbox{tr} ( (\hat{\Sigma} - \Sigma) \hat{\mathrm v} \hat{\mathrm v}^T ) - \mbox{tr} ( (\hat{\Sigma} - \Sigma) \mathrm v \mathrm v^T )\\ &= \mbox{tr} ( (\hat{\Sigma} - \Sigma) (\hat{\mathrm v} \hat{\mathrm v}^T - \mathrm v \mathrm v^T) )\\ &= \langle \hat{\Sigma} - \Sigma, \hat{\mathrm v} \hat{\mathrm v}^T - \mathrm v \mathrm v^T \rangle\end{array}$$

Now use the inequality in Rugh's answer.