What is the relationship between $w^TAW$ and $w^TA^{+}w$?

76 Views Asked by At

I am trying to understand what is the relationship between the scalar products $$w^TAw \ \ \text{and} \ \ w^TA^{+}w,$$ where $A$ is a symmetric, non-negative definite matrix and $A^{+}$ is its pseudoinverse. Below, I give some relationship between the two using the Riesz-representation theorem and I wonder whether this is correct, but I would also be interested if there is something more to say on the relationship between the two.

On any Hilbert-space $(H,\langle \cdot, \cdot \rangle_H)$ we have by the Riesz-representation theorem that a linear functional $f \in H^*$ can be expressed by $$ f(v) = \langle w, v \rangle_H$$ for some $w \in H$. Furthermore, it holds that the dual norm of $f$, given by $$ \vert \vert f \vert \vert_{H^*} = \sup \left\{ \vert f(v) \vert \bigg \vert v \in H, \vert \vert v \vert \vert_H = 1 \right \}$$

is given by $$ \vert \vert f \vert \vert_{H^*} = \vert \vert w \vert \vert_H.$$

Now, consider the case where the space in question is $H = \mathbb{R}^n$ and $A \in \mathbb{R}^{n \times n}$ is a symmetric, positive definite matrix. Then we can consider the space $(H, \langle \cdot, \cdot \rangle_A)$ where $$\langle v, w \rangle_A = v^T A w.$$

We then can calculate the dual norm explicitly: By using Riesz on $\mathbb{R}^n$ with the standard euclidean scalar product, we can identify ${\mathbb{R}^n}^*$ with $\mathbb{R}^n$, where $w \in \mathbb{R}^n$ induces the mapping $$ v \mapsto w^Tv.$$ Then we get that $$ \vert \vert w \vert \vert_{H^*} = \vert \vert w' \vert \vert_H$$ where $w'$ is such that $$ v \mapsto w^Tv = w'^T A v.$$

We see that $w' = A^{-1}w$ and thus we get $$ \vert \vert w \vert \vert_{H^*} = \vert \vert w \vert \vert_{A^{-1}}.$$ Thus we also have

$$w^TA^{-1}w = \sup \left\{ (w^T v)^2 \bigg \vert v \in \mathbb{R}^n, v^T A v = 1 \right\}.$$

Now assume that $A$ is a symmetric, non-negative definite matrix.

Then we have that $\mathbb{R}^n = \operatorname{im}(A) \oplus \operatorname{ker}(A)$. In particular, we can define a scalar product $\langle \cdot, \cdot \rangle_A$ on $\operatorname{im}(A)$ as above. Also, we can define the pseudo-inverse $A^+$, which is defined as

$$A^+w = (A\vert_{\operatorname{im}(A)})^{-1} P_{\operatorname{im}(A)}w$$ with $P_{\operatorname{im}(A)}w$ being the orthogonal projector to $\operatorname{im}(A)$.

Then for $w = u + e$, $u \in \operatorname{im}(A), e \in \operatorname{ker}(A)$ we have

$$w^TA^{+}w = u^T (A\vert_{\operatorname{im}(A)})^{-1} u = \sup \left\{ (u^T v)^2 \bigg \vert v \in \operatorname{im}(A), v^T A v = 1 \right\}$$