How to prove $ \|x_k-A^{-1}b\|_A={\rm min}_{x\in \chi}\|x-A^{-1}b\|_A$?

Question

$A$ is an $n\times n$ positive definite matrix, $\chi$ is a $k$-dimensional subspace of $\mathbb{R}^{n\times n}$. Then, for $x_k\in \chi$, \begin{align*} \|x_k-A^{-1}b\|_A={\rm min}_{x\in \chi}\|x-A^{-1}b\|_A \end{align*} holds if and only if $r_k=b-Ax_k$ is perpendicular to $\chi$. Here, $b\in \mathbb{R}^n$ is any vector, $\|x\|_A=\sqrt{x^TAx}$.

Answer 1

You are missing a subscript $A$ in the second norm.

Let $$f(x)=\|x-A^{-1}b\|_A^2 = (x-A^{-1}b)^\top A (x-A^{-1}b) = x^\top A x-2b^\top x + b^\top A^{-1} b.$$ If $x_k$ is the minimizer over $\chi$ then $$(\nabla f(x_k))^\top (x-x_k) \ge 0,\quad \forall x \in \chi$$ else there exists some sufficiently small $c>0$ such that taking a step $c(x-x_k)$ from $x_k$ decreases the value of $f$, contradicting the fact that $x_k$ minimizes $f$ on $\chi$.

We have $\nabla f(x_k) = 2(Ax_k - b)= - 2r_k$, so the above condition implies $$r_k^\top (x-x_k) \le 0,\quad \forall x \in \chi.$$

We just need to conclude that this inequality is actually equality for all $x \in \chi$. Indeed if for some $x$ we have $r_k^\top (x-x_k)<0$, then since $\chi$ is a subspace, $x' := -x+2x_k \in \chi$ as well, but $r_k^\top (x'-x_k) = -r_k^\top (x-x_k) > 0$, a contradiction.