Let
- $n\in\mathbb N$
- $A\in\mathbb C^{n\times n}$ be invertible
- $b\in\mathbb C^n$
- $x_0\in\mathbb C^n$
- $r_0:=Ax_0-b$
Moreover, let $$\mathcal K_i:=\operatorname{span}\left\{A^0r_0,\ldots,A^{i-1}r_0\right\}\;\;\;\text{for all }i\in\mathbb N\;.$$
Let $i\in\mathbb N$ and $$y_i:=\underset{y\in\mathcal K_i}{\operatorname{arg min}}\left\|Ay-r_0\right\|\;.$$ I want to show that $$r_i:=Ay_i-r_0\perp A\mathcal K_i\tag1\;.$$
Using the result of my other question (with $B=I$, $x=r_0$ and $U=A\mathcal K_i$), I'm only able to obtain $$y_i-r_0\perp A\mathcal K_i\tag2\;.$$
So, how can I show $(1)$?
Say we have a finite-dimensional (for simplicity) space $U$ with an inner product $(\cdot,\cdot)$ and the associated norm $\|\cdot\|$ and its subspace $V$. We have a fixed $u\in U$ and are looking for a $v\in V$ such that $$ \|u-v\|=\min_{\tilde{v}\in V}\|u-\tilde{v}\|. $$ This is equivalent to the orthogonality condition $$ (u-v,\tilde{v})=0\quad\forall \tilde{v}\in V. $$
Now put $U=\mathbb{C}^n$, $u=r_0$, $v=Ay$, $V=A\mathcal{K}_i$, and the Euclidean inner product.