Show that points $x_i$ produced by conjugate gradient method stay in $W + x_0$, where $W$ is a space spanned by independent eigenvectors

27 Views Asked by At

Consider problem of minimizing $$f(x) = \frac{1}{2}x^TAx - b^Tx$$ with $A$ positive definite. Write $x_0 - x_{\infty}$ (where $x_{\infty}$ is a optimal point) as $\sum y_i$, where each $y_i$ is an eigenvector of $A$ corresponding to eigenvalue $\lambda_i$, and all $\lambda_i$ are different. Let $W$ be a space spanned by $y_i$. Justify that $x_i$ produced by conjugate gradient method stay in $W + x_0$.