Estimating rate of decay of residual norms in gradient descent

32 Views Asked by At

I am using gradient descent to solve the linear system $Ax=b$, where matrix $A$ is symmetric and positive definite. More precisely, I am attempting to solve the following quadratic program

$$\text{minimize} \quad \frac{1}{2}x^TAx-b^Tx$$

A logarithmic plot of the residual norms ($\rho_k$) after $K$ steps forms a straight line sloping down to the right. I was asked to produce a least-squares approximation to $\log(\rho_k)$ where $k \in \{K, \dots, N\}$ in order to determine $\beta$ and $\gamma$ where

$$\log(\rho_k) \approx \beta + \gamma(k-K)$$

and am not sure what to do. Any help would be appreciated.

enter image description here