If $$ f(\mathbf{x})=\frac{1}{2} \mathbf{x}^{T} \mathbf{Q} \mathbf{x}-\mathbf{b}^{T} \mathbf{x} $$
where $\mathbf{Q} \in \mathbb{R}^{n \times n}$ is symmetric and positive definite; $\mathbf{x}, \mathbf{b} \in \mathbb{R}^{n} .$ Given an initial point $ \mathbf{x}_{0} $ and the following iterative scheme
$$\mathbf{x}_{k+1}=\mathbf{x}_{k}+\alpha_{k} \mathbf{d}_{k} $$
With exact step size
$$ \alpha_{k}=\arg \min_{\alpha>0} f\left(\mathbf{x}_{k}+\alpha \mathbf{d}_{k}\right) $$
with $\mathbf{g}_{k+1}=\mathbf{Q} \mathbf{x}_{k+1}-\mathbf{b}$ and direction $\mathrm{d}_{k}$ defined as follows
$$ \mathbf{d}_{k+1} =-\gamma_{k+1} \mathbf{g}_{k+1}+\mathbf{d}_{k} $$
for $k=-1,0,1, \ldots$, with $\mathbf{d}_{-1}= 0$, $\mathbf{d}_{k+1}^{T} \mathbf{Q} \mathbf{d}_{k}=0$ , $\gamma_{0}=1$
a) Calculate $\gamma_{k+1}$ based on $\mathbf{g}_{k+1}, \mathbf{d}_{k}$, $\mathbf{Q}$.
b) Verify that $\mathbf{g}_{k+1}=\mathbf{g}_{k}+\alpha_{k} \mathbf{Q} \mathbf{d}_{k}$
c) Show that $\mathbf{g}_{k+1}^{T} \mathbf{d}_{k-1}=0$ and $\mathbf{g}_{k+1}^{T} \mathbf{g}_{k}=0$
d) Show that $\mathbf{d}_{k+1}$ is a descending direction, for $k>-1$.
I have already proven parts a) and b), but I can't solve parts c) and d), for the part a) I get $$ \gamma_{k+1}=\frac{\mathbf{d}_k^T \mathbf{Q} \mathbf{d}_k }{\mathbf{g}_{k+1}^{T} \mathbf{Q} \mathbf{d}_k} $$
For part c) and d) I tried substituting the given expressions, but I get something recursive, which makes me think that I should proceed by induction over $k$.
The proof is a bit longer...
Let us minimize function $f(\mathbf{x})=\frac{1}{2}\mathbf{x}^{\text{T}}\mathbf{Q}\mathbf{x}-\mathbf{x}^{\text{T}}\mathbf{b}$, where $\mathbf{Q}$ is positive definite. The gradient yields $\mathbf{g}\equiv\nabla f(\mathbf{x})=\mathbf{Q}\mathbf{x}-\mathbf{b}$.
Theorem (algorithm of the conjugate gradient method): Let starting point $\mathbf{x}_0$ be given and the initial direction is chosen as a steepest descent: \begin{align} \mathbf{d}_0=-\mathbf{g}_0~~~~~~~~~~~~~(0) \end{align} Let us calculate \begin{align} \mathbf{x}_{k+1}&=\mathbf{x}_k+\alpha_k\mathbf{d}_k,~\alpha_k=\frac{\mathbf{g}_k^{\text{T}}\mathbf{g}_k}{\mathbf{d}_k^{\text{T}}\mathbf{Q}\mathbf{d}_k}~~~~~~~~~~~~~(1)\\ \mathbf{d}_{k+1}&=-\mathbf{g}_{k+1}+\beta_k\mathbf{d}_k,~\beta_k=\frac{\mathbf{g}_{k+1}^{\text{T}}\mathbf{g}_{k+1}}{\mathbf{g}_k^{\text{T}}\mathbf{g}_k}~~~~~~~~~~~~~(2) \end{align} where \begin{align} \mathbf{g}_k=\mathbf{Q}\mathbf{x}_k-\mathbf{b}~~~~~~~~~~~~~(3) \end{align} for $k=1,2,\dots$. Then \begin{align} \mathbf{g}_i^{\text{T}}\mathbf{g}_j&=0~~\forall i\neq j, ~~(\text{symmetric in }i,j)~~~~~~~~~~~~~(4)\\ \mathbf{g}_i^{\text{T}}\mathbf{d}_j&=0~~\forall i>j~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~(5)\\ \mathbf{d}_i^{\text{T}}\mathbf{Q}\mathbf{d}_j&=0~~\forall i\neq j ~~(\text{symmetric in }i,j)~~~~~~~~~~~~~~~~(6) \end{align} Moreover is $\mathbf{d}_{k+1}$ the descending direction, i.e. $\mathbf{d}_{k+1}^{\text{T}}(-\mathbf{g}_{k+1})\ge0$.
Proof: We start with \begin{align} \mathbf{g}_{k+1}=_{(3)}\mathbf{Q}\mathbf{x}_{k+1}-\mathbf{b}=_{(1)} \mathbf{Q}\mathbf{x}_{k}-\mathbf{b}+\alpha_k\mathbf{Q}\mathbf{d}_k=_{(3)}\mathbf{g}_k+\alpha_k\mathbf{Q}\mathbf{d}_k. ~~~~~~~~~~~~~(7) \end{align} Let us prove Eqs.(4,5,6) by induction. We prove only inequalities $j<i$, the case $j>i$ for Eqs.(4,6) is a consequence of symmetry. In the first step we show: \begin{align} \mathbf{g}_1^{\text{T}}\mathbf{g}_0&=_{(7)}(\mathbf{g}_0+\alpha_0\mathbf{Q}\mathbf{d}_0)^{\text{T}}\mathbf{g}_0=_{(0,1.2)}\mathbf{g}_0^{\text{T}}\mathbf{g}_0+\frac{\mathbf{g}_0^{\text{T}}\mathbf{g}_0}{\mathbf{d}_0^{\text{T}}\mathbf{Q}\mathbf{d}_0}\mathbf{d}_0^{\text{T}}\mathbf{Q}(-\mathbf{d}_0)=0~~~~~~~~~~~~~(8)\\ \mathbf{g}_1^{\text{T}}\mathbf{d}_0&=_{(0)}-\mathbf{g}_1^{\text{T}}\mathbf{g}_0=_{(8)}0~~~~~~~~~~~~~(9)\\ \mathbf{d}_1^{\text{T}}\mathbf{Q}\mathbf{d}_0&=_{(2)}(-\mathbf{g}_1^{\text{T}}+\beta_0\mathbf{d}_0^{\text{T}})\mathbf{Q}\mathbf{d}_0=_{(2.2,7)}\left(-\mathbf{g}_1^{\text{T}}+\frac{\mathbf{g}_1^{\text{T}}\mathbf{g}_1}{\mathbf{g}_0^{\text{T}}\mathbf{g}_0}\mathbf{d}_0^{\text{T}}\right)\frac{\mathbf{g}_1-\mathbf{g}_0}{\alpha_0}=_{(0,8)}\frac{-\mathbf{g}_1^{\text{T}}\mathbf{g}_1+\mathbf{g}_1^{\text{T}}\mathbf{g}_1}{\alpha_0}=0 \end{align} Let us do the second induction step: let us suppose, that Eqs. (4-6) are valid for $\forall~j<i$ for fixed index $i$ and we want to prove it for $\forall~j<i+1$: \begin{align} \mathbf{g}^{\text{T}}_{i+1}\mathbf{g}_j&=_{(7)}(\mathbf{g}_i+\alpha_i\mathbf{Q}\mathbf{d}_i)^{\text{T}}\mathbf{g}_j=_{(2)}\underbrace{\mathbf{g}_i^{\text{T}}\mathbf{g}_j}_{=0}+\alpha_i\underbrace{\mathbf{d}_i^{\text{T}}\mathbf{Q}(-\mathbf{d}_j+\beta_{j-1}\mathbf{d}_{j-1})}_{=0}=0~~~~~~~~~~~~~(11)\\ \mathbf{g}^{\text{T}}_{i+1}\mathbf{d}_j&=_{(2)}\mathbf{g}^{\text{T}}_{i+1}(-\mathbf{g}_j+ \beta_{j-1}\mathbf{d}_{j-1})=_{(7)}-\underbrace{\mathbf{g}^{\text{T}}_{i+1}\mathbf{g}_j}_{=0~(11)}+\beta_{j-1}\underbrace{(\mathbf{g}_i^{\text{T}}+\alpha_i\mathbf{d}_i^{\text{T}}\mathbf{Q})\mathbf{d}_{j-1}}_{=0}=0 \\ \mathbf{d}_{i+1}^{\text{T}}\mathbf{Q}\mathbf{d}_j&=_{(2)}(-\mathbf{g}_{i+1}^{\text{T}}+\beta_i\mathbf{d}_i^{\text{T}})\mathbf{Q}\mathbf{d}_j=_{(7)}-\mathbf{g}_{i+1}^{\text{T}}\frac{\mathbf{g}_{j+1}-\mathbf{g}_j}{\alpha_j}+\beta_i\mathbf{d}_i^{\text{T}}\mathbf{Q}\mathbf{d}_j=_{(1.2,11)}\nonumber\\ &-\frac{\mathbf{g}_{i+1}^{\text{T}}\mathbf{g}_{j+1}}{\mathbf{g}_j^{\text{T}}\mathbf{g}_j}\mathbf{d}_j^{\text{T}}\mathbf{Q}\mathbf{d}_j+\beta_i\mathbf{d}_i^{\text{T}}\mathbf{Q}\mathbf{d}_j \end{align} which in case $j=i$ identically equals to zero and in case $j<i$ (i.e. $j+1<i+1$) it is zero due to Eq.(11) and assumption due to the induction step. This finishes the proof by induction.
Let us finally show, that $\mathbf{d}_{k+1}$ is the descending direction: \begin{align} \mathbf{d}_{k+1}^{\text{T}}(-\mathbf{g}_{k+1})=_{(2)}||\mathbf{g}_{k+1}||^2-\beta_k\underbrace{\mathbf{d}_{k}^{\text{T}}\mathbf{g}_{k+1}}_{=0}\ge0.~~\spadesuit \end{align}