gradient of gradient is it Hessian?

296 Views Asked by At

Say, I have a function $f(\vec{x}) = \cfrac{1}{2}\vec{x}^{T}Q\vec{x} - \vec{b}^T\vec{x}$, where $Q$ is Symmetric Positive Definite $\in R^{nxn}$.

I want to find $\nabla f(\vec{x} - \nabla f(\vec{x}))$.

So, am I solving this correctly like the following? I know that, \begin{equation}\nabla f(\vec{x}) = Q\vec{x}-\vec{b}\end{equation} So, \begin{equation}\nabla f(\vec{x} - \nabla f(\vec{x})) = \end{equation} \begin{equation}= (Q(\vec{x} - \nabla f(\vec{x})) - \vec{b})\nabla(\vec{x} - \nabla f(\vec{x})) = \end{equation} \begin{equation}= (Q(\vec{x} - \nabla f(\vec{x})) - \vec{b})(\nabla\vec{x} - \nabla \nabla f(\vec{x})) = \end{equation} \begin{equation}= (Q(\vec{x} - \nabla f(\vec{x})) - \vec{b})(\vec{1} - \nabla^2 f(\vec{x})) = \end{equation}

So, can I say that $\nabla \vec{x} = \vec{1}\quad \text{and} \quad\nabla \nabla f(\vec{x}) = \nabla^2 f(\vec{x})$

1

There are 1 best solutions below

2
On BEST ANSWER

You are correct until partial difference of $f(\mathbf{x})$. Next, the vector $\mathbf{x} - \nabla f(\mathbf{x})$ is replaced for $\mathbf{y}$. Therefore,

$$ \begin{align} \mathbf{y} &=\mathbf{x}-\nabla f(\mathbf{x}) \\ &=\mathbf{x}-(Q\mathbf{x}-\mathbf{b}) \\ &=(E-Q)\mathbf{x}+\mathbf{b} \end{align} $$

Then, $E$ is an identical matrix. So, the calculation $\nabla f(\mathbf{x}-\nabla f(\mathbf{x}))$ can be described as follows.

$$ \begin{align} \nabla f(\mathbf{x}-\nabla f(\mathbf{x})) =& \nabla f(\mathbf{y}) \\ =& Q\mathbf{y} - \mathbf{b} \\ =& Q\{(E-Q)\mathbf{x}+\mathbf{b}\}-\mathbf{b} \\ =& Q(E-Q)\mathbf{x}-(E-Q)\mathbf{b} \end{align} $$