Gradient with respect to x?

Question

What is the gradient (with respect to $x \in \mathbb{R}^n$) of $f(\vec{x}) = e^{\vec{x}^TA\vec{x}+b}$ where $A \in \mathbb{R}^{n\times n}$ is a symmetric matrix and $b$ is a real number?

I know that the derivative of $\vec{x}^TA\vec{x}+b$ is $2\vec{x}^TA$, but I am not sure of how to get the gradient of $f(\vec{x}) = e^{\vec{x}^TA\vec{x}+b}$ with respect to $x$?

Answer 1

This is just the chain rule.

Let:

• $\alpha: \mathbb{R}^n \rightarrow \mathbb{R}$ such that $\alpha(x)=xAx+b$
• $\exp: \mathbb{R} \rightarrow \mathbb{R} $ such that $exp(t)=e^t$

Then you have that

$$ f(x)=(exp \, \circ \, \alpha)(x) $$

thus, buy chain rule for the gradient

$$ \nabla f (x) = \nabla exp(\alpha(x)) \, \cdot \, \nabla \alpha (x) $$

$exp$ being from the reals to the reals, the first gradient is simply a derivative: $ \nabla exp(t) = \frac{d}{dt}|_t e^t = e^t $, while $ \nabla \alpha (x) = \nabla (xAx+b)(x) = \nabla (xAx) +\nabla b = \nabla xAx = 2Ax $ (the latest term should be proven but you can also take it for granted).

In the end:

$$ \nabla f (x)= 2e^{xAx+b} (A\cdot x)$$ where $ 2e^{xAx+b} $ is a number and $ (A\cdot x) $ is a vector.

Answer 2

Sum over repeated indices: since$$\partial_if=f\partial_i\ln f=f\partial_i(x_jx_kA_{jk}+b)=(\delta_{ij}x_kA_{jk}+x_j\delta_{ik}A_{jk})f=(x_kA_{ik}+x_jA_{ji})f,$$it follows $\nabla f=f(A+A^T)x$. Since $A$ is symmetric, this simplifies to $\nabla f=2fAx$.

Answer 3

Define the scalar variable $a = x^TAx+b\;$ and write the function in terms of this variable. $$f=\exp(a)$$ Calculate the differential, change variables from $\color{red}{a\to x,}\,$ and recover the desired gradient. $$\eqalign{ df &= f\,\color{red}{da} \\ &= f\,\big(\color{red}{2x^TA\;dx}\big) \\ &= \big(2fAx\big)^Tdx \\ \frac{\partial f}{\partial x} &= 2fAx \\ }$$