This question was part of my analysis assignment and I was unable to solve it. So, I am asking for help here.
Let $f: \mathbb{R}^{4} \rightarrow \mathbb{R}$ be defined by $f(x)=x^{t} A x$ where $A$ is a $4 \times 4$ matrix with real entries and $x^{t}$ denotes the transpose of $x$. The gradient of $f$ at a point $x_{0}$ necessarily is
- $2 A x_{0}$
- $A x_{0}+A^{t} x_{0}$
- $2 A^{t} x_{0}$
- $A x_{0}$
I used the definition of gradient and assume $x =[x, y, z, w]$ but couldn't match any of the options. So, it's my humble request to you to help me. Thanks!!
Let $x=(x_1,x_2,x_3,x_4)$. By definition $(Ax)_i = \sum_{j=1}^4 A_{ij} x_j$, and therefore $$f(x) = x^t Ax = x\cdot Ax = \sum_{i=1}^4 x_i (Ax)_i = \sum_{i,j} A_{ij} x_ix_j,$$ where $\sum_{i,j}$ means $\sum_{i=1}^4\sum_{j=1}^4$. Differentiating with respect to $x_k$, we have by ordinary product rule in 1D, \begin{align} \partial_{x_k} f(x) &= \partial_{x_k} \sum_{i,j} A_{ij} x_ix_j \\ &= \sum_{i,j} \partial_{x_k} (A_{ij} x_ix_j ) \\ &= \sum_{i,j} A_{ij} [(\partial_{x_k} x_i) x_j + x_i (\partial_{x_k} x_j)] \\ &= \sum_{i,j} A_{ij} (\partial_{x_k} x_i) x_j + \sum_{i,j}A_{ij}x_i (\partial_{x_k} x_j). \end{align}
Since $\partial_{x_k} x_i=1$ if $k=i$ and $0$ if $k\neq i$, i.e. $\partial_{x_k} x_i=\delta_{ik}$, $$ \partial_{x_k} f(x)=\sum_j A_{kj} x_j + \sum_i A_{ik} x_i. $$ The first term gives $Ax$. The second term gives $A^tx$. So the answer is 2.