On the Hessian of a quadratic function

Question

Consider a vector valued function $f(\mathbf{r})$ defined from $f:\mathbb{R}^N \to \mathbb{R}^N$. Let $Q$ be a positive semidefinite matrix of size $N\times N$, which is independent of $\mathbf{r}$. How can we compute the Hessian of the functions (i) $\mathbf{y}^TQf(r)$ and (ii) $f(r)^TQf(r)$, where $\mathbf{y} \in \mathbb{R}^{N \times 1}$ is another vector independent of $\mathbf{r}$.

Answer 1

[Solving (i)]

Using Einstein Notation:

$[\nabla f\,]_{ij} = \frac{\partial\,f_i}{\partial\,r_j}$

---

$g_k = \frac{\partial\,(y_i\,Q_{ij}\,f_j)}{\partial\,r_k} = y_i\,Q_{ij}\frac{\partial\,f_j}{\partial\,r_k} = y_i\,Q_{ij}\,[\nabla f\,]_{jk}$

so, the column vector that will be the gradient of the function is:

$g = [\nabla f\,]^T\,Q^T\,y$

diferentiating it again we have the Hessian:

$H_{ij}=\frac{\partial}{\partial\,r_j}\left(\frac{\partial\,f_l}{\partial\,r_i} Q_{kl}\,y_k\right) = \frac{\partial^2 f_l}{\partial\,r_i\,\partial\,r_j} Q_{kl}\,y_k$

which I don't know how to write only with matricial products... but it can be computed as follows

$s = y^T\,Q$ (row vector)

$P_l = \frac{\partial^2\,f_l}{\partial\,r_i\,\partial\,r_j}$ (those are the Hessian matrices of each component of $f$)

$\boxed{H = \sum_\limits{l} s_l\,P_l}$

---

[Solving (ii)]

For the second case you just have to follow the same steps and apply the Product Rule when necessary.

---

$g_k = \frac{\partial\,(f_i\,Q_{ij}\,f_j)}{\partial\,r_k} = \frac{\partial\,f_i}{\partial\,r_k}Q_{ij}\,f_j + f_i\,Q_{ij}\frac{\partial\,f_j}{\partial\,r_k} = [\nabla f\,]_{ik}\,Q_{ij}\,f_j + f_i\,Q_{ij}\,[\nabla f\,]_{jk}$

$g^T = f^T\,Q^T\,[\nabla f\,] + f^T\,Q\,[\nabla f\,]$

$g = [\nabla f\,]^T\left(Q+Q^T\right)f$

diferentiating it again we have the Hessian:

$p = f^T\,Q$ (row vector)

$q = f^T\,Q^T$ (row vector)

$\boxed{H = [\nabla f\,]^T\left(Q+Q^T\right)[\nabla f\,] + \sum_\limits{l}(p_l+q_l)\,P_l}$