Gradients and functions on matrices

Question

Given a twice differentiable $f: \Bbb R \to \Bbb R$, with continuous second order derivative.

We define $$F(x) = \sum_{i=1}^{m}f(x_i)$$ and $$L(x) = \sum_{i=1}^{m}f( \langle a_i, x \rangle+ b_i),$$ where $\langle \cdot , \cdot \rangle$ is the standard inner product

where $a_1, ..., a_m$ are in $\Bbb R^n$ and $b_1, ..., b_m$ are in $\Bbb R$.

Show that $$\nabla L(x) = A^T \nabla F(Ax+b)$$ and $$\nabla^2L(x) = A^T \nabla^2 F(Ax+b)A$$ where $A$ and $b$ are to be determined along with their respective dimension.

Anyone can tell me how to solve this?

Answer 1

A partial answer. Let $b=(b_1, ..., b_m)^T$ and $$A=\begin{pmatrix}a_1 \cdots\\ a_2 \cdots\\ \vdots \;\;\cdots\\ a_m \cdots \end{pmatrix}$$ (so $A$ is a $m \times n$ matrix).

Then we have that $$\langle a_i, x \rangle + b_i = (Ax + b)_i.$$ So $L(x) = F(Ax + b)$. The first identity follows (you may need to write it all out to see it) and I guess the second one is not too difficult either.

Answer 2

With the riem notations ; in particular, the rows of $A$ are the $({a_i}^T)_i$ and $L(x)=F(Ax+b)$. The function $F:\mathbb{R}^m\rightarrow \mathbb{R}$ has real values ; thus one has the following relations between the derivatives $DF,D^2F$ and the gradients $\nabla(F),\nabla^2(F)$:

${DF_u}^T=\nabla F(u)\in\mathbb{R}^m$ and $\nabla^2(F)(u)$ is the $m\times m$ symmetric matrix associated to the quadratic form $D^2_F(u)$ (the same relations are true for the function $L$).

$DL_x:h\in\mathbb{R}^m\rightarrow DF_{Ax+b}Ah$ and $\nabla L(x)=A^T\nabla F(Ax+b)$.

$D^2L_x:(h,k)\rightarrow D^2F_{Ax+b}(Ak,Ah)=h^T(A^T\nabla^2F(Ax+b)A)k$ and $\nabla^2L(x)=A^T\nabla^2F(Ax+b)A$.