Differentiating a non-linear functional with respect to a vector

Question

I have the functional:

$$F=v^T\times A \times v$$

Where $A$ is a function of $v$. The non-linear system of equations necessary to find $v$ is obtained doing:

$$\frac{\partial F}{\partial v}=0$$

According to the differentiation rules this system could be written in matrix form as:

$$v^T \times A + (A \times v)^T + v^T \times \frac{\partial A}{\partial v} \times v =0 $$

How to calculate $\dfrac{\partial A}{\partial v}$?

Playing with the following example:

$$A=\begin{bmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{bmatrix}$$ $$A_{11}=a$$ $$A_{12}=ab$$ $$A_{21}=a^2b$$ $$A_{22}=ab^2$$

and: $$v^T=\begin{Bmatrix} a & b\end{Bmatrix}$$

What I tried is wrong, which consists on differentiating $A$ for each component in vector $v$ such that $\dfrac{\partial A}{\partial v}$ is a $2 \times 2 \times 2$ matrix:

$$\frac{\partial A}{\partial v}=\begin{bmatrix} \begin{bmatrix} A_{11,a} & A_{12,a} \\ A_{21,a} & A_{22,a} \end{bmatrix} \\ \begin{bmatrix} A_{11,b} & A_{12,b} \\ A_{21,b} & A_{22,b} \end{bmatrix}\end{bmatrix}$$

I believe the final shape is correct. Does anyone know where my mistake is?

Answer 1

You have that $F$ is a quadratic form. You can write, for any dimension

$$F= \sum_{i,j}A_{ij}v_iv_j.$$

So for your example you have:

$$F=A_{11}v_1^2+(A_{12}+A_{21})v_1v_2+A_{22}v_2^2.$$

So now, you just use chain rule ... right?

$$ \frac{\partial F}{\partial v}=\begin{bmatrix} \frac{\partial F}{\partial v_1}\\ \vdots \\ \frac{\partial F}{\partial v_n} \end{bmatrix}. $$

Answer 2

Everything looks fine to me. As for the part $\frac{\partial A}{\partial v}$, I don't think you need to write it explicitly, once you know its exact shape. Because all you want is to obtain a row vector $v^T\frac{\partial A}{\partial v}v$, whose $k$-th component can be computed as:

$$\sum_{i,j}\frac{\partial A_{ij}}{\partial v_k}v_iv_j$$

Particularly, in your example:

$$\begin{align}v^T\frac{\partial A}{\partial v}v&=\left(A_{11,1}v_1^2+(A_{12,1}+A_{21,1})v_1v_2+A_{22,1}v_2^2,A_{11,2}v_1^2+(A_{12,2}+A_{21,2})v_1v_2+A_{22,2}v_2^2\right)\\\end{align}$$

Answer 3

@Saullo, you can do a formal calculus. Note that $\dfrac{∂v}{∂v_i}=e_i$, the $i^{th}$ vector of the canonical basis. Then $\dfrac{∂F}{∂v_i}=e_i^TAv+v^TAe_i+v^T\dfrac{∂A}{∂v_i}v=e_i^T(A+A^T)v+v^T\dfrac{∂A}{∂v_i}v$ where $\dfrac{∂A}{∂v_i}$ is the matrix $[\dfrac{∂A_{j,k}}{∂v_i}]_{j,k}$.