Matrix Derivative Problem

Question

Sorry for boring you my friends. I am haunted by a question of matrix derivative.

$q$ is a vector of dimension $n\times1$;

$A_1$, $A_2$, $...$ $A_n$ and $B$ are matrix with constant coefficients of dimension $n\times n$;

$A_1q$ , $A_2q$ $...$ and $A_nq$ become $n$ columns of matrix $\left[ A_1q \space A_2q ... \space A_nq \right]$ whose dimension is $n \times n$.

I would like to perform the following matrix derivative: $$\frac{\partial}{\partial q}\left( \left[ A_1q \space A_2q ... \space A_nq \right]Bq\right)^\text{T}$$

The first part of derivative is done, but I don't know how to perform derivative on the second part.

$$\frac{\partial}{\partial q}\left( \left[ A_1q \space A_2q ... \space A_nq \right]Bq\right)^\text{T} = B^\text{T}\left[ A_1q \space A_2q ... \space A_nq \right]^\text{T}+ q^\text{T}B^\text{T}\frac{\partial}{\partial q}\left(\left[ A_1q \space A_2q ... \space A_nq \right]^\text{T} \right)$$

Thanks in advance for taking a look!

P.S. Thanks for your response.Let $p$ and $q$ are all n-element vector.

$$ \frac{\partial p}{\partial q} = \left[ \begin {array}{ccc} \frac{\partial p_1}{\partial q_1}&...&\frac{\partial p_1}{\partial q_n} \\ ...&...&...\\ \frac{\partial p_n}{\partial q_1}&...&\frac{\partial p_n}{\partial q_n}\end {array} \right]$$

Answer 1

I find it easiest to study this when the matrix and vector products are rewritten as sums over coordinates (Einstein sums, if you allow).

The object you want to differentiate is the transpose of a vector $v$, where

$$v_i = (A_l)_{ik}q_k B_{lm}q_m,$$

and differentiation is defined by

$$\left(\frac{\partial p}{\partial q}\right)_{ij} = \frac{\partial p_i}{\partial q_j}$$

so let's just write (the transposition can be left for later)

$$\begin{aligned} \left(\frac{\partial v}{\partial q}\right)_{ij} &= \frac{\partial}{\partial q_j} \left((A_l)_{ik}q_k B_{lm}q_m\right) \\ &= (A_l)_{ik}\delta_{jk} B_{lm}q_m + (A_l)_{ik}q_k B_{lm}\delta_{jm} \\ &= (A_l)_{ij} B_{lm}q_m + (A_l)_{ik}q_k B_{lj}. \end{aligned}$$

The second term can be read as the $(i,j)$-th component of $[A_1q A_2q \ldots A_nq] B$, as you found. The first term will need to introduce a new notation because now each element of the $A_k$ must be accessed separately – that's what it says. It's a matrix whose $(i,j)$-th component is the scalar product of

$$a_{ij}^{\rm T} B q,$$

where $a_{ij}$ is a vector formed of $(i,j)$-th elements of the matrices $A_1, A_2, \ldots, A_n$ in order.

Written as a single multiplication, you'd need a rank-3 tensor here. Composed of slices given by the $A$ matrices as $T_{ijk} = (A_k)_{ij}$, your problem has a natural formulation as

$$\frac{\partial}{\partial q_j} T_{lik}q_kB_{lm}q_m = T_{lij}B_{lm}q_m + T_{lik}q_kB_{lj},$$

it's the lack of the contraction $T_{li\circ} q_\circ$ that has no representation in your formalism except splitting into coordinate components and recombining.

Answer 2

Denote $C(q)=\left[ A_1q \space A_2q ... \space A_nq \right]$. If $B=(e_1, \dots, e_n)$ is the canonical basis of the space and $B^*=(e^1, \dots, e^n)$ the dual basis, the image of a vector $v$ under the map $C(q)$ is $$D(q)=C(q)v=\sum_{i=1}^n [e^i(v)] A_i q $$ This is a sum of linear maps. Their derivatives are therefore themselves and $$D^\prime(q)h = \sum_{i=1}^n [e^i(v)] A_i h = [A_1 h \ A_2 h \ \dots \ A_n h ]v$$

Based on that you get $$\frac{\partial}{\partial q_i}\left(\left[ A_1q \space A_2q ... \space A_nq \right] \right)=\left[ A_1 e_i \space A_2 e_i ... \space A_n e_i \right]$$

Answer 3

Your subscripted $A_k$ matrices are actually components of a third-order tensor, which I'll call ${\mathcal A}$.

It will be convenient to define the vector $b=Bq,\,\,$ whose differential is $$\eqalign{ db &= B\,dq \cr }$$ Then in index notation, you're interested in the gradient of this vector $$\eqalign{ y_{i} &= b_{j}\,{\mathcal A}_{jik}q_{k} \cr }$$ Dropping the indices, you can write $$\eqalign{ y &= b^T{\mathcal A}q \cr\cr dy &= b^T{\mathcal A}\,dq + db^T{\mathcal A}q \cr &= b^T{\mathcal A}\,dq + dq^TB^T{\mathcal A}q \cr &= b^T{\mathcal A}\,dq + (B^T{\mathcal A}q)^T\,dq \cr\cr \frac{\partial y}{\partial q} &= b^T{\mathcal A} + (B^T{\mathcal A}q)^T \cr\cr }$$ The first term can be written as a weighted sum of your $A_k$ matrices $$\eqalign{ b^T{\mathcal A} &= \sum_{j=1}^n b_j\,A_j \cr }$$ And you already know how to write the second term, since $$\eqalign{ {\mathcal A}q &= [A_1q\,\,A_2q\,...A_nq\,] \cr (B^T{\mathcal A}q)^T &= ({\mathcal A}q)^TB \cr &= [A_1q\,\,A_2q\,...A_nq\,]^TB \cr }$$