Partial derivative of matrix-vector product

Question

I have to compute the following:

$$ \frac{\partial}{\partial \mathbf{A_p}} \sum_{p=1}^P \mathbf{A_p}\mathbf{y}(t-p), $$

with $\mathbf{A}$ being a matrix and $\mathbf{y}$ a vector. Do I have to write the matrix-vector product as a sum?

Answer 1

Consider the simple vector-valued function $$\eqalign{ f &= Ay \cr }$$ or in index notation $$\eqalign{ f_i &= A_{ij}y_j \cr }$$ Find its differential and gradient wrt $A$ $$\eqalign{ df_i &= dA_{ij}\,y_j \cr\cr \frac{\partial f_i}{\partial A_{km}} &= \frac{\partial A_{ij}}{\partial A_{km}}\,y_j \cr &= \delta_{ik}\,\delta_{jm} \,y_j \cr &= \delta_{ik}\,y_m \cr }$$ As expected, the gradient is a 3rd order tensor.

The funtion you are interested in is $$\eqalign{ \frac{\partial}{\partial A_{q}}\,\,\sum_{p=1}^{N} A_p\,y(t-p) \cr }$$ Assuming all of the $A_p$ are independent, only the $A_q$-term survives differentiation.

Apply the previous result by substituting $$\eqalign{ A &\rightarrow A_q \cr y &\rightarrow y(t-q) \cr }$$