I have a matrix $A$ which is $n$ by $m$ and a vector $b$ which is $m$ by 1. I have the following expression:
$\frac{\delta}{\delta b} \mathbf{1}^{T}f(Ab)$
Where $\mathbf{1}$ is a vector of ones which is $n$ by $1$, $f$ is a function which is applied elementwise and $^{T}$ represents transpose.
How can I express this just in terms of $f'$, $A$ and $b$? And does anyone have any resources on vector differentiation that could help me.
Let $y=Ab,\,$ then the differential of the elementwise function $f(y)$ is given by $$\eqalign{ df &= f'\odot dy \cr }$$ where $\odot$ is the elementwise/Hadamard product.
Now find the differential and gradient of the cost function. $$\eqalign{ \phi &= 1:f \cr d\phi &= 1:df = 1:(f'\odot dy) = f':dy = f':A\,db = A^Tf':db \cr \frac{\partial\phi}{\partial b} &= A^Tf' \cr }$$ where : is the trace/Frobenius product, i.e. $\,A:B={\rm Tr}(A^TB)$