I'm currently working on an university assignment where we have to calculate the derivative of several formulas. However, we are to derive the formulas by components of vectors and matrices and I'm not sure how this is correctly done.
I did some research on this by reading this post but found it hard to find any proper information on it as most questions only regard derivatives w.r.t. to a vector or a matrix but not w.r.t. components of them.
The formulas I'm especially having problems with are the following ones: $$\frac{\partial}{\partial W_{i,j}}(Wx+b)$$ $$\frac{\partial}{\partial b_i}(Wx+b)$$ and $$\frac{\partial}{\partial x_i}(Wx+b)$$ $x,b \in \mathbb{R}^d$ and $W \in \mathbb{R}^{d \times d}$.
According to the post linked above I came to the assumption that $\frac{\partial}{\partial b_i}(Wx+b) = e_i$ where $e_i$ is the $i$-th standard basis vector of $b$. But as we're not in scalar calculus, I'm not entirely sure whether this is correct or if I also need to calculate a derivative for the $Wx$ part? Because for normal derivatives, if the variable used for differentiation is not "part of a term" (i.e. no multiplication or divison), then that term is set to $0$. However, I'd like to know if this also applies in this case?
If this assumption is correct, I figure that $\frac{\partial}{\partial x_i}(Wx+b)$ should result in a vector containing the product of $W_{0,i}$ up to $W_{n,i}$ with the $i$-th standard basis vector of $x$. Or in other words, the $i$-th column of $W$. But how would I denote this in a correct mathematical form, as I guess e.g. $W_{n,j}$ would not be a correct expression for this.
Finally, comming to the first formula, I'm a bit confused about the calculating the derivative w.r.t. the component of a matrix. As a generalization from the derivative w.r.t. the component of a vector, my assumption would be that we end up with a "standard basis matrix" that has a $1$ at the $i,j$-th position and $0$ anywhere else. Is this understanding correct or am I on the wrong track here?
Thanks for any help in understanding this properly in advance.