I have a problem with a multi-dimensional chain rule. I can't understand what's wrong!
Well, let's consider such partial derivative:
$\partial_k(F\circ G)(x)$, where $F:\mathbb{R}^d\rightarrow\mathbb{R}$ and $G:\mathbb{R}^d\rightarrow\mathbb{R}^d$. So, the composition $F\circ G:\mathbb{R}^d\rightarrow\mathbb{R}$.
Thus, the partial derivative should be just a real number. However, with the chain rule I somehow get a vector.
Firstly I derive $G$. It's a vector-valued function, like $(G_1, G_2,...,G_d)$, so the partial derivative is a vector $\partial_kG=(\partial_kG_1, \partial_kG_2,...,\partial_kG_d)$.
And now I should multiply it on $F$ derivative which is a number, so finally I get a vector.
I'm totally confused, help me please to deal with it! :)
By the chain rule (please look the following notation symbolically) you have: $$d_x(F\circ G)(x)=d_xF(G(x))=d_{G}F(G)\cdot d_xG(x)$$ Note that $d_xG$ is a vector and $d_G F(G)$ is a gradient of a scalar function, which is also a vector. Therefore their product gives a scalar.