How to partially differentiate w.r.t. a vector?

Question

I have to prove that for two vectors $w$ and $b$ the following identity holds true

$$\frac{\partial(w^Tb)}{\partial w} = \frac{\partial(b^Tw)}{\partial w} = b$$

But I can't seem to understand how can I partially differentiate a function w.r.t. a vector? Does it mean that I differentiate each of its terms? As in if $w = [x_1, x_2, x_3]$ then does

$$\frac{\partial{x}}{\partial{w}} = \frac{\partial{}}{\partial{x_1}}\left(\frac{\partial{}}{\partial{x_2}}\left(\frac{\partial{x}}{\partial{x_3}}\right)\right)?$$

Answer 1

It means the gradient: $$ \frac{\partial f}{\partial w} = \left( \frac{\partial f}{\partial w_1} , \dots, \frac{\partial f}{\partial w_n} \right) . $$ (If $\partial(w^T b)/\partial w$ is going to equal $b$, it had better be a vector, right?)

Answer 2

There is no such thing as "partial differentiation with respect to a vector variable". Given a function $f:\>{\mathbb R}^n\to{\mathbb R}$, a point $p$ in the domain of $f$, and a tangential vector $w\in T_p$ there is the Gâteaux derivative $$D_w f(p):=\lim_{t\to0},{f(p+t w)-f(p)\over t}=df(p).w\ ,$$ also called the directional derivative of $f$ in direction $w$. If $x$ is the declared coordinate variable in ${\mathbb R}^n$ then $df(p)$, being a functional on $T_p$, appears as $$df(p).X=a_1X_1+\ldots a_nX_n=\nabla f(p)\cdot X$$ with $$a_i={\partial f\over\partial x_i}(p)\qquad(1\leq i\leq n)\ .$$ The vector $$\nabla f(p)=\left({\partial f\over\partial x_1},\ldots,{\partial f\over\partial x_n}\right)_p$$ is called the gradient of $f$ at $p$, and is, sometimes, denoted by ${\partial f \over\partial x}(p)$.