A notational confusion on gradient

Question

Given a parametrized function $f_{w}: \Bbb R ^{m} \to \Bbb R ^{k}, w \in \Bbb R^d$, I see in a book the following notation $\bigtriangledown ^ {w} f_{w}(.)$ denote its gradient w.r.t. $w$. What is the meaning of it ? Also, in other place, the notation $\bigtriangledown ^ {w} g(.)$ is used where $g$ is a scalar-valued function. Confused with the meaning of it.

Answer 1

So you have in fact a function $$F:\quad{\mathbb R}^d\times{\mathbb R}^m \to{\mathbb R}^k, \qquad (w,x)\mapsto f_w(x)\ .$$ I'd think that $\nabla^w f_w(x_0)$ refers to changes of $F$ when $w$ undergoes an infinitesimal change, and $x_0$ stays fixed, in other words, that you take the gradient just with respect to the $w$-variables. Unpacking it all it looks like so: The function $F$ appears as $$F(w,x)=f_w(x)=\bigl(f_1(w_1,\ldots ,w_d\,;x_1,\ldots, x_m),\ldots,f_k(w_1,\ldots ,w_d\,;x_1,\ldots, x_m)\bigr)\ .$$ Its gradient with respect to $w$ has $d$ components, each of these itself a vector with $k$ components. The $j$'th component of this gradient is given by $$\bigl(\nabla^w f_w\bigr)_j=\left({\partial f_1\over\partial w_j},{\partial f_2\over\partial w_j},\ldots,{\partial f_k\over\partial w_j}\right)\qquad(1\leq j\leq d)\ .$$ A coordinate free way of writing this would be $$F(w+W,x)-F(w,x)=\nabla^w F(w,x)\cdot W+\ o(|W|)\qquad(W\to0)\ .$$ Here $W$ is a variable increment vector attached at the point $w$.