derivatives of a vector of functions with respect to a vector

Question

Let $\vec W \in \mathbb R^3$. What is the general solution to:

$$\frac{\partial}{\partial \vec{W}} \begin{pmatrix} f(\vec W) \\ g(\vec W) \end{pmatrix} $$

I think that in the case where $f$ and $g$ are linear I could rewrite: $$\begin{pmatrix} f(\vec W) \\ g(\vec W) \end{pmatrix} =A\cdot \vec W $$ for some suitable matrix $A$ and then this would break down to:

$$\frac{\partial}{\partial \vec{W}}A\cdot \vec W=A $$

Is this correct? So my question really aims for the general case, i.e. when $f$ and $g$ are not necessarily linear.

It feels kind of wrong to take the derivatives along the rows given that $\vec W$ is a column-vector.

Answer 1

I think I've found what I needed on wikipedia and it is actually quite simple:

$$ \frac{\partial \mathbf{y}}{\partial \mathbf{x}} = \begin{bmatrix} \frac{\partial y_1}{\partial x_1} & \frac{\partial y_1}{\partial x_2} & \cdots & \frac{\partial y_1}{\partial x_n}\\ \frac{\partial y_2}{\partial x_1} & \frac{\partial y_2}{\partial x_2} & \cdots & \frac{\partial y_2}{\partial x_n}\\ \vdots & \vdots & \ddots & \vdots\\ \frac{\partial y_m}{\partial x_1} & \frac{\partial y_m}{\partial x_2} & \cdots & \frac{\partial y_m}{\partial x_n}\\ \end{bmatrix} $$

Hence in my case this it would look like this:

$$ \frac{\partial}{\partial \vec{W}} \begin{pmatrix} f(\vec W) \\ g(\vec W) \end{pmatrix} = \begin{bmatrix} \frac{\partial f(\vec W)}{\partial W_1} & \frac{\partial f(\vec W)}{\partial W_2} & \frac{\partial f(\vec W)}{\partial W_3}\\ \frac{\partial g(\vec W)}{\partial W_1} & \frac{\partial g(\vec W)}{\partial W_2} & \frac{\partial g(\vec W)}{\partial W_3}\\ \end{bmatrix} $$ The fact that the vector is a vector of functions doesn't add anything special to it...

Answer 2

What you write is a (dangerous) abuse of notation. In dimensions superior to 1, the concept of derivative depends on the direction see wikipedia

In the particular case where it is linear, you can see that:

$\frac{d}{dt} f(\vec a + t \vec W) = \frac{d}{dt} (A \vec a + t A \vec w ) = A \vec w$

You can also look at the gradient (taking partial derivatives), which is probably more what you're after.