Differentiation of a scalar function w.r.t. a vector

Question

I'd like to know how to take a differentiation w.r.t. a vector as follows: $$\frac{\partial \log f({\bf x}-{\bf y})}{\partial {\bf y}}$$ where $f:{\mathbb R}^p \to {\mathbb R}$ and ${\bf x},{\bf y}\in{\mathbb R}^p$. My calculation is $$\frac{\partial \log f({\bf x}-{\bf y})}{\partial {\bf y}} =-\frac{f'({\bf x}-{\bf y})}{f({\bf x}-{\bf y})}{\bf I_p}$$ But it should be a vector because this is a differentiation of a scalar function $f(\cdot)$ with respect to a vector ${\bf y}$. Any comments would be appreciated.

Answer 1

The notation $\frac{\partial f(\bf x)}{\partial \bf x}$ is a shorthand for the vector with components $\frac{\partial f(\bf x)}{\partial \bf x_i}$ so in more standard notation we have $\frac{\partial f(\bf x)}{\partial \bf x} \equiv {\bf \nabla} f({\bf x})$ $-$ the gradient of $f$.

Using this for your problem we find

$$\frac{\partial \log f({\bf x- y})}{\partial {\bf y}_i} = \frac{1}{f({\bf x- y})} \frac{\partial f({\bf x- y})}{\partial {\bf y}_i} = \frac{1}{f({\bf x- y})} \frac{\partial f({\bf x- y})}{\partial ({\bf x- y})_i}\frac{{\partial ({\bf x- y})_i}}{\partial {\bf y}_i} = -\frac{[\nabla f({\bf x- y})]_i}{f({\bf x- y})}$$ so

$$\frac{\partial \log f({\bf x- y})}{\partial {\bf y}} = -\frac{\nabla f({\bf x- y})}{f({\bf x- y})}$$

where $\nabla f({\bf x- y})$ means the gradient of $f$ evaluated at $({\bf x- y})$. This is the same as you have found if we interpret $f'$ as $\nabla f$ (and we don't need ${\bf I}_p)$. However note that for a function $f: \mathbb R^p \to \mathbb{R}$ of several variables there is no such thing as $f'$ (if interpreted as a single variable derivative; what variable is this with respect to?). The natural generalization of $f'$ from the single variable case is ${\bf \nabla} f$, the vector of all the possible partial derivatives of $f$.