Derivative of a function divided by its norm, i.e., $\phi(x) = \frac{f(x)}{\|f(x)\|}$

Question

Setting

$f:\mathbb{R}^n\to\mathbb{R}^n$ and $\|\cdot \|$ be the usual Euclidean norm. I would like to compute the derivative with respect to $x$ of $$ \phi(x) = \frac{f(x)}{\|f(x)\|} $$

My Attempt at a Solution

$$ \nabla_x \frac{f(x)}{||f(x)||} = \frac{\nabla_x f(x)}{||f(x)||} + f(x)\nabla_x\left(f(x)^\top f(x)\right)^{-\frac{1}{2}} = \frac{\nabla_x f(x)}{||f(x)||}-\frac{1}{2}\frac{f(x)}{||f(x)||^3} 2f(x)^\top \nabla_x f(x) = \left(I - \frac{f(x)f(x)^\top}{||f(x)||^2}\right)\frac{\nabla_x f(x)}{||f(x)||} $$ However I am very unsure about this. In particular, I have a feeling that the second term would be $f(x)^\top \nabla_x f(x)$ and hence lead to $$ \nabla_x \frac{f(x)}{||f(x)||} = \frac{\nabla_x f(x)}{||f(x)||} - \frac{\nabla_x f(x)}{||f(x)||} = 0 $$ However this wouldn't make much sense cause surely the gradient is not $0$ for every function.

Answer 1

$ \def\p{{\partial}} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\c#1{\color{red}{#1}} $Separate $f$ into two components: $\;$ its length $(\lambda)$ and direction $(\phi)$ $$\eqalign{ \lambda^2 &= \|f\|^2 &\quad\implies\quad \lambda\,d\lambda = \c{f^Tdf} \\ \phi &= \lambda^{-1} f \\ }$$ Calculate the differential of $\phi$ $$\eqalign{ d\phi &= \lambda^{-1} df - \lambda^{-2}f\,{d\lambda} \\ &= \lambda^{-1} df - \lambda^{-3}f\,{(\lambda\,d\lambda)} \\ &= \lambda^{-1}I\, df - \lambda^{-3}f\,\c{(f^Tdf)} \\ &= \lambda^{-1}\left(I - \phi\phi^T\right)df \\ }$$ Now differentiate with respect to $x$ $$\eqalign{ \grad{\phi}{x} &= \left(\frac{I - \phi\phi^T}{\lambda}\right)\grad{f}{x} \\ }$$ So your initial solution was absolutely correct. And you can prove the famous result about unit vectors being perpendicular to their gradients $$\eqalign{ \phi^T\left(\grad{\phi}{x}\right) &= \left(\frac{\phi^T-({\tt1})\phi^T}{\lambda}\right)\grad{f}{x} \\ &= \left(\frac{0}{\lambda}\right)\grad{f}{x} \\ &= 0 \\ }$$

Answer 2

As the bets for gradient are low, let's compute the total derivative of $\phi$ as follows: $$ \begin{align} d_p\phi(v)&= \frac{d_pf(v)\|f(p)\|-f(p)\frac{1}{2\|f(p)\|}\cdot2\langle d_pf(v),f(p\rangle}{\|f(p)\|^2}\\ &=\frac{1}{\|f(p)\|}\left(d_pf(v)-\frac{f(p)}{\|f(p)\|}\langle \frac{f(p)}{\|f(p)\|},d_pf(v)\rangle, \right), \end{align}$$ which is quite a pleasant result as it is the perpendicular of $d_pf(v)$ to $f(p)$, scaled by the length of $f(p)$.

Answer 3

I will present the calculation in standard coordinates. Your $\phi$ is the composition of $u(x)=x/\|x\|$ and $f(x)$, i.e. $$ \phi_i(x) = u_i(f(x)), \quad u_i(x) = \frac{x_i}{\|x\|}.$$ For $x$ away from the origin, as $u_i(x) = -\partial_i {\|x\|}$, and since $(1/x)' = -1/{x^2}$, product and chain rule gives $$ \partial_k u_i = \frac{\delta_{ik}}{\|x\|} - \frac{x_i x_k}{\|x\|^3}.$$ Or, if you prefer, $\nabla u = \frac1{\|x\|}I_n - \frac1{\|x\|^3}xx^T$. So long as $f$ avoids the origin, chain rule gives $$ \partial_j \phi_i = \sum_{k=1}^n\partial_k u_i(f(x))\partial_j f_k(x) = \frac{\partial_jf_i(x)}{\|f(x)\|} - \frac1{\|f(x)\|^3}\sum_{k=1}^n f_i(x) f_k(x) \partial_j f_k $$ i.e. the Jacobian is $$ \nabla \phi = \frac{\nabla f}{\|f\|} - \frac1{\|f\|^3}ff^T\nabla f.$$

---

Regarding your final comment on the feeling: the error in your intuition is that you replaced $ff^T\in \mathbb R^{n\times n}$ with the scalar $f^T f = \|f\|^2$. This is in fact (as you basically said) what makes the two terms different. In fact, this is already apparent from the above formula for $\nabla u = \partial_i\partial_j \|x\|$.

As $\|x\|$ "scales" like one power of $x$, two derivatives should "scale" like $x^{1-2}=1/x$. Which is precisely what we see, except in the case of dimension $n=1$ where the derivative is indeed exactly $0$ whereever defined (!!) In higher dimensions, there are simply more functions that have the same scaling, leading to your error.