Say I have a radial basis function $g : \mathbb{R}^M \to \mathbb{R}^L$, i.e. the value of $g(x)$ depends only on $|x|$. Furthermore, let $g$ be differentiable at $a \in \mathbb{R}^M$, i.e. it has a Jacobian Matrix $Dg(a) \in \mathbb{R}^{L \times M }$. I want to show that for any $v \in \mathbb{R}^M$ that is orthogonal to $a$, $Dg(a)(v) = 0$.
The question somewhat reminds me of questions on circular motion in classical physics, and in that context, you have expressions such as $(r\cos(\phi)+r\sin(\phi)) $. In this general $L$ by $M$ context, I can't find an opening to this question. Where do I use the radial property of $g$ for example? How does that relate to the Jacobian?
Let $r=\lVert x \rVert$, and $G(x)=g(r)$. In components we can compute $$ \partial_i \sqrt{r^2} = \frac{1}{2\sqrt{r^2}} \partial_i r^2 = \frac{x_i}{r}. $$ Then by the chain rule, $$ \partial_i G_j(x) = \partial_i g_j(r) = g_j'(r) \frac{x_i}{r}. $$ Putting $x=a$ gives $$\partial_i G_j(a) = g'_j(\lVert a \rVert) \frac{a_i}{\lVert a \rVert}. $$ It is then clear that $$DG(a)(v) = g'_j(\lVert a \rVert) \frac{a \cdot v}{\lVert a \rVert}, $$ which is zero if and only if $a\cdot v =0$.
One can do the calculation using only Jacobians using the chain rule in the form $D(G\circ N)(a) = DG(N(a))DG(a), $ where $N(x)=r$ is the norm. Then $N(x+v)^2 = N(x)^2 + 2x\cdot v + o(N(v)), $ and then the chain rule gives $DN(x)(v) = x \cdot v/N(x)$ as before, and we again have $DG(x)(v) = g'(x)x \cdot v/N(x)$.