Consider the entropy function over the simplex,
$f(x) = \sum\limits_{i = 1}^J x_i \log(x_i), x_i \geq 0, \sum\limits_{i = 1}^J x_i =1$ and $f(x) = +\infty$ everywhere else
This function is given in Boyd and Vanderberg's book Pg. 93.
My question is whether this function is differentiable, i.e. does $\nabla f(x)$ exist?
I was guessing it does exist, because taking a derivative with respect to each $x_i$ gives me $(\nabla f(x))_i = (1 + \log(x_i))$.
But I saw some discussion online (can't find it now, maybe here: https://johncarlosbaez.wordpress.com/) that this function is not differentiable. Which also makes sense, because the sharp corner of this function.
So if it is not differentiable, then what is the $(1 + \log(x_i))$ term after I take its gradient as usual?
While Robert's answer is correct as he precisely stated it, there is a reason that we often say that $f(x)=x\log x$ is not differentiable.
In a convex optimization setting at least (and likely in other domains), we require that differentiable functions have an open domain; and if the boundary of the domain is non-empty, the function must approach infinity on that boundary. that is, $$x\rightarrow\mathop{\textrm{Bd}}(\mathop{\textrm{dom}}(f)) \quad\Longleftrightarrow\quad f(x)\rightarrow +\infty ~~\text{or}~~f(x)\rightarrow-\infty$$ After all, if this were not the case, there would be an obvious continuous extension of the function onto the boundary, or that the function's domain was "artificially restricted" in some sense. For convex functions, this boundary criterion means that the function serves as a barrier function for its domain.
So, for example, the function $f(x)=-\log x$, with a domain $x\in(0,+\infty)$, is differentiable at every point on its domain. It also approaches $+\infty$ as $x\rightarrow 0$, which is the sole boundary point in its domain. So it is differentiable according to our criteria.
On the other hand, $f(x)=x\log x$ can naturally be defined to satisfy $f(0)=0$ by appealing to continuity. So its natural domain is $x=[0,+\infty)$. And while it is differentiable for positive $x$ as Robert states, it is not differentiable at 0, therefore the function is not differentiable.
You could drop our $f(0)$ convention and define the domain to be $x\in(0,+\infty)$, but then you are left with a function that does not approach $+\infty$ on the boundary of its domain.
The barrier requirement may seem arbitrary but it has important theoretical and practical consequences in convex analysis and optimization. For instance, because the derivative of $x\log x$ approaches $+\infty$ near the origin, it violates the continuity assumptions used to prove the convergence of certain optimization methods.