The function $p:\mathbb{R}\to[0, 1]$ is such that $\sum_{x \in \mathcal{X} \subset \mathbb{R}}p(x)=1$, where $\mathcal{X}$ is a countable set.
Let $$H(p) = -\sum_{x \in \mathcal{X}}p(x)\log[p(x)]$$ where we define $p(x)\log[p(x)] = 0$ wherever $p(x) = 0$.
I am interested in solving for $p$ in the problem $$\max H(p) \text{ subject to }\sum_{x \in \mathcal{X}}p(x)=1\text{.}$$
My thoughts:
I can't use calculus techniques. Notably, $p$ is not continuous, and is therefore not differentiable. If this weren't the case, I would use Lagrange multipliers.
I've honestly never run into an optimization problem before where I couldn't use derivatives, so I'm not even sure where to begin.
Take $p_n = ({1 \over n},\cdots,{1\over n},0,\cdots)$, then $H(p_n) = -\log {1 \over n}$ and $\sup H(p_n) = \infty$, so $H$ is unbounded.