Why does $\mathbb{E}\left[u(X)|X\right] = u(X)$?
Since $X$ is a random variable why does it seem like the function of $X$ it is being treated like a constant in this case?
In other words, why does it seem like this is happening: $$\mathbb{E}\left[u(X)|X\right] = u(X)\cdot\mathbb{E}[1|X] = u(X)\cdot 1? $$
You are conditioning on the value of $X$. So if you know that the value of $X$ is $x$ then what do you expect the value of $u(X)$ to be?
More generally (notably in the continuous random variable case), $Y=E(u(X)|X)$ is the unique random variable which is $X$-measurable and such that for any measurable $A\subset {\Bbb R}$: $$ \int_{X\in A} Y \; dP = \int_{X\in A} u(X) \; dP$$ Now $u(X)$ satisfies perfectly these requirements.