Let $s$ be an integer greater than $6$. A solid cube of side $s$ has a square hole of side $x < 6$ drilled directly through from one face to the opposite face (so the drill removes a cuboid). The volume of the remaining solid is numerically equal to the total surface area of the remaining solid. Determine all possible integer values of $x$.
I've figured out the first bit, setting the equations. Would it be easier if I said "Find the solutions of $s$ and $x$ such that $s^3-x^2s=6s^2-2x^2+4xs$"? (bearing in mind that $s$ must be $>6$ and an integer and $x$ must be $<6$).
So we just need to write an equation and solve for $x$. The remaining volume is easy: It's just $s^3-sx^2$. The surface area is a little more complicated, but not too hard. We have $6s^2-2x^2$ for the outside surface area and $4sx$ for the inside surface area.
Hence we have the equation
$$s^3-sx^2=6s^2-2x^2+4sx$$
which is the same as the one you got.
Now solving the quadratic for $x$, we get
$$\begin{align} x&=\frac{-4s\pm\sqrt{16s^2-4(s-2)(6s^2-s^3)}}{2(s-2)} \\&=\frac{-4s\pm\sqrt{4s^2(4-(s-2)(6-s))}}{2(s-2)} \\&=\frac{-2s\pm s\sqrt{4-(s-2)(6-s)}}{s-2} \\&=\frac{-2s\pm s\sqrt{s^2-8s+16}}{s-2} \\&=\frac{-2s\pm s(s-4)}{s-2} \end{align}$$
Note that we can discard the negative branch because it gives a negative result for $x$. Hence $$\begin{align} x&=\frac{-2s+ s(s-4)}{s-2} \\&=\frac{s^2-6s}{s-2} \\&=s-\frac{4s}{s-2} \end{align}$$
This leaves us the problem to solve for what integers $s>6$ does $(s-2)\mid 4s$. To do this, note that $4(s-2)<4s$ and $5(s-2)=5s-10$ is greater than $4s$ whenever $s>10$, so it suffices to check the integers $s=7,8,9,10$.
Doing this we get the single solution $s=10$ which gives us $x=5$.