I want to show that $\underline{\int_{a}^{b}} f \ d \alpha \leq \overline{\int_{a}^{b}} f \ d \alpha$
So I want to show that $\sup L(P,f, \alpha) \leq \inf \ U(P, f, \alpha)$. Can I just suppose that $\sup L(P,f, \alpha)> \inf \ U(P, f, \alpha)$ and come up with a contradiction?
I would first prove this lemma: If $A$ and $B$ are nonempty sets of real numbers such that $a\leq b$ for all $a\in A$ and $b\in B$, then $\sup A\leq \inf B$. You can prove the lemma using only the facts that $\sup A$ is less than or equal to every upper bound for $A$, and $\inf B$ is greater than or equal to every lower bound for $B$.
To apply the lemma to your problem, you can proceed as Didier indicated by showing that if $P$ and $Q$ are arbitrary partitions of $[a,b]$, then $L(P,f,\alpha)\leq U(Q,f,\alpha)$. This would be clear if you had $P=Q$, but typically you don't. However, notice what happens when you refine partitions: Adding more points to a partition can only make the lower sum larger and the upper sum smaller. So what happens if you take a partition that includes both $P$ and $Q$?