Let $1 \leq p < \infty$ and $1/p + 1/q = 1$. Then if $l$ is a bounded linear functional on $L^p(E, d\mu)$ where $\mu$ is a $\sigma$-finite measure, $l(f) = \int_Efgd\mu$ for some $g \in L^q(E,d\mu)$ with $\|g\|_q = \|l\|_{op}$. The statement follows from the Radon-Nikodym Theorem, which requires $\sigma$-finiteness.
Is it true that if we let $p>1$ we can get rid of $\sigma$-finiteness in the proof? Is there a reference for this? What is the idea of the proof?
Yes, there is a complete proof for $p>1$ in Folland's book. Roughly speaking, the proof is split into three steps:
The last step is somehow performed by fixing a $\sigma$-finite subset $F$ of $E$ and applying the Radon-Nikodym Theorem on $F$. Then one must show that it is possible to let $F$ "grow" in a countable way. You can find all the details in Theorem 6.15