I am familiar with Rudin's book's proof of the fact that, in $\sigma$-finite measure spaces and for $p\in[1,+\infty)$, the dual space of $L^p$ is $L^q$ where $p,q$ are conjugate, i.e. $\frac{1}{p}+\frac{1}{q}=1$. I should be able to understand it - I haven't read through the whole of it carefully enough to be sure just yet but will in a while. The question is anyway not about the given proof, but rather about an alternate and way simpler proof I thought up which, seen as both my teacher and the book go through a number of extra contortions to prove the theorem, seems likely to be flawed.
Let me sketch the book's proof. First, it starts with finite measure spaces. It takes indicator functions as a start, and proves that, for a continuous linear functional $\Phi:X\to\mathbb{R}$, $X$ being the space on which we construct $L^p(X,\mathcal{E},\mu)$, we have that $\lambda(E):=\Phi(\chi_E)$ is a measure on $X$, and is absolutely continuous with respect to $\mu$, thus finding a measurable function which is a density for the measure $\lambda$, that is, the Radon-Nikodym derivative $g=\frac{\mathrm{d}\mu}{\mathrm{d}\lambda}$ of $\mu$ with respect to $\lambda$, thanks to the Radon-Nikodym theorem. So he says well, this shows that, for every measurable indicator function $\chi$, one has: $$\Phi(\chi)=\int\limits_X\chi\cdot g\mathrm{d}\mu,$$ so if we prove this holds for every function in $L^p$ (i.e. the equality holds when $\chi$ is substituted with any $f\in L^p$), the result is proven. The fact it holds for indicators means it holds for simple functions, by linearity of both sides. Then he says this implies the same for all $L^\infty$ functions, and goes on, splitting things into two cases, to show this can be further extended to all $L^p$, and that $\|\Phi\|=\|g\|_q$.
I was thinking: he has proven that the simple functions, call their family $S$, are dense in $L^p$ for all $p$, i.e. $\overline S=L^p$, where the closure is done in $L^p$. Can't we just say that, by continuity of both sides, $\{\Phi=T_g\}$ is closed in $L^p$, where $T_g(f)=\int fg\mathrm{d}\mu$, and thus, as that set contains $S$, it must contain its $L^p$-closure, which is $L^p$, and conclude the proof that way? I mean, $\Phi$ is continuous by hypothesis. The integral looks very "innocent" and likely to be continuous, even without proving $g\in L^q$.
Let's see if that right-hand side is actually continuous. It is definitely continuous on $L^\infty$, since the approximations of $L^\infty$ functions by simple functions are uniformly converging to the functions, so the limit and integral can be permuted, giving continuity. Take $f\geq0$ with $f\in L^p$. We have $s_n\geq0$ simple $L^p$ functions such that $s_n\to f$ pointwise. The convergence is monotone, so by the monotone convergence theorem we have that limit and integral indeed commute. If $f$ is not non-negative, I can split $X=P\cup N$, where $P=\{f\geq0\}$ and $N=\{f<0\}$. These are disjoint. I can easily construct simple functions $s_n\to f$ such that $s_n\downarrow f$ on $N$ and $s_n\uparrow f$ on $P$, or in short, $|s_n|\uparrow|f|$ everywhere. By the monotone convergence theorem: $$\int_Xgs_n\mathrm{d}\mu=\int_P|gs_n|\mathrm{d}\mu-int_N|gs_n|\mathrm{d}\mu\uparrow\int_P|fg|\mathrm{d}\mu-\int_N|fg|\mathrm{d}\mu=\int_Xfg\mathrm{d}\mu.$$ This gives us the continuity of the right-hand side, as we wanted. Btw, even if $g$ is not non-negative, I can split $X$ in the same fashion to get convergence for non-negative $f$s, and then I could split $X$ into 4 for the general case, the argument would be the same but with 4 pieces instead of 2.
But then, one may say, why should functions in $L^1\smallsetminus L^q$ not give contnuous functionals? My only answer is that those would not be $X\to\mathbb{R}$ but rather $X\to\mathbb{R}\cup\{+\infty,-\infty\}$. However, that would require to prove every $f\in L^1\smallsetminus L^q$ gives such a functions, i.e. that for all such $f$s there is a $g\in L^p$ such that the integral of the product is infinite, which seems a long way to go. so maybe I should just stick to Rudin's proof.
I anyway ask this because I'm curious. What's wrong here? Am I missing something obvious?
Update: Reading more carefully, I realized the "contortions" I was referring to, and the case splitting, are to prove the equality of the norms, and that $g\in L^q$. Then Rudin says «it follows that they are both continuous». So my question remains: what would happen if $g\not\in L^q$? Would continuity fail? OK, maybe the functional wouldn't be bounded, but would it be continuous?
What I seem to gather from the comments is that my argument is fine, but one needs to prove $g\in L^q$, otherwise the associated functional is not in the dual. That is, it doesn't take values in $\mathbb{R}$ because it takes $\infty$ as a value, and $\infty\not\in\mathbb{R}$. However, if I consider $\mathbb{R}^\ast=\mathbb{R}\cup\{+\infty,-\infty\}$ and endow it with the topology making all "open" intervals open, or more precisely having "open" intervals as a base, then the functional associated to this $g$ should always be continuous, right? Though it would not be defined on all $L^p$ functions since some may change sign and have both the integral of the positive and the negative part of $fg$ diverge, meaning the integral can't be defined.