I have some questions on duality and Legendre-Fenchel Transform, and I hope some of you can help (perhaps providing some references as well). All the books/references I looked at on functional analysis are either not precise enough or lacking on this level of generality.
Assume that I have the space $\mathcal{X}$ which is a normal topological space with the sup-norm. Then the Riesz-Markov-Kakutani theorem (or at least, the only version of it I could find in a book: [DS58, Theorem IV.6.2, pag. 262]) tells me that if $C(\mathcal{X})$ is the space of all bounded continuous real functions on $\mathcal{X}$ then the topological dual $C(\mathcal{X})^\star$ is equal to $rba(S)$, the space of regular bounded additive set functions defined on the field generated by the closed sets with respect to the total variation norm. (What about $C_0$, $C_c$ or $C_b$? Is there a reference specifying the dual of these spaces? If I add the constraint of $\mathcal{X}$ being compact and consider the space of function vanishing at infinity, for instance. In the thesis [JL18], there is such a result in Theorem 2.4.2, page 44, but no proof or reference.)
Given that the set of finite signed measures $\mathcal{M}(\mathcal{X})$ is included in $rba(\mathcal{X})$ then given a measure $\nu\in \mathcal{M}(\mathcal{X})$ I should be able to have, given $\psi:C(\mathcal{X})\to\mathbb{R}$ through the Legendre-Fenchel transform $\psi^\star:C(\mathcal{X})^\star \to \mathbb{R}$ the following characterisation: \begin{equation} \psi^\star(\nu) = \sup_{f\in C(\mathcal{X})} \langle f,\nu\rangle -\psi(f). \end{equation}
My question then is, I can always compute $(\psi^\star)^\star$, and if $\psi$ is convex and lower-semi continuous, I should have that $\psi=(\psi^\star)^\star$ but then $(\psi^\star)^\star$ would be defined over $(C(\mathcal{X})^\star)^\star=(rba(\mathcal{X}))^\star$ which is not necessarily $C(\mathcal{X})$, right? So if my $\psi$ is convex and lsc how can I write this precisely \begin{equation} \psi(f) = \sup_{\nu\in ...} \nu(f)-\psi^\star(\nu)? \end{equation}
Does it hold? In some cases it does seem to hold. For instance if I select $\psi^\star(\nu)$ to be the Kullback-Leibler divergence $\psi^\star(\nu) = D(\nu\|\mu)$ where $\mu$ is fixed beforehand, then I have by the Donsker-Varadhan representation that \begin{equation} D(\nu\|\mu) = \sup_{f\in C_c(\mathcal{X})} \nu(f) - \log\int e^fd\mu. \end{equation} But can I write, for some $f$ \begin{equation} \log\int e^fd\mu=\sup_{\nu \in ...} \nu(f) - D(\nu\|\mu)? \end{equation} In the book [DZ], Equation (6.2.14) on page 264, the authors do this but it is unclear to me how rigorous and correct this actually is.
The last question is, given that one has \begin{equation} \psi(f) = \sup_{\nu\in ...} \nu(f)-\psi^\star(\nu)? \end{equation}
in $\mathbb{R}^n$ for instance I have that for a given $f$ I can always find a $\hat{\nu}$ such that $\psi(f)=\hat{\nu}(f)-\psi^\star(\hat{\nu})$, but does this stop being true in more general spaces? Are there any references to Legendre-Fenchel duality in more general spaces?
References
[DS58] Nelson Dunford, Jacob T. Schwartz, Linear Operators, Part I: General Theory, Interscience, 1958.
[JL18] Jingbo Liu, Information Theory from a Functional Viewpoint, 2018, https://dataspace.princeton.edu/bitstream/88435/dsp01g158bk964/1/Liu_princeton_0181D_12396.pdf
[DZ] Amir Dembo, Ofer Zeitouni, Large Deviations Techniques and Applications, https://link.springer.com/book/10.1007/978-3-642-03311-7#toc