I need help to find some compact sets for my engineering problem. Through this page I learned quite much about it however since I have neither read a book yet nor have an experience I am not able to evaluate the situation in an optimal way. I hope you can help me.
In a question that I asked, Is this set compact? I learned that
The set
$$ {\cal{F}}=\{g:S(g,f)\leq \epsilon\} $$
based on the squared Hellinger distance
$$ S(g,f)=H^2(g,f)=\frac{1}{2}\int_{\mathbb{R}}\left(\sqrt{g(y)}-\sqrt{f(y)}\right)^2 \mbox{d}y $$
is not compact.
My first question:
what is/are missing here to get a compact set? In other words what
modification can I make to get it compact?
according to counterexamples it seems that even Hellinger distance, which is a metric, will not give me a compact set.
In another question How to prove that the space created by pointwise Bernoulli random variables are compact I learned that the set of all decision rules $\Delta$ seems also non-compact.
In short a decision rule $\delta\in\Delta$ is a non decreasing continuous function which is $0$ for $y\in(-\infty,y_l)$, increasing in $[0,1]$ for $y\in[y_l,y_u)$, and $1$ for $y\in(y_u,\infty)$.
Now I am in a trouble because I have the minimax theorem which holds for compact sets for the existence of a unique solution.
According to Von Neumann's minimax theorem, I have $$\max_{x\in X} \min_{y\in Y}f(x,y)=\min_{y\in Y} \max_{x\in X} f(x,y)$$ for some compact sets $X$ and $Y$
in my problem $X={\cal{F}}$ and $Y=\Delta$ where $\Delta$ is the set of all decision rules $\delta\in\Delta$.
However In this paper http://arxiv.org/pdf/0707.2926.pdf with a slight difference, where $X$ is constructed by means of KL-divergence instead of squared Hellinger distance, it is claimed that there exists a unique solution for Von Neumann's minimax theorem!
His argument for the compactness of $\Delta$ was that it satisfies infinity norm. However $y^2/(y^2+1)$ as an example given by @did shows that the maximum does not exist.
Now I want to get something working but all I get is something not working.
My second question:
What minimal assumptions I can have such that X and Y are compact
sets or at least one of them is compact.
Normally whenever I get some nominal density $f$ and construct ${\cal{F}}$, and choose a decision function $\delta$ for the minimization, the solution is unique. However I have mathematically either incorrect or incomplete expressions if I try to type the things as I observe.
I would really appreciate your help in this matter.
Thank you very much.