In particular I was trying to make sense of the expression:
$$ dist(V,f) = \inf_{v \in V} \| v - f \|_{\mathcal M}$$
which seems to require me to extend the notion of infimum to metric spaces and things that look like minimization problems.
To start this search I reviewed the definition of infimum from Walter Rudin's book (here paraphrased):
Suppose $(P, \leq)$ is an ordered set, $S \subset P$ and $S$ is bounded from bellow. Suppose there exists an $\alpha \in P$ with the following properties:
- $\alpha$ is a lower bound. i.e. $\forall x \in S, \alpha \leq x$. i.e. $\alpha \in LowerBounds(S)$
- If $\gamma > \alpha$ then $\gamma \in P$ is not a lower bound. i.e. if $\gamma > \alpha \implies \gamma \notin LowerBounds(S)$.
which makes sense and the notation:
$$ \alpha = \inf S$$
makes sense to me. Similarly I think:
$$ \alpha = \inf_{ x \in P} S $$
makes sense to me too, just being more explicit over which set (i.e. $P$) is the infimum and with respect to what set (i.e. $S$) we are taking the infimum.
However, I am having difficulties extending this notation because in a metric space we don't explicitly have an ordered set nor do we have the explicit sets $P$ and $S$ defined.
It seems that for this to make sense or be equivalent we would have to define $P = \mathbb R$ the reals and $S_f = \{ \| v - f \|_{\mathcal M} : f \in V\}$ which seems in this case that $S_f \subset \mathbb R$ and the ordered set is induced by the standard order on the number line. The issue in this case might be that multiple different $v \in V$ could map to the same real number depending on the nature of $V$. So in this case it seems that the infimum would be the smallest numerical/real value for $\| v - f \|_{\mathcal M}$ and that specific number is unique but it seems multiple different $v \in V$ could achieve it (at least in principle). So the value achieving the infimum seems to not be unique.
However, something else worries me. It worries me that each value of $v \in V$ decreases $\| v - f \|_{\mathcal M}$ a little bit more and more, making it a sequence that decreases forever and that approaches some value (say $L_{B_f}$). If this is true it seems that the infimum wouldn't exist, at least not using element from $V$, right? Would I have to change $V$ in this case? Do I really need the word infimum if its not achievable? I know that sometimes one cannot achieve the minimum so we use the word infimum. Is this one of those examples?
To make it clear my main worries are two:
- What the role of $\| v - f\|_{\mathcal M}$ and $V$ is. It seems that $\| v - f\|_{\mathcal M}$ is just for us to define the set we are looking to define a lower bound and that $V$ is just an index set.
- It seems that infimum has a small assumption that I just realized (that isn't clear until you "view it" as an optimization problem. In this case $\alpha = \inf S \iff \alpha = \inf_{ x \in P} S $ which is important to make explicit what "the minimization is over". It seems that the minimization is always over the set $P$ i.e. the larger set that $S$ is a susbet of. But if the minimization was over something else that did not have say, a largest element itself and we had: $ \inf_{x \in Q} S$ where we are looking to lower bound $S$ with $Q$ but $Q$ itself might a subset of $P$ but not have a largest element. In this case the "infimum" would not be defined. Maybe this is not what the definition of infimum is and it should be that we look for a lowest value that actually exists. Otherwise, there is largest lower bound...right?
Actually, you don't need to be concerned about the metric space at all. The metric space only serves as the index set for the collection of real numbers $|| v-f||_{\mathcal{M}}$.