My question might seem related to this one. However, that question asks how $s=1$ was obtained and why $t>s$ was introduced. My question, on the other hand, is about why $s=1$ proves the theorem. I tried to revive that question to no avail. If anybody can give an intelligible, full answer to that question, I'll delete my post.
So, here's my question.
I'm following Hormander's "The analysis of linear partial differential operators I".
Hormander states the theorem as follows:
$$||\vec f(y)-\vec f(x)|| \leq |y-x| \sup \{||\vec f'(x+t(y-x))||, 0 \leq t \leq 1 \}$$
Then he gives the proof as follows.
Let $ M > sup \{ ||\vec f'(x+t(y-x))||, 0 \leq t \leq 1 \} $ and set $ E = \{ t; 0 \leq t \leq 1, ||\vec f(x+t(y-x))-f(x)|| \leq Mt|x-y| \} $.
Then, he justifies that $E$ has a largest element and says that if $t>s$ and $t-s$ is sufficiently small, we have
$$ ||\vec f(x+t(y-x)) - \vec f(x)|| \leq ||\vec f(x+t(y-x) - \vec f(x+s(y-x)|| + ||\vec f(x+s(y-x)) - \vec f(x)|| \leq M|(t-s)(y-x)| + Ms|y-x| = Mt|y-x| $$
Then he concludes that $s=1$ and claims it proves the theorem.
I have managed to understand why it follows from the given inequality that $s=1$. What I can't understand is why that proves the theorem.
Here's how I decipher Hormander's mathematics in this theorem.
The theorem states that the approximation can be less than or equal to the derivative. The proof, then, goes as what if we could find cases where it is bigger. Assume the approximation is bigger than the derivative, but, still, less than some arbitrary number $M$. Assume, there exists the biggest segment of the function - between the values of the parameter $t=0$ and $t=s<1$ - where the approximation is bigger than the derivative but smaller than $M$. We say, then, what if we could find a bigger segment for which the approximation is bigger than the derivative and smaller than $M$. And, indeed, we can. We keep doing that till we reach $s=1$. Then, we can state that for the entire segment of the function - between $t=0$ and $t=1$ - it is true that $||\vec f(x+t(y-x))-f(x)|| \leq Mt|x-y|$.
According to my logic, I do not see how it proves the theorem. The only thing it proves is that the approximation is always smaller than some arbitrary number $M$. But since $M$ is bigger than the derivative, the approximation can still be bigger than the derivative. Moreover, I can do the same trick with any number $M$: bigger, equal to or smaller than the derivative. In other words, we picked $M$ and did not connect it to the derivative in the course of the proof.
P.S. When I say "approximation", I mean $||\vec f(y)-\vec f(x)|| / |y-x|$. When I say "derivative", I mean $sup \{||\vec f'(x+t(y-x))||, 0 \leq t \leq 1 \}$.
My comments to the accepted answer and to what I was confused about.
I'm not a mathematician, I'm a mechanical engineer. We always derive equations. We want to understand equations deeply, fundamentally, understand what the "physics" behind them is. That's what we do in all our mechanical engineering courses. I wasn't even thinking about it, that was something I wasn't realizing, something I learnt subconsciously. Thus, when I encountered the proof of the mean value inequality, I expected to see the derivation of the mean value inequality.
That's what I was looking for. I couldn't understand that one can proof a theorem without the complete derivation. That's why I was trying to find a deeper meaning of going from $M$ to $M_0$: I thought the derivation was hidden somewhere there.
It turns out, it is a rather common trick in mathematics to prove something not by deriving it, but by showing that it is true.
That was a, fundamentally, unfamiliar and unintuitive approach to me.
Here's what is done in the proof. We assumed $M > sup \{ ||\vec f' (x + t(y-x)||, 0 \leq t \leq 1 \}$. And we proved $||\vec f(y) - \vec f(x)|| > M |x-y|$. If it works for $M$, it works for $M_0$ as well (see very nice @JackyChong's comment on that). That's it. That's all we want: to show that the original inequality $$||\vec f(y)-\vec f(x)|| \leq |y-x| \sup \{||\vec f'(x+t(y-x))||, 0 \leq t \leq 1 \}$$ does hold. We don't care about deriving it. We simply showed that it is true. It is very simple and very different from what I got used to. That is why I just couldn't wrap my head around this absolutely simple idea: don't derive, just show that it works.
I think this post can be of big importance for the people, like me, who are coming to mathematics without proper mathematical training because it explains this fundamental idea of mathematical proves.
I believe this is a proof by the continuity method. I will help you by filling out the gaps in the proof.
Fix $x, y \in I$ with $x\ne y$ and consider $M_0:=\sup_{0\le t\le 1}\|f'(x+t(y-x))\|$. If $M_0=\infty$, then there is nothing to prove since $\|f(x)-f(y)\|$ is finite and $|x-y|\ne 0$.
Assume $M_0<\infty$. Fix $M>M_0$ and define the set \begin{align} E = \{t \in [0, 1]\mid \|f(x+\tau (y-x))-f(x)\|\le M \tau|x-y| \quad \text{ for all } \tau \le t\}. \end{align} Clearly, $E$ is nonempty since $0 \in E$. Let us also note that $E$ is closed in $[0, 1]$: suppose $\{t_n\} \subset E$ such that $t_n \rightarrow t_\ast \in [0, 1]$, then it follows from the continuity of $f$ that \begin{align} \|f(x+t_\ast(y-x))-f(x)\|=\lim_{n\rightarrow \infty}\|f(x+t_n(y-x))-f(x)\|\le \lim_{n\rightarrow \infty} M t_n|x-y| = M t_\ast |x-y|. \end{align}
Also, since $E\subset [0, 1]$, then $E$ has a supremum, say $s$. But since $E$ is closed then $s \in E$.
Here's the proof. Assume by contradiction that $s<1$, then we can choose $t\in (s, 1)$, i.e. $t>s$, with $t-s$ sufficiently small (let us quantify this later). Hence it follows that \begin{align} \|f(x+t(y-x))-f(x)\| \le \|f(x+s(y-x))-f(x)\|+\|f(x+t(y-x))-f(x+s(y-x))\|=: I_1 + I_2. \end{align}
Notice that the first term $I_1$ is readily bounded by $Ms|x-y|$ since $s \in E$, i.e. \begin{align} \|f(x+s(y-x))-f(x)\| \le M s|x-y|. \end{align} For the second term $I_2$, by the definition of differentiability, we have that \begin{align} \|f(x+t(y-x))-f(x+s(y-x))-f'(x+s(y-x))(t-s)(y-x)\|=o(|(t-s)(y-x)|) \end{align} when $t-s$ is sufficiently small, i.e. we have that \begin{align} \|f(x+t(y-x))-f(x+s(y-x))\| \le |(t-s)(y-x)|\|f'(x+s(y-x))\|+o(|(t-s)(y-x)|) \le M |(t-s)(y-x)| \end{align} when $t-s$ is sufficiently small.
This means that \begin{align} \|f(x+t(y-x))-f(x)\| \le&\, \|f(x+s(y-x))-f(x)\|+\|f(x+t(y-x))-f(x+s(y-x))\|\\ \le&\, Ms |y-x| + M(t-s)|y-x| = M t|y-x|. \end{align}
Well, here we can conclude that there exists $t>s$ such that \begin{align} \|f(x+t(y-x))-f(x)\|\le Mt|y-x| \end{align} which contradicts the fact that $s$ is the supremum of $E$. Hence $s\ge 1$. But since we are working only in $[0, 1]$, then it follows that $s=1$.
Thus, we have shown that for any $M>M_0$, we have the inequality \begin{align} \|f(y)-f(x)\|\le M|y-x|. \end{align} Finally, take the limit as $M\rightarrow M_0$, we arrive at the desired theorem.