Consider a continuous function $F:\Bbb R_{\ge0}\times\Bbb R_{\ge0}\setminus\Delta_{\Bbb R_{\ge0}^2}\to\Bbb R_{\ge0}$ where $\Delta_{\Bbb R_{\ge0}^2}:=\{(x,x):x\ge0\}$ is the diagonal of $\Bbb R_{\ge0}^2$.
Suppose moreover $F$ symmetric, that is $F(t,s)=F(s,t)\;\;\forall (s,t)\in\Bbb R_{\ge0}\times\Bbb R_{\ge0}\setminus\Delta_{\Bbb R_{\ge0}^2}$ and $F(0,t)>0$ for all $t>0$.
Is it true that $$ \lim_{T\to0+}\left[\sup_{0<t\le T}F(0,t)\right] =\lim_{T\to0+}\left[\sup_{0\le s<t\le T}F(s,t)\right]\;\;? $$ It seems clear that RHS is $\ge$ than LHS; is the converse true?
I have a proof, but in the comments there is a counterexample: where is the problem?
PROOF: We distinguish two cases for the value of RHS.
Let us suppose first $\lim_{T\to0+}\left[\sup_{0\le s<t\le T}F(s,t)\right]=c\in\Bbb R_{\ge0}$.
If $c=0$ then the conclusion is trivial. Suppose thus $c>0$.
Now for every $n\in\Bbb N$ there exists $\delta_n>0$ such that $$ \left|\sup_{0\le s<t\le\delta_n}F(s,t)-c\right|<\frac1n $$ and for every $n\in\Bbb N$, fixed $\delta_n$, there exists $\{(s_m^{(\delta_n)},t_m^{(\delta_n)})\}_{m\ge1}\subset\Bbb R_{\ge0}^2\setminus\Delta_{\Bbb R_{\ge0}^2}$ such that $$ F(s_m^{(\delta_n)},t_m^{(\delta_n)})\stackrel{m\to+\infty}{\longrightarrow} \sup_{0\le s<t\le\delta_n}F(s,t). $$ from which we get $$ \lim_{n\to+\infty}F(s_n^{(\delta_n)},t_n^{(\delta_n)}) =\lim_{T\to0+}\left[\sup_{0\le s<t\le T}F(s,t)\right]=c\; $$ (moreover it is clear that $s_n^{(\delta_n)},t_n^{(\delta_n)},\delta_n\to0$). Then, by continuity, $\forall\varepsilon>0\;\;\exists N_{\varepsilon}\ge1$ such that $$ \left|F(0,t_n^{(\delta_n)})-F(s_n^{(\delta_n)},t_n^{(\delta_n)})\right|<\frac{\varepsilon}2\;\;\;\;\;\forall n\ge N_{\varepsilon} $$ and $$ \left|F(s_n^{(\delta_n)},t_n^{(\delta_n)})-c\right|<\frac{\varepsilon}2\;\;\;\;\;\forall n\ge N_{\varepsilon}, $$ from which we get $$ \left|F(0,t_n^{(\delta_n)})-c\right|<\varepsilon\;\;\;\;\;\forall n\ge N_{\varepsilon}, $$ and thus $$ \lim_{T\to0+}\left[\sup_{0<t\le T}F(0,t)\right] =\lim_{n\to+\infty}\left[\sup_{0<t\le t_n^{(\delta_n)}}F(0,t)\right] \ge \lim_{n\to+\infty}F(0,t_n^{(\delta_n)})=c $$ which allows to conclude, since clearly $\lim_{T\to0+}\left[\sup_{0<t\le T}F(0,t)\right]\le c$.
The case $\lim_{T\to0+}\left[\sup_{0\le s<t\le T}F(s,t)\right]=+\infty$ is only slightly different: since $$ T\mapsto\sup_{0\le s<t\le T}F(s,t) $$ is not decresing, it is clear that $$ \sup_{0\le s<t\le T}F(s,t)=+\infty\;\;\;\forall T>0. $$ In particular fixing $n\ge1$ we have that $\sup_{0\le s<t\le \frac1n}F(s,t)=+\infty$, thus there exists $\{(s_m^{(n)},t_m^{(n)})\}_{m\ge1}\subset\Bbb R_{\ge0}^2\setminus\Delta_{\Bbb R_{\ge0}^2}$ such that $$ \lim_{m\to+\infty}F (s_m^{(n)},t_m^{(n)})=+\infty, $$ and since this holds true for every $n\ge1$ we can write \begin{equation} \lim_{n\to+\infty}F (s_n^{(n)},t_n^{(n)})=+\infty. \end{equation} Now, being clearly $s_n^{(n)},t_n^{(n)}\to0$ as $n\to+\infty$ and since $F$ is continuous, $\forall\varepsilon>0$ $\exists N_{\varepsilon}\ge1$ such that $$ \left|F(0,t_n^{(n)})-F(s_n^{(n)},t_n^{(n)})\right|<\varepsilon\;\;\;\;\;\forall n\ge N_{\varepsilon}; $$ in particular we get $$ F(0,t_n^{(n)})>F(s_n^{(n)},t_n^{(n)})-\varepsilon\;\;\;\;\;\forall n\ge N_{\varepsilon}, $$ from which $$ \lim_{n\to+\infty}F (0,t_n^{(n)})=+\infty. $$ Then we can conclude as in the previous case: $$ \lim_{T\to0+}\left[\sup_{0<t\le T}F(0,t)\right] =\lim_{n\to+\infty}\left[\sup_{0<t\le t_n^{(n)}}F(0,t)\right] \ge \lim_{n\to+\infty}F(0,t_n^{(n)})=+\infty. $$
FINALLY: I wanted to treat my problem in general, but originally it was $$ F(s,t)=\frac{|f(t)-f(s)|}{|t-s|^{\lambda}} $$ for some $0<\lambda\le1$ and for a function $f$ which is $\alpha$-Holder continuous for some $\alpha$.
THE MOST IMPORTANT THING: If my proof is wrong, what kind of hypothesis we can put in order to obtain the result?
THE GENESIS OF THIS POST: $f$ originally was the difference of two solutions of the following scalar SDE: $$ x_t=\underbrace{\xi_0+\int_0^tb(s,x_s)\,ds+\int_0^t\sigma(s,x_s)\,dW_s^H}_{=:z_t}+y_t\; $$ where $$ y_t=\sup_{s\in[0,t]}\left(z_s\right)^{-}. $$ and $$ b:\Bbb R_{\ge0}\times\Bbb R\to\Bbb R $$ and $$ \sigma:\Bbb R_{\ge0}\times\Bbb R\to\Bbb R $$ are bounded measurable functions, called drift and diffusion coefficient respectively. We will assume the following hypotesis on them:
$$ |b(t,x)-b(t,y)|\le K_0|x-y|\;\;\forall x,y\in\Bbb R,\;\forall t\in[0,T] $$
$$ |\sigma(t,x)-\sigma(t,y)|\le K_0|x-y|\;\;\forall x,y\in\Bbb R,\;\forall t\in[0,T] $$
$$ |\sigma(t,x)-\sigma(s,x)|\le K_0|t-s|^{\nu}\;\;\forall x\in\Bbb R,\;\forall s,t\in[0,T] $$ and $W^H$ is the fractional Brownian motion of Hurst index $1/2<H<1$; moreover the integral is the YOUNG INTEGRAL, which is defined as the limit of the usual Riemann Sums, or it can be equivalently viewed as follows (here we deal with deterministic case: we think to $W^H$ as a fixed path of the fBM): if $h$ is a $\lambda$-Holder real function, for $\lambda>1-H$, then it can be proved that $$ \int_s^th_u\,dW_u^H=h_s(W_t^H-W_s^H)+\Lambda_{st}((h_c-h_a)(W_b^H-W_c^H)) $$ where $a,c,b$ are mute variables and $\Lambda:\mathcal{ZC}_3^{\mu}\to\mathcal{C}_2^{\mu}$ is the inverse of $\delta:\mathcal{C}_2^{\mu}\to\mathcal{ZC}_3^{\mu}$ which is defined as $$ (\delta h)_{sut}:=-h_{ut}+h_{st}-h_{su} $$ and $$ \mathscr C_2^{\mu}:=\{h\in\mathscr{C}_{2}\;:\;\|h\|_{\mu}<+\infty\} $$ where $\mathscr{C}_{2}$ is the $\Bbb R$-vector space of all functions $h:[0,T]^2\to\Bbb R$ continuous and such that they vanish on diagonal. Moreover $$ \|h\|_{\mu}:=\sup_{\substack{s,t\in[0,T]\\s\neq t}}\frac{|h_{ts}|}{|t-s|^{\mu}} $$ and $$ \mathscr C_3^{\mu}:=\{h\in\mathscr{C}_{3}\;:\;\|h\|_{\mu}<+\infty\} $$ where $\mathscr{C}_{3}$ is the $\Bbb R$-vector space of all functions $h:[0,T]^3\to\Bbb R$ continuous such that $h_{sut}=0$ whenever $s=u$ or $u=t$ and $\|\cdot\|_{\mu}$ is a suitable norm.
Thus \begin{align*} f_t &=x_t^{(1)}-x_t^{(2)}\\ &=\int_0^t(b(u,x_u^{(1)})-b(u,x_u^{(2)}))\,du+ \int_0^t(\sigma(u,x_u^{(1)})-\sigma(u,x_u^{(2)}))\,dW_u^H +y_t^{(1)}-y_t^{(2)} \end{align*}
and $$ F(0,t)=\frac{|f_t|}{t^H} $$ IMPORTANT since the original exponent is $\rho>1$, then it would be enough to prove that this last limit is nonzero, in fact $$ \frac{|f_t|}{t^{\rho}}=\frac{|f_t|}{t^{H}}\frac1{t^{\rho-H}} $$
The statement "Then by continuity, $\forall \epsilon>0$, $\exists N_\epsilon$..." is wrong. If a function $F$ is continuous on an open set and $\xi_n$ is a sequence converging to a boundary point then $F(\xi_n)$ need not be Cauchy (which in fact is the claim you make). For the argument to hold you need uniform continuity, but then $F$ extends continuously to $(0,0)$ which is not what you are interested in. A concrete example is: $ F(s,t) = \frac{s}{t}$ for $0\leq s<t$ and $F(s,t) = \frac{t}{s}$ for $0\leq t < s$. You have $\limsup_T F(0,T)=0$ and $\limsup_{s,t} F(s,t)=1$.
The application you have in mind is, however, somewhat different. So let $$ F(s,t) = \frac{f(s)-f(t)}{|s-t|^\lambda}$$ with $f$ an $\alpha$-Hölder continuous function.
If $ 0 < \lambda < \alpha \leq 1 $ then $F$ is in fact uniformly continuous, since you have: $$ |F(s,t) | \leq \frac{ C|t-s|^\alpha}{|t-s|^\lambda} = C |t-s|^{\alpha-\lambda}.$$ so there is no problem.
In the case $0<\lambda=\alpha\leq 1$ the conclusion is, however, wrong. To constuct a counter-example, fix $k>0$ and define for $A>0$ the spike of amplitude $A$: $$ \phi_A(x) = \max\{0, A - k |x|^\alpha\}$$ This non-negative function is bounded by $A$, has support in $|x|\leq (A/k)^{1/\lambda}$ and has $\alpha$-Hölder constant $k$ (independent of $A$): $$ \sup_{x\neq y} \frac{|\phi_A(x)-\phi_A(y)|}{|x-y|^\alpha} = k .$$
Now, define $$ f(t) = \sum_{n\geq 1} \phi_{\frac{1}{n^{2\lambda}}} (t-\frac{1}{n}) .$$ The spikes have disjoint support for $k$ large enough and $f(1/n)=1/n^{2\lambda}$ so the function verifies: $f(t)\leq t^{2\lambda}$, $t\geq 0$. In particular, $$ \limsup_{T\rightarrow 0^+} F(0,T) = \limsup_T f(T)/T^\lambda \leq C \limsup_T T^{2\lambda-\lambda} = 0. $$ On the other hand we have when $\alpha=\lambda$: $$ \limsup_{T\rightarrow 0^+} \sup_{0<t<s\leq T} F(t,s)= \limsup_T \sup_{0<t<s\leq T} \frac{|f(s)-f(t))|}{|s-t|^\lambda} = k $$ since arbitrarily close to the origin we have a spike with Hölder constant $k$. By letting $k$ increase with $n$ you may also construct an example where the first limsup is zero and the last infinity. In other words in order to obtain the result you need better control on the function $f$ when approaching the origin.
According to one of your comments elsewhere on this page it looks as if you may be interested in the case $\alpha=\lambda=1/2$ and $f(t)$ being e.g. a standard Brownian motion. In that case I believe that a.s. both limits are infinity (so in a sense you are then right). It is well-know that the $\limsup_{T}\sup_{s,t}$ is infinity. This follows for example from Lévy's modulus of continuity. see e.g. Peres and Morters, Thm 1.14. I believe that is also true a.s. for the point wise limit $\limsup_T F(0,T)$ but couldn't find a direct proof. There exists points where the limit is finite (sometimes called $\alpha$ slow times) but it seems that they have zero Lebesgue measure, although possibly Hausdorff dim 1 (?). But I am far from sure about this last point.