Definitions
$f:\mathbb{R}\rightarrow \mathbb{R}$ is increasing if $x\leq y\implies f(x)\leq f(y)$.
$f:\mathbb{R}\rightarrow \mathbb{R}$ is called increasing at $p\in \mathbb{R}$ if there exists an $\epsilon >0$ such that $f(x)\leq f(p)\leq f(y)$ for all $x\in (p-\epsilon, p)$ and $y\in (p,p+\epsilon)$.
Proof
$\implies$
$(\forall \epsilon>0)(\forall p \in \mathbb{R})\big(x\in (p-\epsilon, p)\wedge y\in (p,p+\epsilon)\implies x\leq y\implies f(x)\leq f(y)\big)$
$\impliedby $
$(\forall c\in [x,y])(\exists \epsilon_c >0 )(\forall \alpha,\beta)\big(\alpha \in (c-\epsilon_c)\wedge \beta\in (c,c+\epsilon)\implies f(\alpha\leq f(c)\leq f(\beta)\big)$
Let $\mathcal U=\bigcup_{c\in [x,y]}(c-\epsilon_c,c+\epsilon_c)$. Then $[x,y]\subseteq \mathcal{U}$ and $f(x)\leq f(y)$.
I felt like my second proof has some logical gap in it. For example how to show that every two elements $a,b,$ in the open cover $\mathcal{U}$ has the property that $a\leq b\implies f(a)\leq f(b)$?
I found out a hint from question below that is related to my question but he didn't explain enough about how to use the compactness to globalized the local property.
Show that a function that is locally increasing is increasing?
My updated attempt:
Since $[x,y]$ is closed and bounded, there is a finite open subcover $\bigcup_{i=1}^n (c_n-\epsilon_n,c_n+\epsilon_n)$ that $f$ is increasing in each iterval.
So $x\in (c_i-\epsilon_i,c_i+\epsilon_i)$. All points $\tilde{x}$ greater than $x$ in this interval satisfy $f(\tilde{x})\geq f(x)$. Since this interval is open, $c_i+\epsilon_i$ belongs to another interval $c_j-\epsilon,c_j+\epsilon_j)$.
Now if we take union of these two sets $(c_i-\epsilon_i,c_i+\epsilon_i)\cup (c_j-\epsilon,c_j+\epsilon_j)$. All points $\tilde{x}$ greater than $x$ in this interval must continue to satisfy $f(\tilde{x})\geq f(x)$. This is because $\mathcal{I}_{ij}=(c_i-\epsilon_i,c_i+\epsilon_i)\cap (c_j-\epsilon_j,c_j+\epsilon_j)\neq \emptyset$. All points $\tilde{x}_i\in (c_i-\epsilon_i,c_i+\epsilon_i)$ is less than or equal to some points $\tilde{x}_{ij}\in \mathcal{I}_{ij}$ which in turn is less than or equal to some points $\tilde{x}_j\in (c_j-\epsilon_j,c_j+\epsilon_j)$. So we have $f(\tilde{x}_i)\leq f(\tilde{x}_{ij}\leq f(\tilde{x}_j)$.Thus, this new interval remains increasing the whole interval.
Repeat this procedure to obtain a larger interval. You will only need to do this procedure at most $n$ times to reach interval that covers $[x,y]$. Thus, $f$ is increasing in $[x,y]$. Since this is true for all $x,y$, so $f$ is increasing in $\mathbb{R}$.
Suppose that $f$ is increasing at every point but not increasing. Then there are numbers $x$ and $y$ susch that $x<y$ and $f(x)>f(y)$. Let$$S=\{t\in[x,y]\mid f(x)\leqslant f(t)\}.$$The set is is not empty (since $x\in S$) and it is bounded (since $S\subset[x,y]$). Let $s=\sup S$. If $s<y$, then ww know that $[s,s+\varepsilon_s)\subset S$, which contradicts the fact that $s=\sup S$. So, $s=y$. But this is impossible, since, for each $z\in(y-\varepsilon_y,y]$ $f(z)<f(y)<f(x)$ but, on the other hand, $(y-\varepsilon_y,y]$ must contain an element of $S$. So, we get a contradiction and therefore $x<y\implies f(x)\leqslant f(y)$.