Here is Prob. 17, Sec. 6.1, in the book Introduction To Real Analysis by Robert G. Bartle & Donald R. Sherbert, 4th edition:
Let $f \colon I \to \mathbb{R}$ be differentiable at $c \in I$. Establish the Straddle Lemma. Give $\varepsilon > 0$ there exists $\delta (\varepsilon) > 0$ such that if $u, v \in I$ satisfy $c-\delta(\varepsilon)<u\leq c \leq v < c+\delta(\varepsilon)$, then we have $\left\lvert f(v) - f(u) - (v-u)f^\prime(c) \right\rvert \leq \varepsilon (v-u)$. [Hint: The $\delta(\varepsilon)$ is given by Definition 6.1.1. Subtract and add the term $f(c) - c f^\prime(c)$ on the left side and use the Triangle Inequality.]
Here is Definition 6.1.1 in Bartle & Sherbert, 4th edition:
Let $I \subset \mathbb{R}$ be an interval, let $f \colon I \to \mathbb{R}$, and let $c \in I$. We say that a real number $L$ is the derivative of $f$ at $c$ if given any $\varepsilon > 0$ there exists $\delta(\varepsilon)>0$ such that if $x \in I$ satisfies $0 < \lvert x-c \rvert < \delta(\varepsilon)$, then $$ \tag{1} \left\lvert \frac{ f(x) - f(c) }{ x-c} - L \right\rvert < \varepsilon. $$ In this case we say that $f$ is differentiable at $c$, and we write $f^\prime(c)$ for $L$.
In other words, the derivative of $f$ at $c$ is given by the limit $$ \tag{2} f^\prime(c) = \lim_{x \to c} \frac{ f(x) - f(c) }{ x-c} $$ provided this limit exists. (We allow the possibility that $c$ may be the endpoint of the interval.)
My Attempt:
As $f$ is differentiable at $c \in I$, so there is a real number $f^\prime(c)$ such that given any real number $\varepsilon > 0$ we can find a real number $\delta(\varepsilon) > 0$ such that $$ \left\lvert \frac{ f(x) - f(c) }{ x-c} - f^\prime(c) \right\rvert < \varepsilon \tag{1} $$ or $$ \left\lvert \frac{ f(x) - f(c) - (x-c) f^\prime(c) }{ x-c} \right\rvert < \varepsilon \tag{1'} $$ for all $x \in I$ which satisfy $$ 0 < \lvert x-c\rvert < \delta(\varepsilon), $$ which is equivalent to $$ c-\delta(\varepsilon) < x < c+ \delta(\varepsilon) \ \mbox{ and } \ x \neq c. $$
So if $x \in I$ and $0 < \lvert x-c \rvert < \delta(\varepsilon)$, then upon multiplying both sides of (1') by $\lvert x-c\rvert$, we get $$ \left\lvert f(x)-f(c) - (x-c)f^\prime(c) \right\rvert < \varepsilon \lvert x-c \rvert. \tag{2} $$ And for $x=c$ both sides of (2) equal $0$. Therefore we can conclude that $$ \left\lvert f(x)-f(c) - (x-c)f^\prime(c) \right\rvert \leq \varepsilon \lvert x - c \rvert \tag{2'} $$ for all $x \in I$ for which $c-\delta(\varepsilon) < x < c+\delta(\varepsilon)$.
From (2') we conclude that if $u, v \in I$ and $c-\delta(\varepsilon) < u \leq c \leq v < c + \delta(\varepsilon)$, then we have $$ \left\lvert f(u)-f(c) - (u-c)f^\prime(c) \right\rvert \leq \varepsilon \lvert u-c \rvert = \varepsilon ( c-u ) \tag{2*} $$ and also $$ \left\lvert f(v)-f(c) - (v-c)f^\prime(c) \right\rvert \leq \varepsilon \lvert v-c \rvert = \varepsilon (v-c), \tag{2**} $$ and therefore $$ \begin{align} & \ \ \ \ \left\lvert f(v) - f(u) - (v-u) f^\prime(c) \right\rvert \\ &= \left\lvert f(v) - f(c) + f(c) - f(u) - (v-c + c - u) f^\prime(c) \right\rvert \\ &= \left\lvert \left( f(v)-f(c) - (v-c)f^\prime(c) \right) + \left( f(c) - f(u) - (c-u)f^\prime(c) \right) \right\rvert \\ &\leq \left\lvert f(v)-f(c) - (v-c)f^\prime(c) \right\rvert + \left\lvert f(c) - f(u) - (c-u)f^\prime(c) \right\rvert \\ &\leq \varepsilon ( v-c ) + \varepsilon ( c-u ) \qquad \mbox{[ using (2*) and (2**) above ] } \\ &= \varepsilon (v-u). \end{align} $$
Is this proof good enough? Or, are there any problems in it?
Where has this lemma originated? Any applications of this lemma? Some references please.
Your proof is correct! It's better to ask separately about "Where has this lemma originated? Any applications of this lemma?"
Here's Bartle's partial solution in his manual on p 44.