In the proof of lemma 2 in Gelfand & Fomin, he says that if $$\int^b_a[\alpha (x)-c]^2dx=0,$$ then it follows that $\alpha (x)-c=0$ for all $x\in[a,b]$, where $\alpha(x)$ is assumed to be continuous in $[a,b]$.
I can't figure out why this is the case.
I know in some special case the integral is zero will imply the integrand is zero, but I can't find the exact theorem I need in the web.'
I am writting a more verbose answer here using the answer provided below.
Suppose $\alpha(x_0)-c>0$ for some $x_0\in [a,b]$. By continuity of $(\alpha(x)-c)$ at $x_0$, there exists $\delta>0$ such that, $$\text{for }\epsilon= \frac{\alpha(x_0)-c}{2}, |x-x_0|<\epsilon \implies \alpha(x)-c>\frac{\alpha(x_0)-c}{2}>0\text{ for all }x\in (x_0-\delta, x_0+\delta).$$
Next, $$\int^b_a[\alpha(x)-c]^2dx=\int^{x_0-\delta}_a [\alpha (x)-c]+\int^{x_0+\delta}_{x_0-\delta} [\alpha (x)-c]+\int^{b}_{x_0+\delta} [\alpha (x)-c]\geq \int^{x_0+\delta}_{x_0-\delta} [\alpha (x)-c]$$
And $$ \int^{x_0+\delta}_{x_0-\delta} [\alpha (x)-c]^2dx\geq \int^{x_0+\delta}_{x_0-\delta} \Big[\frac{\alpha (x_0)-c}{2}\Big]^2dx=2\delta \Big[\frac{\alpha(x_0)-c}{2}\Big]^2>0.$$ This contradicts our hypothesis that $\int^b_a[\alpha (x)-c]^2dx=0$. Noted the first inequality above uses the monotonicity of riemann integral.
Suppose $\;\alpha(x_0)-c\neq 0\;$ for some $\;x_0\in [a,b]\implies\;$ by continuity of $\;\alpha(x)\;$ and thus of $\;\alpha(x)-c\;$ , there exists an interval $\;I_0:=(x_0-\epsilon,\,x_0+\epsilon)\subset[a,b]\;$ s.t. $\;\alpha(x)-c\neq0\;\;\forall\,x\in I_0\;$ , and thus
$$\int_a^b|\alpha(x)-c|\,dx\ge\int_{x_0-\epsilon}^{x_0+\epsilon}|\alpha(x)-c|\,dx>0\implies\text{ contradiction}$$
Fill in details, for example: how to (slightly) mend the above if $\;x_0=a\;\,or\,\;\,x_0=b\;$ and etc.