Let $f\in \mathcal C^2([a,b]\times \mathbb R\times \mathbb R)$. I have to show that if $\bar u\in \mathcal C^2([a,b])\cap X$ is a minimizer of $$\inf_{u\in X}\left\{\int_a^b f(x,u(x),u'(x))dx\right\},$$ for $X=\{u\in \mathcal C^1([a,b])\mid u(a)=\alpha \}$, then $$\begin{cases}\frac{d}{dx}f_\xi=f_u\\ f_\xi(b,\bar u(b),\bar u'(b))=\alpha \end{cases}.$$
We suppose WLOG that $u(a)=0$. The proof goes as the proof of Euler-Lagrange Equation. Except that at the end, I get $$\int_a^b \left(f_u-\frac{d}{dx}f_\xi\right)v+f_{\xi}(b, \bar u(b),\bar u'(b))v(b)=0.$$ for all $v\in X$. Now, it's written that using fundamental theorem of calculs of variation, we get $$\begin{cases}\frac{d}{dx}f_\xi=f_u\\ f_\xi(b,\bar u(b),\bar u'(b))=0\end{cases}.$$
I would agree if $$\int_a^b \left(f_u-\frac{d}{dx}f_\xi\right)v=0,$$ but it's not what we have... so how can it work ?
The fundamental lemma says this (see : https://en.wikipedia.org/wiki/Fundamental_lemma_of_calculus_of_variations )
If : $\int_a^b f(x)h(x) dx = 0 $ for all compactly supported smooth functions $h(x) \text{ on } (a,b)$ , then this implies $\implies f(x)=0$.
Your $'h(x)'=v$ , and your $'f(x)'=f_u-\frac{d}{dx}f_\xi $ .
Your question was why can we assume $\int_a^b \left(f_u-\frac{d}{dx}f_\xi\right)v dx$ must be $0$ ?
"Compactly supported" means "vanishes outside" $(c,d)$ for some $c,d$ such that $a<c<d<b$. So in that case the fundamental theorem applies.
When you say : $ \forall \enspace v \in X : \int_a^b \left(f_u-\frac{d}{dx}f_\xi\right)v dx +f_{\xi}(b, \bar u(b),\bar u'(b))v(b)=0 $ that means that also this is valid for all compactly supported $v \in X$. But for all compactly supported $v$ we have $v(b)=0 $ implying that : $f_{\xi}(b, \bar u(b),\bar u'(b))v(b)$ must be zero for all compactly supported $v$. Then we see that $\int_a^b \left(f_u-\frac{d}{dx}f_\xi\right)v dx$ must also be zero for all compactly supported $v$ and we can apply the fundamental theorem.
It follows that $f_u-\frac{d}{dx}f_\xi=0$ and then also : $f_{\xi}(b, \bar u(b),\bar u'(b))=0$.