Let $$I(u)=\frac{1}{2}\int_{(a,b)} u'^2\quad \text{and}\quad J(u)=\int_{(a,b)} u^2,$$ with $u\in W_0^2(a,b)$. I want to minimize $$H(u)=\frac{I(u)}{J(u)}.$$
Way 1 :
Using Lagrange multiplier, I know that there is a constant $\lambda $ s.t. $$\int_{(a,b)} u'v'=2\lambda \int_{(a,b)} uv,$$ for all $v\in H_0^1(a,b)$.
Way 2 :
I minimize $\log(H(u))=\log(I(u))-\log(J(u))$. $$\frac{H'(u)}{H(u)}=\frac{\int_{(a,b)} u'v'}{\int_{(a,b)}u'^2}-\frac{2\int_{(a,b)}uv}{\int_{(a,b)}u^2},$$ and thus a necessary condition is $$\frac{H'(u)}{H(u)}=0\iff \int_{(a,b)}u'v'=2\frac{\int_{(a,b)}u'^2}{\int_{(a,b)}u^2}\int_{(a,b)}uv.$$
So, to be coherent with the way 1 I suppose that I can set $\lambda =\frac{\int_{(a,b)} u'^2}{\int_{(a,b)} u^2}.$ But my problem is that such a $\lambda $ depending on $u$, and thus I don't really understand why we can say that $\lambda $ is constant.
The multiplier $\lambda$ being a constant means it is not a function of $t$ or whatever your variable is called. Think of a finite dimensional case: if we minimize, say, $\sum x_i^4$ subject to $\sum x_i^2 = 1$, the Lagrange multiplier method tells us that the gradient of the first function is a multiple of the second: $$ (4x_1^3,\dots, 4x_n^3) = \lambda(2x_1,\dots, 2x_n) \tag1 $$ Here $\lambda$ is constant in the sense that we do not have separate $\lambda_1, \lambda_2,\dots$ for the components $x_1, x_2, \dots$. But the value of $\lambda$ may well depend on $x$ in the sense that the system (1) could have multiple solutions (multiple critical points on the hypersurface $\sum x_i^2 = 1$), and at those points the values of $\lambda$ will probably be different.
Same in your case. In case that multiple functions $u$ are stationary for this extremal problem, each will have its own constant $\lambda$.
Both of your approaches yield the same result. The first one, with $$ \int_{(a,b)} u'v'=2\lambda \int_{(a,b)} uv,\quad \forall v\in H^1_0 $$ also tells you what $\lambda$ is, if you put $v=u$ in this equation.
The end result is that $u''+2\lambda u = 0 $ in the weak sense; but the equation tells us that the regularity of $u$ is self-improving (if $u$ is square-integrable, so is $u''$, but then $u$ is better, and then $u''$ is better, etc), so $u\in C^\infty$ and in fact $u$ is a trigonometric function with zero values at the endpoints.