My Question: Find first and second variation of following functional $$J(u)=\int_{t_0}^{t_1}(1+u'^2)\sqrt{u}dt$$
My Answer: I found first variation of the functional as $$\delta J=\int_{t_0}^{t_1}\big[\frac{(1-u'^2)}{2\sqrt{u}}- 2u''\sqrt{u} \big]\delta u dt$$ Is it right?
But I can' t find second variation of the functional. Could you help me?
Your expression for the first variation is right. The integrand for the second variation is given by
$$ \frac14\frac{(\frac{d}{dt}u(t))^2-4(\frac{d^2}{dt^2}u(t))u(t)-1}{u(t)^{3/2}} $$
This result is obtained using the Euler-Lagrange equation with the form
$$\frac{\partial L}{\partial u}-\frac{d}{dt}\left(\frac{\partial L}{\partial u'}\right)+\frac{d^2}{dt}\left(\frac{\partial L}{\partial u''}\right)=0$$
where $L$ is given by the integrand in the first variation
$${\color{blue}L}=\frac{(1-u'^2)}{2\sqrt u}-2u''\sqrt{u}$$
More formally we write
$$\delta^2 J=\int_{t_0}^{t_1}\big[\frac14\frac{(\frac{d}{dt}u(t))^2-4(\frac{d^2}{dt^2}u(t))u(t)-1}{u(t)^{3/2}}]\delta^2 u dt$$