For example, let $$ I[u]=\int_M |\nabla u|^2+u dV $$ It's not hard to compute the variation of $I[u]$. If $I[u]$ reach the minimum at $u_0$, I can get that $$ i(t)=I[u_0+tv] \\ i'(0)=0 \\ i'(0)=\int_MvdV $$
But if add some constraint condition, for example $$ \int_M u^2dV=1 $$ How to compute the variation ?
Actually, in your first example, I would expect that (without doing the calculations)
$$i^\prime(0) = -\int_M (2\Delta u -1)vdV$$ assuming $M$ is compact without boundary.
The case of constraints can often be dealt with by using Lagrange Multipliers. The side condition (assuming it is a regular continuously differentiable functional in a neighbourhood of a minimizer or critical point) defines a submanifold of codimension $1$ (or possibly higher, if you have more than one constraint) in which the solution is searched for. Usually this means that (in a Hilbert space setting) the gradient of the functional which defines the constraint is orthogonal to the set defined by the side conditions. See, e.g., https://en.wikipedia.org/wiki/Lagrange_multiplier and/or do a Web search for 'Lagrange multiplier calculus of variations'.