Imagine I am trying to optimise some function of the variables $x_i$ with Lagrange's method of undetermined multipliers and arrive at the following relation
$$ \sum_i c_i dx_i = 0 $$
I've always accepted the argument that each coefficient must evaluate to 0 as very intuitive, but I don't think I can prove it.
My idea is that if you want your problem to have a unique solution then the one with all $c_i = 0$ must be the one.
What is the exact reason behind the idea that all coefficients must be zero?
The notation you are using is most likely to be confusing for you if you are not properly comfortable with the theory of differential forms, which I doubt you are if you are asking a question about Lagrange multipliers. Let us stick to traditional multivariable calculus notation.
We are looking at extremizing the scalar function $f$ subject to the scalar constraint $g=0$.
The first point is that the directional derivative of $f$ in the direction of a unit vector $u$ is $\nabla f \cdot u$. Therefore if $\nabla f \cdot u$ is positive for a $u$ in the tangent (hyper)plane of the surface $g=0$, then one can follow the direction of $u$ to increase $f$ and follow the direction of $-u$ to decrease $f$. Thus such a point cannot be an extremum.
The second point is that the tangent (hyper)plane of the surface $g=0$ is exactly the vectors which are perpendicular to $\nabla g$. Thus we need to have $\nabla f \cdot u = 0$ whenever $u$ is perpendicular to $\nabla g$. One can prove using linear algebra that this is equivalent to $\nabla f$ being parallel to $\nabla g$ (for instance one can use the rank-nullity theorem).
For a careful proof one should notice that we actually cannot typically move in the direction of the tangent (hyper)plane and stay on the surface $g=0$ (unless $g=0$ is a plane, or at least is locally a plane). But you can move in such a direction and then project back down to the surface and still find a point with a larger/smaller value of $f$.