
I tried the substitution but got for the second bracket one plus k squared times x squared, and why did the G term disappear. Thank you.

I tried the substitution but got for the second bracket one plus k squared times x squared, and why did the G term disappear. Thank you.
First, the reason is that it's an integral of the form $\sqrt{G.....}$, so the G is just a coefficient and nonessential, so they just omitted it in the next line, because they were focusing on the substitution.
Also, there is a negative sign because they are substituting $\displaystyle k^2 = -\frac{n}{m}$. Perhaps you may have accidentally forgotten to include the negative.