Editted work.
I am attempting to find the maximum curvature for the space curve mentioned below.
space curve: $\vec{r}(t) = <3t, 4t, e^{2t}>$
From the space curve, I'm able to find that:
$\vec{r}' = <3,4,2e^{2t}>$
$|\vec{r}'| = \sqrt{25 + 4e^{4t}}$
$\vec{r}' \times \vec{r}'' = <16e^{2t}, -12e^{2t}, 0>$
$|\vec{r}' \times \vec{r}''| = 20e^{2t}$
By plugging in the above, the general curvature is:
$k = \frac{|\vec{r}' \times \vec{r}''|}{|\vec{r}'|^{3}} = \frac{20e^{2t}}{(25+4e^{4t})^{3/2}}$
Find the point of max curvature by:
Setting $\frac{dk(t)}{dt} = 0$ to find critical values.
Without loss of generality, set numerator of quotient rule = 0,
$[(25+4e^{4t})^{3/2}]\frac{d}{dt}(20e^{2t}) - 20e^{2t}\frac{d}{dt}[(25+4e^{4t})^{3/2}]$
Just to keep it simple, let $\Re = 25+4e^{4t}$,
$\Re^{3/2}(40e^{2t}) - 20e^{2t}[\frac{3}{2}\Re^{1/2}\frac{d\Re}{dt}] = 0$
Simplify by taking out the $20e^{2t}$,
$\Re^{3/2}(2e^{2t}) - \frac{3}{2}\Re^{1/2}\frac{d\Re}{dt}$
Simplify by taking out the $\Re^{1/2}$,
$\Re(2e^{2t}) - \frac{3}{2}\frac{d\Re}{dt} = 0$
With $\Re = 25 + 4e^{4t}$ and $\frac{d\Re}{dt} = 16e^{4t}$,
$(25+4e^{4t})(2e^{2t}) - \frac{3}{2}(16e^{4t}) = 0$
$\Rightarrow50e^{2t} + 8e^{6t} - 24e^{4t} = 0$
$\Rightarrow25 + 4e^{4t} - 12e^{2t} = 0$
When subbing in $x = e^{2t}$, I end up with unreal numbers when I try to solve, which cannot be used in my attempt to find max curvature. Can anyone try to point me in the right way or see if I somehow messed up in my calculations. Thank you!
It appears that the curvature formula should be $$ \kappa = \frac{\lVert r'\times r''\rVert}{\lVert r'\rVert^3} $$ From your equation you have $$ r' = (3,4,2\mathrm{e}^{2t}) $$ this leads to $$ \lVert r'\rVert = \sqrt{3^2 + 4^2 + 4\mathrm{e}^{4t}} = \sqrt{25 + 4\mathrm{e}^{4t}} $$ This is different to what you have in your equation above.
$$ \lVert r'\times r''\rVert = \left(\matrix{\mathbf{\mathrm{e}_{x}} & \mathbf{\mathrm{e}_{x}} & \mathbf{\mathrm{e}_{x}}\\ 3 & 4 & 2\mathrm{e}^{2t} \\ 0 & 0 & 4\mathrm{e}^{2t}}\right) = \lVert\left(16\mathrm{e}^{2t}, -12\mathrm{e}^{2t},0\right)\rVert = \sqrt{16^2\mathrm{e}^{4t} + (-12)^2\mathrm{e}^{4t}} = 20\mathrm{e}^{2t} $$ so you are trying to maximise $$ \kappa = \frac{20\mathrm{e}^{2t}}{(25 + 4\mathrm{e}^{4t})^{3/2}} $$ you can proceed like you did above $$ 2\kappa + 20\mathrm{e}^{2t}\frac{d}{dt}\frac{1}{(25 + 4\mathrm{e}^{4t})^{3/2}} = 2\kappa + 20\mathrm{e}^{2t}\left(-\frac{3}{2}(25 + 4\mathrm{e}^{4t})^{-5/2} \cdot 16\mathrm{e}^{4t}\right) $$ note I think you did not account for the fact you are working with (using your variable names) $\Re^{3/2}$ as opposed to $\Re^{-3/2}$ which have different properties when taking the derivative.
ultimately we end up with $$ \dot{\kappa} = 2\kappa -\frac{3}{2}\frac{16\mathrm{e}^{4t}}{25 + 4\mathrm{e}^{4t}}\kappa = 0 $$ so we end up with either $\kappa = 0$ which is trivial $$ 2 -\frac{3}{2}\frac{16\mathrm{e}^{4t}}{25 + 4\mathrm{e}^{4t}} = 0 $$ which ends up as $$ \mathrm{e}^{4t} = \frac{50}{16} = \frac{25}{8} $$