The Milstein scheme to approximate the solution of an SDE is $$ Y_{n+1} = Y_n + a\Delta_t + b\Delta W_t + \frac{1}{2} bb' ((\Delta W)^2 - \Delta) $$ where $\Delta_t$ is the time step size (usually equal for all $t$), $\Delta W_t = W_t - W_{t-1}, W_t$ denotes Wiener process, $b'(x) = d/dx \ \ b(x)$.
Suppose I want to use the Milstein scheme for the SDE $$ dX_t = aX_t \ dt + bX_t \ dW_t $$ where both $a$ and $b$ are constant. Will the Milstein scheme in this case just reduce to the Euler-Maruyama scheme, since $b' = 0$?
no
Here the $b$ in the first equation is a function $b(x).$ In the second equation the $b$ is a constant and a different $b$ and in fact $b(x)$ in the first sense equals $b$ times $x$ in the second sense.
I.e. if we change notation for the second SDE
$$ dX_t = \mu X_t \ dt + \sigma X_t \ dW_t $$
then $b(x) = \sigma x$ and $b' = \sigma. $
If you want to solve this SDE better to work out the process for $\log X_t$ and solve that directly.