Constraints: $a,b>0$
Here is what I have so far:
In order to get the shortest distance from the origin, we set $f(x,y,z)=x^2+y^2+z^2$ subject to the constrains $xyz=a$ and $y=bx$. By Lagrange multipliers, let $g(x,y,z,t,s)=x^2+y^2+z^2+t(xyz-a)+s(y-bx)$. We can solve
\begin{cases} \frac{\partial g}{\partial x}=2x+tyz-bs=0 \\ \frac{\partial g}{\partial y}=2y+txz+s=0 \\ \frac{\partial g}{\partial z}=2z+txy=0 \\ \frac{\partial g}{\partial t}=xyz-a=0 \\ \frac{\partial g}{\partial s}=y-bx=0, \end{cases}
and get the relation
\begin{cases} x^6=\frac{2a^2}{b^2(1+b^2)} \\ y=bx \\ z=\frac{a}{bx^2}. \end{cases}
Therefore, the shortest distance is $\sqrt{x^2+y^2+z^2}$
However, when I plug into the expression above, I get this very long expression. According to the textbook the answer should be $3\sqrt{\frac{a(1+b^2)}{2b}}$. Is it just a LOT of algebra that I need to do to get to the previous expression, or have I erred somewhere or is the textbook wrong (the textbook is known to have many errors)?
From $y=bx$ you get $x^2+y^2+z^2=x^2(1+b^2)+z^2$ and $xyz=a\iff xbxz=a\iff x^2=\frac{a}{bz}$, hence determine the extrema of $\frac{a}{b z}(1+b^2)+z^2$.