I am trying to understand how to convert the general form of an ellipsoid equation: $$ (x^2 + y^2)/a^2 + (z^2)b^2 = 1 $$ to the equation required by SolTrace: $$ Z(x,y) = \frac{c(x^2 + y^2)}{1 + [1 - \kappa c^2(x^2 + y^2)]^{0.5}} $$
could anyone explain what $\kappa$ and $c$ are? 
May I assume that you meant $z^2/b^2$ rather than $\left(z^2\right)b^2$ in your first equation? That would make more sense to me because then $a$ and $b$ would be the half-axes of the ellipsoid. If so, please correct your post.
Assuming that, the first change to make is to replace $z$ with $Z-b$ so that in the $(x,y,Z)$ coordinate system the center is at $(0,0,b)$ as shown in the diagram. The next thing to do is solve for $Z$, taking the negative square root in order to get the bottom half of the ellipsoid.
The last trick you are going to need in order to reach the stated form is to simultaneously multiply and divide $1-\sqrt{1-\frac{x^2+y^2}{a^2}}$ by $1+\sqrt{1-\frac{x^2+y^2}{a^2}}$ and then simplify. Matching the two forms, you should find that $\kappa=\frac{a^2}{b^2}$. You should then be able to understand why $\kappa<1$ and $\kappa>1$ correspond to the given images.