I'm trying to find the Möbius transform that maps the circles in the image below ($|z| = 6$, and $|z- 4.5i| = 1.5$) to concentric circles. Comparing with a similar problem where the circles lie on the real axis, I came up with:
$$w = z/(z-6i)$$
Is this correct? If so, why? If not, why not?
On
It's impossible to map circles that have a common point to circles that don't. You can map them to parallel lines, which is what your map does. Indeed, sending $6i$ to $\infty$, you mapped any circle passing through $6i$ to a line.
Recall that orthogonal lines/circles are mapped to orthogonal lines/circles. Since the imaginary axis is orthogonal to both of these circles, let's find its image... which is easy: the real axis, since $iy/(iy-6i)$ is always real.
Thus, both circles are mapped to vertical lines. To find exactly what they are, pick a point on each: where does $3i$ go, where does $-6i$ go...
Since these two circles touch each other, there is no Möbius transformation that takes them to two circles that don't touch each other. Otherwise what you suggest could be done.
If $w = \dfrac{z}{z-6i}$ then $w=\infty$ when $z=6i$, since you get $0$ in the denominator and something else in the numerator. If a Möbius transformation maps one point on a circle to $\infty$, then it maps the circle to a straight line. Since that point is on both circles, they both get transformed to straight lines. The two circles don't meet except at $6i$; therefore the two lines don't meet except at $\infty$, so they are parallel. If you find the images of two points on each circle other than the point that gets mapped to $\infty$, then you'll know exactly which two straight lines they are.
Some people draw a distinction between the meanings of the terms "Möbius transformation" and "Möbius transform". What we're talking about above are "Möbius transformations" or "linear fractional transformations". The Möbius transform, on the other hand, transforms one function of a positive integer to another via the Möbius $\mu$ function.