I understood the method of transforming a finite sized base-k numbers to another base (j) through the use of successive divisions
For example
$$12_{10} = 12/2 + 0*2^0 = 6/2 + 0*2^1 + 0*2^0 = 3/2 + 1*2^2 + 0*2^1 + 0*2^0 = 1100_2$$
But what about generalizing this to consider infinite expansions in some base?
I can't seem to reason a good strategy involving remainders for this problem.
You can never write down all of an infinite expansion -- the best you can hope for is to approximate the number as closely as you need for a particular purpose, or, in some cases, to specify a pattern that repeats ad infinitum in the digits.
With this in mind, converting an infinite expansion from base $k$ to base $j$ is no more problematic then writing down the expansion in the first place. You can determine it up to any given digit after the radix point by performing your procedure for finite expansions on a truncation of your base $k$ expansion.
Let me elaborate: say that you have a number $x$, represented in base $k$ by a possibly infinite expansion, which you can compute to any finite number of digits after the radix point (for instance $(.3333....)_{10}$ represents the number $\frac{1}{3}$, and given any natural number $m$ you can compute its base $10$ expansion to $m$ digits after the radix point).
To approximate $x$ in base $j$, you first truncate the base $k$ expansion of $x$ to include $m$ places after the radix point. That is, you approximate $x$ by a string of digits $(a_na_{n-1}.. a_0.b_1...b_m)_k$. Note that the number $y$ represented by this approximation differs from $x$ at most $k^{-m}$.
Now, suppose that you convert your expansion for $y$ from base $k$ to base $j$. How many digits of $y$ in base $j$ match those of $x$ in base $j$? To answer this, suppose that $l$ is the maximal natural number so that $j^l$ divides $k^m$. It's easy to see that your base $j$ expansion of $y$ matches that of $x$ at least up to the $(l-1)$-st digit after the radix point. You can run this argument backwards to get as many digits as you like of the expansion for $x$ in a given base, if that's what you are looking to do.
Note that numbers with finite expansions in one base need not have finite expansions in another, and vice versa: $$(.3333333....)_{10} = (.1)_{3}.$$
So, really, the only way to avoid this "infinity problem" when converting between basis is to restrict yourself to whole numbers.