Take two different boxes, one with $N$ red balls and one with $N$ blue balls. Remove balls one at a time from either box with equal probability. When only one color is left, the (expected value of the) number of balls should be about $\sqrt{N}$ when $N$ is large. I'm not sure how to show this. Intuitive heuristics are welcome, but I'm really after a rigorous solution.
One thought is to note that the number of trials required to draw $N$ blue balls is asymptotic to $N$, so we can look at $$P(S - F | S > F)$$
where $S$ and $F$ represent the numbers of successes and failures in a binomial random variable with parameters $n=N$ and $p=1/2$. As $N$ tends to infinity, the normal approximation to the binomial gives a half-normal distribution. Its expected value is the standard deviation, which here is $\sqrt{N}$. This seems very sloppy, though, and I'm not sure the approximation is valid.
Something feels off here, why would it be $\sqrt{N}$? Are you sure you've stated the problem correctly, where each column has only one color?
Notice that this process is reversible: you can start with 0 balls in both bins and add balls with equal chance on each step, and when a column fills up, you must put balls in the non-full one. To see this, imagine that each time you take out a red ball, you replace it with a ghost-red ball, and similarly with a ghost-blue ball, which don't get picked in your process. The distribution of red,blue balls at a given time T is the same in distribution as the number of ghost red,blue balls at time N-T.
Thus your question is equivalent asymptotically to how long it takes both columns to have at least one ball. Thus the waiting time has distribution something like $(1/2)^t$, for $N$ large, which has expected value 2.