Introduction
The Kelly betting criterion is a betting strategy for repeated games of chance which works by wagering a fixed proportion of one's bankroll each time. That is, suppose I play a game of chance where I wager an amount $a$, win with probability $p$ and payout $(1+b)a$, and lose with probability $(1-p)$ with payout $0$. The Kelly betting criteria then advises wagering the proporition $$p+\frac{p-1}{b}$$ of my bankroll at each opportunity, and this formula is derived from maximizing the expected value of the logarithm of wealth. See Wikipedia for more details.
Question
How do we get to the point of deciding to maximize the expected value of the logarithm of wealth?
I'm also interested in why one should choose such a strategy - why should we just bet a fixed amount each time and not adjust our strategy to events as they occur?
Work
My (probably somewhat naïve) attempt at working this out has an error somewhere but I'm not sure where it is. My idea is that if one bets a fixed proportion, one can actually calculate the expected change in bankroll over some fixed term: if we bet a proportion $f$ of our bankroll each time, winning multiplies our stash by $1+bf$ and losing multiplies it by $1-f$. Starting with, say, one unit of money, and running for $n$ trials, the expected value to finish with is $$2^{-n}\sum_{i=0}^{n} \binom{n}{i} p^i(1-p)^{n-i}(1+bf)^i(1-f)^{n-i}=2^{-n}(1+f(bp+p-1))^n$$ which doesn't maximize at all like the Kelly criteria. What error am I making here?
Your error is you are maximizing expected terminal wealth after $n$ trials.
That solution is trivial and not particularly useful. For $p > 1/2$, bet all accumulated wealth on each trial. Unfortunately the probability of ruin is $1 -p^n$ which rapidly approaches $1$ with increasing $n$. If this seems paradoxical and is puzzling, then you should read about the St. Petersburg paradox.
The approach "doesn't maximize ... like the Kelly criterion" because it is not the Kelly criterion which seeks to maximize the expected logarithm of terminal wealth $\log W_n$ and, equivalently, the expected geometric growth-rate $(W_n/W_0)^{1/n}$. For independent and identically distributed trials this result is independent of $n$.
To the question of why it was decided to maximize the expected logarithm of wealth, the answer is it is one approach that has many desirable properties. For example, the expected time to reach a goal is less than any other betting strategy (proportional or non-proportional).