Some background information. I'm trying to estimate four probabilities $(p_1, p_2, p_3, p_4)$ (sum is equal to one) from nine numbers $(n_1, \dots, n_9)$ using Maximum Likelihood. (All $n_i$'s are greater than or equal to zero and at least one $n_i$ is positive.) The likelihood is given by
$$ \mathcal{L} % = \Bigl( \frac{n!}{\prod n_i !} \Bigr) \propto p_1^{2n_1} (p_1 p_2)^{n_2} p_2^{2n_3} (p_1 p_3)^{n_4} (p_1 p_4 + p_2 p_3)^{n_5} (p_2 p_4)^{n_6} p_3^{2n_7} (p_1 p_2)^{n_8} p_4^{2n_9} $$
The following equations result from equating the partial derivatives of the log-likelihood to zero (let's define $n = n_1 + n2 + \cdots + n_9$):
$$ f_1(p_1, p_2, p_3, p_4) = p_1 p_4 (2n_1+n_2+n_4+n_5 - 2 n p_1) - 2 p_2 p_3 (2 n p_1-(2n_1+n_2+n_4)) = 0 $$
$$ f_2(p_1, p_2, p_3, p_4) = 2 p_2 p_3 (n_2+2n_3+n_5+n_6 - 2 n p_2) - p_1 p_4 (2 n p_2-(n_2+2n_3+n_6)) = 0 $$
$$ f_3(p_1, p_2, p_3, p_4) = 2 n_5 p_2 p_3+n_4 (2 p_2 p_3+p_1 p_4) + (2 p_2 p_3+p_1 p_4) (2 n_7+n_8-2 n p_3) =0 $$
$$ f_4(p_1, p_2, p_3, p_4) = n_5 p_1 p_4-(2 p_2 p_3+p_1 p_4) (2 n p_4-(n_6+n_8+2 n_9)) = 0 $$
These equations can be solved numerically using Newton's method; it's very efficient. But I'm interested in having a light-weight, if less efficient method. So I took a fixed-point approach and devised these recurrence relations
$$p_1^{(n+1)} = p_1^{(n)} + \frac{1}{n} f_1(p_1^{(n)}, p_2^{(n)}, p_3^{(n)}, p_4^{(n)})$$
$$p_2^{(n+1)} = p_2^{(n)} + \frac{1}{n} f_2(p_1^{(n)}, p_2^{(n)}, p_3^{(n)}, p_4^{(n)})$$
$$p_3^{(n+1)} = p_3^{(n)} + \frac{1}{n} f_3(p_1^{(n)}, p_2^{(n)}, p_3^{(n)}, p_4^{(n)})$$
$$p_4^{(n+1)} = p_4^{(n)} + \frac{1}{n} f_4(p_1^{(n)}, p_2^{(n)}, p_3^{(n)}, p_4^{(n)})$$
Experimentally (I tested millions of random input vectors $(n_i)$), I observed:
They converge (though slowly compared to Newton's method, ~15 steps vs 3 steps)
If you start with a probability vector $(p_1^{(0)}, p_2^{(0)}, p_3^{(0)}, p_4^{(0)})$, it looks as if you keep getting a meaningful vector of probabilities, i.e., the $p_i$'s add up to one (this follows from the fact that the $f_i$'s add up to zero) and all $0 \leq p_i \leq 1$.
So things are looking good experimentally. But any attempt at proving convergence has failed so far, so I wonder if these equations can be shown to converge. They look as if there should be an elegant way to prove that, but I've failed to find it. (If I'm missing something obvious, please forgive me and be kind, as I'm not a mathematician.)