I am trying to find a least squares approximation for $a,b$ for the following data set $$g(x)=a+be^x,\quad \begin{array}{|l|l|l|l|} \hline i & 0 & 1 & 2 \\ \hline x & 1 & 2 & 3 \\ \hline y & 0 & 2 & 2 \\ \hline \end{array}$$
We want to find a stationary point of $$f(a,b)=\sum_i(a+be^{x_i}-y_i)^2$$ so we need $$f'(a,b)=\begin{pmatrix}\sum _{i}\:2\left(a+be^{x_i}-y_i\right)\\ \sum _i\left (a+be^{x_i}-y_i\right)2e^{x_i}\end{pmatrix}$$ Now we have to solve the the non linear system $f'(a,b)=0$. The jacobi matrix is $$J_{f'}=\begin{pmatrix}6&\sum _i2e^{x_i}\\ \sum \:_i2e^{x_i}&\sum \:_i2e^{2x_i}\end{pmatrix}=\begin{pmatrix}6&2\left(e+e^2+e^3\right)\\ 2\left(e+e^2+e^3\right)&2\left(e+e^4+e^6\right)\end{pmatrix}$$
Now the newton-raphson iteration? Choose the starting $x^0=(a_0,b_0)=(0,0)$ and apply $$x^1=x^0-\begin{pmatrix}6&2\left(e+e^2+e^3\right)\\ \:2\left(e+e^2+e^3\right)&2\left(e+e^4+e^6\right)\end{pmatrix}^{-1}\begin{pmatrix}-4\\ \:-4e^2-4e^3\end{pmatrix}=\begin{pmatrix}\frac{16e^5+8e^4+8e^3-8e}{8e^6-8e^5-8e^3-4e^2+12e}\\ \frac{-16e^3-16e^2+8e}{8e^6-8e^5-8e^3-4e^2+12e}\end{pmatrix}$$ now reaching this horrifying expression makes me believe I did something horribly wrong. Is this way of solving this problem correct?
I used $QR$ factorization rather than attempting to solve the normal equations. We have $$\begin{bmatrix}1&e\\1&e^2\\1&e^3\end{bmatrix}\begin{bmatrix}a\\b\end{bmatrix}\approx\begin{bmatrix}0\\2\\2\end{bmatrix}$$ After $QR$ factorization we have $$\begin{bmatrix}\frac1{\sqrt3}&\frac{-e-2}{\sqrt{6(e^2+e+1)}}\\ \frac1{\sqrt3}&\frac{-e+1}{\sqrt{6(e^2+e+1)}}\\ \frac1{\sqrt3}&\frac{2e+1}{\sqrt{6(e^2+e+1)}}\end{bmatrix}\begin{bmatrix}\sqrt3&\frac e{\sqrt3}(e^2+e+1)\\ 0&\frac{e(e-1)}3\sqrt{6(e^2+e+1)}\end{bmatrix}\begin{bmatrix}a\\b\end{bmatrix}=QR\begin{bmatrix}a\\b\end{bmatrix}\approx\begin{bmatrix}0\\2\\2\end{bmatrix}$$ Multiplying both sides on the left by $Q^T$ we arrive at $$\begin{bmatrix}\sqrt3&\frac e{\sqrt3}(e^2+e+1)\\ 0&\frac{e(e-1)}3\sqrt{6(e^2+e+1)}\end{bmatrix}\begin{bmatrix}a\\b\end{bmatrix}=\begin{bmatrix}\frac4{\sqrt3}\\ \frac{2(e+2)}{\sqrt{6(e^2+e+1)}}\end{bmatrix}$$ Dividing both sides on the left by $$\begin{bmatrix}\sqrt3&0\\ 0&\frac{e(e-1)}3\sqrt{6(e^2+e+1)}\end{bmatrix}$$ We have $$\begin{bmatrix}1&\frac e3(e^2+e+1)\\ 0&1\end{bmatrix}\begin{bmatrix}a\\b\end{bmatrix}=\begin{bmatrix}\frac43\\ \frac{e+2}{e(e-1)(e^2+e+1)}\end{bmatrix}$$ Finally, subtracting $\frac e3(e^2+e+1)$ times the second row from the first our solution is $$\begin{bmatrix}a\\b\end{bmatrix}=\begin{bmatrix}\frac{e-2}{e-1}\\ \frac{e+2}{e(e-1)(e^2+e+1)}\end{bmatrix}$$ Comparing numerically with the solution originally obtained from solving the normal equations, we have agreement as best we can tell, so we conclude that the solution in the original question should have worked in one step because the system of equations was in fact linear! So what went wrong? The problem was that the matrix $$\begin{bmatrix}-4\\-4e^2-4e^3\end{bmatrix}$$ Should have been $$\begin{bmatrix}-8\\-4e^2-4e^3\end{bmatrix}$$ Fix that and simplify and you should get the solution @Moo and I agree on.
EDIT: Not to mention that also you should have $$J_{f^{\prime}22}=2(e^2+e^4+e^6)$$ Rather than what you have there.